I've read the entire post and I still don't know how he did it. And his books are incredibly well written.
The day after I announced publicly I'm writing a book, my wife announced she was pregnant. I still continued writing but the motivation of the first day was halfed. Two weeks later, she told me it was twins, I quickly decided to get a stable job. I still wrote in the evening and weekends.
The kids came earlier then expected, and we had to spend 2 months in the NICU. Writing took a backseat. It turned into a day or two a week. When the kids finally came home... If I could get a paragraph in every week I was lucky.
At one year old, I check the last modified date on the files and it says a month ago. The quarantine helped me get a few paragraph here and there every night. But a good part of my routine has been reading the book to remember what it was about.
I'm really impressed with how Bob Nystrom managed to get 200k words in with life fiercely trying to get in the way.
Your first book was amazing, and I can't wait to get this one too in my hands.
> we had to spend 2 months in the NICU. Writing took a backseat.
That sounds entirely like the right priority to me. Your book isn't going anywhere and kids change so fast at that age that it would be a shame to miss out.
> I used to do graphic design, and I have this weird tic where any time I see something that looks handwritten, I look for multiple instances of the same letter to see if they are different or if the design just used a handwriting font. It’s almost always a handwriting font and I die a little inside to see the illusion evaporate.
Not a graphic designer, but I do the exact same thing and feel the exact same way... Happy to see I'm not crazy alone
And his look like really neat hand-drawn diagrams! I certainly appreciate the degree of craftsmanship and care displayed throughout both books
I once published a magazine on typography and someone asked me how I got the cool sketched-with-pencil look for the headline on one article. I sketched it with a pencil. (It was especially ironic in that the article was about an aggressively digital type foundry).
It's a fascinating video but what struck me the _most_ was the manual kerning of the handwriting after it was scanned.
I know I'd have just copy/pasted all those identical echo() cells. Amazing to watch.
Yeah... it's one of those things where once I noticed the kerning was off I basically had to fix it. In many of the illustrations, I did a better job of writing with good spacing and then I didn't need to do as much fixup in Photoshop. But with the illustration I recorded for the video, I was so worried about making a mistake that I spread the letters out way too much.
Wow. I was already impressed with the book, but after reading about what was happening behind the scenes, I'm truly in awe. I especially appreciate the work you put into making sure that the code snippets are accurate to the actual source code. Far too many books omit crucial steps or contain faulty code.
I also found it really heartening how you kept working through the tough times. We're all going through some challenging times right now and I have found that working on my projects (including a compiler!) has given me a small bit of satisfaction.
Awesome, thanks muchly, Bob! Reading and learning from "Crafting Interpreters" inspired four of my own fun projects (the first three on github.com/benhoyt): littlelang, my first little language interpreter, in Go; loxlox, a Lox interpreter written in Bob's Lox language; goawk, a full AWK interpreter written in Go; and my current unreleased project, a Turbo Pascal to Go converter with the goal of transpiling the reconstructed ZZT source to Go.
> I liked the hand-drawn look. It furthered my meta-goal of making the material more approachable, more human. But I wanted to up the quality. I wanted them to be more intricate and contain more information. I wanted the drawings to be more detailed. Less margin doodles and more, well, illustrations. Maybe even some lowercase letters.
This is one of the most unusual aspects of the book, and something that immediately drew me to it. I'm curious how Nystrom hit on that idea, whether he experimented with vector graphics alternatives, and if so, what did he take away from those experiments.
The method described in the video looks extremely laborious. That said, the quality of the result speaks for itself.
I poured a lot of effort into creating something similar for my book, but could never get it quite right. So, I backed away to diagrams in Inkscape using squares and curves. The result has been well received, but I've always slightly regretted not being able to create something as elegant as what Crafting Interpreters has been able to achieve.
At some stage in the future, I'll try again I'm sure!
I "hand-draw" graphics for technical articles, but on an iPad, which works well for me. Here's how I came to my current style, by experiment and reverse engineering: https://philipkiely.com/essays/useful_charts.html
On creating the diagrams (buy scratch on pen and paper):
> I make mistakes sometimes, usually when lettering like “upvaluels” here. I fix that in Photoshop after scanning.
That's a pro tip, right there. All too often I get caught up in using the tool/program I'm in to do something completely, where it causes inefficient work loops. Part of that is realizing I'm stuck in a loop in my mind, and need to expand my horizons. Familiarity lets you know when to abandon one approach and use another (or at least when and where to shave off the rough bits you're subjecting yourself too). As an example, this doesn't happen to be in the shell. I'll happily take any textual output I'm given and process it through one or more tools to manipulate it, or if it comes to it, write a Perl one liner (or even convert that to a small Perl program with better syntax if it gets more complex) to manipulate and massage data, even if the purpose it to throw it back into the same program.
That may or may not be the most efficient way to tackle the problem, but it's a known solution, as opposed to not only not knowing how to to it in the tool I was using initially, but not knowing how to efficiently determine if it can even be done in that tool.
In this case, he's using pen and paper. He makes a mistake in the lettering after making sure the rest of the structure is correct, so he's got 90% of it correct and to his liking. He can start from scratch which is time consuming, or use some other physical medium tools (like white-out) to fix it, but it may not look quite right, or he can annotate the error with the fix and he can use a separate tool designed for quick and accurate visual manipulation (as opposed to creation, which it does but for many not as easily as with pen and paper) and layer a very effective fix in place afterwards.
Recognizing where and when you can do this effectively is likely one of the most beneficial skills you can train. It comes naturally with experience in an area, but being able to apply it to almost anything you do is a skill in itself.
> That's a pro tip, right there. All too often I get caught up in using the tool/program I'm in to do something completely, where it causes inefficient work loops.
I was caught in that loop with the first couple of chapters where I felt like the drawing itself had to be perfect and I would redraw them if I made a mistake. I got wrapped up in some sort of purist ideal (an affliction I am highly susceptible to).
Eventually I realized that all that matters is the final image, so I got better about taking advantage of Photoshop. Each line and letter in the diagram was still drawn by hand on paper, but sometimes I rearrange or fix mistakes in Photoshop.
> get readers to understand there’s no magic in there and nothing keeping them out
This was a key feature of the book for me; reading it really helped me grok that compilers/interpreters are just programs. Very complicated programs, perhaps, but still many of the same concerns that apply to my day job writing applications. It's code all the way down, and that feels like a hugely useful insight to have. Plus I think I now have the language bug myself. :)
Well done, Bob, and congratulations! Can't wait to be able to buy a copy.
And what's more they're simpler programs than most web applications. A compiler is conceptually a pure function. No need to worry about provisioning anything, any APIs, Docker images. A compiler is just a function from one array of bytes to another array of bytes.
Thank you so much. They are very interesting and well presented. I hope munificent is able to pick yet another interesting topic after this one has concluded!
I usually don't buy computer books anymore because I can't really finish them. I went through this post and signed-up for the mailing list to notify when its available in physical form. The effort he put in is an inspiration - if he has the tenacity and this much pure love towards this work, I have to try harder to overcome my inertia and read it once it is available.
I am definitely amazed and grateful like others by the work involved in writing this book. The attention to detail is visible from the first parts of the README. Clear explanations for everything.
I am also a sucker for clean written Makefiles. I have "Managing Projects with GNU Make" book that I keep in my to-read list (shamefully putting reading it off).
Checking out the Makefile in the repository was a delight.
Once the print and eBook editions are ready, buying one of those will be the best way to say thanks. But, honestly, right now my family and I are in so much better shape than a lot of people. I'm able to work from home in a well-paid field. We're all safe and healthy.
So if you feel like doing something kind for someone, instead of throwing some cash my way, consider donating to a food bank or other local charity that can help people financially impacted by COVID-19.
Bob, you absolutely rock. Congratulations on this massive accomplishment. I thought back to the sporadic times we hung out and you'd mention the book or illustrating almost in passing... I had no idea the depth of what you were trying to achieve. Then, finally, you did! And you did it astride many unique and challenging passages in life. I especially appreciate how you built it into a habit. You knew you'd finish, you just had to keep going. Right on!!
Holy smokes. I thought I couldn’t love your output anymore than I already did, but this is something else. This little behind the scenes is... enlightening? inspiring? overwhelming?
Thank you for everything you’ve been through to provide this wonderful gift to our community.
Great writing. This should replace the syllabus for very many Comp Sci / Soft Eng "Intro to Compilers/Interpreters" courses as many do not come close to this level of detail.
I'm considering it for the course I teach! Although one thing that draws me is it doesn't have too much extraneous detail off the main narrative thread.
I currently use parts of the textbook Programming Language Pragmatics [1], one of the semi-standard texts used by a lot of universities (plus some of my own course notes). That book isn't really readable straight through though. It's 992 pages long, and some of the chapters get bogged down in a ton of coverage of the landscape of design and implementation choices. As a reference book that has some pros: you can look up something like looping constructs, and get a very detailed tour of how languages from Algol-68 through Modula-2 and C# have taken different design and implementation strategies. But that's not the same as reading something as your first introduction to a subject.
> I currently use parts of the textbook Programming Language Pragmatics
That was the first PL book I read. I really liked it, but you're right that it's like a survey of the entire landscape. For a first book, I personally like getting a single guided tour so that I feel like I'm going somewhere and not just looking at everything from a distance.
> In the center, you have the new code being added in this snippet. It may have a few faded out lines above or below to show you where to insert it in the existing code. There is also a little blurb telling you which file and where in the file it goes. If it says “replace _ lines”, there was some previous code between the faded lines that you need to remove and replace with this snippet.
> ...lox/Scanner.java in scanToken() replace 1 line
How are these done? Are they compiled from actual diffs, or what? I looked at the source but the CSS/HTML just puts `source-code-narrow` in a class and doesn't tell me anything about them.
The build system figures out all of this location information automatically based on the snippet markers in the code.
The code is pretty hairy since it's a pile of Python that sort of accreted organically as I was writing the book. I have been sorely tempted to rewrite the whole thing but abstained because I wanted to focus on writing.
The basic idea is that the build system walks every line of code. As it steps through, it keeps track of:
* What is the current source file. This is pretty easy to track. :)
* What is the current stack of snippet markers? The marker comments support nesting because it makes it easier to, say, introduce a function in one chapter and then insert a couple of lines into the middle later. The topmost snippet on the stack indicates which snippet owns the current line of code.
* What is the current surrounding declaration? Using some fairly hacky regexes, it tracks whether it is inside a class, method, struct, etc.
Given that, the build system knows for every line of code which snippet it first appears in, which snippet it disappears in if it's one that gets replaced, and the surrounding declaration if any.
When compiling a chapter, when it inserts a snippet, it finds all of the lines owned by that snippet. It also grabs some of the preceding and following lines (which is made trickier by the fact that some of the surrounding lines that may be textually present in the actual source file may not exist in the code as it is at the point in time that that snippet is inserted). And it looks for any lines that are removed by that snippet so that it knows what number to put in the "replace _" annotation.
From that, it can create the HTML that shows the code for the snippet, some surrounding lines if needed, and a location annotation describing the file, declaration, and number of replaced lines.
It's a pretty neat tool, though the implementation is fairly hacky. It's just good enough to produce the correct output for this one book and this book only. :)
Wow. Overall, I wonder if it wouldn't've been easier to write the code samples as diffs, and then compile them appropriately to a regular code sample + summary?
Congrats on finishing the book! This has been a real joy to read in installments. I will look forward to being able to sit down with a print edition in the future. Hats off to you sir. Cheers.
What a fantastic blog post - both technically as well as deeply human and touching. Really looking forward to seeing the physical book coming to life.
This part makes it clear the passion and attention to details that the author pours into his books:
"I hand letter everything. It takes a long time. I used to do graphic design, and I have this weird tic where any time I see something that looks handwritten, I look for multiple instances of the same letter to see if they are different or if the design just used a handwriting font. It’s almost always a handwriting font and I die a little inside to see the illusion evaporate.
Well, this is my damned book and no reader will ever feel that disappointment. Every single fucking letter in every one of the illustrations was hand lettered and is unique."
I had the pleasure of starting your book a few days ago and I am really impressed. Every aspect of it is excellent: it feels comprehensive but easy to read, the content is very well-organised and the illustrations are very nice.
I will definitely buy the book once it is available :)
How does this work? Spending 4 years, full time from what it seems, on a book with a (let's face it) niche audience. While having (again, it seems from the post) a family with children, living in a first world country, flying regularly (I'm using this as a shortcut for 'getting out of the house for things more expensive than a hike in the woods'). So that's an opportunity cost of 200k for a book that would a runaway success if it made... 50k over its lifetime? Is there a backstory I'm missing here - is the author an already well-known internet celebrity living off Patreon or something like that (I couldn't find anything on the blog that indicates that)
> Spending 4 years, full time from what it seems, on a book with a (let's face it) niche audience.&
Four years, yes, but much less than full time. Usually around an hour a day, often less.
I work full time at Google and the book was a (very involved) hobby.
> that's an opportunity cost of 200k*
Yes, there's definitely a financial opportunity cost. But when I see someone working a job they hate eight hours a day, I often wonder what the satisfaction opportunity cost of that is. What could they be doing instead that would be more meaningful to them?
I'm unlikely to make enough money from the book to justify the time (though my first book made me much more than I expected), but I do think it will bring enough other rewards to be worth the time.
The author has a full-time job at Google and apparently wrote the book as a passion project, not for the money. The knowledge within is an incredible contribution to the public good.
Loved this book, it's amazing work. It's already been helpful to me when implementing a toy language, even without working through the book (which I still intend to do).
Just wanted to say congratulations on finishing the last chapter of the book. It’s a huge step and hope to see it printed later down the road. Stay safe.
I went through most of the Java half of this using c# right around the time that portion was completed. I think it may be time to build the C half, the question now becomes doing it in c or C++ (but still hand writing the hash table and similar things instead of falling back to the STL)
that's a fantastic read! thanks for sharing the process with us. really surprised that the code was complete the very first year and the real work was to put all that in writing, snippets, and illustrations.
> Writing this all out makes me sound like a crazy person. What the hell am I doing with my life? Or, more importantly, what could I have been doing instead of doing all that?
That's why your book is so good! Singular focus on one subject like this either causes insanity or genius.
Examples: redis, Linux, sqlite, dwarf fortress, etc. I (believe, not sure) all were started by a single person who was insane by any measure to start those projects, but they became something that people love.
I feel like this might be a required but insufficient condition for what Bob has achieved here. With both of his books, he has taken a somewhat dry topic (object oriented patterns, and now compilers / interpreters), and turned them into books anyone can understand. More importantly, he has somehow made them interesting, in a way that I have rarely seen before.
I suspect you're right, the focus probably helps with the process of distilling the core concepts (which are necessary to convey meaning), as well as the writing style. Picking the right analogies, the right illustrations, then executing them to perfection. I guess the passion for the topic shines through in his writing. Massive congrats to Bob!
Because the topics were interesting to him, and his goal was to convey what made them interesting. All great teachers have this in common regarding the subjects they teach.
This book is great, the only thing missing is how to write fully featured type checker. Hard to find a resource on type checking that has the same practical style as this book.
By the way, does anyone know of any good tools to convert the whole site into a single PDF with working links? Using pandoc or something? If not, I'll probably sit down and write a script myself.
The day after I announced publicly I'm writing a book, my wife announced she was pregnant. I still continued writing but the motivation of the first day was halfed. Two weeks later, she told me it was twins, I quickly decided to get a stable job. I still wrote in the evening and weekends.
The kids came earlier then expected, and we had to spend 2 months in the NICU. Writing took a backseat. It turned into a day or two a week. When the kids finally came home... If I could get a paragraph in every week I was lucky.
At one year old, I check the last modified date on the files and it says a month ago. The quarantine helped me get a few paragraph here and there every night. But a good part of my routine has been reading the book to remember what it was about.
I'm really impressed with how Bob Nystrom managed to get 200k words in with life fiercely trying to get in the way.
Your first book was amazing, and I can't wait to get this one too in my hands.
That sounds entirely like the right priority to me. Your book isn't going anywhere and kids change so fast at that age that it would be a shame to miss out.
Not a graphic designer, but I do the exact same thing and feel the exact same way... Happy to see I'm not crazy alone
And his look like really neat hand-drawn diagrams! I certainly appreciate the degree of craftsmanship and care displayed throughout both books
As a someone put it on YouTube, the time-lapse video of the entire process is like ASMR for geeks https://www.youtube.com/watch?v=iN1MsCXkPSA
I also found it really heartening how you kept working through the tough times. We're all going through some challenging times right now and I have found that working on my projects (including a compiler!) has given me a small bit of satisfaction.
Really looking forward to going through this.
https://tylerayoung.com/2017/01/23/notes-on-game-programming...
> I liked the hand-drawn look. It furthered my meta-goal of making the material more approachable, more human. But I wanted to up the quality. I wanted them to be more intricate and contain more information. I wanted the drawings to be more detailed. Less margin doodles and more, well, illustrations. Maybe even some lowercase letters.
This is one of the most unusual aspects of the book, and something that immediately drew me to it. I'm curious how Nystrom hit on that idea, whether he experimented with vector graphics alternatives, and if so, what did he take away from those experiments.
The method described in the video looks extremely laborious. That said, the quality of the result speaks for itself.
http://journal.stuffwithstuff.com/2014/04/22/zero-to-95688-h...
At some stage in the future, I'll try again I'm sure!
> I make mistakes sometimes, usually when lettering like “upvaluels” here. I fix that in Photoshop after scanning.
That's a pro tip, right there. All too often I get caught up in using the tool/program I'm in to do something completely, where it causes inefficient work loops. Part of that is realizing I'm stuck in a loop in my mind, and need to expand my horizons. Familiarity lets you know when to abandon one approach and use another (or at least when and where to shave off the rough bits you're subjecting yourself too). As an example, this doesn't happen to be in the shell. I'll happily take any textual output I'm given and process it through one or more tools to manipulate it, or if it comes to it, write a Perl one liner (or even convert that to a small Perl program with better syntax if it gets more complex) to manipulate and massage data, even if the purpose it to throw it back into the same program.
That may or may not be the most efficient way to tackle the problem, but it's a known solution, as opposed to not only not knowing how to to it in the tool I was using initially, but not knowing how to efficiently determine if it can even be done in that tool.
In this case, he's using pen and paper. He makes a mistake in the lettering after making sure the rest of the structure is correct, so he's got 90% of it correct and to his liking. He can start from scratch which is time consuming, or use some other physical medium tools (like white-out) to fix it, but it may not look quite right, or he can annotate the error with the fix and he can use a separate tool designed for quick and accurate visual manipulation (as opposed to creation, which it does but for many not as easily as with pen and paper) and layer a very effective fix in place afterwards.
Recognizing where and when you can do this effectively is likely one of the most beneficial skills you can train. It comes naturally with experience in an area, but being able to apply it to almost anything you do is a skill in itself.
I was caught in that loop with the first couple of chapters where I felt like the drawing itself had to be perfect and I would redraw them if I made a mistake. I got wrapped up in some sort of purist ideal (an affliction I am highly susceptible to).
Eventually I realized that all that matters is the final image, so I got better about taking advantage of Photoshop. Each line and letter in the diagram was still drawn by hand on paper, but sometimes I rearrange or fix mistakes in Photoshop.
This was a key feature of the book for me; reading it really helped me grok that compilers/interpreters are just programs. Very complicated programs, perhaps, but still many of the same concerns that apply to my day job writing applications. It's code all the way down, and that feels like a hugely useful insight to have. Plus I think I now have the language bug myself. :)
Well done, Bob, and congratulations! Can't wait to be able to buy a copy.
And what's more they're simpler programs than most web applications. A compiler is conceptually a pure function. No need to worry about provisioning anything, any APIs, Docker images. A compiler is just a function from one array of bytes to another array of bytes.
Thank you so much. They are very interesting and well presented. I hope munificent is able to pick yet another interesting topic after this one has concluded!
I am also a sucker for clean written Makefiles. I have "Managing Projects with GNU Make" book that I keep in my to-read list (shamefully putting reading it off). Checking out the Makefile in the repository was a delight.
Is there any way to pay you for this book? :-)
Once the print and eBook editions are ready, buying one of those will be the best way to say thanks. But, honestly, right now my family and I are in so much better shape than a lot of people. I'm able to work from home in a well-paid field. We're all safe and healthy.
So if you feel like doing something kind for someone, instead of throwing some cash my way, consider donating to a food bank or other local charity that can help people financially impacted by COVID-19.
Thank you for everything you’ve been through to provide this wonderful gift to our community.
I currently use parts of the textbook Programming Language Pragmatics [1], one of the semi-standard texts used by a lot of universities (plus some of my own course notes). That book isn't really readable straight through though. It's 992 pages long, and some of the chapters get bogged down in a ton of coverage of the landscape of design and implementation choices. As a reference book that has some pros: you can look up something like looping constructs, and get a very detailed tour of how languages from Algol-68 through Modula-2 and C# have taken different design and implementation strategies. But that's not the same as reading something as your first introduction to a subject.
[1] https://www.cs.rochester.edu/~scott/pragmatics/
That was the first PL book I read. I really liked it, but you're right that it's like a survey of the entire landscape. For a first book, I personally like getting a single guided tour so that I feel like I'm going somewhere and not just looking at everything from a distance.
> In the center, you have the new code being added in this snippet. It may have a few faded out lines above or below to show you where to insert it in the existing code. There is also a little blurb telling you which file and where in the file it goes. If it says “replace _ lines”, there was some previous code between the faded lines that you need to remove and replace with this snippet.
> ...lox/Scanner.java in scanToken() replace 1 line
How are these done? Are they compiled from actual diffs, or what? I looked at the source but the CSS/HTML just puts `source-code-narrow` in a class and doesn't tell me anything about them.
The code is pretty hairy since it's a pile of Python that sort of accreted organically as I was writing the book. I have been sorely tempted to rewrite the whole thing but abstained because I wanted to focus on writing.
The basic idea is that the build system walks every line of code. As it steps through, it keeps track of:
* What is the current source file. This is pretty easy to track. :)
* What is the current stack of snippet markers? The marker comments support nesting because it makes it easier to, say, introduce a function in one chapter and then insert a couple of lines into the middle later. The topmost snippet on the stack indicates which snippet owns the current line of code.
* What is the current surrounding declaration? Using some fairly hacky regexes, it tracks whether it is inside a class, method, struct, etc.
Given that, the build system knows for every line of code which snippet it first appears in, which snippet it disappears in if it's one that gets replaced, and the surrounding declaration if any.
When compiling a chapter, when it inserts a snippet, it finds all of the lines owned by that snippet. It also grabs some of the preceding and following lines (which is made trickier by the fact that some of the surrounding lines that may be textually present in the actual source file may not exist in the code as it is at the point in time that that snippet is inserted). And it looks for any lines that are removed by that snippet so that it knows what number to put in the "replace _" annotation.
From that, it can create the HTML that shows the code for the snippet, some surrounding lines if needed, and a location annotation describing the file, declaration, and number of replaced lines.
It's a pretty neat tool, though the implementation is fairly hacky. It's just good enough to produce the correct output for this one book and this book only. :)
This part makes it clear the passion and attention to details that the author pours into his books:
"I hand letter everything. It takes a long time. I used to do graphic design, and I have this weird tic where any time I see something that looks handwritten, I look for multiple instances of the same letter to see if they are different or if the design just used a handwriting font. It’s almost always a handwriting font and I die a little inside to see the illusion evaporate.
Well, this is my damned book and no reader will ever feel that disappointment. Every single fucking letter in every one of the illustrations was hand lettered and is unique."
I will definitely buy the book once it is available :)
Four years, yes, but much less than full time. Usually around an hour a day, often less.
I work full time at Google and the book was a (very involved) hobby.
> that's an opportunity cost of 200k*
Yes, there's definitely a financial opportunity cost. But when I see someone working a job they hate eight hours a day, I often wonder what the satisfaction opportunity cost of that is. What could they be doing instead that would be more meaningful to them?
I'm unlikely to make enough money from the book to justify the time (though my first book made me much more than I expected), but I do think it will bring enough other rewards to be worth the time.
Congratulations on completing your work.
In other words I need some
'Crafting "Crafting Interpreters"' Interpreters
That's why your book is so good! Singular focus on one subject like this either causes insanity or genius.
Examples: redis, Linux, sqlite, dwarf fortress, etc. I (believe, not sure) all were started by a single person who was insane by any measure to start those projects, but they became something that people love.
If this is insanity, please keep going insane!
I suspect you're right, the focus probably helps with the process of distilling the core concepts (which are necessary to convey meaning), as well as the writing style. Picking the right analogies, the right illustrations, then executing them to perfection. I guess the passion for the topic shines through in his writing. Massive congrats to Bob!
By the way, does anyone know of any good tools to convert the whole site into a single PDF with working links? Using pandoc or something? If not, I'll probably sit down and write a script myself.