I feel like spaced repetition is exactly what you shouldn't do. Spaced repetition is great for rote memorization of arbitrary facts but programming is about how to think about and solve problems.
If you master the methods and ideas, you should be able to derive the answers on the spot. That's better than memorization because then you can actually deal with real interviewers who 1) Want to hear your train of thought and 2) Will give you random changes or variants to problems.
It's not about memorizing solutions, because in reality when are you ever going to reverse a linked list or balance a red-black tree? No, so if you're going to put in the effort anyway, you should learn the concepts and understand when to apply them so your knowledge is actually applicable. This will actually make you a better programmer, and you'll do better with the interviews that are actually good (I speak with experience giving interviews at FB).
Learning the concepts and using SRS are not only not mutually exclusive, they're complementary. There's no quicker way to "grok" a deep technical or mathematical insight than to simply expose yourself to it in a diversity of cognitive contexts. SRS works incredibly well because it generates this diversity as a function of regularly scheduled time. Think of it as entropy generation coupled with heavy reinforcement.
Right. I learned what binary trees were in college and didn't directly use them much afterward. Since interviews sometimes ask about trees, I started my practice of solving tree problems by practicing how to invert a binary tree. I practiced it so much that I can invert a binary tree from memory now, so that I won't get bounced from an interview like the inventor of Homebrew was from a Google interview.
Most of the concepts I learned, although I don't remember learning Kadane's algorithm for solving the maximum subarray problem in school, so I've learned some things like that. I can also implement a solution for an associated Leetcode problem in one line of code, so I wound up memorizing that too.
OP says "If you master the methods and ideas, you should be able to derive the answers on the spot", but I hear from a lot of people that does not happen, even (implicitly) in this thread. Any how, even if I use OPs method, if I do the same problem over and over and over again I'm going to wind up memorizing it any how.
Learning is like nutrition. There is no ”one true way", instead there are lots of people and each one has their own approach to learning.
Some people are closer to each other, and upon realizing that they believe they've found a pattern. But it's like when people tell you that you need to supplement x in your diet. It's possible that x helped you. And it's even possible that you'll find other people with the same nutritional deficit, creating the illusion that everyone would benefit from supplementing. But that's just what it is: a premature extrapolation from anecdotes
I was at Google when that happened, and had some inside sources (ie gossip from the guys who interviewed him, who said they didn't even ask any tree questions, and memegen).
I tried to find some outside sources, but nothing about Howell admitting his 'artistic license' outright. So take my fallible memory with a large pinch of salt.
So, inverting is just swapping the left and right pointers for the entire tree? Why would you ever do that? That will flip the ordering of the tree and none of your search or balancing functions will work anymore. Shut up! Don't ask questions! Uh, ok.
I thought we were done with the myth that once we understand something, we don't need to memorize. I've come to the realization that memorization is a core competency. The more I have memorized, whether it's syntax or concepts or problems and how to solve them, the better equipped I am and the more productive I am in the future when faced with even novel problems. Memorization saves me insane amounts of time in day to day programming and gives me an edge in being able to solve and understand problems. I don't just have a deep understanding of the code bases I work with, I have a lot of details memorized about what parts there are, how they work, how they're interconnected, certain technical details, etc. And sure, I could just check, but when having meetings about technical problems and planning our projects, it's an insane help to know so much off the top of my head and that I only need to check a handful of things.
Spaced repetition is nothing other than intentionally refreshing something you currently know, to ensure that in a month or a year or longer, you'll still know it. It's not your place to judge what to memorize or not. I'd suggest you should also use spaced repetition on the methods and ideas, since we're very prone to forgetting that too. I would also never criticize anybody for memorizing something, since it's literally just doing intentionally what we would otherwise do haphazardly and by accident.
This is a common misconception not supported by scientific evidence.
What experimental psychology and neuroscience research has found is that the better people memorize topics, the better they understand and reason about them.
Edit: Of course, you can memorize things without any effort to understand. But if you are making these efforts, having an efficient and effective memorization technique can help a LOT.
One of the things that puzzles me is when I see say Anki flashcards for mathematics things that I never learned like that. I never intentionally learned anything in mathematics in that was and it feels very 'wrong' to me when I see people trying to learn like that. I did a lot of practice looking back, but there was absolutely no conscious memorisation - the nearest I got was at university level when I would sometimes run through the main steps in a long proof before the exams, but it would be at quite a high level when I did.
I don't know if I'm just lucky in terms of how my brain works in some way and I found I had a pretty good 'natural' memory when it came to things like learning languages, and I appreciate that this is totally anecdotal. That book looks interesting however and as somebody who teaches, I'm interested in what works for everybody, not just me.
SRS is not meant as the basis of learning. You study and learn first, then you input what you've learned in SRS, and it'll help you to solidify those learnings and keep them readily available when you need them in practice.
I am always deathly afraid of forgetting a definition/key result in an exam.
Whenever I practice, I always have my notes in front of me and I focus on puzzling through the problem at hand rather than testing my memory.
But in an exam, for me there us always the possibility of failing to recall some fact. To combat this, I record every not-completely-obvious fact into Anki and aggressively practice it in the run-up to the exam.
The multiplication tables and two digit arithmetic are 100% memorization, and more challenging multiplication/arithmetic is just learning to break the problems into memorized chunks and then combine them. Math is 100% memorization.
Spaced repetition is a good way to remember anything, including methods. Just review the method (maybe with a small practice problem) on an SRS-based schedule, to hone your skill. When you encounter an issue in the wild, having committed to memory ways to solve it will speed up the actual execution.
You’re partially right that it’s not about memorizing solutions; it’s about committing patterns to memory.
I well doubled my income after using SRS to master leetcode/algo/ds patterns. Implementing SRS was a catalytic moment in my interview prep training that really accelerated the volume of patterns I held in the RAM of my brain.
I disagree. To be fair, I used to think this too, but then when I started to actually used spaced repetition in the process of learning more complex subjects I became surprised by how much more fluid my reasoning became.
Writing good flashcards is about
- noticing similarities and differences,
- exploring variations of a concept,
- finding generalisations of related concepts,
- learning properties of a concept, etc.
All these things are things you use when reasoning also about more complex ideas.
In other words, spaced repetition is an efficient way to raise the baseline for what counts as a "fundamental fact" which makes it possible to think at a higher level of abstraction.
That’s a fair perspective but consider that spaced-repetition can be an “MVP” of the broader concept of adaptive / personalized learning. You track some state about your knowledge of the world, in this case your performance on a flash card, and optimize the next piece of educational content around it.
It feels obvious to me that, in theory, we could do a lot more to leverage your current world knowledge to recommend the next piece of educational content, to optimize not just your understanding but also other things like how engaged you are. For example, a system could throw you an easier question if it helps you focus longer.
When I see the state of current interview prep it’s mostly “here’s a big list of questions” , perhaps tagged by difficulty and grouped under related topics such as “graph problems” or “tree problems” but I’m personally convinced a more sophisticated system could serve you the best possible next problem to stretch your brain in the right way.
Spaced rep is simply the proven starting point. The rest of adaptive learning has a bit of a troubled past because it was trendy to VCs , but pushed on educators and presented more as an alternative to a teacher rather than something to augment a teacher. I worked for a company that raised 100M+ to work on it but the CEO was great at terms sheets but uninterested in actually building a great education project.
But the reason that I joined the company was the high level idea still resonates with me. Surely many HN users have a big list of things they want to learn- perhaps about LLMs. But a typical course will have zero knowledge on where you’re starting as a student . It might bore you with stuff you know already or take for granted key prerequisites and skip them.
Anyway, I’m currently building my own adaptive learning platform but focused on helping professional poker players learn game theory . My idea being that it’s an easier ed tech app to bootstrap as the knowledge very directly translates to money. And really the same criticism applies because you can’t truly memorize a game tree , it’s more important that you build a high level conceptual understanding. But , as much as rote memorization deserves to be maligned when done in isolation, it’s not so bad when done as part of a broader learning strategy. For example, you can’t memorize vocab to learn Spanish but certainly knowing 5000 words in Spanish is a very nice starting point compared to not knowing any words. And tools like spaced rep have been proven by research to help with that goal and I view as a pathway to more broad adaptive learning strategies.
> Spaced repetition is great for rote memorization of arbitrary facts
Traditionally, yes. But several people use the SR algorithm to simply do the same problems over and over (practice), without the memorization component.
Looking at a typical SR algorithm, you're not really going to memorize doing the same medium/high Leetcode problem unless you actively try to memorize it. The interval sequence will be something like: 2, 5, 13, 35, 120, etc. By repeatedly solving the problem so few times in such a long period (and while solving other problems at the same time): You're not really going to memorize it accidentally.
Bullshit. You can't get to mastery without memorizing (at least) the fundamentals into muscle memory.
Can you just "master" the 7 notes and what octaves are and all other theory to be able to play beethoven? Or should you play a few songs repeatedly till they're muscle memory practicing in parts or full and then moving over to other songs?
Should you learn that the integral means the area under the graph to "master" integration or do you still need to run through dozens of same/similar problems till you can solve them without an error?
You need to memorize but its a fools errand to start and stop at memorizing leetcode problems. You need to get the fundamental problems (binary search, bfs, dfs, sliding window) solve them repeatedly to a point it just flows naturally and then apply it to problems that build on them and still repeat those till they get natural too.
I believe this is incorrect, primarily because this approach has never worked for me. Incidentally, just today I found an article that has put this feeling into words really well:
I think that’s a bit missing the mark. The emphasis in the story, contrary to the parent comment, is that rote memorization and repeated problem solving, lead to understanding. Whereas most American education claims the exact opposite: that understanding comes before fluency.
A lot of my personal opinion follows, and I'd be very, very happy to hear dissenting opinions! I believe there are at least two valid goals for learning that are relevant to memorization:
1) Learning for mastery / competency to apply the skills to real life endeavors
2) Learning so you can meet some standard necessary to enable certain opportunities in life
It seems your opinion comes from a personal value bias towards #2 and I share that bias, but perhaps not as dominantly as you do (at the moment). For both of these, memorization can be very helpful, even optimal. But the application of spaced repetition would be different depending on the goal.
On memorization applied to learning for genuine personal mastery:
I believe memorization is optimal for foundational axiomatic knowledge. A good example is quick addition and times tables (and other arithmetic). I believe:
1) People should consider it a necessary life skill to be able to perform reasonably complex arithmetic manually in your head. For example, 4462+9241=13703 or 25120=3000. Many people will disagree on the nuances on where this begins and ends, and that's okay. For example some people will feel that people should be able to multiple 27123 in their head, others would feel it's okay for something like that to be done on paper. But for me, the point is that there exists a category like "things you should be able to do, but it's fine not to have memorized". More importantly, I believe it would be suboptimal to spend a significant amount of your life memorizing an increasing sample of these, but it's reasonable to spend time maintaining the skill of manual mental arithmetic and probably reasonable to spend time improving the speed and accuracy with which you do it.
-- 1a) A nuanced subset of this skill example is "estimation", for situations where it's appropriate. It might take longer for some people to manually calculate 27123 (or whatever) than for them to pull out their cell phone or ask their Apple Watch. If the situation only needs them to have an upper and lower bound within they might break it down to {lower_bound = 25120 = ((256)2)10 = (150210) = 3000; upper_bound = 30125 = 3055 = 150510/2 = 75010/2 = 7500/2 = 3750} and they could determine the answer is around 3350 or so, but absolutely 100% definitely between 3000 and 3750. And they could still do this type (if not degree) of estimation faster (and more stealthily, which can be important sometimes, like during negotiations/haggling) than pulling out a cellphone and calculating.
2) People absolutely cannot do arithmetic if they do not have certain foundational axioms memorized. If you do not have 1+1=2, 2+1=3, 3+1=4, etc memorized (counting), you literally cannot do arithmetic. That would be an absurdly low-functioning adult and essentially means you don't know how to count. But technically* you only need to be able to count to add, subtract, and multiply (by maintaining two counts simultaneously) any real numbers. Obviously just counting is a helpful, but not optimal, level of memorized knowledge to be competent at arithmetic. It would plainly be helpful to also memorize things like 7+6=13 and 152 = 30, and have instant recall for these rather than re-calculating them manually every time you need them. Which brings me to:
-- 2a) I believe it is helpful to memorize more than what many people think it is helpful to memorize. We generally are taught in primary school to memorize times tables up to 10x10. But perhaps it would be useful to extend this beyond 10x10, at minimum for some additional prime numbers which might show up frequently and cannot be quickly factored to smaller numbers. I could see it being super nice to have my times tables memorized up to 100x100, and additional addition/substraction memorized. I feel this way because despite having done well in three semesters of calculus and differential equations, I have always had a few "holes" in my brain's lookup table for certain simple addition and multiplication - specifically it really slows me down to need to manually calculate 86 (I have to do 832) or add 16+7 (I personally have to do 20+4+3). It's very frustrating to hit these basic roadblocks. Often I wonder how much happier I'd be if, at a young age, I had personally chosen to endeavor to memorize an unusually large amount of basic integer combinations, rather than just focus on memorizing the minimum necessary to get 100% on my K-5 coursework. Why do I not know 86? Because I took the less ambitious route of just breaking it down to 446 or 823 every time. Which brings me to:
On memorization for learning so that you can meet some standard necessary to enable certain opportunities in life:
1) I think we can all agree that some things are worth striving for, but are gated behind formal or informal tests which aren't relevant to the skills and temperament that actually enable people who do those things to do them well. For many people, that's "leetcode", but it can also be things like a college graduate memorizing and practicing ad nauseam a powerpoint delivery for a technical role for which they'll never formally present anything in their entire career. It could also be memorizing certain key motifs in a history or politics or English class which are actually wrong. The student may even know* its wrong, but it must still be memorized and regurgitated to get the best possible grade for their transcript.
1a) I think we can all agree that when cramming for an exam for which knowledge is not particularly desired to be retained, the best memorization technique is the one that takes the least amount of time to remember the facts just long enough to regurgitate them for the exam. I've discovered that this method is not the most commonly evangelized "spaced" repetition, but rather a Leitner system implemented in a way such that you actually minimize the time spacing as much as possible. For me, this uses digital or physical flash cards. Make a deck containing all facts you want to be able to maccess n days later during the exam. Go through the deck one at a time and any you get correctly, move to a new pile on the right; any you get wrong, move to a new pile on the left. Keep going repeating this with the left-most pile without breaks or pauses until you are getting enough % correct to get the desired score on the exam (with some safety margin), or you get exhausted or stop improving (then sleep and do it again the next day). This takes up far less actual time than spaced repetition, but you'll also forget the information fairly quickly as well and retain much less of it permanently.
--------------
To bring it back to the actual topic of memorizing leetcode: For me personally, I think memorizing some leetcode can be useful in three ways.
1) (For both mastery and credentialism) This is stretching the definition of leetcode a lot, but it's worth mentioning. I think it's a good idea to memorize syntax and functions/methods of the language, libraries, frameworks, and tools that you intend to use. I switch languages a lot at work, and I switch jobs a lot. It genuinely shocks me how much I can accomplish in some languages for which I can't even write a syntactically correct for-loop. Like if you put a blank paper and a pen in front of me, I could not write a for-loop -- but I can copy/paste one loop from language documentation or google result or ChatGPT and then copy/paste that line from my source code over and over until I have a working API or application or website from scratch in under a day. But I hate it, and not knowing all the syntax makes me slower. Not knowing what methods are available for a certain class is very frustrating. I don't need to have it memorized to do my job, but when I'm working in a language that I do know super-well, I have a much happier time doing the job.
2) (For mastery) Some leetcode could be considered "merely slightly more complex fundamental concepts". Nobody would argue that you should be able to write a for loop from memory in your language, but I think it's reasonable to say that it could very well be useful to be able to have instant recall of some amount of slightly more complex algorithms rather than needing to google them, like implementing sliding windows using trees. Or (for low-resource embedded programming) things like exponentially weighted moving average. Etc. What these are depends on what you desire to have mastery over, but memorization via spaced repetition can ensure that these "leetcode" techniques are always available to you the instant you need to implement them.
3) (For credentialism) Yes, it would be wonderful if we all had the brainpower to instantly figure out all leetcode problems fluently on the fly without awkward pauses. But most programmers can't, and it's better for the interviews if you can quickly, decisively, and confidently present everything that the interviewers are looking for. For this, I believe it can be genuinely worth spending some of your life memorizing leetcode if it means the difference between making $120,000 vs. $225,000 for the next five years. You probably won't ever use two-sum or flood-fill in your real job, but if it's the requirement to get that job, it can make a massive difference in your quality of life. Offloading brain-processing by memorizing leetcode also allows you to spend more of your higher-level thinking on connecting with your interviewers, figuring out the best presentation style tailored to them, and optimizing emotional expression during the leetcode portions of the interviews. Basically, leetcode isn't the only "hard" part of interviews, and it's useful to free up as much brainpower as possible available for the other challenging things that you need to handle simultaneously as well. Most of those other things don't have a "ceiling", any additional marginal efficacy might make the difference. I believe spaced repetition is best here instead of time-minimized Leitner method, because not only is it helpful to still have much of this memorized two years later when you might desire to interview again, but also because the complexity of the problems makes a time-minimized approach fairly infeasible.
On the topic of memorising arithmetic, I have found it surprisingly useful to memorise a few base ten logarithms. While most people have a good sense of numbers as an arithmetic progression, the geometric perspective is often more useful. (Things like how 3 is halfway from 1 to 10, but 10 is halfway from 1 to 100. (This is just the square root, of course, since that is a power of a half. But it's also useful to know that 2 is one-third of the way to 10, and 1.6 is a tenth of the way to 100. I.e. knowing some logarithms, you can mentally compute arbitrary fractional powers in your head.))
The goal of our site is to teach smart people how to quickly master leetcode.
We put in a year's worth of thought to make things as simple as possible, and are super proud of our content. Recently, two people used our site to land Amazon. Check it out!
DeriveIt has ~100 questions and ~40 lessons. It's up to you to decide how much practice you need, but users typically aren't limited by "not enough" practice problems.
Not sure if it's worth mentioning, but we made sure to cover all the ideas you learn from the Blind75 and NeetCode150. It's not like you're missing out - we're just:
- organized better
- higher quality solutions
(==> more efficient, hence not thousands of problems)
I continue to get puzzled with leetcode type things. Will coding exercises like this become obsolete someday or will the old guards continue to push for such things to be used during interviews? Today coding Copilots can easily replace leetcode "winners"
It replaced “did you go to a top university” and “do you know our exact stack and tools” as the top hiring practice. Now anyone can compete for the same jobs from anywhere in the world. It’s a leveller, in my mind.
I applied to Atlassian as a 50 year old from South Africa who has very broad and deep experience but has never worked in big tech. Failed at the last interview (of 5) but I found the process and feedback very fair and much better than others I’ve seen elsewhere. The focus was on collaboration and communication and not being a jerk, along with the problems to be solved. I enjoyed it.
The people / companies using it to hire will tend to hire people who studied leetcode.
The companies that don't use leetcode / don't use it as a strong signal will continue not using it.
In my opinion, leetcode is a poor signal if used as a binary decision ("were they right or wrong?"). The more important thing is communication and how they worked through the problem. I've heard this multiple places and I absolutely agree: attitude over aptitude. You can teach knowledge, you can't teach attitude or ability to problem solve.
The companies that use them as a strong signal are the ones that will be absolutely demolished in engineering because they are on a fast track for a staff full of rote memorization rather than strong creative problem solving. They'll be handicapped when it comes to solving problems that aren't covered by leetcode because no one there bothered to learn it.
> The companies that use them as a strong signal are the ones that will be absolutely demolished in engineering because they are on a fast track for a staff full of rote memorization rather than strong creative problem solving.
What metric are you using to measure this? I think its a positive correlation at best as far as good business outcomes are concerned, and maybe uncorrelated at worst. Google and Meta have been known to ask contrived puzzle questions for years, even before Leetcode (and Meta now is infamous for having a high standard for Leetcode interviews), and I do not see Engineering being "demolished" here, as far as results are concerned.
What people don't realize is that Leetcode selects for generally positive traits, no matter how its solved. In my mind, here are things LC selects for:
1. Actual innate skill. If someone never practiced leetcode before but can solve an arbitrary new coding question, they probably have a pretty decent aptitude for problem solving.
2. Determination. If someone practiced leetcode for hours a day just to pass an interview, thats commendable. Does it really matter that they don't "know" the problems if they studied hard and passed? They might be more likely to work hard on the job given the right rewards.
Leetcode style interviewes were basically initially created as a thinly veiled aptitude test, and in theory they are still good at that. If you _can't_ solve an easy to medium leetcode question then what does that say about you, assuming it basically is the inverse of the two singals above?
> I've heard this multiple places and I absolutely agree: attitude over aptitude. You can teach knowledge, you can't teach attitude or ability to problem solve.
Seems like society should try to figure this out.. if attitude is so important, why can’t we cultivate it systematically?
The way to cultivate it is letting people figure things out on their own and rewarding atypical solutions that arrive at the same conclusion.
A lot of education tries to deliver in a specific box and a specific way and any other method is punished. Society would likely get more value if education was more diverse in perspective and methods. It took me way too long to realize that I didn't care about anything but computers. I started doing better in my classes when I framed everything through my lens rather than viewing it through the instructor's lens.
I don't know the correlation but I think a lot of entrepreneurial families tend to have good problem-solving children or maybe it is a selection bias among friends. It could be genetic but I think these families tend to reward / not punish ingenuity. In addition, I believe part of the equation is leaving children alone to figure things on their own. It sucks but I think that letting your kids jump into the deep end of the proverbial pool makes them better for it even if they flail for a bit.
Give them boundaries but give them the space to figure it out and make mistakes. I know it drove my parents crazy that me and my siblings took everything apart and argued systematically but it paid off in our adulthood.
> Seems like society should try to figure this out.. if attitude is so important, why can’t we cultivate it systematically?
We do. It's called culture.
But culture doesn't optimize for something so narrow since it exists in a much wider context than just "what's good for knowledge workers in a capitalist system".
If candidates know going in that it's a leetcode interview it filters for "do you want to work here so much that you'll spend a month or two drilling on these exercises". Which can be a useful filter if there are thousands of qualified applicants.
Not sure what you meant. Leetcode type things solves a problem of past. Future will be copilots doing lot of grunt coding work and human value add would be instructing copilots the right way, bringing broader contextual information. Solving binary search tree problems during interviews will eventually go away (once old guards go away)
We agree, solving binary search tree problems do not need solving. That's an old problem. Problem is if you need people to choose and adapt algorithms for a specific new problem, copilot will have a hard time and so you will have a hard time if you don't know how the algorithms work and never adapted one. Because copilots are not good at choosing the correct algorithm and adapting it for new problems, as far as my experience with the competitive programming course I took goes anyway.
If you have an old problem, copilot can solve it, true.
I recently noticed that many people that solve leetcode style problems to learn programming patterns tend to save links to problems they'd like to try again later on google docs/notes.
However, this doesn't help them remember to try these problems again or differentiate between problems that are more urgent to revise.
Alt+Tab is practically instant and I already use Anki for all my SRS needs, so having a separate SRS for a niche topic as a Chrome-only browser extension is just a lose-lose for me.
Related: executeprogram.com is an excellent learning platform that combines spaced repetition with interactive lessons targeted at professional developers.
Small feedback regarding the website and the "Try Lanki for Free" heading. It makes it seem like we need pay after a trial period. But there is no pricing information I could find. Calling out the price would be helpful information
Some alternatives not mentioned here that I am aware of are Mnemosyne Proj [1] and org-drill [2]. I have only ever used anki though. I am going to try org-drill because anki has over 40 dependencies I would not need to compile if I just switched to org-drill. The one nice thing about anki is the community cards you can download, but at the same time I feel that an important part of learning is making the cards yourself.
FSRS 4.5 landed in stable Anki release earlier this year for Desktop and modile clients. Strongly recommend you give it a try if you're reading this. It's a per-deck setting near the bottom in the Advanced deck settings (not application settings). You can integrate the FSRS 4.5 scheduler into your software.
The anki decks/cards are fully customisable to add any fancy extra feature.
Otherwise one can make their own cards (I often use LLMs to convert knowledge into cards and then import them into my account) for just about anything.
super similar - was inspired by anki! this extension shows up directly on the leetcode website as a starting point, but looking to expand to all of ankis-type features soon
Just fyi for the dev - on mobile, the slide in menu that opens on clicking the burger icon on the top right is empty.
Cool idea btw. What do you think about incorporating the "patterns" associated with problems into the repetition algorithm? For example, if I find 2 pointer problems hard, then the next 2 pointer problem will be recommended sooner.
I can totally see what you're solving for - the friction of switching between an anki deck and leetcode to achieve the same thing with existing tools.
If you want one data point about how I prepare for coding interviews - I just solve Neetcode's list and the blind 75 list (minus the overlap between the two). For reference I got 5 offers in quick succession after being laid off in early 2023.
for now, it will always 5 recommend problems you have attempted before, prioritizing the ones you solved both (1) longest time ago and the ones that were (2) hardest for you. so it will not recommend anything you haven't tried. would you want it to stop recommending a problem once you've mastered it? maybe an indication on the tool so you can mark it as mastered?
Seems like a smart tool for enhancing coding skills. Tagging LeetCode problems by difficulty and having a tailored review plan based on spaced repetition is a strategic approach to learning.
I'm about to start the leet code grind for interviews coming up and I'll give this a shot. I found your feedback form on your website but would be nice to post it in your post here as well. Cheers.
“Leetcode grind” means solving Leetcode problems. Interviewing for many software engineering positions in Silicon Valley these days require doing well on Leetcode-style coding exercises where one has a limited amount of time to solve a problem.
What makes software engineering interviews a grind these days is that many employers don’t care about your thinking process; they want an optimal solution to the problem within the time limit without errors, since chances are high that an applicant will come up with the optimal solution. In addition, the high compensation for many Silicon Valley employers has made these positions very desirable and thus ultra-competitive. The same type of applicants who grinded for high SAT and AP scores in high school and who grinded in college for 3.7+ GPAs don’t feel discouraged grinding some more for a six-figure job with life-changing amounts of RSUs once vested and other perks. You can grind hard, but chances are high that somebody else spent even more time studying than you. These types of interviews are similar in spirit to the employment exams that some companies have in Japan during job-hunting season for upper-level college students.
So, aiming for a FAANG position, as well as software engineering positions at many other companies in Silicon Valley, essentially require studying not unlike preparing for a GRE subject test for graduate school admissions.
Hey I just want to add, in addition to your advice here, there are other important aspects to interviewing beyond performance on Leetcode questions. In other words, it's sometimes necessary, but not sufficient. Most senior jobs require system design interviews, conversations about leadership, and more.
I think "grinding" on Leetcode also entails learning and practicing a lot of irrelevant topics for many jobs, but there are newer resources out there that help you focus on what you need to know for specific roles. I don't like to plug my own company, but Exponent[1] is designed to help with this more 'well-rounded' interview prep for different types of engineering paths, and we also have a mock interview feature that helps you iron out your communication skills with other candidates.
I would like to try exponent, but the only free "lessons" I see are the intros to each section. I want to see a real, representative sample of the content I'd get before I'm willing to open my wallet.
> for many software engineering positions in Silicon Valley these days
Do US most companies outside of Silicon Valley not use leetcode style interviews? Every single software interview I've done in Canada had at least one round of leetcode programming exams.
I've hardly dipped my toe in SV and have almost entirely worked in the Eastern half of the United States.
Leetcode questions do happen but typically they are guidelines for a further discussion rather than the entire signal. By that I mean, they are used to investigate problem solving, communication, and personality.
It tends to be the bigger places have more rote tests. One company posed the "If you were re-making Instagram, design me the ability to handle millions of page views over the span of an hour" or something like that. Apparently CDNs, caching, and load balancers wasn't the answer...
Never bothered to look it up because I spent what felt like an hour trying to figure out why this guy thought a CDN wasn't good enough and whether he was _trying_ to get a rise out of me.
What only made it funnier is this place had nothing to do with images.
I think outside of silicon valley (and companies emulating that culture), US companies still give coding problems but they're usually not leetcode style, more of just an implementation problem, if that makes sense. Instead of checking that you have some obscure algorithm memorized they simply want to see you code a bit.
But I see it's already been mentioned here in another thread.
Do you rank the problems solved by the user only, or all leetcode problems?
Because I think that a nice thing would be for your tool to first suggest one pb in each category, and then start recommending the category where you're the weakest.
If you master the methods and ideas, you should be able to derive the answers on the spot. That's better than memorization because then you can actually deal with real interviewers who 1) Want to hear your train of thought and 2) Will give you random changes or variants to problems.
It's not about memorizing solutions, because in reality when are you ever going to reverse a linked list or balance a red-black tree? No, so if you're going to put in the effort anyway, you should learn the concepts and understand when to apply them so your knowledge is actually applicable. This will actually make you a better programmer, and you'll do better with the interviews that are actually good (I speak with experience giving interviews at FB).
Most of the concepts I learned, although I don't remember learning Kadane's algorithm for solving the maximum subarray problem in school, so I've learned some things like that. I can also implement a solution for an associated Leetcode problem in one line of code, so I wound up memorizing that too.
OP says "If you master the methods and ideas, you should be able to derive the answers on the spot", but I hear from a lot of people that does not happen, even (implicitly) in this thread. Any how, even if I use OPs method, if I do the same problem over and over and over again I'm going to wind up memorizing it any how.
Some people are closer to each other, and upon realizing that they believe they've found a pattern. But it's like when people tell you that you need to supplement x in your diet. It's possible that x helped you. And it's even possible that you'll find other people with the same nutritional deficit, creating the illusion that everyone would benefit from supplementing. But that's just what it is: a premature extrapolation from anecdotes
Just for the record, that never actually happened. It's just a thing the homebrew guy made up.
(I know, the rest of your points are unaffected by this. I just feel annoyed when we casually warm up old myths.)
I tried to find some outside sources, but nothing about Howell admitting his 'artistic license' outright. So take my fallible memory with a large pinch of salt.
See https://news.ycombinator.com/item?id=15713801 for some discussion, and to see how 'invert a binary tree' wasn't even an established term that anyone at Google would have used.
https://leetcode.com/problems/invert-binary-tree/
Spaced repetition is nothing other than intentionally refreshing something you currently know, to ensure that in a month or a year or longer, you'll still know it. It's not your place to judge what to memorize or not. I'd suggest you should also use spaced repetition on the methods and ideas, since we're very prone to forgetting that too. I would also never criticize anybody for memorizing something, since it's literally just doing intentionally what we would otherwise do haphazardly and by accident.
What experimental psychology and neuroscience research has found is that the better people memorize topics, the better they understand and reason about them.
For those interested, this book has the best compilation I've seen of such research: https://www.amazon.com/Memory-Alan-Baddeley/dp/1138326097
Edit: Of course, you can memorize things without any effort to understand. But if you are making these efforts, having an efficient and effective memorization technique can help a LOT.
I don't know if I'm just lucky in terms of how my brain works in some way and I found I had a pretty good 'natural' memory when it came to things like learning languages, and I appreciate that this is totally anecdotal. That book looks interesting however and as somebody who teaches, I'm interested in what works for everybody, not just me.
Whenever I practice, I always have my notes in front of me and I focus on puzzling through the problem at hand rather than testing my memory.
But in an exam, for me there us always the possibility of failing to recall some fact. To combat this, I record every not-completely-obvious fact into Anki and aggressively practice it in the run-up to the exam.
I well doubled my income after using SRS to master leetcode/algo/ds patterns. Implementing SRS was a catalytic moment in my interview prep training that really accelerated the volume of patterns I held in the RAM of my brain.
You can view the cards I made at https://github.com/yfSWn5KP/algo_and_ds
There’s also system design cards at https://github.com/yfSWn5KP/sys_design
Writing good flashcards is about
- noticing similarities and differences,
- exploring variations of a concept,
- finding generalisations of related concepts,
- learning properties of a concept, etc.
All these things are things you use when reasoning also about more complex ideas.
In other words, spaced repetition is an efficient way to raise the baseline for what counts as a "fundamental fact" which makes it possible to think at a higher level of abstraction.
It feels obvious to me that, in theory, we could do a lot more to leverage your current world knowledge to recommend the next piece of educational content, to optimize not just your understanding but also other things like how engaged you are. For example, a system could throw you an easier question if it helps you focus longer.
When I see the state of current interview prep it’s mostly “here’s a big list of questions” , perhaps tagged by difficulty and grouped under related topics such as “graph problems” or “tree problems” but I’m personally convinced a more sophisticated system could serve you the best possible next problem to stretch your brain in the right way.
Spaced rep is simply the proven starting point. The rest of adaptive learning has a bit of a troubled past because it was trendy to VCs , but pushed on educators and presented more as an alternative to a teacher rather than something to augment a teacher. I worked for a company that raised 100M+ to work on it but the CEO was great at terms sheets but uninterested in actually building a great education project.
But the reason that I joined the company was the high level idea still resonates with me. Surely many HN users have a big list of things they want to learn- perhaps about LLMs. But a typical course will have zero knowledge on where you’re starting as a student . It might bore you with stuff you know already or take for granted key prerequisites and skip them.
Anyway, I’m currently building my own adaptive learning platform but focused on helping professional poker players learn game theory . My idea being that it’s an easier ed tech app to bootstrap as the knowledge very directly translates to money. And really the same criticism applies because you can’t truly memorize a game tree , it’s more important that you build a high level conceptual understanding. But , as much as rote memorization deserves to be maligned when done in isolation, it’s not so bad when done as part of a broader learning strategy. For example, you can’t memorize vocab to learn Spanish but certainly knowing 5000 words in Spanish is a very nice starting point compared to not knowing any words. And tools like spaced rep have been proven by research to help with that goal and I view as a pathway to more broad adaptive learning strategies.
Traditionally, yes. But several people use the SR algorithm to simply do the same problems over and over (practice), without the memorization component.
Looking at a typical SR algorithm, you're not really going to memorize doing the same medium/high Leetcode problem unless you actively try to memorize it. The interval sequence will be something like: 2, 5, 13, 35, 120, etc. By repeatedly solving the problem so few times in such a long period (and while solving other problems at the same time): You're not really going to memorize it accidentally.
> ...
> That's better than memorization
Bullshit. You can't get to mastery without memorizing (at least) the fundamentals into muscle memory.
Can you just "master" the 7 notes and what octaves are and all other theory to be able to play beethoven? Or should you play a few songs repeatedly till they're muscle memory practicing in parts or full and then moving over to other songs?
Should you learn that the integral means the area under the graph to "master" integration or do you still need to run through dozens of same/similar problems till you can solve them without an error?
You need to memorize but its a fools errand to start and stop at memorizing leetcode problems. You need to get the fundamental problems (binary search, bfs, dfs, sliding window) solve them repeatedly to a point it just flows naturally and then apply it to problems that build on them and still repeat those till they get natural too.
https://nautil.us/how-i-rewired-my-brain-to-become-fluent-in...
Who would ever claim otherwise?
1) Learning for mastery / competency to apply the skills to real life endeavors
2) Learning so you can meet some standard necessary to enable certain opportunities in life
It seems your opinion comes from a personal value bias towards #2 and I share that bias, but perhaps not as dominantly as you do (at the moment). For both of these, memorization can be very helpful, even optimal. But the application of spaced repetition would be different depending on the goal.
On memorization applied to learning for genuine personal mastery:
I believe memorization is optimal for foundational axiomatic knowledge. A good example is quick addition and times tables (and other arithmetic). I believe:
1) People should consider it a necessary life skill to be able to perform reasonably complex arithmetic manually in your head. For example, 4462+9241=13703 or 25120=3000. Many people will disagree on the nuances on where this begins and ends, and that's okay. For example some people will feel that people should be able to multiple 27123 in their head, others would feel it's okay for something like that to be done on paper. But for me, the point is that there exists a category like "things you should be able to do, but it's fine not to have memorized". More importantly, I believe it would be suboptimal to spend a significant amount of your life memorizing an increasing sample of these, but it's reasonable to spend time maintaining the skill of manual mental arithmetic and probably reasonable to spend time improving the speed and accuracy with which you do it.
-- 1a) A nuanced subset of this skill example is "estimation", for situations where it's appropriate. It might take longer for some people to manually calculate 27123 (or whatever) than for them to pull out their cell phone or ask their Apple Watch. If the situation only needs them to have an upper and lower bound within they might break it down to {lower_bound = 25120 = ((256)2)10 = (150210) = 3000; upper_bound = 30125 = 3055 = 150510/2 = 75010/2 = 7500/2 = 3750} and they could determine the answer is around 3350 or so, but absolutely 100% definitely between 3000 and 3750. And they could still do this type (if not degree) of estimation faster (and more stealthily, which can be important sometimes, like during negotiations/haggling) than pulling out a cellphone and calculating.
2) People absolutely cannot do arithmetic if they do not have certain foundational axioms memorized. If you do not have 1+1=2, 2+1=3, 3+1=4, etc memorized (counting), you literally cannot do arithmetic. That would be an absurdly low-functioning adult and essentially means you don't know how to count. But technically* you only need to be able to count to add, subtract, and multiply (by maintaining two counts simultaneously) any real numbers. Obviously just counting is a helpful, but not optimal, level of memorized knowledge to be competent at arithmetic. It would plainly be helpful to also memorize things like 7+6=13 and 152 = 30, and have instant recall for these rather than re-calculating them manually every time you need them. Which brings me to:
-- 2a) I believe it is helpful to memorize more than what many people think it is helpful to memorize. We generally are taught in primary school to memorize times tables up to 10x10. But perhaps it would be useful to extend this beyond 10x10, at minimum for some additional prime numbers which might show up frequently and cannot be quickly factored to smaller numbers. I could see it being super nice to have my times tables memorized up to 100x100, and additional addition/substraction memorized. I feel this way because despite having done well in three semesters of calculus and differential equations, I have always had a few "holes" in my brain's lookup table for certain simple addition and multiplication - specifically it really slows me down to need to manually calculate 86 (I have to do 832) or add 16+7 (I personally have to do 20+4+3). It's very frustrating to hit these basic roadblocks. Often I wonder how much happier I'd be if, at a young age, I had personally chosen to endeavor to memorize an unusually large amount of basic integer combinations, rather than just focus on memorizing the minimum necessary to get 100% on my K-5 coursework. Why do I not know 86? Because I took the less ambitious route of just breaking it down to 446 or 823 every time. Which brings me to:
On memorization for learning so that you can meet some standard necessary to enable certain opportunities in life:
1) I think we can all agree that some things are worth striving for, but are gated behind formal or informal tests which aren't relevant to the skills and temperament that actually enable people who do those things to do them well. For many people, that's "leetcode", but it can also be things like a college graduate memorizing and practicing ad nauseam a powerpoint delivery for a technical role for which they'll never formally present anything in their entire career. It could also be memorizing certain key motifs in a history or politics or English class which are actually wrong. The student may even know* its wrong, but it must still be memorized and regurgitated to get the best possible grade for their transcript.
1a) I think we can all agree that when cramming for an exam for which knowledge is not particularly desired to be retained, the best memorization technique is the one that takes the least amount of time to remember the facts just long enough to regurgitate them for the exam. I've discovered that this method is not the most commonly evangelized "spaced" repetition, but rather a Leitner system implemented in a way such that you actually minimize the time spacing as much as possible. For me, this uses digital or physical flash cards. Make a deck containing all facts you want to be able to maccess n days later during the exam. Go through the deck one at a time and any you get correctly, move to a new pile on the right; any you get wrong, move to a new pile on the left. Keep going repeating this with the left-most pile without breaks or pauses until you are getting enough % correct to get the desired score on the exam (with some safety margin), or you get exhausted or stop improving (then sleep and do it again the next day). This takes up far less actual time than spaced repetition, but you'll also forget the information fairly quickly as well and retain much less of it permanently.
--------------
To bring it back to the actual topic of memorizing leetcode: For me personally, I think memorizing some leetcode can be useful in three ways.
1) (For both mastery and credentialism) This is stretching the definition of leetcode a lot, but it's worth mentioning. I think it's a good idea to memorize syntax and functions/methods of the language, libraries, frameworks, and tools that you intend to use. I switch languages a lot at work, and I switch jobs a lot. It genuinely shocks me how much I can accomplish in some languages for which I can't even write a syntactically correct for-loop. Like if you put a blank paper and a pen in front of me, I could not write a for-loop -- but I can copy/paste one loop from language documentation or google result or ChatGPT and then copy/paste that line from my source code over and over until I have a working API or application or website from scratch in under a day. But I hate it, and not knowing all the syntax makes me slower. Not knowing what methods are available for a certain class is very frustrating. I don't need to have it memorized to do my job, but when I'm working in a language that I do know super-well, I have a much happier time doing the job.
2) (For mastery) Some leetcode could be considered "merely slightly more complex fundamental concepts". Nobody would argue that you should be able to write a for loop from memory in your language, but I think it's reasonable to say that it could very well be useful to be able to have instant recall of some amount of slightly more complex algorithms rather than needing to google them, like implementing sliding windows using trees. Or (for low-resource embedded programming) things like exponentially weighted moving average. Etc. What these are depends on what you desire to have mastery over, but memorization via spaced repetition can ensure that these "leetcode" techniques are always available to you the instant you need to implement them.
3) (For credentialism) Yes, it would be wonderful if we all had the brainpower to instantly figure out all leetcode problems fluently on the fly without awkward pauses. But most programmers can't, and it's better for the interviews if you can quickly, decisively, and confidently present everything that the interviewers are looking for. For this, I believe it can be genuinely worth spending some of your life memorizing leetcode if it means the difference between making $120,000 vs. $225,000 for the next five years. You probably won't ever use two-sum or flood-fill in your real job, but if it's the requirement to get that job, it can make a massive difference in your quality of life. Offloading brain-processing by memorizing leetcode also allows you to spend more of your higher-level thinking on connecting with your interviewers, figuring out the best presentation style tailored to them, and optimizing emotional expression during the leetcode portions of the interviews. Basically, leetcode isn't the only "hard" part of interviews, and it's useful to free up as much brainpower as possible available for the other challenging things that you need to handle simultaneously as well. Most of those other things don't have a "ceiling", any additional marginal efficacy might make the difference. I believe spaced repetition is best here instead of time-minimized Leitner method, because not only is it helpful to still have much of this memorized two years later when you might desire to interview again, but also because the complexity of the problems makes a time-minimized approach fairly infeasible.
For the curious, I write a little more about it here: https://two-wrongs.com/learning-some-logarithms.html
The goal of our site is to teach smart people how to quickly master leetcode.
We put in a year's worth of thought to make things as simple as possible, and are super proud of our content. Recently, two people used our site to land Amazon. Check it out!
Not sure if it's worth mentioning, but we made sure to cover all the ideas you learn from the Blind75 and NeetCode150. It's not like you're missing out - we're just:
- organized better
- higher quality solutions
(==> more efficient, hence not thousands of problems)
I applied to Atlassian as a 50 year old from South Africa who has very broad and deep experience but has never worked in big tech. Failed at the last interview (of 5) but I found the process and feedback very fair and much better than others I’ve seen elsewhere. The focus was on collaboration and communication and not being a jerk, along with the problems to be solved. I enjoyed it.
The people / companies using it to hire will tend to hire people who studied leetcode.
The companies that don't use leetcode / don't use it as a strong signal will continue not using it.
In my opinion, leetcode is a poor signal if used as a binary decision ("were they right or wrong?"). The more important thing is communication and how they worked through the problem. I've heard this multiple places and I absolutely agree: attitude over aptitude. You can teach knowledge, you can't teach attitude or ability to problem solve.
The companies that use them as a strong signal are the ones that will be absolutely demolished in engineering because they are on a fast track for a staff full of rote memorization rather than strong creative problem solving. They'll be handicapped when it comes to solving problems that aren't covered by leetcode because no one there bothered to learn it.
What metric are you using to measure this? I think its a positive correlation at best as far as good business outcomes are concerned, and maybe uncorrelated at worst. Google and Meta have been known to ask contrived puzzle questions for years, even before Leetcode (and Meta now is infamous for having a high standard for Leetcode interviews), and I do not see Engineering being "demolished" here, as far as results are concerned.
What people don't realize is that Leetcode selects for generally positive traits, no matter how its solved. In my mind, here are things LC selects for:
1. Actual innate skill. If someone never practiced leetcode before but can solve an arbitrary new coding question, they probably have a pretty decent aptitude for problem solving.
2. Determination. If someone practiced leetcode for hours a day just to pass an interview, thats commendable. Does it really matter that they don't "know" the problems if they studied hard and passed? They might be more likely to work hard on the job given the right rewards.
Leetcode style interviewes were basically initially created as a thinly veiled aptitude test, and in theory they are still good at that. If you _can't_ solve an easy to medium leetcode question then what does that say about you, assuming it basically is the inverse of the two singals above?
Seems like society should try to figure this out.. if attitude is so important, why can’t we cultivate it systematically?
The way to cultivate it is letting people figure things out on their own and rewarding atypical solutions that arrive at the same conclusion.
A lot of education tries to deliver in a specific box and a specific way and any other method is punished. Society would likely get more value if education was more diverse in perspective and methods. It took me way too long to realize that I didn't care about anything but computers. I started doing better in my classes when I framed everything through my lens rather than viewing it through the instructor's lens.
I don't know the correlation but I think a lot of entrepreneurial families tend to have good problem-solving children or maybe it is a selection bias among friends. It could be genetic but I think these families tend to reward / not punish ingenuity. In addition, I believe part of the equation is leaving children alone to figure things on their own. It sucks but I think that letting your kids jump into the deep end of the proverbial pool makes them better for it even if they flail for a bit.
Give them boundaries but give them the space to figure it out and make mistakes. I know it drove my parents crazy that me and my siblings took everything apart and argued systematically but it paid off in our adulthood.
We do. It's called culture.
But culture doesn't optimize for something so narrow since it exists in a much wider context than just "what's good for knowledge workers in a capitalist system".
If you have an old problem, copilot can solve it, true.
I recently noticed that many people that solve leetcode style problems to learn programming patterns tend to save links to problems they'd like to try again later on google docs/notes.
However, this doesn't help them remember to try these problems again or differentiate between problems that are more urgent to revise.
Built this tool to help solve this inefficiency!
It's a cool project, though!
It's genuinely amazing for what it covers. A fantastic way to quickly get up to speed for JS/TS, SQL, and basic Python.
Edit: I just remembered it takes like 500(?) data points to train the algo... so maybe not.
(Jarrett's the author of fsrs, fyi.)
[1]: https://mnemosyne-proj.org/features
[2]: https://orgmode.org/worg/org-contrib/org-drill.html/
Otherwise one can make their own cards (I often use LLMs to convert knowledge into cards and then import them into my account) for just about anything.
Cool idea btw. What do you think about incorporating the "patterns" associated with problems into the repetition algorithm? For example, if I find 2 pointer problems hard, then the next 2 pointer problem will be recommended sooner.
https://alpine-aspen-2ab.notion.site/Lanki-da1481129bfa4d3ca...
If you want one data point about how I prepare for coding interviews - I just solve Neetcode's list and the blind 75 list (minus the overlap between the two). For reference I got 5 offers in quick succession after being laid off in early 2023.
I take it you solve puzzles by writing code. How does it help with job interviews really?
What makes software engineering interviews a grind these days is that many employers don’t care about your thinking process; they want an optimal solution to the problem within the time limit without errors, since chances are high that an applicant will come up with the optimal solution. In addition, the high compensation for many Silicon Valley employers has made these positions very desirable and thus ultra-competitive. The same type of applicants who grinded for high SAT and AP scores in high school and who grinded in college for 3.7+ GPAs don’t feel discouraged grinding some more for a six-figure job with life-changing amounts of RSUs once vested and other perks. You can grind hard, but chances are high that somebody else spent even more time studying than you. These types of interviews are similar in spirit to the employment exams that some companies have in Japan during job-hunting season for upper-level college students.
So, aiming for a FAANG position, as well as software engineering positions at many other companies in Silicon Valley, essentially require studying not unlike preparing for a GRE subject test for graduate school admissions.
I think "grinding" on Leetcode also entails learning and practicing a lot of irrelevant topics for many jobs, but there are newer resources out there that help you focus on what you need to know for specific roles. I don't like to plug my own company, but Exponent[1] is designed to help with this more 'well-rounded' interview prep for different types of engineering paths, and we also have a mock interview feature that helps you iron out your communication skills with other candidates.
1. https://www.tryexponent.com
Do US most companies outside of Silicon Valley not use leetcode style interviews? Every single software interview I've done in Canada had at least one round of leetcode programming exams.
Leetcode questions do happen but typically they are guidelines for a further discussion rather than the entire signal. By that I mean, they are used to investigate problem solving, communication, and personality.
It tends to be the bigger places have more rote tests. One company posed the "If you were re-making Instagram, design me the ability to handle millions of page views over the span of an hour" or something like that. Apparently CDNs, caching, and load balancers wasn't the answer...
Never bothered to look it up because I spent what felt like an hour trying to figure out why this guy thought a CDN wasn't good enough and whether he was _trying_ to get a rise out of me.
What only made it funnier is this place had nothing to do with images.
Thank you
score = ( timeSinceLastAttempt ^ 2 ) * difficulty;
Anything else you'd want factored in?
But I see it's already been mentioned here in another thread.
Do you rank the problems solved by the user only, or all leetcode problems?
Because I think that a nice thing would be for your tool to first suggest one pb in each category, and then start recommending the category where you're the weakest.