I see how the problem maps onto this. It seems like the main issue in the original problem is that you have a fixed number of eggs. But for git bisect-find there are unlimited eggs, but we want to use as few as possible.
It is also quite different because there is a bias that most targets are probably sooner. If you have a 10 year old repo it is more likely that an obvious bug is in the last month than the first month. (Otherwise the optimal would always just be to test the first commit and then assuming that it is good just bisect from there).
if you don't know what commit is good, just go way way way back. Go back 10,000 commits instead of 1,000. The magic of bisection search is that opening your search window by 10x only adds 3 (well, 3.332) extra bisections.
The actual way to do it is to go back in exponentially larger steps until you reach a good commit. 1, 2, 4, 8, ... Then you already know that the bad commit is between 2^{k-1}st and 2^{k}th commit.
Basically you'd pass a command to the script and it would track down the last commit that did not fail the command.
It'd search HEAD~4, HEAD~8, HEAD~16... until it found a good commit, then bisect.
The problem that I found is that bugs that have already been fixed still triggered in the past, and the only solution that I found to that was to branch the first buggy commit, apply a fix and squash and then rebase on top of that.
Not the best practice, but it saved me a lot of time.
Yeah. I've been considering something like `git bisect-find start 14d` as people often have some sort of notion for how long something was likely to fly under the radar. If it is an obvious bug maybe a few days, if it is subtle maybe start by jumping back a month.
> I do wonder if jumping back by twice as much is optimal.
Some time ago, the idea came to me of weighting commits by diff size (as in LOC added + LOC removed, no thought to binary files), and using that to pick the halfway point for bisection. I'm not familiar with Git internals so I left it there. But I imagine that, while it wouldn't make a huge difference in either case, it would benefit your tool more than it would a regular bisect.
I've also pondered about custom functions to rank each commit. Maybe you can mark some sub-directory of the repo as 8x more likely to cause trouble than others and that would effect the split point decision. For example if this seems like a UI bug mark the frontend/ directory as extra suspicious. But don't completely rule out the backend as it may be triggering the frontend bug.
Do people really use bisect in their professional life?
Can you sensibly use it without an enforcement of every PR being squashed, as otherwise you have not working, half baked commits all the time while bisecting? And you can’t compile, not even saying about stating if your bug is still reproducible or not.
Yes, I use it rarely, but regularly. It is simply the fastest way to discover when a behavior was introduced.
I'm not sure what un-squashed PRs have to do with half-baked commits. Inexperienced devs tend to make too few commits, not too many. I've never known anybody to make commits that don't compile except by a rare accident. It's one of the few rules of version control that people tend to obey without having to make an effort to change their existing habits.
Non-compiling intermediate commits is basically the normal workflow for most people I've reviewed, unless they're rewriting history. One commit will do something like refactor an interface, the next will modify a distinct subset of callers, etc. Sometimes those intermediates don't compile for any number of reasons. I'll personally rewrite history for teams that want clean commits, but definitely not universally.
How do you develop that you don't have broken intermediate working states or thoughts that you can't complete by the end of the day?
You can wait, but why would you? Your intermediate work doesn't have to be clean and it's occasionally nice to have a "reset everything to this morning" button without digging into reflog.
How would reflog help without a ref having been created?
And I just don’t know what’s special about “this morning” in your sentence. If theres some important intermediary state you’d like to keep track of, by all means commit it. But to commit just because the day has ended is odd to me.
Going to blame sleep deprivation for that one. Took your comment to realize reflog wouldn't help.
To give a recent example, I was fixing a cryptographic function and had a bunch of constraints in my head. Dumped that into comments and didn't read them carefully enough the next morning. Immediately wrote a change that subtly broke one of those constraints. Having a restore point was helpful. Similarly, I can also say "this experiment I've been pursuing for a couple days went off-track, let's reset and take advantage of hindsight".
It's not something I use frequently, but it's convenient and free.
Often times I will "have to" commit WIP work to check out another branch (to test something, code review, urgent bug fix...). The other main option would be start but it isn't as organized as branches. I guess work trees would also work if they is your workflow.
But I would amend these WIP commits rather than just leaving them as they happen to occur.
That is what I meant, typo + phone seems to have resulted in "start".
The problem for me is that stashes aren't associated with a branch and I find them very hard to manage. I end up just using stash for very temporary things (like moving code to a different branch). Once something has been stashed for more than 10min I basically forget it exists.
You can report a commit as good, bad, or unkown (`git bisect skip`), and git bisect will do "the right thing", even for non-linear histories. You can also automatically report un-testable commits during `git bisect run`. Of course, you may end up with a range of commits where a regression was introduced rather than a specific commit.
Yes! But for me it's very rare (< 1/yr) and I'm like a kid in a toy store every time I get to bust it out. My employer uses squash commits so that's not a problem for us, but you're right that commits far enough back will be unlikely to be in a buildable state.
Yes. It is very helpful for seeing what change introduced a bug to help understand why it happened, what could be undone by simple fixes and what assumptions were broken.
If you use a simple merging workflow --first-parent can avoid testing inside feature branches if you find they are commonly broken. You can also just skip broken commits if they are rare.
Also I thought git bisect prefered testing merges before inside the merged branch, but I can't find official documentation in this on my phone. Either way I have never seen this to be much of a problem. (Including in nixpkgs which merges branches every which way)
Can stgit patches be pushed to a private branch on a central remote like GitHub or Bitbucket in case you need to work on the stack from multiple computers?
I'm interested in #2 here - I've often struggled with writing a test to identify a bug, and then using that new test with bisect to find the commit where that test would first fail.
It depends on your type of software. I use it all the time for mathematical software.
I have an input I keep around, which checks "is the program, very basically, doing what it should?", which is compatible with all previous versions. I use that when to 'git bisect skip' commits which don't build, or if when built aren't functioning at all. It's not 100% perfect, but works well for me.
Not everyone likes squashed/rebase workflows and it's frustrating to have to police people. A lot of groups use a workflow where all commits to branches of global interest are strictly pull requests.
For those who are in this situation, consider that what you can do is test _only_ merge commits - that is, put something like this into a script:
if git log -1 --pretty=%B | grep -q 'Merge pull request'; then
# tests here
exec the_actual_test.sh
else
exit 125
fi
... then "git bisect run script.sh". This skips the intermediates and only tests merge waypoints.
Edit: someone below notes --first-parent option, which basically accomplishes the same thing.
Git is for 1. Helping with conflict resolution when multiple people work on a piece of code, and 2. Tracking changes to software over time, to aid in bug fixing, patching older versions, and understanding why the code looks the way it does.
If you're intentionally committing code that doesn't work then you are ruining both the ability for others to work with the code, and your own ability to use the history for anything.
You _can_ use it as a poor man's backup solution (I do this myself), but the key then is to have a clear separation between a "work in progress" branch that only you work in, and the collaborative or long-lived branches. Before you merge a work-in-progress branch (or make a pull request), you need to make sure that each commit that remains passes the unit tests. You can use an interactive rebase for this.
The problem with Git is that it is actually a rich man's backup solution. It lets you do very detailed snapshotting and manipulation across versions. So when working on something it can be very helpful to have many commits during the work.
But I agree that it is usually best to then "refine" those into nice, working, logical chunks before sharing with others.
> Yes of course, every commit on your main branch should pass CI.
Or when you fix a bug first you put the regression test in a new commit first then the fix, that way you can show that a) the test finds the bug that is b) fixed by the subsequent commit. This means you need to git bisect skip such commits, but that's fine.
Even better, if your testing framework supports it, mark the test as expecting failure in your regression commit, then remove the xfail in the commit that fixes the bug.
I actually prefer to add a passing test asserting the bug exists in one commit, then fix the bug and reverse the sense of the test in a second commit. The second commit is now more or less self documenting -- this case used to produce this buggy behavior but now produces correct behavior -- and the fact that the test originally passed provides proof that the test is capable of detecting a regression.
Yeah, I've used it a bunch. No project I've ever worked on has made it a practice to merge not working, half-baked commits at all. Squashing before merging like you mention has been pretty much universal on the codebases I've worked on; if it's not done on a given PR, it's usually because of an explicit rebase to split things up manually in a way that wouldn't break this.
Though I cannot find the original, well-written article I read some years ago, this link covers the problem for those interested: https://www.geeksforgeeks.org/egg-dropping-puzzle-dp-11/
It is also quite different because there is a bias that most targets are probably sooner. If you have a 10 year old repo it is more likely that an obvious bug is in the last month than the first month. (Otherwise the optimal would always just be to test the first commit and then assuming that it is good just bisect from there).
Basically you'd pass a command to the script and it would track down the last commit that did not fail the command.
It'd search HEAD~4, HEAD~8, HEAD~16... until it found a good commit, then bisect.
The problem that I found is that bugs that have already been fixed still triggered in the past, and the only solution that I found to that was to branch the first buggy commit, apply a fix and squash and then rebase on top of that.
Not the best practice, but it saved me a lot of time.
Putting in a number of months/years ago to start from would be a good middle ground.
Some time ago, the idea came to me of weighting commits by diff size (as in LOC added + LOC removed, no thought to binary files), and using that to pick the halfway point for bisection. I'm not familiar with Git internals so I left it there. But I imagine that, while it wouldn't make a huge difference in either case, it would benefit your tool more than it would a regular bisect.
Can you sensibly use it without an enforcement of every PR being squashed, as otherwise you have not working, half baked commits all the time while bisecting? And you can’t compile, not even saying about stating if your bug is still reproducible or not.
I'm not sure what un-squashed PRs have to do with half-baked commits. Inexperienced devs tend to make too few commits, not too many. I've never known anybody to make commits that don't compile except by a rare accident. It's one of the few rules of version control that people tend to obey without having to make an effort to change their existing habits.
How do you develop that you don't have broken intermediate working states or thoughts that you can't complete by the end of the day?
And I just don’t know what’s special about “this morning” in your sentence. If theres some important intermediary state you’d like to keep track of, by all means commit it. But to commit just because the day has ended is odd to me.
To give a recent example, I was fixing a cryptographic function and had a bunch of constraints in my head. Dumped that into comments and didn't read them carefully enough the next morning. Immediately wrote a change that subtly broke one of those constraints. Having a restore point was helpful. Similarly, I can also say "this experiment I've been pursuing for a couple days went off-track, let's reset and take advantage of hindsight".
It's not something I use frequently, but it's convenient and free.
But I would amend these WIP commits rather than just leaving them as they happen to occur.
The problem for me is that stashes aren't associated with a branch and I find them very hard to manage. I end up just using stash for very temporary things (like moving code to a different branch). Once something has been stashed for more than 10min I basically forget it exists.
If you use a simple merging workflow --first-parent can avoid testing inside feature branches if you find they are commonly broken. You can also just skip broken commits if they are rare.
Also I thought git bisect prefered testing merges before inside the merged branch, but I can't find official documentation in this on my phone. Either way I have never seen this to be much of a problem. (Including in nixpkgs which merges branches every which way)
squashing PRs is not the only way to avoid broken half-baked commits.
Use something like stgit and it's easy to create a stream of 100s of commits that all work and make sense without squashing any commits together.
I used git bisect yesterday.
Yes.
> Can you sensibly use it without an enforcement of every PR being squashed, [...]
I insist on linear, clean history, i.e., rebase workflows. No merge turds.
You train people/enforce in CI that all commits on `main` compile. People shouldn't not be committing broken code. Squash if they can't manage that.
1) You can `skip` a bad commit if it happens
2) You can branch and rebase a unit test into the history to use a scripted bisect run
Has this technique got a googlable name?
I think the following is enough to explain the concept. It's not quite how I do things, but it's mostly right
https://www.pauline-vos.nl/fix-bugs-%E2%9A%A1-fast-with-regr...
I have an input I keep around, which checks "is the program, very basically, doing what it should?", which is compatible with all previous versions. I use that when to 'git bisect skip' commits which don't build, or if when built aren't functioning at all. It's not 100% perfect, but works well for me.
Not everyone likes squashed/rebase workflows and it's frustrating to have to police people. A lot of groups use a workflow where all commits to branches of global interest are strictly pull requests.
For those who are in this situation, consider that what you can do is test _only_ merge commits - that is, put something like this into a script:
... then "git bisect run script.sh". This skips the intermediates and only tests merge waypoints.Edit: someone below notes --first-parent option, which basically accomplishes the same thing.
Yes it is. What am I doing wrong?
If you're intentionally committing code that doesn't work then you are ruining both the ability for others to work with the code, and your own ability to use the history for anything.
You _can_ use it as a poor man's backup solution (I do this myself), but the key then is to have a clear separation between a "work in progress" branch that only you work in, and the collaborative or long-lived branches. Before you merge a work-in-progress branch (or make a pull request), you need to make sure that each commit that remains passes the unit tests. You can use an interactive rebase for this.
But I agree that it is usually best to then "refine" those into nice, working, logical chunks before sharing with others.
Share finished bits of work that help others; keep the snapshot of your messy workshop with the table full of things and tools on a private branch.
Or when you fix a bug first you put the regression test in a new commit first then the fix, that way you can show that a) the test finds the bug that is b) fixed by the subsequent commit. This means you need to git bisect skip such commits, but that's fine.
And yeah, I use it all the time.
https://git-scm.com/docs/git-bisect
I generally mandate linear history for repos and squash all commits. I've frankly never understood people who don't.
Premature announcement?