Felix' tech blog

Who Moved My Cheese

Dec 30, 2014
3 minute read

Finding out why Travis fails your latest PR - 2 minutes. Reproducing locally and figuring out how to fix the issue - 3 hours. Squashing and rebasing to master, then finding out that the issue reappeared due to the rebase - priceless.

What is it now?

The work on PUP-3341 had basically been finished weeks earlier. The mailing list had consented (or something like that) and implementation had gone rather swimmingly. As always, the tests had been the most challenging aspect. Imagine my chagrin when Travis failed on my branch.

I quickly discerned where I had gone wrong and touched the code up. A local spec run was succesful. Yay! Do the dance:

git commit -am fixup
git rebase -i origin/master

Squashed the fix into the appropriate commit in my branch and pushed to GitHub. This should have been it.

Upon revisiting the list of my PRs I was disappointed. Had it not worked? Were my changes not pushed? Nope, the checksums matched my local repository. Had I made a mistake when verifying the fix locally? Engage time machine:

git reflog

Checking out the state before that rebase cleared things up: Yes, my fix to the spec had worked. No, it would not work after the rebase. Some upstream change re-broke my test. What fun.

Running bisection

I had presented a bisection in an earlier post but let’s do this again, because

  • it bears repeating
  • a local branch breaking courtesy of upstream is especially nasty

The idea for bisecting this kind of issue is based around this workflow:

  1. Check out the respective commit to test.
  2. Apply the branch.
  3. Run whatever test and save its result.
  4. Clean the WORKDIR so that the next iteration can commence.
  5. Return the saved result.

An easy way to apply changes from a branch to the current HEAD, with the option to trivially undo that, is git-cherry-pick in no-commit mode:

#!/bin/bash

git cherry-pick -n 3c8fb2417 .. 253f52296
bundle exec rspec spec/integration/application/apply_spec.rb:44
RC=$?
git checkout -f
exit $RC

The actual test is running rspec through Bundler, of course. The hashes belong to the head of my branch when it still worked, and its branch point at the time.

Resolution

For the curious, I once again ended up with a change of Josh’s, which is not surprising, seeing as he has done lots of work on the environment code throughout the whole switch to directory environments. Full disclosure, I have no clear idea how 968695 broke this test, but suspecting that the changes in test_helper.rb might be involved, I tried not fiddling with the Puppet[:environmentpath] setting in favor of working with what rspec gave me. This is what I ended up with.

A bit of an anticlimax, I know. In other cases, this mode of bisection yields more helpful results. I probably won’t keep you posted though - repitition is good, but let’s not get carried away. How about “more on this story as something outrageously amazing happens some time in the future, which I cannot not post about”.