Archive for October, 2016

I’ve been working on a small change to the open source project I mentioned, Zulip. The code required was trivial, but I spent most of my time figuring out a rational order of operations and being confused about how to accomplish them with the version control system.

I’m certainly no stranger to source code version control, but mainly with commercial systems. Git, used by many open source projects, isn’t much like the others I have experience with. The Zulip project has documentation about how they use git and github, the website that provides an online version of that system. But, of course, not everything can be anticipated in advance (and documentation can have bugs too.)

The funny thing is, I’ve used github some before. But only in a very narrow way. If you have a private repository where you directly submit changes to a main branch, there’s not a lot to it. Combining together different changes can be a nuisance, but that’s going to happen in any system like this. And if there’s only one or a few people making changes, the need for it can be nearly avoided with a tiny amount of discipline.

Where things get complicated is when you have a bunch of people working all at once. Even more when they are only loosely coordinated. Git assumes developers will work on what they want, and a handful of administrators will direct traffic with incoming requests to include new code in the main repository.

One thing that I was misunderstanding is how (little) git thinks about branches. Branches are normal things in version control, you make one to take a copy of existing code so you can safely modify it away from the central, primary, version.

In some systems, this is a resource-intensive operation where each branch is literally a copy of everything. Git doesn’t work that way. Since it functionally costs little to make a branch, branching is encouraged. You have your own copy of the code at a particular point in time. Both you and other people can make changes independently on different branches. You make some more branches. In the git universe, that’s no big deal. Time marches forward.

After you do your thing with your branch, you probably want to somehow get it back into the main repository. I’m most familiar with merging, where the system compares two parallel but not identical sets of source code and figures out if the changes are neatly separated enough for it to safely mash them together for you. Some automagical stuff happens, and the result becomes the latest version. (This latest revision is typically called “HEAD”.)

If not, you get to do it by hand. Use a merge-intensive version control system for a while, and you will absolutely find yourself dealing with a horrific mess to unravel. Merging is ugly but, if you are used to it, it’s a known ugly. That’s a certain kind of comfort. You can do that with git if you want. Many people do.

And many people don’t.

One thing about branches: many systems consider HEAD the be-all and end-all picture of reality. You might not be happy with the most recent version of your branch, you could keep a pointer to the revision you’d rather have, but it’s always the most recent version. If you don’t like it, you make a change and now you have a new HEAD. Time always moves forward. Re-writing history, to the extent that it can be done, is only for the most dire of emergencies.

Git has something called “rebase.” You can use it in a couple different ways, but it’s basically the version control equivalent of a Constitutional Convention: everything is on the table. You don’t like the commit message from three changes ago? Rebase. Want to not have those 47 typo-fixing revisions you created? Rebase. It’s also an alternative to merging, where your other branch’s changes are stuck on the end after HEAD and any changes that were made between the time you branched and now are patched in to your code that didn’t get them. (If you want a real explanation, here’s a PDF that helped me understand how rebase works.)

Coming from a merge-land where HEAD is sacred, this terrifies me. You are going into the past and messing with history, and that Just Isn’t Done. Admit that you checked in something with the commit message “shit is broke” and move on.

When branches are expensive and you don’t want to make too many of them, you have to protect the integrity of the ones you have. The idea of something like rebase is dangerous, and with great power comes great responsibility.

When branches are cheap, and you make one because you feel like watching what happens when you delete the database maintenance subsystem? Well, have fun. Clean up after yourself when you are done. It’s not exactly a different universe, but you think about some things in different ways. I’m not entirely there yet, but rewriting history is apparently one of those things.

In making my code change, I ran into a bunch of small things I didn’t understand. I was concerned that I’d do something that would make a mess, and it would be hard to clean up. I didn’t yet know the commands that would have helped. I didn’t understand the multiple purposes of others. I was entirely terrified by the idea of rebase. (I still mostly am, to be honest.)

I made a small mess attempting to merge in an environment that was expecting a rebase. And then halfway in I attempted to cancel but it was applied anyway. There were a few mysteries as things seemed to behave inconsistently. Some of it would have been easier if I had thought to create a branch to try something, against my previous conditioning.

So that happened.

I’ve, of course, considered such services for a long time. My first serious identity theft episode (besides credit cards) was about 15 years ago, when I was informed by my mortgage loan officer that I would not be getting that top-tier rate we had previously discussed.

There were items sent to collections I had never heard of. Addresses reported where I had not lived. There was an unscrupulous collections agency who took my report of fraud, attached to their record the full correct contact info they required me to give them, and submitted it again to the credit agencies as valid.

Among other things, the thieves signed up for local telephone service. But the phone company had No Earthly Idea where they might be located and apologized that they would be unable to help me on that issue, Thank You And Have A Nice Day. A police department in a state I never lived in refused to accept a report except in person. I couldn’t get anyone to tell me if the drivers license number on one of the credit applications meant someone applied for one in my name. My own state and local authorities wanted nothing to do with it, because the visible crime happened elsewhere. “You could try calling the FBI, but they are only interested in cases over a million dollars.”

At one point, when I was having a rather convoluted “discussion” with one of the credit bureaus, I offered to come to their office with paper copies of documents supporting my request to remove the fraudulent items. The main corporate office was ten minute’s walk from my workplace. They offered to call the police if explored that possibility.

This took several years to fully clean up, continuing even after I moved to California. I still have to assume that my personal information is sitting out there, waiting for someone else to abuse it. For all practical purposes, I have a lifetime subscription to credit reports on demand.

So let’s just say I’ve gotten pretty good at this. It’s a giant pain in the ass, but not enough to pay someone a monthly fee for the rest of my life (and probably after.) Particularly when the services available consisted of little more than automated credit report checking. Once in a while something happens, I spend a few weeks arguing about it with various companies, and then it goes away. (Until next time.)

So what changed?

Well, you might have noticed I know a thing or two about computers. Keeping them safe and secure, to the best of my abilities and time available. You would not be surprised to learn that I like backups. Backups! Backups as far as the eye can see! Backups that run hourly. Backups that are swapped out whenever something has the slightest suggestion of a hardware blip. Backups that live in my travel bag. Backups that live at my mother’s house. And backups that live in my car.

My usual “offsite backup” stays in the car glovebox. Every so often, I try for at least monthly, I take it inside and refresh it. We do have a storage unit, I could keep it there, but it’s far less convenient. That means it would be updated less often, and monthly is already not that great.

My laptop backup is encrypted, as are all of my USB hard drives if possible. My server backup is one of those that is not, because the OS version is too old. So my glovebox backup is one USB drive with two volumes, one encrypted and one not.

The unencrypted server backup always concerns me a bit. If someone knowledgable got it, it has all the information necessary to royally screw with my server. That’s a problem. But eventually that server will be going away, replaced with something better. And it’s a basic machine that runs a few websites and processes my outbound email. (I haven’t hosted my own inbox in years.) Yeah, having some archived files of ancient email released would not be fun. But that’s the extent of anything that would impact my actual personal life.

I’d rather not have my backup drive stolen out of the car, sure. It would be annoying, both for the car and having to lock down my server. But it wouldn’t be the end of the world.

So that’s not it, what else? (I’m guessing, at this point, you have some idea that there will be a car chapter to this story.)

A few weeks ago, my spouse decided that this offsite backup thing wasn’t such a bad idea. The thought of having to use it, because the house burned down or all our stuff was stolen, is not pretty. But it’s better to have something in that situation than have nothing. And it’s not that difficult to remember to update and put back once in a while. So he did.

Given that he’s the inspiration for the “tinfoil hat” userpic I have on one of my social media accounts, I presumed it was encrypted. He has many years’ experience in professional system administration and is far, far more paranoid than I am. Nothing with a name or address is discarded intact. He insists the shredding goes to a facility where he can watch it being shredded. When I moved to California, he would not use the cheap 900 MHz cordless phone I brought with me because it was insecure. He doesn’t like my passwords because sometimes I have to choose ones that are capable of being manually typed within two or three tries.

Guess what. Oops.

A few days ago, someone broke into our car and ransacked the glovebox. The only things taken were a small bag of charging cables and two hard drives, mainly because there was nearly nothing else to be had. (This is, by far, not my first rodeo.) Car documents, paper napkins, and some random receipts were scattered about.

One of those hard drives is my spouse’s unencrypted laptop backup.

First I dealt with the immediate problem of filing a police report, which took about 20 minutes on the phone. It is a process that is at least highly efficient, since it is almost certainly useless in getting our stuff back or even in identifying a suspect. But to be able to discuss this with my insurance company, it needed to be done.

Then came the discussion on what, exactly, was on that hard drive: it’s a copy of his user directory. So it didn’t contain system passwords, but that was about the only good thing that could be said. He uses a password manager for many things, but it’s not possible to protect everything that way. Years of email, confidential documents, client project details, credit card statements, tax returns, the medical documents I needed him to scan for me while I was out of town. All there. I handle most of the household finances, so a great many more items are instead on my machine. But sometimes you have to share, and things get passed around.

It’s almost certain that the thief didn’t care about the data. But wherever those drives get dumped, or whoever they are sold to, somebody very easily could. Names, addresses past and present, names and addresses of family members, birth dates, social security numbers, financial account numbers, everything necessary to utterly ruin our financial lives.

I’ll have more to say in other posts: which service I chose, what happened with the car, and how this story develops. But that explains why now, after many years of not being impressed with paid monitoring services, I now have forked over my money for one.