Two weeks collaborating with a machine

Non-clickbait title: I’ve been using GitHub Copilot for two weeks, and I loved it.

How I got here

I started evaluating GitHub Copilot for use at Measure Studio about two weeks ago. Both I and a few other members of the engineering community there were originally invited into the test period, and we all found the suggestions then… lacklustre, at best. They were usually out of place. No adherence to local code style, tendency to call nonexistent methods, and an overwhelming bias towards ActiveSupport methods in non-Rails Ruby codebases all were deal-breakers for us.

It was supposed to be under constant development since then, however, so I decided to take it for a spin. I fell in love on the very first day of evaluation: while it had a bias towards Rails-based code for the first hour of so, a few dismissed suggestions in it started suggesting the correct syntax. As it saw more and more files within the project, it caught on to the style and methods we use.

I suppose it is to be expected of a model that learns. I take pride in writing predictable code, valuing it over clever code whenever possible. It then stands to reason that Copilot would eventually figure out what’s going on. But what I did not expect was the speed at which it adjusted. By the end of the first day, we were in lock-step: I started a method, Copilot finished it. This culminated in Copilot suggesting an - admittedly simple - transformer class from scratch, with no other input than the context of what I was doing before and the class name.

Writing tests

Another area where it seems to excel is writing tests. Preface: I’m one of those people who write tests after the code. I tried TDD, and while I believe it has its uses for very narrowly scoped components, it has never worked for me when executing an entire feature, or “story” in Agile terms, simultaneously. Copilot was quick to suggest fairly complete happy-path specs, and provided decent suggestions as I added more test cases for areas it missed. Gold star for writing a complete spec for a RuboCop cop I wrote for internal use, too; in fact, you could argue it was Copilot that taught me about using RuboCop Commisioner.

And it’s fast. It’s almost unbelievably fast. One of the selling points is “completions fast enough to use as you type”, and to Microsoft’s credit - that is 100% true. I type pretty dang quickly when in the zone, and it still manages to keep up.

How it’s going

As more days passed, this “collaboration” between Copilot and I felt more and more natural. I focused on building the features; Copilot took care of the nitty-gritty. I focused on thinking up great test cases; Copilot filled in the blanks. I wrote a migration name, Copilot suggested the rest of it.

You may notice I’m anthropomorphizing Copilot a bit — and with good reason. As I got used to its constant presence, I forgot I was working with a piece of software. It instead started feeling like I was pair programming with a junior, but still capable programmer. I let Copilot drive for a while, then came in and fleshed out the finer points - then we switched again. It’s hard to describe with words. It’s not a completely new way of working, as the AI vendors would like us to believe, but it is transformative.

It fits so neatly into my workflow, in fact, that when I disabled Copilot over the weekend to work on this very blog (it loves injecting it’s bit into these blog posts), it felt very weird trying to code on Monday and not having these suggestions pop up. I felt like a piece of my toolkit was missing, and that was after only a week.

How this meshes with GitHub, the company

I talked about leaving GitHub last week, and pointed out that Copilot was one of the best tools that came out of that company. I firmly believe that. While there was a lot of controversy around GitHub scraping repositories for training - which I still find in poor taste - the end result, or rather its current iteration, is pretty fantastic. It’s probably the one tool I’d be sad to see go if I were to completely close out my GitHub presence.

Conclusion

I won’t urge you to try Copilot. There are other vendors, like Codeium, if you’d like something deployable to your organization’s cloud. Copilot won’t read your entire codebase, but it will look at the files you work on and transmit them to Microsoft. If that’s something that you’d rather not happen (because, perhaps, your code is not on GitHub, where Microsoft has access to all of it anyway), you’ll have to figure out whether it’s worth it yourself. Samsung banned chatbots, presumably with good reason. Codeium is free as in beer, which means the training data they get from live users must be worth quite a bit of money.

But if you’re comfortable with the thought, give it a try. Maybe you’ll like it. For my part, I’m feeling way more productive with an automated junior developer guessing my thoughts as they flow into the IDE. And I do like that.