Why Timehop Chose Go to Replace Our Rails App
Here at Timehop, we’ve been running Go in production for over a year and a half and have fallen in love with it. We’ve gone from having our median response time of 700ms with our Rails app to a 95th percentile response time of 70ms. We do this while serving over six million unique users every single day.
A couple of weeks ago, I sat down with my friends Luke & Kayvon to talk about how and why we use Go. They wanted some perspective on how we’ve used it and whether it makes sense for them to use it at Universe, their new startup.
I thought I’d share the highlights of our conversation in hopes that it would be useful to other engineering teams considering Go.
What prompted you to first consider Go?
We originally built Timehop as, like many startups, a Rails app. Then we started growing extremely quickly and what we built in Rails couldn’t keep up.
A lot of what we do lends itself to being parallelized. Whenever a user opens the app, we gather data from all the years they have content. The queries we issue to our database are independent of each other, which makes them very easily parallelized. We tried making it work with Ruby, but ultimately, Ruby doesn’t have true multithreading and the abstractions to work around that limitation felt brittle.
When we set out to explore new languages we had three things at the top of our wish list: easy concurrency, true parallelism, and performance.
Why did Go win compared to the other languages you considered?
We looked at a few other languages (Node.js primarily), but ultimately Go had sold us on three major things:
- Performance — Go code is compiled down to machine code. There is no VM or interpreter that adds overhead.
- Static typing — Turns out computers are way better than humans at a whole class of errors when you know what type a variable is. Who knew?
- Sane, readable concurrency — Goroutines and channels make concurrent code easy to read and reason about. Those constructs also make safe concurrent code without needing explicit locks. Also, no callback spaghetti.
Those were the initial points that really sold us on it. As time went on, we added to the list:
- Dead-simple deployment — it compiles down to a single binary with all of its dependencies. More on that later.
- Amazing toolchain — Go also includes tons of amazing tools, not the least of which is the code formatter`go fmt`. It has eliminated code formatting debates, and with it, an untold amount of wasted developer-hours.
- Extremely robust standard library — we’ve found that we haven’t needed a ton of third party libraries. The libraries provided by the language are extremely well thought out.
Go checked off all of the boxes for our requirements — and then some. The additional benefits made it a clear winner in our book.
Were there any surprises — positive or negative — once you started using Go?
We were all hesitant about losing productivity in the switch. Ruby is a very expressive programming language which allowed us to write a lot of code quickly. We were concerned we’d lose that switching to a type safe, compiled language.
We didn’t need to be concerned. Very quickly after the switch we found ourselves writing Go code just as fast and a lot safer. Go’s type safety prevented a lot of the fat fingered mistakes that are all too common in Ruby.
Compiling our code also turned out not to be an issue — our largest Go app compiles in ~2.5 seconds at worst.
How did the team ramp up? What’s your advice to help teams go through this process smoothly?
I wrote a post when I was learning the language and it’s been the starting point for the team.
TL;DR: Tour of Go, Effective Go, and reading the standard library.
What are Go’s weaknesses?
Dependency management.
Go has a convenient import scheme where you include packages by their location, ie.
import "github.com/timehop/golog/log"
You can pull down the code with a simple “go get” command in your terminal. While convenient, it can also be deployment headache because it pulls the HEAD from the repo. This can get in the way of shipping a feature because someone else’s code changed locations or had breaking API changes.
The most popular dependency management tool right now is Godep. At a high level, Godep pulls all of your dependencies and then vendors them into your project — essentially copying your code into your project so that you always have a working copy of your dependencies.
It is worth mentioning that this weakness is by design. The creators of Go have specifically avoided building a dependency system because they wouldn’t know what a general solution would look like to others.
What Go libraries are critical to deployment on the modern web?
When we first started writing Go, we googled around for “Rails for Go.” We quickly realized that was overkill for building out JSON API.
All of our web services simply use the standard net/http library and Gorillamux for routing. Others seem to do the same.
What are options for hosting? How does deployment work?
We started on Heroku because our Rails app were hosted there as well. It was simple to deploy using the Go buildpack.
We eventually migrated to EC2 and deployment was just as easy. Go apps compile down to a single binary, so it can be as simple as scp-ing the binary to a box and running it.
Our current process is:
- Push code to Github
- Run tests via Travis
- On success, build the binaries, tar them, and upload to S3
The app servers simply have to pull down the tar, unpack it, and run it (we do this via Chef)
We haven’t needed to use this, but it also makes compiling binaries for different architectures as easy as:
$ GOOS=linux GOARCH=arm go build main.go
That means you could easily build your app for many types of operating systems and embedded devices.
Is the language suited for building APIs?
Yes. The Go encoding libraries make writing APIs stupidly simple. Take, for example, a User struct (which is similar to a class, but distinctly not):
type User struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Password string `json:"-"`
}
Those tags after those fields define how they’re going to be serialized out to JSON. No need for custom serialization functions.
To output an instance of a User, it is simply:
u := User{FirstName: "Abe", LastName: "Dino", Password: "p4ssw0rd"}
jsonBytes, _ := json.Marshal(u)
How does the language deal with polymorphism and modularization?
Go isn’t an object-oriented language — it doesn’t have any type hierarchy. While it initially took some getting used to, it quickly became a positive. Because of the lack of inheritance, it naturally encourages composition.
It also has really nice interface system. Unlike other languages where the class has to explicitly declare that it is implementing an interface, in Go, a struct simply is. It’s all very philosophical. We’ve found it to be one of the most powerful features of the language. You can read more about why here.
How important is Google’s involvement in the project?
It’s huge — having Google release and continue to invest in the language is great for everyone. It’s clear they’re using it internally and that can only mean continued improvement to the language, tools, and ecosystem.
In addition to the backing of Google, the community has grown by an insane amount since we started. It’s been amazing to see the community come alive, which has been a boon the amount of open source code and content now available in go.
Hopefully, this was helpful for teams trying to decide whether Go is worth a look. There are lots of questions whenever you consider a new language, and hopefully we’ve answered some of those here. We love writing Go and have had a ton of success doing so.
Also, we have a few open source projects, check ‘em out. And if you want to write Go for millions of daily users, we’re hiring. ☺