Technology at Flyclops

Using gofmt and goimports in IntelliJ IDEA

posted in Code on 2016-06-14 12:34:10 EDT by Dave Martorana

IntelliJ IDEA is arguably the best Go IDE on the planet. The go-lang-idea-plugin turns IDEA into a full-featured IDE, and does so incredibly well. In fact, since moving to it, I’m not sure how I used anything else.

In any case, the one thing really missing from the IDE is auto-formatting with gofmt or formatting+imports with goimports - but it’s pretty easy to get working.


First, install IDEA and the go-lang-idea-plugin… plugin. Next, install the “File Watchers” plugin from Intellij:

"File Watchers" plugin

Automatically calling goimports

Unfortunately, you’ll need to set up a file watcher for each project, but it’s not too tedious. With your project open, go to the Preferences screen, and open the “Tools” tree. Select “File Watchers” and click the + button to create a new file watcher.

Fill in the information similarly to below (make sure “Program” points to your goimports binary):

goimports set up

A few tips:

  • Putting -w in the “Arguments” field will tell goimports to overwrite the file, not dump its output to stdout.
  • Make sure to un-check “Immediate file synchronization” or the IDE will spaz out on you
  • I like to check “Trigger watcher regardless of syntax errors,” but YMMV

That’s it. Click save on any document in your project, and (assuming there are no errors that break goimports) your document will be formatted and imports will be automatically filled in.

This last little step is all that Sublime Text or Atom (or Vim-Go) had on IntelliJ IDEA. After this, in my humble opinion, IntelliJ IDEA stands alone as the best Go IDE out there.

Note: To use gofmt instead of goimports, simply replace the “Program” field with the path to gofmt instead of goimports and you’ll get the results you’re looking for.

Flyclops sites are now all 100% static and served over HTTPS exclusively

posted in Devops on 2016-04-27 16:28:37 UTC by Dave Martorana

Attempting to put my money where my mouth is, I just finished moving all three of our websites to a new, static-only, HTTPS-secured infrastructure.

Most recent setup

I moved the site to a statically-rendered site some time ago, serving it over Amazon S3. That went fairly well, but it’s a very minimal website, and doesn’t require anything dynamic.

The tech blog, which you’re reading now, was hosted on Wordpress, as are most blogs in this world. But over time, “features” became unnecessary (or even fairly useless) - think comments. Other features, like caching plug-ins, were nothing more than a means to an end - near-statically loading sites. But it was kludgy.

Our press kit site was based on presskit() 2.0, a PHP-based press-kit solution that is fantastic.


LetsEncrypt was released out of beta. This was a turning point. I spend a lot of time making noise about three-letter agencies and snooping and the necessity of encryption across as much of life as possible. Considering Flyclops is a profitable enterprise, there wasn’t much excuse for not having HTTPS across all our sites to begin with. With LetsEncrypt being production-ready, there was no more excuse at all.

Shortly thereafter, Amazon AWS released AWS Certificate Manager. AWS Certificate Manager offers free, auto-renewing SSL certificates for anything hosted on AWS services. That was now two free SSL services to choose from. Being hosted almost entirely on AWS, ACM was a natural choice - it’s also easier to manage from a renewal standpoint, in that there is no renewal overhead as with LetsEncrypt. There are plenty of tools that make LetsEncrypt renewing almost entirely hands-off, but that would still be something to manage.

What to do with PHP?

Now we had to choose - spin up servers to run PHP-based websites in AWS? That sounded plenty unnecessary. Instead, I took a closer look at Hugo, the Go-based static site generator I had glanced at when I moved the main website to a static site. I quickly ported the static site to Hugo, and was very happy with the process. (It helps that we recently finished moving our entire backend infrastructure to Go-written services - Hugo felt like a natural fit after that.)

I became determined to get our blog there as well, and installed the Wordpress to Hugo Exporter plug-in (thank you all you wonderful open-source devs out there). I exported my content, and spent about a day or so setting up templates, partials, taxonomies, and archetypes in Hugo. The process wasn’t entirely painless, but “minimally” painful. All-in-all, easier than I expected.

All that was left was the press kit. Lo-and-behold, I was not the first person to want to push the wonderful presskit() to a static environment, and I happened upon dopresskit-static on Github. A fork and a few tweaks later, and I had all our websites generating static output.


Static sites need static hosting, and S3 was a natural fit… except it really isn’t a totally natural environment. To boot, you can’t use AWS Certificate Manager-based SSL certificates with S3. Besides, Amazon has this sweet CDN service called Cloudfront that would deliver static content so much faster than S3.

The setup

First, I generated the following SSL certificates in AWS Certificate Manager:

  • One for and
  • One wildcard certificate for *

I set up three S3 buckets (one per site) to act as origins for content. I then set up three Cloudfront distributions, one for each bucket, with the appropriate CNAMEs. Cloudfront distributions let you specify:

  • CDN origin - an S3 bucket, or BYO
  • A secure certificate from AWS Certificate Manager
  • CNAMEs for the distribution
  • A default file to serve (index.html)
  • What to do/serve for different HTTP error codes (404 page). Hint: Depending on how you set up S3 origins, they will return 403 errors for missing documents, not 404.
  • Whether to serve both HTTP/HTTPS, HTTPS only, or redirect HTTP to HTTPS (we use redirect)

Distributions take a few minutes to set up, and once they were published, I went into DNS (conveniently, Route53 on AWS) and updated the,,, and to point to the appropriate CDN distributions.


All that was left to do was to set up publishing. At first, I just assumed I’d push public content to S3 manually as I updated it. But then I took a look at the “Deployments” capabilities of our git service, Beanstalk by Wildbit. It turns out, Beanstalk’s deployments are incredibly powerful and feature-rich. Publishing, selectively, even, a git branch to an S3 bucket, with pattern-matched inclusions and exclusions, automatically or manually, was extremely easy to set up.

(Don’t use Beanstalk for git hosting? Check out DeployBot!!)

And bonus - it will automatically invalidate any updated files on Cloudfront! That is spectacular!

Settings Click the image above for a look at all the features

Now I write content, use Hugo to generate the static version of the site, commit the changes to git, push them to my Beanstalk remotes, and a few seconds later, they’re published!

So now, at least, my money is now firmly where my mouth is - all of our sites are HTTPS only. To boot, they’re completely static, and CDN-served from an edge node close to you.

Now we can concentrate on more important things, and not worry about constant security updates to Wordpress and PHP, hosting issues, SSL costs or renewals, site or page speeds, ease of crawl-ability for web crawlers, and a host of other things.

Not too shabby for a couple days work.

The easiest way to install NSQ on FreeBSD

posted in Devops on 2015-05-15 19:38:27 UTC by Dave Martorana

This is the easiest way to install NSQ on FreeBSD. There are other, probably more proper ways of doing so, but it’s as close as you can get to the Linux install instructions.

First, you’ll need bash if you don’t already have it:

# Install BASH for installation
pkg install -y bash

# Install Go
pkg install -y go

Make sure you set up your Go paths:

# Set up Go paths
export GOPATH="<gopath>"
export PATH=$PATH:$GOPATH/bin

Next, install GPM, the package manager Bitly chose to use with NSQ. In order to install GPM and use it, you’ll need to temporarily mount fdescfs, the file-descriptor file system for FreeBSD. You don’t need this as a permanent mount, so I use it in the install script only:

# Install GPM
git clone gpm
cd gpm
git checkout -b v1.3.2
make install

Now that we have GPM set up, we can move to NSQ:

# Install
go get -d || true
cd "$GOPATH/src/"
git checkout -b v0.3.5
gpm install
go get ./...

That’s it. nsqd, nsqlookup, etc., will be in your $GOPATH/bin directory.

#nsq  #freebsd

JsonTree Sublime Text 3 plugin

posted in Code on 2014-03-31 18:13:04 UTC by Dave Martorana

We use Sublime Text (23) pretty heavily at Flyclops.

We (and especially I) deal with a lot of JSON files. In my case, the majority of our backend is defined by AWS CloudFormation template files - and man, can they be long. So I wanted a way to browse them quickly.

So, I wrote this wonky, almost-working plugin for Sublime Text 3 (which is a “beta” but substantially better than 2) called JsonTree. It works by hitting a key combination (cmd-ctrl-j in OS X, or ctrl-alt-j in Windows/Linux) and brings up a tree-view of your JSON document that is browsable and searchable. Screenshot:


JsonTree has been accepted into the Sublime Package Repository. Grab the Package Control plugin and install JsonTree from there.

This is the second open-source code bit from Flyclops.

GoCM: Super-simple asynchronous GCM listener in Go

posted in Code on 2014-03-18 19:44:19 UTC by Dave Martorana

Build Status

We have been using pyapns, a very light-weight, fast, Python/Twisted based asynchronous push notification server for Apple Push Notification Services. It sent our very first push notification ever, and has sent well over 1.5 billion more since. I originally stumbled upon it while reading the Instagram Engineering Blog, and this post in particular.

With pyapns, we toss off an alert asynchronously and never block the API from returning as quickly as possible. It’s rock-solid.

GCM, or Google Cloud Messaging, is the technology that is used to send notifications to Android devices. Unlike APNS, GCM has an HTTP REST interface to GCM servers that is used to send “downstream messages” (server-to-device).

The problem is that sending a message to GCM over HTTP is a synchronous call. Our average API response time is ~70ms, but sending a message synchronously added between 125-250ms on top of that.

So we needed to make GCM just as “fast” as APNS.

It’s likely not obvious yet, but we’re slowly beginning to move pieces of our monolithic Python-based API server application to a multi-service architecture built primarily on Go. At the speed that we’re answering API requests, we’ve hit the need a massively concurrent language that’s also capable of taking advantage of multi-core, multi-CPU systems. But more on this in later post.

No matter what language our API is built on, now or in the future, pyapns will likely continue to run our APNS push notifications. So, we needed something similar for GCM.

Enter GoCM.

GoCM follows some of the basic conventions of pyapns, but almost none of the architecture. It accepts a push request and returns immediately, handing the work off to a series of asynchronous goroutines. It can handle thousands of messages at a time, and doesn’t block the API.

GoCM listens via the HTTP protocol on port 5601 (or one of your choosing) and can accept messages to send from any application written in any language - from a BASH script to Python to C++ and everything inbetween.

It also has a (currently minor) reporting interface, and gracefully handles GCM canonical responses, allowing for them to be handled asynchronously as well.

Open Source

While we rely on a lot of open source software, and contribute code to many OSS projects, GoCM is the first piece of OSS originally authored at Flyclops that is being released for public use. If you’d like to give GoCM a try, head over to our GitHub repository and check it out.

GoCM is enabling us to grow with our Android traffic while still keeping those nice, fast response times. Maybe you can get some use out of it too.

GCM is also beginning to push it’s long-open-connection solution for messaging that uses CSS/XMPP. However, there are several hurdles - first, it requires special device permissions, and with hundreds of thousands of copies of our games in the wild, it would take months to get them all updated to use the newer technology - and until then, we’d have to continue sending messages over the HTTP REST interface. Secondly, we have no need currently for GCM’s XMPP bidirectional protocol.