So maybe you've outgrown Heroku, or perhaps you want more power, flexibility or control. Or, maybe you just want to tell your friends that you built your own Heroku in 10 minutes. Whatever your reasons, I'd love you show you how simple it can be to create a multi-host Heroku-like system using EC2, Docker and Longshoreman. With that said, let's get going!
Longshoreman is a deployment system that allows you to build a Docker-powered app cluster on multiple machines. Some of it's features include: zero-downtime rollouts, deployment history and rollbacks, app instance scaling and more great stuff. Please read more about Longshoreman or visit the GitHub page if you're interested.
Today we're going to use EC2 to create a 3 node setup—where 2 nodes will run our Docker containers, route traffic between themselves and allow cluster administration (deployment, scaling, etc.) from the Longshoreman CLI. A load balancer will sit in front of our instances and will distribute traffic between them.
The load balancer is the main entry point for web traffic into the system. We're using AWS's Elastic Load Balancer in this example.
/_pinglocation for the health check. Longshoreman will respond with 200 if it's alive.
lsm-lb. Allow connections
Let's stand up a couple of EC2 instances. For this example we're going to avoid Amazon's VPC option for simplicity's sake. However it's can be a good idea to use private networking when it's available (we do in production).
Select the HVM version of Ubuntu 14.04.
Longshoreman can work with multiple domains (all pointed at the same LB), but one common use case is to launch an application to a subdomain. For example, launching an app to
*.mikejholly.com is configured with a CNAME record pointing to the LB. You'll also need to reserve a subdomain for communication between the Longshoreman and it's CLI tool. We'll be using
lsm.mikejholly.com in this example.
Longshoreman stores the state of the cluster in Redis. I'm going to use a micro EC2 instance for this example. You can also use the AWS ElastiCache or RedisLabs for a quick solution.
Launch a micro instance (I'm using Ubuntu 14.04 here).
Give it a bit more storage than the default (I used 32GB).
Configure the Security Group to allow connections to the default Redis port of 6379. Connections should only be allowed from the
lsm-node Security Group.
Launch and install Redis with
sudo apt-get install redis-server. Save the EC2 public DNS name for the next step.
/etc/redis/redis.conf to allow the server to accept connections from the outside world.
Longshoreman works by deploying instances of Docker powered applications to 1 or more servers. It does this using a CLI application just like Heroku. Since traffic to our applications and to the Longshoreman API (coming from the CLI) both flow through the same service, we use a custom domain to tell Longshoreman that we're talking to it and not one of our apps. Longshoreman uses a custom setting called
CONTROLLER_HOST to determine whether incoming requests are meant for the API or an app instance.
SSH into your new instance.
sudo apt-get update.
Install Docker with
curl -sSL https://get.docker.io/ubuntu/ | sudo sh.
Enable the Docker Remote API by adding the line below to
DOCKER_OPTS="-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock"
Restart Docker with
sudo service docker restart.
Download and run Longshoreman using the following commands:
export REDIS_HOST=ec2-255-255-255-255.us-west-2.compute.amazonaws.com (use your own Redis host here!)
export CONTROLLER_HOST=lsm.mikejholly.com (use your own controller host here!)
sudo docker run -d -p 80:80 -e REDIS_HOST=$REDIS_HOST -e REDIS_PORT=6379 -e CONTROLLER_HOST=$CONTROLLER_HOST longshoreman/longshoreman:0.10.4
You can optionally pass the
-e DEBUG=longshoreman flag if you want Longshoreman to print debug information.
We now have a fully operational Docker powered application cluster managed by Longshoreman! But ... it can't really do anything without an application, so let's push a very contrived example app to the cluster.
Install the Longshoreman CLI with
npm install -g longshoreman
longshoreman init to configure your credentials. Enter the Longshoreman controller host domain (with
http://) and your token. The token is auto-generated and is stored in Redis. To retrieve the token, run
redis-cli $REDIS_HOST GET token.
To make Longshoreman aware of your nodes run
longshoreman hosts:add <host-ip> for each host you've created.
Add a new service or application to your cluster
longshoreman apps:add demo.mikejholly.com.
longshoreman --app demo.mikejholly.com envs:set NAME=Mike to configure your application's runtime settings.
longshoreman --app demo.mikejholly.com deploy longshoreman/demo to deploy the first version of your application!
Launch your own version to see what the app does! ;)
Note: Make sure to replace
demo.mikejholly.com with the domain you're using for your cluster and app!
Thanks a lot for following along! Please give Longshoreman a star on the GitHub page if you think it's cool. We definitely enjoyed building it.
PS. Longshoreman is sponsored by Wayfinder and runs our production applications.