Scoutapark.com

A story of pragmatic dockerization

Created by:
Dan Sheppard
Paul Czarkowski / @pczarkowski

For DevOps Days Austin

Who are we ?

Dan Sheppard

Founder @ ScoutaPark

Who are we ?

Paul Czarkowski

Cloud Engineer @ Bluebox

Who are we ?

Paul Czarkowski

Snarky DevOps Princess
@ Scoutapark

WTF are we talking about?

What is ScoutaPark?
Business Challenges
How we use Docker

What is ScoutaPark?

ScoutaPark is a free to use website that connects parents, dog owners, and other park visitors to parks with the exact features they are looking for.

Let's talk business

Goals

Make money
Improve park experience for park patrons
But mostly make money

Obstacles

No Money
No Time
No Developers

Obstacle: No Money

Cheap/Free resources for startups
University programs
Incubator programs
Fiver, Craigslist, Elance, Freelancer
* Trello,Slack,bitbucket,RAX startup,etc
* Arizona State Startup School
* Edson Innovation Student Incubator
* Negotiate the fees

Obstacle: No Time

Be smart about choices, only implement critical features
Understanding the problem > Solving the problem
Present > Future
Devop all the things

Obstacle: No Developers

Pragmatic Language Choices ( PHP )
Assume everything needs explained
Cheap outsourced developers for initial prototypes/MVP
Always be on the lookout for the dev partner

Golden rules of Docker

that you should ignore

http://github.com/factorish

Dockerfile


FROM scoutapark/base

RUN \
  DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y wget && \
  wget -O - http://dl.hhvm.com/conf/hhvm.gpg.key | apt-key add - && \
  echo deb http://dl.hhvm.com/ubuntu trusty main | tee /etc/apt/sources.list.d/hhvm.list && \
  apt-get update && apt-get install -yq \
  hhvm-fastcgi nginx runit mysql-client

RUN \
  curl -sSL -o /usr/local/bin/etcdctl https://s3-us-west-2.amazonaws.com/scoutapark/etcdctl-v0.4.6 \
  && chmod +x /usr/local/bin/etcdctl \
  && curl -sSL -o /usr/local/bin/confd https://s3-us-west-2.amazonaws.com/scoutapark/confd-0.7.1-linux-amd64 \
  && chmod +x /usr/local/bin/confd

ADD . /scoutapark
CMD ["/scoutapark/docker/bin/boot"]
VOLUME /scoutapark

Docker Compose


nginx:
  build: .
  command: /scoutapark/docker/bin/boot nginx
  links:
   - mysql
   - hhvm
  ports:
   - "8081:8080"
hhvm:
  build: .
  command: /scoutapark/docker/bin/boot hhvm
  links:
   - mysql
  ports:
   - 9000
mysql:
  image: orchardup/mysql
  ports:
   - "3306:3306"

Reality kicks in

Developers not ready for this.
Data Persistence.
I'm not building this for me.

Config Management is dead.

Long live Config Management

Docker + Config Management

Chef,Puppet,Ansible
CM + Docker gives the best of both worlds.
Docker ~= packaging format
* At this point, all major CM has very functional docker integration.
* Using CM and Docker together gives you a way to move at your own pace to a more `docker` way of doing things.
* Start by using CM for managing backing services and state and to manage docker container running.
* Docker becomes roughly equiv of a packaging format. Start/Stop container not much diff to a service.

Docker Cookbook

https://supermarket.chef.io/cookbooks/docker

Installs Docker
Provides resources for most docker actions.
Puppet and Ansible both have roughly equivalent tooling.
* installs docker on most OSes
* LWRPs for build, run, push, etc
* anything can do in docker can do via cookbook resources
* at this point, all CM has similar stories.

Docker Cookbook


docker_container 'registry' do
  detach  true
  port    '5000:5000'
  action [:run]
end

docker_image 'scoutapark_app' do
  repository 'scoutapark/app:#{version}'
  registry   '127.0.01:5000'
  action [:pull]
end

docker_container 'scoutpark_app' do
  image "127.0.01:5000/scoutapark/app:#{version}"
  port '8080:8080'
  env 'MYSQL_HOST=10.0.0.22'
  action [:run]
end

Scoutapark Cookbook

Cookbook lives in app repo /cookbook
Vagrantfile for development environment
Meez for cookbook skeleton, testing tooling.
two role recipes - 'web' and 'database'
web can do nginx+hhvm, apache+mod_php, docker
database can do mysql or RAX CloudDB
local .chef/knife.rb

Scoutapark Cookbook

vagrant up
kitchen test
knife-rackspace

CM + Docker

Docker logging

log to stdout/stderr
set log files in nginx to /dev/stdout and /dev/stderr
* key is to never write logs to a file in the container
* most app configs allow setting logfile location. used /dev/stdout|stderr

Docker logging


# /etc/nginx/nginx.conf

daemon off;
user app app;
pid /app/nginx.pid;
error_log /dev/stderr;
access_log /dev/stdout;
...

Docker logging

Logspout!

sends docker logs -> external syslog


docker_container 'gliderlabs/logspout' do
  detach  true
  volume '/var/run/docker.sock:/tmp/docker.sock:ro'
  command syslog://logs.papertrailapp.com:55555
  name 'logspout'
end

Docker monitoring

Docker uses libcontainer for cgroups/namespaces.
metrics exposed in weird places in /proc
Example - /sys/fs/cgroup/memory/lxc/[longid]/
Hard to track where all of these metrics live.

* docker used to use lxc, now accesses cgroups/namespaces via libcontainer
* container metrics do get exposed by OS, but are hard to find.
* Can use collectd or sensu or similar to collect these metrics
* Docker 1.5 comes with some basic metrics in docker API

Docker monitoring

CAdvisor

Native docker metrics
Web UI
Rest API
InfluxDB and Prometheus outputs.
Probably needs to support Graphite.
* Cadvisor was written at google for kubernetes, but runs well for any dockered system
* Has a Web UI to look at local metrics to the machine
* Has a rest API that you can interrogate for metrics
* supports sending data to InfluxDB and Prometheus.
* needs to add support for more metrics storage engines

Docker monitoring

CAdvisor


docker_container 'google/cadvisor:latest' do
  detach  true
  volume [
    '/:/rootfs:ro',
    '/var/run:/var/run:rw',
    '/sys:/sys:ro',
    '/var/lib/docker/:/var/lib/docker:ro'
  ]
  port '127.0.0.1:8080:8080'
  name 'cadvisor'
end

Docker monitoring

CAdvisor::InfluxDB::Grafana

Configs in Containers

confd

templating engine written in GO.
key/value pairs from a number of supported storage engines.
Environment Variables, ETCD, Consul, etc.

Configs in Containers


# conf.d/app_Config_database.php.toml

[template]
src   = "app_Config_database.php"
dest  = "/scoutapark/app/Config/database.php"
owner = "app"
group = "app"
mode  = "0750"
keys = [
  "/db"
]
check_cmd = "apachectl configtest"
reload_cmd = "apachectl restart"

Configs in Containers


# templates/app_Config_database.php

class DATABASE_CONFIG {
  public $default = array(
    'datasource' => 'Database/Mysql',
    'persistent' => false,
    'host' => '{{ getv "db/host" }}',
    'login' => '{{ getv "db/user" }}',
    'password' => '{{ getv "db/pass" }}',
    'database' => '{{ getv "db/name" }}',
    'prefix' => 'scout_',
    'encoding' => 'utf8',
  );
}

Configs in Containers


#!/bin/bash
# bin/boot

export DB_HOST=${DB_HOST:-"mysql"}
export DB_PORT=${DB_PORT:-"3306"}
export DB_USER=${DB_USER:-"scoutapark"}
export DB_PASS=${DB_PASS:-"scoutapark"}
export DB_DATA=${DB_NAME:-"scoutapark"}

confd -onetime -config-file /scoutapark/docker/confd_env.toml
exec nginx
wait

Configs in Containers


$ docker run -d \
  -e DB_HOST=10.0.0.1 \
  -e DB_USER=scout \
  -e DB_PASS=scout \
  -e DB_NAME=scout \
  --name scoutapark_web \
  -p 8080:8080 \
  scoutapark/web bin/boot

Scoutapark.com

A story of pragmatic dockerization

Who are we ?

Who are we ?

Who are we ?

WTF are we talking about?

What is ScoutaPark?

Let's talk business

Goals

Obstacles

Obstacle: No Money

Obstacle: No Time

Obstacle: No Developers

Golden rules of Docker

that you should ignore

http://github.com/factorish

Dockerfile

Docker Compose

Reality kicks in

Config Management is dead.

Long live Config Management

Docker + Config Management

Docker Cookbook

Docker Cookbook

Scoutapark Cookbook

Scoutapark Cookbook

CM + Docker

Docker logging

Docker logging

Docker logging

Logspout!

Docker monitoring

Docker monitoring

CAdvisor

Docker monitoring

CAdvisor

Docker monitoring

CAdvisor::InfluxDB::Grafana

Configs in Containers

confd

Configs in Containers

Configs in Containers

Configs in Containers

Configs in Containers

The Future

The Future

Service Discovery

The Future

Docker based platform

The Future

Data Persistence

Links

THE END

Dan Sheppard

Paul Czarkowski / @pczarkowski