Agenda
Troubleshooting Containers
Overview of Security Practices
Private Registry
Intro to Docker Machine
Intro to Docker Swarm
Intro to Docker Compose
Building micro service applications with Docker
Container logging
View the output of the containers PID 1 process: docker logs <container name>
View and follow the output: docker logs -f <container name>
Limit the output: docker logs -f —tail 5 <container name>
Container application logs
Typically, apps have a well defined log location
Map a host folder to the application’s application log folder in the container
In this way, you can view the log generated in the container from your host folder
Run a container using nginx image and mount a volume to map the /nginxlogs folder in the host to the /var/log/nginx folder in the container: docker run -d -P -v /nginxlogs:/var/log/nginx nginx
Check container logs
Run a new container using the tomcat image: docker run -d -P tomcat
View the container log: docker logs <container id>
On your host machine, create a folder /container/logs/nginx
Run a new container using the NGINX image and mount the /container/logs/nginx folder into /var/log/nginx: docker run -d -P -v /container/logs/nginx:/var/log/nginx nginx
Look inside your /container/logs/nginx folder and notice the new log files from the container
Inspecting a container
docker inspect command displays all the details about a container
Outputs details in JSON array
Use grep to find a specific property
Display all details of the specified container: docker inspect <container name>
Display the IP address of the specified container: docker inspect <container name> | grep IPAddress
Format: docker inspect —format [{.NetworkSettings.IPAddress}] <container name>
Starting and Stoping Decker daemon
If you started Docker as a service, use service command to stop, start and restart the Docker daemon
sudo service docker stop
sudo service docker start
sudo service docker restart
If not running as a service, run Docker executable in daemon mode to start the daemon: sudo docker -d &
If not running as a service, send a SIGTERM to the Docker process to stop it
Run ‘pidof docker’ to find the Docker process PID
sudo kill $(pid of docker)
Docker daemon upstart configuration file
Located in /etc/default/docker
Use DOCKER_OPTS to control the startup options for the daemon when running as a service
Restart the service for changes to take effect: sudo service docker restart
Start daemon with log level of debug and allow connections to an insecure registry at the domain of my_server.org : DOCKER_OPTS=“—log-level debug —insecure-registry my_server.org:5000”
Docker daemon logging
Start the docker daemon with —log-level parameter and specify the logging level
Levels are (in order from most verbose to least):
Debug
Info
Warn
Error
Fatal
Run docker daemon with debug log level (log written on terminal): sudo docker -d —log-level=debug
Configuring in DOCKER_OPS (log output will be written to /var/log/upstart/docker.log): DOCKER_OPTS=“—log-level debug”
Linux containers and security
Docker helps make applications safer as it provides a reduced set of default privileges and capabilities
Namespaces provide an isolated view of the system. Each container has its own
IPC, network stack, root file system etc…
Processes running in one container cannot see and effect processes in another container
Control groups (Cgroups) isolate resource usage per container
Ensures that a compromised container won’t bring down the entire host by exhausting resources
Quick security considerations
Docker daemon needs to run as root
Only ensure that trusted users can control the Docker daemon
Watch who you add to docker group
If binding the daemon to a TCP socket, secure it with TLS
Use Linux hardening solution
Apparmor
SELinux
GRSEC
Private Registry
Allows you to run your own registry instead of using Docker Hub
Multiple options
Run registry server using container
Docker Hub Enterprise
Two versions:
Registry v1.0 for Docker 1.5 and below
Registry v2.0 for Docker 1.6
Setting up a private registry
Run the registry server inside a container
Use the registry image at https://registry.hub.docker.com/u/library/registry
Image contains a preconfigured version of registry v2.0
Run a new container using the registry image: docker run -d -p 5000:5000 registry:2.0
Push and pull from private registry
First tag the image with host IP or domain of the registry server, then run docker push
Tag image and specify the registry host: docker tag <image id> my_server.net:5000/my-app:1.0
Push image to registry: docker push my_server.net:5000/my-app:1.0
Pull image from registry: docker pull my_server.net:5000/my-app:1.0
List tags: curl -v -X GET http://localhost:5000/v2/mynginx/tags/list
Docker machine overview
Docker machine is a tool that automatically provisions Docker hosts and installs the Docker Engine on them
Create additional hosts on your own computer
Create hosts on cloud providers(e.g. Amazon AWS, DigitalOcean etc…)
Machine creates the server, installs Docker and configures the Docker client
Installing Machine
Download the binary for the operating system at https://github.com/docker/machine/releases/tag/v0.2.0
Place the binary into a folder of your choice
Add the folder to your system environment PATH
Creating a host
Use 'docker-machine create’ command and specify the driver to use
Use virtual box driver if creating hosts on a Windows or Mac
Need to have Virtual Box installed (https://www.virtualbox.org/)
Create a host named “testiest” on the current machine, using Virtual Box: docker-machine create —driver virtual box testhost
Provisioning hosts in the cloud
Each cloud provider has different options on the docker-machine create command
See https://docs.docker.com/machine/#drivers as reference
Example with DigitalOcean
docker-machine create —driver digitalocean —digitalocean-access-token <your access token> —digitalocean-size 2gb testhost
List machines: docker-machine ls
Docker machine SSH
Allows us to connect to a provisioned host using SSH
Logs in using the SSH key that is created when creating the machine
Connect to host3 using SSH: docker-machine ssh host3
What is Docker Swarm
Docker Swarm is a tool that clusters Docker hosts and schedules containers
Turns a pool of host machines into a single virtual host
Ships with simple scheduling backend
Supports many discovery backends
Hosted discovery
etcd
Consul
ZooKeeper
Static files
https://docs.docker.com/swarm/discovery
Setup process (using hosted discovery)
On the machine that you will use as the Swarm master, run a command to create the cluster
Start Swarm master
For each node with Docker installed, run a command to start the Swarm agent
Note: Agents can be started before or after the master
Installing and running Swarm
Most convenient option is to use the Swarm image on Docker Hub https://registry.hub.docker.com/u/library/swarm/
Swarm container is a convenient packaging mechanism for the Swarm binary
Swarm containers can be run from the image to do the following
Create a cluster
Start the Swarm manager
Join nodes to the cluster
List nodes on a cluster
Create the Swarm cluster
'swarm create’ command will output the cluster token
Token is an alphanumeric sequence of characters that identifies the cluster when using the hosted discovery protocol
Copy this number somewhere
Run a container using the swarm image. We run the create command of the Swarm application inside and get the output on our terminal. —rm means to remove the container once it has finished running.
docker run —rm swarm create
Start the Swarm manager
Run a container that run the ‘swarm manager’
Make sure to map the swarm port in the container to a port on the host: docker run -d -P swarm manage token://<cluster token>
Connect a node to the cluster
Run a container that funs the ‘swarm join’ command
Specify the IP address of the node and the port the Docker daemon is listening on
Note: Your Docker daemon on the machine must be configured to listen on a TCP port instead of just on the unix socket.
docker run -d swarm join —addr=<node ip>:<daemon port> token://<cluster token>
sudo service docker stop
sudo vim /etc/default/docker
sudo service docker start
DOCKER_HOST=localhost:2375
export DOCKER_HOST
Connect the Docker client to Swarm
Point your Docker client to the Swarm manager container
Two methods:
Configuring the DOCKER_HOST variable with the Swarm IP and port
Run docker with -H and specify the Swarm IP and port
Look at the container port mapping to find the Swarm port
Configure the DOCKER_HOST variable
export DOCKER_HOST=127.0.0.1:<swarm port>
Run docker client and specify the daemon to connect to
docker -H tcp://127.0.0.1:<swarm port>
Checking your connected nodes
Run ‘docker info’
Since client is connected to Swarm, it will show the nodes
Run a container in the cluster
Standard ‘docker run’ command
Swarm master decides which node to run the container on based on your scheduling strategy
https://docs.docker.com/swarm/scheduler/strategy
Running ‘docker ps’ will sow which node a container is on
What is Compose
Docker Compose is a tool for creating and managing multi container applications
Containers are all defined in a single file called ‘docker-compose.ml'
Each container runs a particular component / service of your application.
For example:
Web front end
User authentication
Payments
Database
Container links are defined
Compose will spin up all your containers in a single command
Configuring the Compose yml file
Defines the services that make up your application
Each service contains instructions for building and running a container
Example
javaclient:
build: . (building using Dockerfile in current directory)
command: java HelloWorld
links:
-redis
redis:
image: redis (Use the latest redis Image from Docker Hub)
Build and image instruction
‘build' defines the path to Dockerfile that will be used to build the image
Container will be run using the image build
‘image’ defines the image that will be used to run the container
All services must have either a build or image instruction
Running your application
Use ‘docker-compose up’
Up command will
Build the image for each service
Create and start the containers
Install docker-compose
https://docs.docker.com/compose/install/
curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose- uname -s - uname -m > /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose