Notes & Commands from my first real attempt at learning Docker (…but I probably spent more time fiddling with markdown styling & css of this post)

List Containers

docker ps syntax

docker ps [OPTIONS]

ps-docs

docker ps

# CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                              NAMES
# e9e1f48613d9        amazon/dynamodb-local   "java -jar DynamoDBL…"   9 seconds ago       Up 8 seconds        8000/tcp, 0.0.0.0:8000->8008/tcp   festive_lamport

List Docker Images

docker images syntax

docker images [OPTIONS] [REPOSITORY[:TAG]]

images-docs

docker images

# REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
# amazon/dynamodb-local     latest              d0e7e8b4a50a        8 months ago        506MB
# node                      7.2.1-alpine        a1c188c2c5e1        2 years ago         55.3MB

Pull an image or a repository from a registry

docker pull syntax

docker pull [OPTIONS] NAME[:TAG|@DIGEST]

pull-docs

docker pull amazon/dynamodb-local
#           [NAME]
#           amazon/dynamodb-local

Run a command in a new container

docker run syntax

docker run [OPTIONS] IMAGE [COMMAND][arg...]

run-docs

docker run -p 8000:8000 amazon/dynamodb-local
           ├ [Options]
           # -p (Host:Container)
                        ├ [Image]
                        # amazon/dynamodb-local

docker run options

Name, shorthand	Default	Description
`--publish` , `-p`		Publish a container’s port(s) to the host. See container-networking
`--detach` , `-d`		Run container in background and print container ID

Run a command in a running container

docker exec syntax

docker exec [OPTIONS] CONTAINER COMMAND [ARG...]

exec-docs

docker exec ...

Example: List files in a running container

docker ps -al

# CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
# a928b1a90fb3        postgres            "docker-entrypoint.s…"   47 hours ago        Up 46 hours         5432/tcp            docker-compose-django_db_1

docker exec a928b1a90fb3 ls

# bin
# boot
# dev
# docker-entrypoint-initdb.d
# docker-entrypoint.sh
# etc
# home
# lib
# lib64
# media
# mnt
# opt
# proc
# root
# run
# sbin
# srv
# sys
# tmp
# usr
# var

Deleting unused containers

docker system prune syntax

docker system prune [OPTIONS]
       ├ [base]
       # system
              ├ [COMMAND]
              # prune

system docs

system prune docs

docker system prune

Example

docker system prune

# WARNING! This will remove:
#  - all stopped containers
#  - all networks not used by at least one container
#  - all dangling images
#  - all dangling build cache
#
# Are you sure you want to continue? [y/N]

y

# Deleted Containers:
# 8bc3730ccf92cde2ba71648487810af163fb3f765a4f9e2e310e5f2eeab1e08e
# ...(truncated)

# Deleted Networks:
# ...(truncated)

# Deleted Images:
# deleted: sha256:6938bb2bf6917e2d4e2e0af781b199e2643b6d1d27b00f993a5c7770faa23c76
# deleted: sha256:47e443d20a18d3720df4afc842529527e0a0fdb18641421c4bf17c3636728e1f
# ...(truncated)

# Deleted build cache objects:
# vbv35t98brzqtai4ryp7iq7hq
# ypjmoyte081rvb2y69sgktqi0
# ...(truncated)

# Total reclaimed space: 754.3MB

Before: docker ps -al

After: docker ps -al

Docker Compose

Notes

compose-docs

What is docker-compose?

“A tool for defining and running multi-container Docker applications”
“You use a YAML file to configure your application’s services”

3 step process to using docker-compose

Define your app’s environment with a Dockerfile so it can be reproduced anywhere.
Define the services that make up your app in docker-compose.yml so they can be run together in an isolated environment.
Run docker-compose up and Compose starts and runs your entire app.

Usage with Django + PostgreSQL

I just started learning Django last week, so coming across this is a perfect coincidence.

compose + django docs

Dockerfile

FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/

Explanation

Statement	Params	What’s going on?
`FROM`	`python:3`	This Dockerfile starts with ha Python 3 parent image
`ENV`	`PYTHONUNBUFFERED` `1`	Set environment variable
`RUN`	`mkdir /code`	Add a new directory to the image
`WORKDIR`	`/code`	Set working directory to `code/` for following `RUN`, `CMD`, `ENTRYPOINT`, `COPY`, `ADD` commands
`COPY`	`requirements.txt` `/code/`	Copy files/dirs from `<src>` to the filesystem of the container, `<dest>` `src` is relative to the source of the context of the build `dest` is an absolute path, or a path relative to `WORKDIR`
`RUN`	`pip install -r requirements.txt`	Run pip install on the container
`COPY`	`.` `/code/`	?

requirements.txt

Django>=2.0,<3.0
psycopg2>=2.7,<3.0

What's this for?

This file is used by the RUN pip install -r requirements.txt command in the Dockerfile.

docker-compose.yml

version: "3"

services:
  db:
    image: postgres
  web:
    build: .
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

Uh, what?

Compose file reference

This describes

the services, db & web, which make up the app.
which Docker images the services use
how they link together
volumes they need mounted inside the containers.
the ports the services expose

Key	Description
`image`	Specify the image to start the container from. Can either be a repository/tag or a partial image ID.
`volumes`	Mount host paths or named volumes, specified as sub-options to a service. Here, the web service uses a volume that is defined using the old string format for mountain a volume. Uses “short syntax”: `HOST:CONTAINER`
`depends_on`	Express dependency between services. `docker-compose up` starts services in “dependency order”. So, `db` starts before `web`

Create a Django Project

Syntax

run [options] [-v VOLUME...] [-p PORT...] [-e KEY=VAL...] [-l KEY=VALUE...] SERVICE [COMMAND] [ARGS...]

docker-compose run docs

Runs a one-time command against a service.

The following command instructs Compose to run django-admin startproject composeexample in a container, using the web service’s image and configuration.

Because the web image doesn’t exist yet, Compose builds it from the current directory, as specified by the build: . line in docker-compose.yml

sudo docker-compose run web django-admin startproject composeexample .
#                       [SERVICE]
#                       web
#                           [COMMAND]
#                           django-admin startproject composeexample
#                           - "create a project called composeexample"
#                                                                 [ARGS]
#                                                                 .

Sample Terminal Log

~/r/docker-compose-django $
~/r/docker-compose-django $ touch Dockerfile
~/r/docker-compose-django $ vim Dockerfile
~/r/docker-compose-django $ touch requirements.txt
~/r/docker-compose-django $ vim requirements.txt
~/r/docker-compose-django $ touch docker-compose.yml
~/r/docker-compose-django $ vim docker-compose.yml
~/r/docker-compose-django $ sudo docker-compose run web django-admin startproject composeexample .
Password:
Creating network "docker-compose-django_default" with the default driver
Pulling db (postgres:)...
latest: Pulling from library/postgres
b8f262c62ec6: Pull complete
fe6da876d968: Pull complete
46b9d53972f5: Pull complete
23a11bddcc75: Pull complete
d6744ba78bdc: Pull complete
8d95423a7aa9: Pull complete
8590ba4183e5: Pull complete
ed97b9b8e039: Pull complete
d9b574d4da1e: Pull complete
04119344259c: Pull complete
774edf2116fc: Pull complete
2d839f35fc94: Pull complete
77d2dd4efe0a: Pull complete
22c5d24859af: Pull complete
Digest: sha256:3dbb3cb945dfe0316dcdd3a75e8a3c6192ce30f87a9952f285b9ba2f02b81982
Status: Downloaded newer image for postgres:latest
Creating docker-compose-django_db_1 ... done
Building web
Step 1/7 : FROM python:3
3: Pulling from library/python
4a56a430b2ba: Pull complete
4b5cacb629f5: Pull complete
14408c8d4f9a: Pull complete
ea67eaa7dd42: Pull complete
4d134ac3fe4b: Pull complete
4c55f6f5d7f0: Pull complete
6ae475e50652: Pull complete
6f4152644229: Pull complete
6933d3d46042: Pull complete
Digest: sha256:9455815814cd05da0fe73fba64dbed5a3dcb582c757b6c7591c49178c3a4398c
Status: Downloaded newer image for python:3
 ---> 02d2bb146b3b
Step 2/7 : ENV PYTHONUNBUFFERED 1
 ---> Running in 4eb161cc1ff9
Removing intermediate container 4eb161cc1ff9
 ---> c47637decb35
Step 3/7 : RUN mkdir /code
 ---> Running in dac0b2776f8c
Removing intermediate container dac0b2776f8c
 ---> 906b1e10b143
Step 4/7 : WORKDIR /code
 ---> Running in 19cecd5bb2d9
Removing intermediate container 19cecd5bb2d9
 ---> 606becca4aea
Step 5/7 : COPY requirements.txt /code/
 ---> 98b4d6c6d83e
Step 6/7 : RUN pip install -r requirements.txt
 ---> Running in 61c6a6b94003
Collecting Django<3.0,>=2.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b2/79/df0ffea7bf1e02c073c2633702c90f4384645c40a1dd09a308e02ef0c817/Django-2.2.6-py3-none-any.whl (7.5MB)
Collecting psycopg2<3.0,>=2.7 (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/5c/1c/6997288da181277a0c29bc39a5f9143ff20b8c99f2a7d059cfb55163e165/psycopg2-2.8.3.tar.gz (377kB)
Collecting sqlparse (from Django<3.0,>=2.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Collecting pytz (from Django<3.0,>=2.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py): started
  Building wheel for psycopg2 (setup.py): finished with status 'done'
  Created wheel for psycopg2: filename=psycopg2-2.8.3-cp37-cp37m-linux_x86_64.whl size=465788 sha256=6d3889e010861b408ec9cc225ca637438d0394685fd4ae5daf324eb79e671eb7
  Stored in directory: /root/.cache/pip/wheels/48/06/67/475967017d99b988421b87bf7ee5fad0dad789dc349561786b
Successfully built psycopg2
Installing collected packages: sqlparse, pytz, Django, psycopg2
Successfully installed Django-2.2.6 psycopg2-2.8.3 pytz-2019.3 sqlparse-0.3.0
Removing intermediate container 61c6a6b94003
 ---> febdc392632b
Step 7/7 : COPY . /code/
 ---> e17ab833112b
Successfully built e17ab833112b
Successfully tagged docker-compose-django_web:latest
WARNING: Image for service web was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.

Connect the database

These steps are Django specific. See Connect the database

Start your Django Project

Syntax

docker-compose up [options] [--scale SERVICE=NUM...] [SERVICE...]

docker-compose up docs

docker-compose up

Sample Terminal Log

~/r/docker-compose-django $ docker-compose up                                                                               11:59:26
Starting docker-compose-django_db_1 ... done
Recreating docker-compose-django_web_1 ... done
Attaching to docker-compose-django_db_1, docker-compose-django_web_1
db_1   | 2019-10-12 15:59:45.658 UTC [1] LOG:  starting PostgreSQL 12.0 (Debian 12.0-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2019-10-12 15:59:45.658 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2019-10-12 15:59:45.658 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2019-10-12 15:59:45.661 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2019-10-12 15:59:45.674 UTC [25] LOG:  database system was interrupted; last known up at 2019-10-12 15:59:09 UTC
db_1   | 2019-10-12 15:59:45.761 UTC [25] LOG:  database system was not properly shut down; automatic recovery in progress
db_1   | 2019-10-12 15:59:45.763 UTC [25] LOG:  redo starts at 0/1645598
db_1   | 2019-10-12 15:59:45.763 UTC [25] LOG:  invalid record length at 0/16455D0: wanted 24, got 0
db_1   | 2019-10-12 15:59:45.763 UTC [25] LOG:  redo done at 0/1645598
db_1   | 2019-10-12 15:59:45.773 UTC [1] LOG:  database system is ready to accept connections
web_1  | Watching for file changes with StatReloader
web_1  | Performing system checks...
web_1  |
web_1  | System check identified no issues (0 silenced).
web_1  |
web_1  | You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
web_1  | Run 'python manage.py migrate' to apply them.
web_1  | October 12, 2019 - 15:59:47
web_1  | Django version 2.2.6, using settings 'composeexample.settings'
web_1  | Starting development server at http://0.0.0.0:8000/
web_1  | Quit the server with CONTROL-C.

Stop your Django Project

Syntax

docker-compose down [options]

docker-compose down docs

docker-compose down
# ctrl + c

17 unapplied migration(s)?

docker-compose run python manage.py migrate
# ERROR: No such service: python

docker-compose run web python manage.py migrate
# Starting docker-compose-django_db_1 ... done
# Operations to perform:
#   Apply all migrations: admin, auth, contenttypes, sessions
# Running migrations:
#   Applying contenttypes.0001_initial... OK
#   Applying auth.0001_initial... OK
#   Applying admin.0001_initial... OK
#   Applying admin.0002_logentry_remove_auto_add... OK
#   Applying admin.0003_logentry_add_action_flag_choices... OK
#   Applying contenttypes.0002_remove_content_type_name... OK
#   Applying auth.0002_alter_permission_name_max_length... OK
#   Applying auth.0003_alter_user_email_max_length... OK
#   Applying auth.0004_alter_user_username_opts... OK
#   Applying auth.0005_alter_user_last_login_null... OK
#   Applying auth.0006_require_contenttypes_0002... OK
#   Applying auth.0007_alter_validators_add_error_messages... OK
#   Applying auth.0008_alter_user_username_max_length... OK
#   Applying auth.0009_alter_user_last_name_max_length... OK
#   Applying auth.0010_alter_group_name_max_length... OK

Recap: “Anatomy of a docker-compose command”

docker-compose run web python manage.py migrate
├ [base]
  # docker-compose run
                   ├ [service]
                   # web
                   # ...see docker-compose.yml
                       ├ [command]
                       # python manage.py migrate

Recap: “Where are my containers?”

docker ps

Glossary

After 2 days of writing this Docker blog post, I figured out how to use Docker, but realized that I still couldn’t define some key terms… So here are some of them… for myself.

Term	Definition
container	A container is a runtime instance of a docker image.
image	Docker images are the basis of containers. An Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime. An image typically contains a union of layered filesystems stacked on top of each other. An image does not have state and it never changes.
layer	In an image, a layer is modification to the image, represented by an instruction in the Dockerfile. Layers are applied in sequence to the base image to create the final image. When an image is updated or rebuilt, only layers that change need to be updated, and unchanged layers are cached locally. This is part of why Docker images are so fast and lightweight. The sizes of each layer add up to equal the size of the final image.
base image	An image that has no parent is a base image.
parent image	An image’s parent image is the image designated in the `FROM` directive in the image’s Dockerfile. All subsequent commands are applied to this parent image. A Dockerfile with no `FROM` directive has no parent image, and is called a base image.

← Home

Docker

List Containers

List Docker Images

Pull an image or a repository from a registry

Run a command in a new container

Run a command in a running container

Deleting unused containers

Docker Compose

What is docker-compose?

3 step process to using docker-compose

Usage with Django + PostgreSQL

This describes

Create a Django Project

Connect the database

Start your Django Project

Stop your Django Project

17 unapplied migration(s)?

Recap: “Anatomy of a docker-compose command”

Recap: “Where are my containers?”

Glossary

Comments