Docker SBeliakou Part01
Docker SBeliakou Part01
2
AGENDA
• Docker Architecture Overview
• Docker Installation and Configuration
• Creating Docker Images
• Running Containers
• Mounting Data from the Host
ARCHITECTURE OVERVIEW
o Docker Architecture
o Docker Components
o Isolation Technologies
o What is a Container?
o What is an Image?
Docker Architecture
Server Registry
Client Docker Daemon Docker Images
unix socket
CLI Tool
tcp
docker tcp socket
build
pull
run
ps
logs
images Containers Images Volumes Networks
port
volume
network
exec
start
stop
inspect
export
...
5
Docker Workflow
6
What is a Container?
7
Virtual Machines vs Containers
8
Isolation Technologies: Functional Requirements
Why How
A process in the container shouldn’t see other processes running Namespaces
in the system and should feel itself as the only process in the
system
We should have facility to manage systems resources – (for Control Groups
example) to provide necessary amount of RAM, CPU shares and so
on to the Container
A process in the container should see/use the only that part of the Chroot
filesystems which is required by it
A process in the container should have enough permissions to Process Capabilities
manage/use kernel
A process in the container should have access to ethernet device Virtual eth
A few containers can expose the same ports, but is shouldn’t Port Binding
cause any problem
A service should have facility to keep data, even it goes down Volumes
accidently
Service should be able to communicate with other services by Docker Network
IP/names
9
Isolation Technology
Namespaces
When you run a container, Docker creates a set of namespaces for that
container. This provides a layer of isolation: each aspect of a container
runs in its own namespace and does not have access outside of it.
10
Isolation Technology
Control Groups
11
Isolation Technology
chroot
is simply isolation
on the filesystem
Union FS
operate by creating layers, making them very lightweight and
fast. Docker Engine uses union file systems to provide the
building blocks for containers. Docker Engine can make use of
several union file system variants including: AUFS, btrfs, vfs, and
DeviceMapper
12
Docker Image. What is this?
An image is an inert, immutable, file that's essentially a snapshot of a container. Images are created with
the build command, and they'll produce a container when started with run. Images are stored in a Docker
registry such as https://hub.docker.com. Because they can become quite large, images are designed to be
composed of layers of other images, allowing a minimal amount of data to be sent when transferring images
over the network
Docker Image
(layers)
13
Docker Storage (Graph) Drivers
UnionFS allows files and directories of separate file systems, known as branches, to be transparently overlaid,
forming a single coherent file system. Contents of directories which have the same path within the merged
branches will be seen together in a single merged directory, within the new, virtual filesystem.
14
Docker Graphdrivers
Given the “image graph” of layer content represents the relationships between various layers, the driver
to handle these layers is called a “graphdriver.”
➢ VFS - does not use a Union FS or CoW, is very valuable for simple validation and testing of other parts
of the Docker engine
➢ AUFS - enables shared memory pages for different containers loading the same shared libraries from
the same layer (because they are the same inode on disk). Currently it's available on Debian and
Ubuntu and not considered as mainstream.
➢ Overlay2 - resolves the inode exhaustion problem as well as a few other bugs that were inherent to
the old design of the original driver. It also enables shared memory between disparate containers
using the same on-disk shared libraries.
➢ Device Mapper - It is quite unlike the union filesystems in that devicemapper works on block devices.
There is no way to get default “out of the box” performance with devicemapper. Some of the features
rely on specific versions of libdevmapper and it requires above-average skill to validate all these
settings on a system.
➢ Btrfs - a disk formatted as a btrfs filesystem is required as the graphdriver root (by default,
/var/lib/docker). It's not supported by RedHat.
15
Docker Infrastructure
docker client dockerd:
The Docker daemon itself. The highest level component in your list and also
the only 'Docker' product listed. Provides all the nice UX features of Docker.
docker-containerd:
It’s a daemon, listening on a Unix socket, exposes gRPC endpoints. Handles
docker (daemon)
all the low-level container management tasks, storage, image distribution,
network attachment, etc...
containerd docker-containerd-ctr:
A lightweight CLI to directly communicate with containerd. Think of it as
how 'docker' is to 'dockerd'.
docker-containerd-shim:
shim shim shim After runC actually runs the container, it exits (allowing us to not have any
long-running processes responsible for our containers). The shim is the
runc runc runc component which sits between containerd and runc to facilitate this.
||| ||| ||| docker-runc:
A lightweight binary for actually running containers. Deals with the low-
docker-proxy
docker-proxy
docker-proxy
16
Docker installation and Configuration
18
Docker on MacOS X and Windows
19
Docker on Windows
Microsoft includes two different types of containers:
And here:
• working-windows-containers-docker-basics
• working-windows-containers-docker-running
• working-windows-containers-docker-stride
20
Local Engineering Environment
21
Installing Docker Service
The docker daemon binds to a Unix socket /var/run/docker.sock which is owned by root:docker.
[vagrant@localhost ~]$ docker info
Got permission denied while trying to connect to the Docker daemon socket at
unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/json:
dial unix /var/run/docker.sock: connect: permission denied
[vagrant@localhost ~]$ id
uid=1000(vagrant) gid=1000(vagrant) groups=1000(vagrant),993(docker)
22
How It Works
Either by using the docker binary or via the API, the Docker client tells the Docker daemon to run a container.
The Docker Engine client is launched using the docker tool with the run option running a new container.
The bare minimum the docker client needs to tell the Docker daemon to run the container is:
✓ What Docker image to build/create the container from, for example, ubuntu
✓ The command you want to run inside the container when it is launched, for example,/bin/bash
You now have a running container! Now you can manage your container, interact with your application and then, when
finished, stop and remove your container.
23
Try Something Simple
[vagrant@localhost vagrant]$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
9db2ca6ccae0: Pull complete
Digest: sha256:4b8ff392a12ed9ea17784bd3c9a8b1fa3299cac44aca35a85c90c5e3c7afacdc
Status: Downloaded newer image for hello-world:latest
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/engine/userguide/
24
Docker Daemon, Hub/Registry Commands
25
Inspecting Docker Configuration
[vagrant@localhost ~]$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.06.0-ce
Storage Driver: overlay2
Backing Filesystem: xfs
Logging Driver: json-file
Runtimes: runc
Default Runtime: runc
Kernel Version: 3.10.0-862.9.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
Docker Root Dir: /var/lib/docker
Registry: https://index.docker.io/v1/
...
☞
[vagrant@localhost]$ ps -eo %a | grep [d]ocker
☞
/usr/bin/dockerd
docker-containerd --config /var/run/docker/containerd/containerd.toml
26
Docker Root Dir
27
Configuring Docker Daemon
/etc/docker/daemon.json
{
"exec-opts": [
"native.cgroupdriver=systemd"
],
"storage-driver": "devicemapper"
}
# systemctl daemon-reload
# systemctl restart docker.service
# docker info | egrep "(Cgroup|Storage) Driver"
Storage Driver: devicemapper
Cgroup Driver: systemd
28
Creating Docker Images
o Build own Image with Dockerfile
o Tagging Images
o Entrypoint and CMD
o Build Arguments
o Multistage Build
o ONEBUILD Instructions
o Health Checks
Working with Images
Lifecycle:
o docker images - shows all images.
o docker import - creates an image from a tarball.
o docker build - creates image from Dockerfile.
o docker commit - creates image from a container, pausing it temporarily if it is running.
o docker rmi - removes an image.
o docker load - loads an image from a tar archive as STDIN, including images and tags.
o docker save - saves an image to a tar archive stream to STDOUT with all parent
layers, tags & versions.
Info:
o docker history - shows history of image.
o docker tag - tags an image to a name (local or registry).
30
What is an Image?
trainings:centos-node
9a6721b97758
ansible:2.6.2
. . .
639b4cd19004
4a45ab92583f
ansible:2.6.1
a3c99315580f
22a01822f3c0
f52300eb0803
sbeliakou/centos:latest
b24939aa8741
469cfcc7a4b3
31
Writing Own Dockerfile
http://flask.pocoo.org/ dockerfile_best-practices
http://flask.pocoo.org/docs/1.0/quickstart/#quickstart sbeliakou/flask-hello
hello.py Dockerfile
FROM python:2.7
from flask import Flask
app = Flask(__name__) RUN pip install Flask
COPY . /
@app.route("/")
def hello(): ENV FLASK_APP=hello.py
EXPOSE 5000
return "Hello World!\n"
CMD flask run --host=0.0.0.0
32
Dockerfile Instructions
o .dockerignore
o FROM - Sets the Base Image for subsequent instructions.
o RUN - execute any commands in a new layer on top of the current image and commit
the results.
o CMD - provide defaults for an executing container.
o EXPOSE - informs Docker that the container listens on the specified network ports at runtime.
NOTE: does not actually make ports accessible.
o ENV - sets environment variable.
o ADD - copies new files, directories or remote file to container. Invalidates caches.
o COPY - copies new files or directories to container.
o ENTRYPOINT - configures a container that will run as an executable.
o VOLUME - creates a mount point for externally mounted volumes or other containers.
o USER - sets the user name for following RUN / CMD / ENTRYPOINT commands.
o WORKDIR - sets the working directory.
o ARG - defines a build-time variable.
o ONBUILD - adds a trigger instruction when the image is used as the base for another build.
o STOPSIGNAL - sets the system call signal that will be sent to the container to exit.
o LABEL - apply key/value metadata to your images, containers, or daemons.
33
Useful Links and Examples
34
Dockerfile
☞
...
Successfully built 50a986f614d5
35
Tagging Images
36
ENTRYPOINT and CMD
...
ENTRYPOINT ["httpd"]
CMD ["-DFOREGROUND"]
➢ If you would like your container to run the same executable every time, then you should consider
using ENTRYPOINT in combination with CMD
https://docs.docker.com/engine/reference/builder/#entrypoint
https://docs.docker.com/engine/reference/builder/#cmd
http://www.johnzaccone.io/entrypoint-vs-cmd-back-to-basics/
37
ENTRYPOINT and CMD
FROM busybox
ENTRYPOINT ["ping"]
CMD ["-c1", "epam.com"]
38
Dockerfile Example: Custom Packer Image
FROM centos:7
ARG PACKER_VERSION=1.2.3
RUN yum install -y \
epel-release \
yum-plugin-ovl \
wget unzip \
rsync \
openssh openssh-clients && \
yum install -y python-pip && \
yum clean all
RUN wget -q https://releases.hashicorp.com/...${PACKER_VERSION}_linux_amd64.zip && \
unzip -q packer_${PACKER_VERSION}_linux_amd64.zip -d /bin/ && \
rm -f packer_${PACKER_VERSION}_linux_amd64.zip
RUN pip install -U ansible ansible-modules-hashivault
RUN useradd packer
USER packer
ENV USER packer
ENTRYPOINT ["/bin/packer"]
CMD ["--version"]
39
A Few More Examples
FROM openjdk:8-jre
ADD customer-contact-service.jar /
EXPOSE 4040
FROM openjdk:8
ENV PORT=8080
EXPOSE 8080
COPY wiremock /wiremock
ENTRYPOINT ["/wiremock/bin/startServer.sh"]
40
Docker Base Images
o scratch – this is the ultimate base image and it has 0 files and 0 size.
o busybox – a minimal Unix weighing in at 2.5 MB and around 10000 files.
o debian:jessie – the latest Debian is 122 MB and around 18000 files.
o alpine:latest – Alpine Linux, only 8 MB in size and has access to a package repository
41
Building With Arguments
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
...
ARG BUILD_NUMBER
ARG JOB_NAME
LABEL build_number="${BUILD_NUMBER}"
LABEL job_name="${JOB_NAME}"
# docker build \
--build-arg BUILD_NUMBER=${BUILD_NUMBER} \
--build-arg JOB_NAME=${JOB_NAME} \
.
42
Using ARG Directive
...
https://github.com/jenkinsci/docker/blob/master/Dockerfile
43
Multistage Build
FROM openjdk:8-jre
COPY --from=builder /build/target/demoapp.jar /opt/
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/opt/demoapp.jar"]
sbeliakou/springboot_example
44
Try Out Online Test
https://vitalflux.com/docker-certification-practice-questions-dockerfile/
https://djitz.com/certification/docker-certified-associate-test-review-questions-set-1-image-creation/
45
Running Containers
o Running in detached mode
o Exposing Ports
o Managing Restart Policy
o Changing Workspace Directory
o Changing Runtime User
o Providing Environment Variables
o Providing Labels
Working with Containers
Lifecycle:
o docker create - creates a container but does not start it.
o docker rename - allows the container to be renamed.
o docker run - creates and starts a container in one operation.
o docker rm - deletes a container.
o docker update - updates a container's resource limits.
47
Working with Containers
Info:
o docker ps - shows running containers.
o docker logs - gets logs from container.
o docker inspect - looks at all the info on a container.
o docker events - gets events from container.
o docker port - shows public facing port of container.
o docker top - shows running processes in container.
o docker stats - shows containers' resource usage statistics.
o docker diff - shows changed files in the container's FS.
Import / Export:
o docker cp - copies files or folders between a container and the local filesystem.
o docker export - turns container filesystem into tarball archive stream to STDOUT.
Executing Commands:
o docker exec - to execute a command in container.
48
docker run reference
https://docs.docker.com/engine/reference/run/
49
Running the Container
# docker run 50a986f614d5 # myhttpd:1.0
^C
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9f761335efe2 50a986f614d5 "/bin/sh -c 'httpd -…" 5 seconds ago Up 4 seconds 80/tcp trusting_kilby
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
74954ff14ec5 myhttpd:1.0 "/bin/sh -c 'httpd -…" 9 seconds ago Up 18 seconds 0.0.0.0:32768->80/tcp fervent_noyce
9f761335efe2 myhttpd:1.0 "/bin/sh -c 'httpd -…" 3 minutes ago Up 3 minutes 80/tcp trusting_kilby
50
Running the Container: Restart Policy
# docker run -d --restart=always --name sleeper centos sleep 5
6c3d24b3f89f13de92e710fcbb5b343b4cb81e7454ecc79445e94f0c5ba31a49
Policy Result
no Do not a utomatically res tart the container when it exi ts. This is the default.
Res tart only i f the container exits wi th a non-zero exit s tatus. Optionally, l imit the number of restart retri es the Docker daemon
on-failure[:max-retries] a ttempts .
Al wa ys restart the container regardless of the exit s tatus. When you s pecify a lways, the Docker da emon will try to res tart th e
always contai ner i ndefinitely. The container wi ll also always s tart on daemon s tartup, regardless of the current state of the contai ner.
Al wa ys restart the container regardless of the exit s tatus, including on daemon s tartup, except i f the container wa s put into a
unless-stopped s topped s tate before the Docker daemon was s topped.
More details
51
Running Containers in Interactive Mode
[root@localhost ~]# docker run centos cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
52
Executing Commands Inside Running Container
# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9626a94669c9 centos "sleep infinity" 11 seconds ago Up 10 seconds priceless_einstein
53
Stopping/Deleting Containers
# docker ps --format "table {{.Image}}\t{{.Names}}\t{{.ID}}\t{{.RunningFor}}\t{{.Status}}"
IMAGE NAMES CONTAINER ID CREATED STATUS
myhttpd:1.0 h8081 fca7f4525bc6 About an hour ago Up About an hour
myhttpd:1.0 h8082 014e5efa5ca9 About an hour ago Up About an hour
# docker rm 014e5efa5ca9
014e5efa5ca9
54
Changing Container's Build Defaults
...
RUN useradd jenkins -u 1000
USER jenkins
1. Default User
# docker run jenkins id
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins)
...
2. Default Workdir WORKDIR /
55
Changing Container's Build Defaults
3. Default Entrypoint ...
# docker run -it myhttpd:1.0 bash ENTRYPOINT ["httpd"]
# echo $? ...
1
4. Environment Variables
# docker run -it -e MYVAR="My Variable" centos env | grep MYVAR
MYVAR=My Variable
# docker run -it --env-file <(env| grep ARM | cut -f1 -d=) centos env | grep ARM
ARM_SUBSCRIPTION_ID=64a3f30f-xxx-xxxx-xxxx-yyyyfe63fe9a
ARM_TENANT_ID=bd5c6713-xxx-yyyy-xxxx-78f2d078e543
ARM_CLIENT_SECRET=xxxxxxxxxxxxxxxx
ARM_CLIENT_ID=808f38ed-xxxx-xxxx-yyyy-ebbce91bcfee
56
Running Containers with Labels
57
Mounting Data from the Host
o Mounting Folders
o Mounting Files
o R/O Mounts
o Mounting TMPFS
Use Cases
Examples:
Run httpd service with custom "index.html"
# ls -l ./
total 40
-rw-r--r-- 1 sbeliakou wheel 119 July 25 22:05 index.html
60
Use Cases
https://hub.docker.com/_/jenkins/
# docker run -p 8080:8080 -p 50000:50000 -v /your/dir:/var/jenkins_home jenkins
61
Mounting tmpfs
A tmpfs mount is temporary, and only persisted in the host memory. When the container stops, the tmpfs
mount is removed, and files written there won’t be persisted.
# docker run -d \
--mount type=tmpfs,destination=/app \
nginx:latest
# docker run -d \
--tmpfs /app \
nginx:latest
https://docs.docker.com/storage/tmpfs/#use-a-tmpfs-mount-in-a-container
62
to be continued
Thank you for your attention!