Useful Docker tips and tricks
Some handy Docker tips in this post, check it out
By: Ajdin Imsirovic 21 February 2021
This article lists some experiences from learning and working with the Docker platform.
Image by Kinsey on Unsplash
1. Resolving error: Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running?
Due to the fact that an app I was working on had a large number of containers running, and also due to the fact that these Docker containers were starting on boot up, this was significantly slowing my boot times and my machine’s general performance. Since the machine’s RAM was already maxed out, the only thing I could do was use
systemctl to prevent the auto-bootup of the said Docker containers. Which worked great, except when I wanted to boot them up manually, I got the error referenced in the above title:
Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running?
The solution was to simply run the following command:
systemctl start docker
Once I did that, my Docker containers obeyed all other commands, which makes perfect sense, because the above command is the one that actually starts the docker daemon.
2. What does
docker run do?
docker run command starts a container:
- we give it an image name out of which to make a container instance
- we give it a process to run (aka “the main process”)
When we run a container we can name it, or if we choose not to, Docker itself will assign a random name to a container.
Once the main process exits, the container is finished. Even if we have other processes started in the said container, when the main process finishes, the container finishes.
Here’s an example of a
docker run with the
ubuntu image and the
docker run ubuntu echo 'Hello'
The above command will output
Hello, than the main process will finish and so the container will finish.
3. What does
docker run --rm do?
When we want to just run something in a container, then delete it when the process is finished, we run the
docker run --rm command.
It’s a one-liner for the following workflow:
docker run <container-name>
- *the container does its work and the main process finishes`
docker rm <container-name>
docker run --rm command is handy because it’s a shortcut for the above process.
4. What does
docker ps do?
docker ps is one of the basic commands in Docker. It checks for running containers. The
ps command is short for “Process Status”. This command was borrowed in Docker from Linux operating system with the same meaning: “Process Status”.
5. Install Docker on Ubuntu 20.04
To install Docker on Ubuntu 20.04, you need to run the following commands:
sudo apt update sudo apt upgrade sudo apt install docker.io sudo usermod -aG docker awv docker --version sudo docker run hello world
If everything works as expected, you’ll get a nice Hello World message in the bash, as well as some explanations of all the steps that Docker took to run the Hello World image on your machine.
Here’s a sample output:
sudo docker run hello-world Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 0a04fdcd24d3: Pull complete Digest: sha256:a451b...d Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/
Now we can run
docker ps command to list running containers.
docker ps -a we can list all the containers.
6. Restart multiple containers on the command line easily using the
Here’s a quick use case. We have three docker containers running. We check their ids using:
docker ps -q
We get the following sample container ids:
abc123123123 def789789789 cde258258258
Now we’d like to restart all three. We could copy paste their id’s and type out a command like this:
docker restart abc123123123 def789789789 cde258258258
But we can do it better, using the
docker restart $(docker ps -q)
The end result is still the same, but the second approach is much better because it saves time and is more generalized (we’re skipping hard-coded values).
7. What’s a docker image?
A Docker image is a file which contains a “recipe” for building a container (a fully functional, self-contained application, which utilizes the host OS kernel).
When we run Docker image, it can become one or multiple instances of that container.
A docker image is:
- immutable (can never change)
- shareable (with other people, e.g on Docker hub)
To list docker images, use:
We get the output similar to this:
REPOSITORY TAG IMAGE ID CREATED SIZE ... ... ... ... ...
The dots above represent actual image data. Here’s what each column means:
- REPOSITORY: where the image came from
- TAG: version number
- IMAGE ID: internal docker id for this specific image
To refer to a specific image, use the combination:
Or use the image id.
The second approach is better because images don’t have to be named (but they must have the
8. How to run the bash app from a Docker image of containerized Ubuntu?
You can do it if the image is of an OS, like Ubuntu.
docker run command uses a specific image to build a container instance from that image:
docker run -ti <image> bash
-ti is an abbreviation for “terminal interactive”.
To exit this image’s bash, just type:
9. Format the output of the
docker ps command
To format the output of the
docker ps command, we use the
docker ps --format=<how-to-format>
For example, if we can replace the
<how-to-format> section with a variable. We can, for example, name that variable like this:
But how do we get the variable into bash?
We just write a simple shell script, like this:
export FORMAT="\nID\t\nIMAGE\t\nCOMMAND\t\nCREATED\t\nSTATUS\t\nPORTS\t\nNAMES\t\n" # this shell will only work on bash
We can name the shell script, for example,
Now we need to make this new script executable. Like this:
chmod +x reformat.sh
Finally, we also need to run this script, like this:
Alternatively, we could have just ran the code from the
reformat.sh script directly in the terminal, like this:
Now we have the
$FORMAT script available on our command line:
The above command returns:
Now we can go back to the first command in this section:
docker ps --format=$FORMAT
10. What happens in a container instance, stays in a container instance
If we spin up two container instances from the same image of, say an Ubuntu OS, and then add a new file to one of these Ubuntu containers, that file is going to only be present in that one Ubuntu OS instance. The file won’t be magically added to the image, nor will it be available in the other container instance.
11. The basic anatomy of docker commands
The way that we can write the docker commands in bash follows a simple pattern:
docker <option> <command> <arguments>
For a list of all commands we can use, just type
To see the detailed description of each command, type:
docker <command> --help
docker run --help
The above command will give detailed info on the Docker’s
To get an overview of what’s happening with Docker on our machine, we can run the
12. Get a list of the most recently stopped containers
The last exited container can be found with:
docker ps -l
The output from the above command returns the following columns:
STATUS column gives us the reason for exit in round brankets, for example
Exited (0), or
13. Make an image out of a container
We can do this using the
docker commit command.
docker run we make a new container from a docker image. Then we might make some changes inside the container instance. We can then commit those changes to a new image, using the
docker commit command.
This way, the immutability of docker images is preserved.
To make an image from the most recently stopped container, we need to get that container’s id, like this:
docker ps -l
Then, we’ll copy the
ID key’s value, for example:
Then we’ll run the
commit command with the above ID provided:
docker commit 1234567890ab
The above command will produce a huge SHA256 hash, which is a unique ID of our new image. To make this more human-readable, we can use tags, like this:
docker tag <the-entire-copy-pasted-sha256-string> <tag-name>
docker tag 123abc234bcd... my-own-image
Docker will also give each of our containers a custom name, listed in the NAMES column, such as:
fervent_sammet determined_ishizaka romantic_mendel amazing_zhukovsky etc...
Because committing and tagging docker images is commonplace, there’s a handy shortcut command that automates this process:
docker commit <name> <tag>
For example, let’s say we got this back from running
docker ps -l:
CONTAINER ID 12cd7701db78 IMAGE my-own-image COMMAND "/hello" CREATED 26 minutes ago STATUS Exited (0) 26 minutes ago PORTS NAMES fervent_sammet
Now we can commit the above container as a new image, like this:
docker commit fervent_sammet my-second-image
Now if we inspected the available images with
docker image, we’ll get the listing of the available images, with the first listed image being the most recently-added one; in our case, the
14. Find a Docker image on the Docker hub
You don’t even need to visit the website; everything can be done from the terminal, like this:
docker search ubuntu
The above command will return a list of images we can download. There are plenty of images there. We’ll just go with the first result, the
To actually download an image, we can do:
docker pull <image-name>
docker pull ubuntu
Here’s the output:
Using default tag: latest latest: Pulling from library/ubuntu 83ee3a23efb7: Pull complete db98fc6f11f0: Pull complete f611acd52c6c: Pull complete Digest: sha256:123abc123abc...cba321 Status: Downloaded newer image for ubuntu:latest docker.io/library/ubuntu:latest
15. What is a detached Docker container?
Detached mode is when a Docker container runs in the background of our terminal. It does not receive input or display output.
For example, we can run our newly downloaded ubuntu image in detached mode, as follows.
First, we’ll locate it using the
Then, we can use the ID to run it, for example, our
ubuntu image id is:
a6318a119b2a, and thus we’ll run:
docker run -d a6318a119b2a
Alternatively, we could use the human-friendly NAME parameter:
docker run -d ubuntu
Either way, we’ll get back that huge SHA string so that we can identify the running container.
Now we can see the most recent running container with
docker ps -l:
docker ps -l
As expected, we get back all the data about that most recently ran container, which, indeed, is the one we’ve just ran with the
-d flag, including the container name.
Now we can attach the currently detached container:
docker attach <container-name>
This way, I’ve gotten into the container that was previously detached.
Additionally, we can convert an attached container into a detached one, using a sequence of two keyboard shortcuts:
- CTRL + p
- CTRL + q
This way, we can exit the container *but the container is still running, detached**. This means that later on, we can attach one more time.
16. Set the maximum memory a container can use
To prevent from too much resources being used, we can run the
--memory flag, with options, for example, like this:
docker run --memory <max-allowed-memory> <image-name> <command>
A great feature is to assign a portion of the
cpu-shares, like this:
docker run --cpu-shares
…the above command is a value that takes into account all the running containers, then assigns the appropriate amount based on the cpu shares. This is more flexible, because if we have two containers and one doesn’t take up any resources, the other will then be able to take more resources because that’s how we specified it should work.
Otherwise, we can set the
--cpu-quota flag to, say 20%, no more, no less, and this will always be the hard-coded value that we’ll be using.
I like to think of it as seats on a bus. In the above scenarios, the former is like saying “spread to as many empty seats available”, and the latter is like saying, “whatever happens, only use your seat”.
17. How to run a process in a Docker container, then stop the running container
Here’s an example of running bash in our
docker run -d -ti ubuntu bash
-ti is an abbreviation for
We can then run the
docker ps command to get the info on the running container(s), then pick the container that was built using the
ubuntu image, and copy that container’s id. Once we have the id in the clipboard, we can run the following command to stop the currently running container:
docker stop a1b2c3d4e5a6b7
a1b2c3d4e5a6b7 SHA stub being the previously copied ID of the then-running container.
18. How to attach to a detached Docker container?
To see all the containers running (including detached ones), we run:
Then, to attach a specific container, we run:
docker attach <container-name>
19. How to add another process to a running container in Docker?
We use this command:
It’s useful for debugging and DB administration.
20. Giving a name to a running container, on-the-fly
If we want to specify a name to a container, at the time we run it, we can use the following command:
docker run --name <our-custom-name> -d <container-image>
For example, we can run our ubuntu image in a container instance we named
docker run --name testname -d ubuntu
21. Inspecting Docker logs to find data about errors in a container
Sometimes we can start a container but it doesn’t do what we expect. We can use the
docker logs command to inspect the logs of that container by passing the container NAME, like this:
docker logs <container-name>
An example of passing a custom name and a wrong command (so that we can inspect the logs):
docker run --name testname -d ubuntu bash -c "ech osomething"
In the above command, we’re running the
- with the custom name of
- detached (specified with
-d) so that it just runs in the background
Additionally, we’ve passed it the process to run,
bash, and we’re running this command:
ech osomething. The command was “announced” using the
Obviously, the above command is wrong: we typed
ech osomething but we should have typed
echo something. The above command won’t run, because the
ech command, contrary to the
echo command, doesn’t exist.
Let’s inspect what happend to make sure:
docker logs testname
ech command was the problem; it’s confirmed by the output:
bash: ech: command not found
Here’s an important note: we need to keep our docker logs slim. If there’s too much stuff being output to the logs, we can slow down docker, even so much that it’s simply unusable.
22. Stop a container with the
kill command moves the container to a stopped state.
Any stopped container can be removed with the
docker kill <container-name> docker rm <container-name>
docker run ubuntu
We get the container’s NAME, in our case it was lucid_newton, so:
docker kill lucid_newton
That stopped our container.
Now, if we inspect the most recent container that was stopped, using:
docker ps -l
… we’d get back that indeed, it was lucid_newton, plus the STATUS is: Exited (137) 26 seconds ago.
Now we can remove it:
docker rm lucid_newton
This will just output the affected container’s name:
That’s it, the container is removed, and the luci_newton name is now again freely available. Otherwise, if we wanted to use the same name sometimes in the future, we’d get a notification saying “that name is already in use”>
23. Make sure your container includes your dependencies (not downloads them at run time)
For example, if we’re using containerized Node.js, and we start running it, and it’s set up so that it downloads its dependecies on start, this can work fine, until…
For example, a library gets removed from the Node.js repositories, and then our container is broken, and can’t run.
24. The importance of naming Docker containers
We can have containers without NAME value specified.
We can also keep some important stuff in such containers.
Don’t do it!
Because, during container clean-up, it’s easier to erase a container that is “unlabelled”, and thus accidentally erase some work that you should’ve kept.
25. Using container ports
Container ports have to do with container networking.
Containers, by default, are set up initially so that they can’t access the internet.
We can group multiple containers into separate local networks, and we can specify exactly how they get connected to each other.
To connect to the internet, we can also explicitly expose a port (aka “publish” a port).
We can expose a port by setting up:
- the internal port that a containerized software is listening on
- the “outside” port that a container is listening on
- the protocol to use (optional)
Here’s an example of exposing the exact same port on the inside and the outside of a container:
We can expose more than one port, by simply specifying another one, like this:
-p 34567:34567 -p 34566:34566 -p 34565:34565
Above, we’ve just specified three inner ports matching three outter ports.
Here’s a command sending data between three different bash instances on the same machine. The first one is the “server”:
docker run --rm -ti -p 34567:34567 -p 34568:L34568 --name the-server ubuntu bash
If we ran
docker run --help, we’d find in the output instructions that:
--rmflag automatically removes the container when it exits,
-t : Allocate a pseudo-tty
-i : Keep STDIN open even if not attached
-p, as just described:
-p, --publish list : Publish a container's port(s) to the host
--name string : Assign a name to the container
Now we can run in the same bash instance, the netcat program, with the
nc -lp 34567 | nc -lp 34568
So we’re listening with one
nc instance on port 34567, and we’re piping that to another
nc instance on port 34568.
In the second bash instance, we’ll run
nc localhost 34567, and in the third, we’ll run
nc localhost 34568, so that, when we type a “message” in the second bash instance, we’ll get that “message” in the third bash instance. The “message” is just some text that we type and that gets passed through by the netcat utility.
To reiterate, we’ll have 3 separate bash instances open. The first one is a server, so we type:
docker run --rm -ti -p 34567:34567 -p 34568:34568 --name the-server ubuntu
the-server container’s bash will open:
nc -lp 34567 | nc -lp 34568
nc is not installed on the system, so:
bash: nc: command not found bash: nc: command not found
This is an opportunity to containerize an app (or a utility in our case).
To do that with our netcat, we’ll just run a docker container that has it installed.
26. How to expose ports dynamically?
The port inside the container is fixed.
The port on the host is chosen from the unused ports.
This allows many containers running programs with fixed ports.
This is often used with a service discovery program.
For example, I can specify only the port inside the container, and let docker choose the port on the outside:
docker run --rm -ti -p 34567 -p 34568 --name the-server ubuntu:14.04 bash
Now we’re inside the
the-server container, and we can run
nc -lp 34567 | nc -lp 34568
Now, in another bash instance, we can check for the
the-server container’s external port:
docker port the-server
Here’s a sample output:
34567/tcp -> 0.0.0.0:32777 34568/tcp -> 0.0.0.0:32776
Now in the other two bash instances, I can run the exernal ports that Docker picked dynamically:
nc localhost 32777
…and in the other instance:
nc localhost 32776
27. It’s enough to run the
docker rm command with the first 4 characters of the id
It’s as simple as, for example, this:
docker rm 76f9
This is great for a quick cleanup.
28. Containers shouldn’t directly refer to a container by an IP address
So how can containers “address” the container that is hosting them?
By using the container’s host name:
29. In Docker, can we use ports with udp protocol instead of the tcp protocol?
We specify them with a slash and the port on the end of the command:
docker run -p <outside-port>:<inside-port>/<protocol-tcp-or-udp)>
docker run -p 23456:23456/udp
Practically, in one bash instance, we’ll run this:
docker run --rm -ti 34567/udp --name the-server ubuntu:14.04 bash
And then in another container we check the port:
docker port the-server # returns, e.g: 34567/udp -> 0.0.0.0:32774
Back in the first one, once inside the container, we’ll run:
nc -ulp 34567
Back in the second:
Hello from second
The message from the second is now passed in to first:
nc -ulp 34567 Hello from second
30. Networking between containers and inspecting Docker networks defaults
There are a number of ways and options of how our Docker containers connect to one another.
For example, once we exposed a port of a container, this opens a network path from the outside of that computer to that container.
Other containers are able to connect to it by going out to the host and then back in along the said network path.
To inspect Docker networks, run:
docker network ls
This is a sample output:
NETWORK ID NAME DRIVER SCOPE abdcabc5fd59 bridge bridge local 1dca4d941c2f host host local f450fbf5a6c4 none null local
Bridge is the network used by containers that don’t specify a preference to put into any other network.
Host is when you want a container to have no isolation at all. This does have some security concerns.
And none is for when a container should have no networking.
31. Build a new Docker network
Here’s the command:
docker network create <network-name>, e.g:
docker network create my-network
To run a machine on our Docker network, we’ll run:
docker run --rm -ti --net my-network
We can also give our machine a name, by appending the
--name <name-we-choose> to the above command, like this:
docker run --rm -ti --net my-network --name my-server
Names are very useful when using private networks in Docker because different containers inside the network can refer to each other by those names, so it makes it very easy for them to find each other.
Now that we have the name, we can run the rest, namely
ubuntu:14.04 bash, so that the entire command now looks like this:
docker run --rm -ti --net my-network --name my-server ubuntu:14.04 bash
Now we can verify that we can send traffic to ourselves:
Indeed, we can, which is proven by the output we get:
PING my-server (172.18.0.2) 56(84) bytes of data. 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=1 ttl=64 time=0.064 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=2 ttl=64 time=0.039 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=3 ttl=64 time=0.064 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=4 ttl=64 time=0.063 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=5 ttl=64 time=0.062 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=6 ttl=64 time=0.063 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=7 ttl=64 time=0.064 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=8 ttl=64 time=0.063 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=9 ttl=64 time=0.036 ms 64 bytes from 31a3b94a4ee1 (172.18.0.2): icmp_seq=10 ttl=64 time=0.064 ms ^C --- my-server ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9224ms rtt min/avg/max/mdev = 0.036/0.058/0.064/0.011 ms
If we try to ping a non-existing server, nothing will happen. So let’ open another bash instance and build another custom server:
docker run --rm -ti --net my-network --name my-second-server ubuntu:14.04 bash
my-second-server on the same network so that
my-second-server can find each other.
Now in my second server, I can write:
nc -lp 1234
And now I can send a message to
hello from my-second-server
my-server, I can received the messages from
my-second-server, and I can send a message to
my-second-server 1234 hi from my-server
Our two containers communicate both ways, and are on the same network.
32. Communicating between Docker networks
I’m going to start up a new terminal here.
Let’s make another network.
docker network create catsonly
This is a network only for cats.
Okay, now let’s go ahead and put the cat machine onto the cat network.
docker network connect catsonly catserver
Okay, now, over here, let’s make things a little more interesting. I’ll start up another terminal, and I’m going to start the new member of our network.
docker run --rm -ti --net catsonly --name bobcatserver bash
It’s like a catserver, only moreso, running bash.
Ok, so from here, I can do
pink catserver, and get data. If I do
ping dogeserver, traffic is not allowed.
From the catserver in the middle, I can ping dogserver, hit CTRL + c, and it works.
Now over here, on the dogserver, we can still only make connections to catserver, but we cannot connect to babcatserver, because the dogserver is not on the catsonly network. There are many, many more options to this that give us fine-grain control over how things connect here. But what was covered here is about 90% of the user cases. The rest will be left up to experience.
33. What are Dockerfiles?
It’s just a file used to build a Docker image.
We run it with
docker build and the
-t tag for “tag”. The below command builds a new Docker image and tags it with the
<name-tag>, in the current folder.
docker build -t <name-tag> .