How To Set Up a Private Docker Registry on Ubuntu 14.04
Introduction
Docker is a great tool for deploying your servers. While docker.io lets you upload your Docker creations to their registry for free, anything you upload is also public. This probably isn't what you want for a non-open source-project.
This guide will show you how to set up and secure your own private Docker registry. By the end of this tutorial you will be able to push a custom Docker image to your private registry, and pull the image securely from a different host.
This tutorial doesn't cover containerizing your own application, but only how to create the registry where you can store your deployments. If you want to learn how to get started with Docker itself (as opposed to the registry), you may want to read the tutorial here.
This tutorial has been tested with all servers (one registry and one client) running Ubuntu 14.04, but may work with other Debian-based distros.
Docker Concepts
If you haven't used Docker before then it's worth taking a minute to go through a few of Docker's key concepts. If you're already using Docker and just want to know how to get started running your own registry, then please skip ahead to the next section.
For a refresher on how to use Docker, take a look at the excellent Docker Cheat Sheet here.
Docker at it's core is a way to separate an application and the dependencies needed to run it from the operating system itself. To make this possible Docker uses containers and images. A Docker image is basically a template for a filesystem. When you run a Docker image with thedocker run command, an instance of this filesystem is made live, and runs on your system inside a Docker container. By default this container can't touch the original image itself, or the filesystem of the host where docker is running. It's a self-contained environment.
Whatever changes you make in the container are preserved in that container itself, and don't affect the original image. If you decide you want to keep those changes, then you can "commit" a container to a Docker image (via the docker commit command). This means you can then spawn new containers that start with the contents of your old container, without affecting the original container (or image). If you're familiar with git then the workflow should seem quite similar: you can create new branches (images in Docker parlance) from any container. Running an image is a bit like doing a git checkout.
To continue the analogy, running a private Docker registry is like running a private Git repository for your Docker images.
Step One — Install Prerequisites
You should create a user with sudo access on the registry server (and on the clients when you get that far).
The Docker registry is a Python application, so to get it up and running we need to install the Python development utilities and a few libraries:
sudo apt-get update
sudo apt-get -y install build-essential python-dev libevent-dev python-pip liblzma-dev
Step Two — Install and Configure Docker Registry
To install the latest stable release of the Docker registry (0.7.3 at the time of writing) we'll use Python's package management utility pip:
sudo pip install docker-registry
Docker-registry requires a configuration file.
pip by default installs this config file in a rather obscure location, which can differ depending how your system's Python is installed. So, to find the path, we'll attempt to run the registry and let it complain:
gunicorn --access-logfile - --debug -k gevent -b 0.0.0.0:5000 -w 1 docker_registry.wsgi:application
Since the config file isn't in the right place yet it will fail to start and spit out an error message that contains a FileNotFoundError that looks like this:
FileNotFoundError: Heads-up! File is missing: /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/config.yml
The registry includes a sample config file called config_sample.yml at the same path, so we can use the path it gave us to locate the sample file.
Copy the path from the error message (in this case /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/config.yml
), and remove the config.yml portion so we can change to that directory:
cd /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/
Now copy the config_sample.yml
file to config.yml
:
sudo cp config_sample.yml config.yml
Docker by default saves its data under the /tmp
directory, which can lead to unpleasantness since the /tmp
folder is cleared on reboot on many flavors of Linux. Let's create a more permanent folder to store our data:
sudo mkdir /var/docker-registry
Now we'll edit the config.yml
file to update any references to /tmp
to /var/docker-registry
. First look for a line near the top of the file that starts withsqlalchemy_index_database
:
sqlalchemy_index_database:
_env:SQLALCHEMY_INDEX_DATABASE:sqlite:tmp/docker-registry.db
Change it to point to /var/docker-registry
like so:
sqlalchemy_index_database:
_env:SQLALCHEMY_INDEX_DATABASE:sqlite:////var/docker-registry/docker-registry.db
Look a bit further down the file for the local:
section, and repeat the process, changing this:
local: &local
storage: local
storage_path: _env:STORAGE_PATH:/tmp/registry
To this:
local: &local
storage: local
storage_path: _env:STORAGE_PATH:/var/docker-registry/registry
The other default values in the sample config are fine, so no need to change anything there. Feel free to look through them. If you want to do something more complex like using external storage for your Docker data, this file is the place to set it up. That's outside the scope of this tutorial though, so you'll have to check the docker-registry documentation if you want to go that route.
Now that the config is in the right place let's try to test the server again:
gunicorn --access-logfile - --debug -k gevent -b 0.0.0.0:5000 -w 1 docker_registry.wsgi:application
You should see output that looks like this:
2014-07-27 07:12:24 [29344] [INFO] Starting gunicorn 18.0
2014-07-27 07:12:24 [29344] [INFO] Listening at: http://0.0.0.0:5000 (29344)
2014-07-27 07:12:24 [29344] [INFO] Using worker: gevent
2014-07-27 07:12:24 [29349] [INFO] Booting worker with pid: 29349
2014-07-27 07:12:24,807 DEBUG: Will return docker-registry.drivers.file.Storage
Great! Now we have a Docker registry running. Go ahead and kill it with Ctrl+C.
At this point the registry isn't that useful yet — it won't start unless you type in the above gunicorn command. Also, Docker registry doesn't come with any built-in authentication mechanism, so it's insecure and completely open to the public right now.
Step Three - Start Docker Registry as a Service
Let's set the registry to start on system startup by creating an Upstart script.
First let's create a directory for the log files to live in:
sudo mkdir -p /var/log/docker-registry
Then use your favorite text editor to create an Upstart script:
sudo nano /etc/init/docker-registry.conf
Add the following contents to create the Upstart script:
description "Docker Registry"
start on runlevel [2345]
stop on runlevel [016]
respawn
respawn limit 10 5
script
exec gunicorn --access-logfile /var/log/docker-registry/access.log --error-logfile /var/log/docker-registry/server.log -k gevent --max-requests 100 --graceful-timeout 3600 -t 3600 -b localhost:5000 -w 8 docker_registry.wsgi:application
end script
For more about Upstart scripts, please read this tutorial.
If you run:
sudo service docker-registry start
You should see something like this:
docker-registry start/running, process 25303
You can verify that the server is running by taking a look at the server.log file like so:
tail /var/log/docker-registry/server.log
If all is well you'll see text similar to the output from our previous gunicorn test above.
Now that the server's running in the background, let's move on to configuring Nginx so the registry is secure.
Step Four — Secure Your Docker Registry with Nginx
The first step is to set up authentication so that not just anybody can log into our server.
Let's install Nginx and the apache2-utils package (which allows us to easily create authentication files that Nginx can read).
sudo apt-get -y install nginx apache2-utils
Now it's time to create our Docker users.
Create the first user as follows:
sudo htpasswd -c /etc/nginx/docker-registry.htpasswd USERNAME
Create a new password for this user when prompted.
If you want to add more users in the future, just re-run the above command without the coption:
sudo htpasswd /etc/nginx/docker-registry.htpasswd USERNAME_2
At this point we have a docker-registry.htpasswd file with our users set up, and a Docker registry available. You can take a peek at the file at any point if you want to view your users (and remove users if you want to revoke access).
Next we need to tell Nginx to use that authentication file, and to forward requests to our Docker registry.
Let's create an Nginx configuration file. Create a new docker-registry file, entering yoursudo password if needed:
sudo nano /etc/nginx/sites-available/docker-registry
Add the following content. Comments are in-line. For more about Nginx virtual host configuration files, see this tutorial.
# For versions of Nginx > 1.3.9 that include chunked transfer encoding support
# Replace with appropriate values where necessary
upstream docker-registry {
server localhost:5000;
}
server {
listen 8080;
server_name my.docker.registry.com;
# ssl on;
# ssl_certificate /etc/ssl/certs/docker-registry;
# ssl_certificate_key /etc/ssl/private/docker-registry;
proxy_set_header Host $http_host; # required for Docker client sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client IP
client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads
# required to avoid HTTP 411: see Issue #1486 (https://github.com/dotcloud/docker/issues/1486)
chunked_transfer_encoding on;
location / {
# let Nginx know about our auth file
auth_basic "Restricted";
auth_basic_user_file docker-registry.htpasswd;
proxy_pass http://docker-registry;
}
location /_ping {
auth_basic off;
proxy_pass http://docker-registry;
}
location /v1/_ping {
auth_basic off;
proxy_pass http://docker-registry;
}
}
And link it up so that Nginx can use it:
sudo ln -s /etc/nginx/sites-available/docker-registry /etc/nginx/sites-enabled/docker-registry
Then restart Nginx to activate the virtual host configuration:
sudo service nginx restart
Let's make sure everything worked. Our Nginx server is listening on port 8080, while our original docker-registry server is listening on localhost port 5000.
We can use curl to see if everything is working:
curl localhost:5000
You should something like the following
"docker-registry server (dev) (v0.8.1)"
Great, so Docker is running. Now to check if Nginx worked:
curl localhost:8080
This time you'll get back the HTML of an unauthorized message:
<html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.4.6 (Ubuntu)</center>
</body>
</html>
It's worthwhile to run these two test commands from a remote machine as well, using the server's IP address instead of localhost, to verify that your ports are set up correctly.
In the Upstart config file we told docker-registry to listen only on localhost, which means it shouldn't be accessible from the outside on port 5000. Nginx, on the other hand, is listening on port 8080 on all interfaces, and should be accessible from the outside. If it isn't then you may need to adjust your firewall permissions.
Good, so authentication is up. Let's try to log in now with one of the usernames you created earlier:
curl USERNAME:PASSWORD@localhost:8080
If it worked correctly you should now see:
"docker-registry server (dev) (v0.8.1)"
Step Five — Set Up SSL
At this point we have the registry up and running behind Nginx with HTTP basic authentication working. However, the setup is still not very secure since the connections are unencrypted. You might have noticed the commented-out SSL lines in the Nginx config file we made earlier.
Let's enable them. First, open the Nginx configuration file for editing:
sudo nano /etc/nginx/sites-available/docker-registry
Use the arrow keys to move around and look for these lines:
server {
listen 8080;
server_name my.docker.registry.com;
# ssl on;
# ssl_certificate /etc/ssl/certs/docker-registry;
# ssl_certificate_key /etc/ssl/private/docker-registry;
Uncomment the SSL lines by removing the # symbols in front of them. If you have a domain name set up for your server, change the server_name to your domain name while you're at it. When you're done the file should look like this:
server {
listen 8080;
server_name yourdomain.com;
ssl on;
ssl_certificate /etc/ssl/certs/docker-registry;
ssl_certificate_key /etc/ssl/private/docker-registry;
Save the file. Nginx is now configured to use SSL and will look for the SSL certificate and key files at /etc/ssl/certs/docker-registry and /etc/ssl/private/docker-registryrespectively.
If you already have an SSL certificate set up or are planning to buy one, then you can just copy the certificate and key files to the paths listed above (ssl_certificate
andssl_certificate_key
).
You could also get a free signed SSL certificate.
Or, use a self-signed SSL certificate. Since Docker currently doesn't allow you to use self-signed SSL certificates this is a bit more complicated than usual, since we'll also have to set up our system to act as our own certificate signing authority.
Signing Your Own Certificate
First let's make a directory to store the new certificates and go there:
mkdir ~/certs
cd ~/certs
Generate a new root key:
openssl genrsa -out devdockerCA.key 2048
Generate a root certificate (enter whatever you'd like at the prompts):
openssl req -x509 -new -nodes -key devdockerCA.key -days 10000 -out devdockerCA.crt
Then generate a key for your server (this is the file we'll later copy to/etc/ssl/private/docker-registry
for Nginx to use):
openssl genrsa -out dev-docker-registry.com.key 2048
Now we have to make a certificate signing request.
After you type this command OpenSSL will prompt you to answer a few questions. Write whatever you'd like for the first few, but when OpenSSL prompts you to enter the "Common Name" make sure to type in the domain of your server.
openssl req -new -key dev-docker-registry.com.key -out dev-docker-registry.com.csr
For example, if your Docker registry is going to be running on the domainwww.ilovedocker.com, then your input should look like this:
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:www.ilovedocker.com Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Do not enter a challenge password. Then we need to sign the certificate request:
openssl x509 -req -in dev-docker-registry.com.csr -CA devdockerCA.crt -CAkey devdockerCA.key -CAcreateserial -out dev-docker-registry.com.crt -days 10000
Now that we've generated all the files we need for our certificate to work, we need to copy them to the correct places.
First copy the certificate and key to the paths where Nginx is expecting them to be:
sudo cp dev-docker-registry.com.crt /etc/ssl/certs/docker-registry
sudo cp dev-docker-registry.com.key /etc/ssl/private/docker-registry
Since the certificates we just generated aren't verified by any known certificate authority (e.g., VeriSign), we need to tell any clients that are going to be using this Docker registry that this is a legitimate certificate. Let's do this locally so that we can use Docker from the Docker registry server itself:
sudo mkdir /usr/local/share/ca-certificates/docker-dev-cert
sudo cp devdockerCA.crt /usr/local/share/ca-certificates/docker-dev-cert
sudo update-ca-certificates
You'll have to repeat this step for every machine that connects to this Docker registry!Otherwise you will get SSL errors and be unable to connect. These steps are shown in the client test section as well.
SSL Test
Let's restart Nginx to reload the configuration and SSL keys:
sudo service nginx restart
Do another curl test (only this time using https) to verify that our SSL setup is working properly. Keep in mind that for SSL to work correctly you will have to use the same domain name you typed into the Common Name field earlier while you were creating your SSL certificate.
curl https://USERNAME:PASSWORD@YOUR-DOMAIN:8080
For example, if the user and password you set up were nik and test, and your SSL certificate is for www.ilovedocker.com, then you would type the following:
curl https://nik:test@www.ilovedocker.com:8080
If all went well, you should see the familiar:
"docker-registry server (dev) (v0.8.1)"
If not, recheck the SSL steps and your Nginx configuration file to make sure everything is correct.
Now we have a Docker registry running behind an Nginx server which is providing authentication and encryption via SSL.
Step Six — Access Your Docker Registry from Another Machine
To access your Docker registry, first add the SSL certificate you created earlier to the new client machine. The file you want is located at ~/certs/devdockerCA.crt
. You can copy it to the new machine directly or use the below instructions to copy and paste it:
On the registry server, view the certificate:
cat ~/certs/devdockerCA.crt
You'll get output that looks something like this:
-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJANiXy7fHSPrmMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
aWRnaXRzIFB0eSBMdGQwHhcNMTQwOTIxMDYwODE2WhcNNDIwMjA2MDYwODE2WjBF
MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50
ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAuK4kNFaY3k/0RdKRK1XLj9+IrpR7WW5lrNaFB0OIiItHV9FjyuSWK2mj
ObR1IWJNrVSqWvfZ/CLGay6Lp9DJvBbpT68dhuS5xbVw3bs3ghB24TntDYhHMAc8
GWor/ZQTzjccHUd1SJxt5mGXalNHUharkLd8mv4fAb7Mh/7AFP32W4X+scPE2bVH
OJ1qH8ACo7pSVl1Ohcri6sMp01GoELyykpXu5azhuCnfXLRyuOvQb7llV5WyKhq+
SjcE3c2C+hCCC5g6IzRcMEg336Ktn5su+kK6c0hoD0PR/W0PtwgH4XlNdpVFqMST
vthEG+Hv6xVGGH+nTszN7F9ugVMxewIDAQABo1AwTjAdBgNVHQ4EFgQULek+WVyK
dJk3JIHoI4iVi0FPtdwwHwYDVR0jBBgwFoAULek+WVyKdJk3JIHoI4iVi0FPtdww
DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAkignESZcgr4dBmVZqDwh
YsrKeWSkj+5p9eW5hCHJ5Eg2X8oGTgItuLaLfyFWPS3MYWWMzggxgKMOQM+9o3+k
oH5sUmraNzI3TmAtkqd/8isXzBUV661BbSV0obAgF/ul5v3Tl5uBbCXObC+NUikM
O0C3fDmmeK799AM/hP5CTDehNaFXABGoVRMSlGYe8hZqap/Jm6AaKThV4g6n4F7M
u5wYtI9YDMsxeVW6OP9ZfvpGZW/n/88MSFjMlBjFfFsorfRd6P5WADhdfA6CBECG
LP83r7/MhqO06EOpsv4n2CJ3yoyqIr1L1+6C7Erl2em/jfOb/24y63dj/ATytt2H
6g==
-----END CERTIFICATE-----
Copy that output to your clipboard and connect to your client machine.
On the client server, create the certificate directory:
sudo mkdir /usr/local/share/ca-certificates/docker-dev-cert
Open the certificate file for editing:
nano /usr/local/share/ca-certificates/docker-dev-cert/devdockerCA.crt
Paste the certificate contents.
Verify that the file saved to the client machine correctly by viewing the file:
cat /usr/local/share/ca-certificates/docker-dev-cert/devdockerCA.crt
If everything worked properly you'll see the same text from earlier:
-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJANiXy7fHSPrmMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
...
...
LP83r7/MhqO06EOpsv4n2CJ3yoyqIr1L1+6C7Erl2em/jfOb/24y63dj/ATytt2H
6g==
-----END CERTIFICATE-----
Now update the certificates:
sudo update-ca-certificates
You should get output that looks like the following (note the "1 added
")
Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
If you don't have Docker installed on the client yet, do so now.
On most versions of Ubuntu you can quickly install a recent version of Docker by following the next few commands. If your client is on a different distro or you have issues then seeDocker's installation documentation for other ways to install Docker.
Add the repository key:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9;
Create a file to list the Docker repository:
sudo nano /etc/apt/sources.list.d/docker.list
Add the following line to the file:
deb https://get.docker.io/ubuntu docker main
Update your package lists:
sudo apt-get update
Install Docker:
sudo apt-get install -y --force-yes lxc-docker
To make working with Docker a little easier, let's add our current user to the Docker group and re-open a new shell:
sudo gpasswd -a ${USER} docker
sudo su -l $USER #(enter your password at the prompt if needed)
Restart Docker to make sure it reloads the system's CA certificates.
sudo service docker restart
You should now be able to log in to your Docker registry from the client machine:
docker login https://YOUR-HOSTNAME:8080
Note that you're using https:// and port 8080 here. Enter the username and password you set up earlier (enter whatever you'd like for email if prompted). You should see a Login Succeeded message.
At this point your Docker registry is up and running! Let's make a test image to push to the registry.
Step Seven — Publish to Your Docker Registry
On the client server, create a small empty image to push to our new registry.
docker run -t -i ubuntu /bin/bash
After it finishes downloading you'll be inside a Docker prompt. Let's make a quick change to the filesystem:
touch /SUCCESS
Exit out of the Docker container:
exit
Commit the change:
docker commit $(docker ps -lq) test-image
If you run docker images now, you'll see that you have a new test-image in the image list:
# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
test-image latest 1f3ce8008165 9 seconds ago 192.7 MB
ubuntu trusty ba5877dc9bec 11 days ago 192.7 MB
This image only exists locally right now, so let's push it to the new registry we've created.
First, log in to the registry with Docker. Note that you want to use https:// and port 8080:
docker login https://<YOUR-DOMAIN>:8080
Enter the username and password you set up earlier:
Username: USERNAME Password: PASSWORD Email:
Account created. Please see the documentation of the registry http://localhost:5000/v1/ for instructions how to activate it.
Docker has an unusual mechanism for specifying which registry to push to. You have to tag an image with the private registry's location in order to push to it. Let's tag our image to our private registry:
docker tag test-image YOUR-DOMAIN:8080/test-image
Note that you are using the local name of the image first, then the tag you want to add to it. The tag is not using https://, just the domain, port, and image name.
Now we can push that image to our registry. This time we're using the tag name only:
docker push <YOUR-DOMAIN>:8080/test-image
This will take a moment to upload to the registry server. You should see output that includesImage successfully pushed.
Step Eight - Pull from Your Docker Registry
To make sure everything worked let's go back to our original server (where you installed the Docker registry) and pull the image we just pushed from the client. You could also test this from a third server.
If Docker is not installed on your test pull server, go back and follow the installation instructions (and if it's a third server, the SSL instructions) from Step Six.
Log in with the username and password you set up previously.
docker login https://<YOUR-DOMAIN>:8080
And now pull the image. You want just the "tag" image name, which includes the domain name, port, and image name (but not https://):
docker pull <YOUR-DOMAIN>:8080/test-image
Docker will do some downloading and return you to the prompt. If you run the image on the new machine you'll see that the SUCCESS file we created earlier is there:
docker run -t -i <YOUR-DOMAIN>:8080/test-image /bin/bash
List your files:
ls
You should see the SUCCESS
file we created earlier:
SUCCESS bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
Congratulations! You've just used your own private Docker registry to push and pull your first Docker container! Happy Docker-ing!