HPC High Performance Computing: 10.1. Create and upload a Docker image to the registry server

by writing the instructions from a Dockerfile, you can build your on images.

Introduction

The following steps are an example to create an own docker image on a local computer/server (not on login or calculation nodes) and upload to the Registry Server of Poblenou.

Before start building process you must know:

about the builder environment:

  • Docker engine and root access on your builder machine.
  • Shell scripting skills.
  • Docker command line knowledge.

about Registry Server:

  • You need a user to upload your image.
  • Access to the private network of Remote Server registry.sb.upf.edu.
  • Uploaded images can't be removed.
  • Uploaded images can be updated

1) Create a directory to build the image

$ mkdir ubuntu-18.04
$ cd ubuntu-18.04/

1.1) Write a dockerfile with the settings

$ vi Dockerfile

FROM       ubuntu:18.04
MAINTAINER John Brown "https://registry.sb.upf.edu"
LABEL authors="John Brown"
LABEL version="18.04"
LABEL description="Basic images with \
openssh-server nmap sudo telnet sssd installed."
RUN apt-get update
RUN apt-get install -y openssh-server nmap sudo telnet sssd
RUN mkdir /var/run/sshd
RUN echo 'root:xxxxxxxxxxxx' |chpasswd
RUN sed -ri 's/^#?PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
RUN mkdir /root/.ssh
RUN apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
EXPOSE 22
CMD    ["/usr/sbin/sshd", "-D"]

It is essential to add "LABEL authors" with your name, surname and email address.

2) Build the image in a local docker-host

$ docker build .

Sending build context to Docker daemon  2.048kB
Step 1/12 : FROM       ubuntu:18.04
18.04: Pulling from library/ubuntu
a48c500ed24e: Pull complete 
1e1de00ff7e1: Pull complete 
0330ca45a200: Pull complete 
471db38bcfbf: Pull complete 
0b4aba487617: Pull complete 
Digest: sha256:c8c275751219dadad8fa56b3ac41ca6cb22219ff117ca98fe82b42f24e1ba64e
Status: Downloaded newer image for ubuntu:18.04
 ---> 452a96d81c30
Step 2/12 : MAINTAINER John Brown "https://registry.sb.upf.edu"
 ---> Running in 2fd5f8e83c17
Removing intermediate container 2fd5f8e83c17
 ---> b984d50d740c
Step 3/12 : RUN apt-get update
 ---> Running in 9c84731ef4ed
...

3) List and check the image

$ docker images

REPOSITORY                    TAG                 IMAGE ID            CREATED              SIZE
...
<none>                        <none>              401153215cb6        25 seconds ago      335MB

4) Rename the image

Change the RESEARCH_GROUP for the acronym of the research group you belong.

$ docker tag <IMAGE_ID> registry.sb.upf.edu/<research_group_acronym>/ubuntu:18.04

$ docker tag 401153215cb6 registry.sb.upf.edu/mygroup/ubuntu:18.04

Note: Use lowercase in research_group_acronym

5) List and check re-tagged image

$ docker images

REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
...
registry.sb.upf.edu/mygroup/ubuntu   18.04               401153215cb6        About a minute ago   335MB

6) Provide credentials to establish a session with the corporate registry server

$ docker login registry.sb.upf.edu

Username (): jbrown
Password: 
Login Succeeded

7) Push the image

$ docker push registry.sb.upf.edu/mygroup/ubuntu:18.04

The push refers to repository [registry.sb.upf.edu/info/ubuntu]
68d59e996e14: Pushed 
b191e5e42292: Pushed 
dcb7bc3f07ca: Pushed 
ffc69fc3fb3e: Pushed 
401b721534a1: Pushed 
a1355d87070f: Pushed 
b337788c3f94: Pushed 
a4d6cb1434aa: Pushed 
059ad60bcacf: Pushed 
8db5f072feec: Pushed 
67885e448177: Pushed 
ec75999a0cb1: Pushed 
65bdd50ee76a: Pushed 
18.04: digest: sha256:54567b1dc80c7e463f9cada74da121dfd71590f0bf1256efc0f1a62e7dcf89e2 size: 3025

8. Check the image in a webbrowser

1. Go to https://registry.sb.upf.edu/

2. Login with your cluster credentials.

9. Convert Docker image to Singularity image

Finally, if we have uploaded a Docker image, it will be necessary to convert the file format (from OCI to SIF). You can do it directly from compute nodes:

$ cd /home/user/
$ singularity pull --docker-login docker://registry.sb.upf.edu/<research-group>/<image>
Use your resarch user as Docker credentials.