docker

1. Using docker in pycharm(remote)

• The tutorial could work when deploying docker in your host and connect it with pycharm2019.2.6, just need to change the ip in pycharm with 127.0.0.1
• command to manage the process work on specified port

 # these two command could check if 8888 port is in using 
 ss -tuln | grep 8888
 netstat -tuln | grep 8888

# this command could find the PID of the process works on 8888 port
lsof -i :8888

# the command could show the mapping of your container
docker port your_container_name
# 22/tcp -> 0.0.0.0:8888
# 22/tcp -> [::]:8888

• reference links
  tutorial 1
  tutorial 2

• run docker

docker run --gpus all -dt --name container_name -p 8889:22 cb908459d986(or image:tag)

• set docker command

apt update
apt install -y openssh-server
echo 'root:daze0822' | chpasswd
sed -i '32s/.*/PermitRootLogin yes/' /etc/ssh/sshd_config
sed -i 's/UsePAM yes/UsePAM no/' /etc/ssh/sshd_config
service ssh restart

• sed command

# sed -i 'range s/object/replace/replace option' file_name
sed -i '1s/object/replace/' file_name # just work on 1st line
sed -i '5,10s/object/replace/' file_name # work on 5-10 lnes
sed -i 's/object/replace/' file_name # work on all the all lines
sed -i '1s/.*/replace/' file_name # replace the 1st line with replace
sed -i 's/.*/replace/' file_name # replace every line with reppace

• to run code in pycharm, it will not work if you just set env variables in docker container, you should set the variables in pycharm, use os.system('env') to show the valid env variables.
  

2. The commands usually used

• copy a file into docker running container(the folder is same)

docker cp /path/to/local/file.txt abc123:/app/data/
docker cp /path/to/local/ abc123:/app/data/

• copy a file inside the container to self system(the container must be
  in running)

docker cp 34ed4ec5f24c:/workspace/Chinese_English_Coreference_Resolution/main.py ./

• delete all the container in docker

docker rm -f $(docker ps -a -q)

• delete a docker image or multiple docker images

docker rmi ec17ff0d3890
docker rmi 724bec4bfb23 9c4ea34a7258 9df32d641ee1 da73ede767da

• continuely running container in background

docker run -dt --name my_container containerID 
docker run -it --name my_container containerID   (just run for a time)

• stop a running container

docker stop <container_id_or_name>

• enter a running container

docker exec -it <container_name_or_id> /bin/bash
# if the container is python, use 
docker attach <container_name_or_id>

• Add –gpus all to use GPU

docker run --gpus all -it --rm --name my_test1 cb908459d986(or image:tag)

3. Set up

• Add user into group

sudo usermod -aG docker username # replace username with the true username

• Check how many users in the docker group

grep /etc/group -e "docker"

• execute Dockerfile

docker build -t your-image-name .

run iamge

docker run -it your-image-name

• commit your container as a image

 docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
 # options
 # -a : the author
 # -c : Creating the image with Dockerfile
 # -m : the comment when committing
 # -p : stop the container temprarily when committing
 # you could also use docker commit --help to view the options
 # example
 # docker commit -a 'dazelu' -m 'restart the container' a813379e8448 liosam-kinetic-xenial:V2 

Using docker container’s python(link)
To connect python in container directly, the port should be exposed.

Connect docker with ssh in the same machine(link)

Run server’s docker container’s jupyter in local brower

ssh -f -N -L 8889:localhost:8888 ludaze@serverIP
# -L localport:localhost:serverport
#-f: This option requests SSH to go into the background just before it executes the command.
#-N: This option tells SSH not to execute any remote commands, which is useful when you only want to set up port forwarding

then run docker container, please note the port 8888 shoud be same as 8888 in ssh -f -N -L 8889:localhost:8888 ludaze@serverIP

docker run -id --name your_container_name -p 8888:8888 image[:tag]

enter docker and run

jupyter notebook --ip 0.0.0.0 --no-browser --allow-root

Rsource

• Pull from tensorflow（link）
• Pull from Pytorch（link）
• Pull from python (link)

Problem and solution

1. Q: Error response from daemon: pull access denied for 2.2.3-gpu, repository does not exist or may require ‘docker login’: denied: requested access to the resource is denied
   Pull completed

A: you should add text as image:tag format

2. Q: Cannot connect: com.intellij.docker.agent.ApiTaskException: COnnot connect to the DOcker daemon at unix:///var/run/docker.sock.ls the docker daemon running? (Details: [13] Permission denied)

3. Q: Cannot connect to remote host: net.schmizz.sshj.transport.TransportException: Connection reset

4. Q: The repository ‘https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease’ is not signed.
   resolution 1 resolution 2

​rm /etc/apt/sources.list.d/cuda.list
rm /etc/apt/sources.list.d/nvidia-ml.list
apt-key del 7fa2af80
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub