Revision as of 18:54, 17 May 2018

questions and comments should go to Zhiwen Zhao zwzhao at jlab.org

general

Users can run docker or singularity container on any machine with no installation needed and obtain consistent result

singularity can load docker image.

singularity

• general
  • website http://singularity.lbl.gov
  • hub https://singularity-hub.org
  • talk "Singularity: Containers for Science, Reproducibility, and HPC" https://youtu.be/DA87Ba2dpNM

• warning
  • if you are running singularity-2.4.x on a centos7.2 host like jlab ifarm and farm, you could have kernel panic if you use ROOT TMD5 and share your host dir

• installation
  • refer to installation instruction and the latest version is recommended.
  • On redhat-kind linux, build an rpm first and install it. at least "squashfs-tools,automake,rpm-build,libtool" are needed to build it.
  • On windows and Mac, the official way is just running a small linux virtual machine with virtualbox, but the virtual machine is not good for graphic application. The better alternative is to use a good but not so small linux virtual machine. If you have one already, use it. If not, see below to download an existing one.

• singularity at jlab ifarm and farm
  • jlab farm and ifarm has singularity installed at /usr/local/singularity, there could be several version installed
  • it will auto bind "/u /w /group /work /cache /volatile /scratch" into any container by default as controlled by "singularity.conf". for example, "/usr/local/singularity/singularity-someverson/etc/singularity/singularity.conf"
  • "module load singularity" or "module load singularity-versionnumber" to load a the latest or a particular version, stop using by "module rm singularity" or "module rm singularity-versionnumber"
  • echo $DISPLAY (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
  • inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actual port with what you have)
  • inside container "xterm" (test if you can pass X11 application from inside container through ssh to your local machine)
  • "cd some_where_with_enough_space" and "setenv SINGULARITY_CACHEDIR ./" to change cache dir from default ~/.singularity. MUST do at jlab ifarm with very limited space at home
  • "setenv http_proxy http://jprox.jlab.org:8082" and "setenv https_proxy http://jprox.jlab.org:8082" if you are on jlab ifarm to download image

• test singularity
  • singularity pull docker://godlovedc/lolcow
  • singularity run lolcow.simg
  • setenv PYTHONHTTPSVERIFY 0 (sometime needed to bypass singularity hub certificate check)
  • singularity pull shub://GodloveD/lolcow
  • singularity run GodloveD-lolcow-master-latest.simg

• note
  • when singularity pull docker image, it pull and build singularity image right away. It can fail sometimes because docker image are made as root user. you have to singularity pull with sudo then

docker

• at linux in general
  • use graphic in linux container with host's Xwindows like this "sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:$PWD --ipc=host container_name" and it share your current dir on host inside container
  • use graphic in linux container with container's Xwindows
    • install x11docker
      • wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
      • sudo bash /tmp/x11docker --update
      • rm /tmp/x11docker
      • install xorg-x11-server-Xephyr by yum or xserver-xephyr by apt-get
    • make sure the container image has fluxbox and xterm installed
    • x11docker --desktop --sharedir $PWD container_name fluxbox
    • right click on the fluxbox window and open xterm
  • if your host is within jlab network but not jlab level1 or level2 linux, mostly likely you will encounter error like "x509: certificate signed by unknown authority" because jlab gateway hijack site certificate with its own. We can solve this by mimic a jlab maintained system as follows
    • do "yum install jlabca", if it fixed the problem, ignore steps below
    • copy all files under /etc/pki/ca-trust/source/anchors/ from a jlab system like ifarm to your system
    • run "update-ca-trust","service docker stop","service docker start" as root
    • refer to https://github.com/moby/moby/issues/8849 for some discussion
  • images are at /var/lib/docker/overlay2
• at centos
  • install the new version something like 17.* from docker repo, not old 1.* from centos extras repo (docker version format changed in 2017/03 [1])
  • you have to run start docker daemon first [see how to start and auto start it https://docs.docker.com/config/daemon/systemd/], then run "sudo docker run hello-world" to test
• at windows
  • docker for windows requires latest window 10 pro 64bit and hyper-V on relative new CPU. docker will enable hyper-V which is disabled by default. Then your virtualbox and maybe vmware will stop working by crashing your windows, refer to https://docs.docker.com/docker-for-windows/install/#what-to-know-before-you-install
  • to use a linux container with xwindows application like xterm, you need to have a xwindow server installed, you can install Xming or VcXsrv, then run XLaunch with display as 0 and "No access control" checked. Finally run "docker run -it -e DISPLAY=host_ip:0.0 container_name". you may find host_ip by "ifconfig.exe"
  • images as one file at "C:\Users\Public\Documents\Hyper-V\Virtual hard disks\"
• at Mac
  • need OSX Yosemite 10.10.3 or above
  • virtualbox can work on the same machine where docker is installed
  • to use a linux container with xwindows application like xterm, you need to have XQuartz installed and enable x11 access by "xhost +host_ip" and run "docker run -it -e DISPLAY=host_ip:0 container_name" . you may find host_ip by "ifconfig en0 | grep inet | awk '$1=="inet" {print $2}'"
• release space by https://stackoverflow.com/questions/44288901/how-to-force-docker-to-release-storage-space-after-manual-delete-of-file-in-volu

a good linux virtual machine to run singularity and docker

• a good linux virtual machine to run singularity and docker, centos7 made with virtualbox
  • download and install virtualbox https://www.virtualbox.org/wiki/Downloads
  • It's nice, but not a must, to run singularity image in shared folder because it keeps the virtual machine size small. Vmware's shared folder doesn't work for this, but virtualbox does
  • download the machine at http://webhome.phy.duke.edu/~zz81/vm/CentOS7_20180303.ova (a centos7 64bit with latest update on 20180303 and singularity-2.4.2 and latest docker installed)
  • import the virtual machine into virtualbox refer to [https://askubuntu.com/questions/588426/how-to-export-and-import-virtualbox-vm-images
  • boot up the linux virtual machine and login with user name "user" without password, just you know root password is "111111"
  • use the installed singularity and docker, or update docker by yum or singularity by compile from source if you want
  • set up a shared folder with name "share" in the machine setting and put any singularity images into the shared folder on host
  • mount the shared folder "sudo mount -t vboxsf -o uid=$uid,gid=$gid share share" and use /home/user/share" as your working dir