Note about container

questions and comments should go to Zhiwen Zhao zwzhao at jlab.org

general

Users can run docker or singularity container on any machine with no installation needed and obtain consistent result

singularity can load docker image.

singularity

• general
  • website http://singularity.lbl.gov
  • hub https://singularity-hub.org
  • talk "Singularity: Containers for Science, Reproducibility, and HPC" https://youtu.be/DA87Ba2dpNM

• warning
  • if you are running singularity-2.4.x on a centos7.2 host like jlab ifarm and farm, you could have kernel panic if you use ROOT TMD5 and share your host dir

• installation
  • refer to installation instruction and the latest version is recommend.
  • singularity-2.4.x can run singularity-2.3.x images and has many new features and we should use it instead of 2.3.x whenever possible
  • On redhat-kind linux, build an rpm first and install it. at least "squashfs-tools,automake,rpm-build,libtool" are needed to build it.
  • On windows and Mac, the official way just running a small linux virtual machine with virtualbox, but the virtual machine is not good for graphic application. The better alternative is to use a good but not so small linux virtual machine. If you have one already, use it. If not, see below to download an existing one.

• singularity at jlab ifarm and farm
  • jlab farm and ifarm has singularity installed at /usr/local/singularity, check what version are installed there, choose latest to run as follows
  • "module load singularity-2.5.1" as 2018/05/14 on ifarm1402
  • echo $DISPLAY (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
  • inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actaul port with what you have)
  • inside container "xterm " (test you can pass X11 application from inside container
  • it will auto bind "/u /w /group /work /cache /volatile /scratch" into any container by default as controlled by "singularity.conf". for example, "/usr/local/singularity/singularity-2.4.6/etc/singularity/singularity.conf"
  • There can be different version installed at /usr/local/singularity/. for example, to load a particular version by "module load singularity-2.4.6", or stop using it by "module rm singularity-2.4.6"

• test singularity
  • cd some_where_with_enough_space
  • "setenv http_proxy http://jprox.jlab.org:8082" "setenv https_proxy http://jprox.jlab.org:8082" if you are on jlab ifarm
  • setenv SINGULARITY_CACHEDIR ./ (change cache dir from default ~/.singularity, MUST do at jlab ifarm with very limited space at home)
  • singularity pull docker://godlovedc/lolcow
  • singularity run lolcow.img
  • setenv PYTHONHTTPSVERIFY 0 (sometime needed to bypass singularity hub certificate check)
  • singularity pull shub://GodloveD/lolcow
  • singularity run GodloveD-lolcow-master-latest.simg

• note
  • when singularity pull docker image, it pull and build singularity image right away. It can fail sometime because docker image are made as root user. you have to singularity pull with sudo

docker

• at linux in general
  • share your current dir in container with option "-v $PWD:$PWD"
  • use graphic in container with host's Xwindows like this "sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --ipc=host container_name"
  • use graphic in container with container's Xwindows
    • install x11docker
      • wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
      • sudo bash /tmp/x11docker --update
      • rm /tmp/x11docker
      • install xorg-x11-server-Xephyr by yum or xserver-xephyr by apt-get
    • make sure the container image has fluxbox and xterm installled
    • x11docker --desktop --sharedir $PWD container_name fluxbox
    • right click on the fluxbox window and open xterm
• at centos
  • install the new version something like 17.* from docker repo, not old 1.* from centos extras repo (docker version format changed in 2017/03 [1])
  • you have to run start docker daemon first [see how to start and auto start it https://docs.docker.com/config/daemon/systemd/], then run "sudo docker run hello-world" to test
  • if your host is within jlab network but not jlab level1 or level2 linux, mostly likely you will encounter error like "x509: certificate signed by unknown authority" because jlab gateway hijack site certificate with its own. We can solve this by mimic a jlab maintained system as follows
    • do "yum install jlabca", if it fixed the problem, ignore steps below
    • copy all files under /etc/pki/ca-trust/source/anchors/ from a jlab system like ifarm to your system
    • run "update-ca-trust","service docker stop","service docker start" as root
    • refer to https://github.com/moby/moby/issues/8849 for some discussion
  • images are at /var/lib/docker/overlay2
• at windows
  • docker for windows requires latest window 10 pro 64bit and hyper-V on relative new CPU. docker will enable hyper-V which is disabled by default. Then your virtualbox and vmware (maybe) will stop working by crashing your windows, refer to https://docs.docker.com/docker-for-windows/install/#what-to-know-before-you-install
  • images as one file at "C:\Users\Public\Documents\Hyper-V\Virtual hard disks\"
• at Mac
  • need OSX Yosemite 10.10.3 or above
  • virtualbox can work on the same machine where docker is installed
• release space by https://stackoverflow.com/questions/44288901/how-to-force-docker-to-release-storage-space-after-manual-delete-of-file-in-volu

a good linux virtual machine to run singularity and docker

• a good linux virtual machine to run singularity and docker, centos7 made with virtualbox
  • download and install virtualbox https://www.virtualbox.org/wiki/Downloads
  • It's nice, but not a must, to run singularity image in shared folder because it keeps the virtual machine size small. Vmware's shared folder doesn't work for this, but virtualbox does
  • download the machine at http://webhome.phy.duke.edu/~zz81/package/CentOS7_20180303.ova (a centos7 64bit with latest update on 20180303 and singularity-2.4.2 and latest docker installed)
  • import the virtual machine into virtualbox refer to [https://askubuntu.com/questions/588426/how-to-export-and-import-virtualbox-vm-images
  • boot up the linux virtual machine and login with user name "user" without password, just you know root password is "111111"
  • use the installed singularity and docker, or update docker by yum or singularity by compile from source if you want
  • set up a shared folder with name "share" in the machine setting and put any singularity images into the shared folder on host
  • mount the shared folder "sudo mount -t vboxsf -o uid=$uid,gid=$gid share share" and use /home/user/share" as your working dir