Revision as of 13:29, 19 March 2018

(testing as 2018/03)

introduction

jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.

SoLID docker images are made to run and develope SoLID software to maintain consistent result without need for installation.

The images are at docker store under account jlabsolid https://store.docker.com/profiles/jlabsolid

They are in 3 layers:

1. jlabsolidbase include a centos base images with some necessary rpms; 
2. jlabsolidapps add various applications above jlabsolidbase; 
3. jlabsolid add implementation of SoLID simulation,reconstruction,analysis above jlabsolidapps to complete solid software release

Read Note_about_container for any general things using container

for SoLID software user

get jlabsolid image
sudo docker pull jlabsolid/jlabsolid:test
location on ifarm, /group/solid/apps/jlabsolid-test.simg
location on web, http://www.phy.duke.edu/~zz81/package/jlabsolid-test.simg 
(jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")

load jlabsolid with singularity anywhere
module load singularity-2.4.2 (only need this on ifarm)
cd your_work_dir
singularity shell jlabsolid-test.simg 
(you are running as yourself in container, not as root)
(your_work_dir at host are mounted in container)

load jlabsolid with docker on your local machine by using your local machine's Xwindows
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:/home/${USER} --ipc=host jlabsolid/jlabsolid:test
(you are running as root in container)
(your home dir is mounted in container)

load it with docker on your local machine by container's Xwindows
install x11docker
   wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
   sudo bash /tmp/x11docker --update
   rm /tmp/x11docker
install xorg-x11-server-Xephyr by yum
x11docker --desktop --homedir ${HOME} jlabsolid/jlabsolid:test fluxbox  
right click on the fluxbox window and open xterm
(you are running as yourself in container, not as root)
(your home dir dir is mounted in container)

run solid_gemc inside container 
env -i tcsh
source /jlabsolidapps/JLAB_ROOT/setup
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC /solid_svn/solid_gemc2
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0                    (batch mode)
(not working on ifarm yet) solid_gemc solid_SIDIS_He3_full.gcard    (graphic mode)
exit or ctrl-c  (leave container)

How to use container to run jlab farm job
examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6

for SoLID software developer

work flow

at your local machine with docker installed,
get the right jlabsolidapps image, 
then add applications and publish next jlabsolidapps image, 
then add solid simulation,reconstruction,analysis code and publish next jlabsolid image
at your local machine use singularity to pull the docker images

The test images are made this way 
jlabsolidbase:test is auto built on docker cloud from this Dockerfile
jlabsolidapps:test is based on jlabsolidbase:test and build manually by adding jlab framework 1.3 [1]
jlabsolid:test is based on jlabsolidapps:test and contained solid svn checkout with compiled solid_gemc

get jlabsolidapps image
sudo docker pull jlabsolid/jlabsolidapps:test
location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
location on web, http://www.phy.duke.edu/~zz81/package/jlabsolidapps-test.simg 
(jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")

load jlabsolidapps with singularity or docker
You should checkout svn on host, then load it via "--bind  $solid_repo_FULL_PATH:/solid_svn" for singularity, "-v $solid_repo_FULL_PATH:/solid_svn" for docker, "--sharedir $solid_repo_FULL_PATH" for x11docker

example

use jlabsolidapps to test application and publish next jlabsolidapps

use jlabsolidapps to test solid code on host, then push change to solid code repo, then and update /solid_svn, publish next jlabsolid

[user@localhost ~]$ cd $HOME
[user@localhost ~]$ svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn
[user@localhost ~]$ setenv solid_repo_FULL_PATH ${PWD}/solid_svn
[user@localhost ~]$ sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $solid_repo_FULL_PATH:/solid_svn --ipc=host jlabsolid/jlabsolidapps:test
[root@f18d5b96626c /]# tcsh
[root@f18d5b96626c /]# (install application or test solid code)
[root@f18d5b96626c /]# exit
[user@localhost ~]$ sudo docker commit f18d5b96626c
sha256:d55987e37b16b9a17e0795df19b2d6dbe368e0d24891e497805f79a1d733ec80
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
<none>                    <none>              d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidapps   test                a1b97160252c        11 hours ago        5.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker tag d55987e37b16 jlabsolid/jlabsolidapps:test
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
jlabsolid/jlabsolidapps   test                d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker login  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo docker push jlabsolid/jlabsolidapps:test  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo singularity pull docker://jlabsolid/jlabsolidapps:test
[user@localhost ~]$ md5sum jlabsolidapps-test.simg
e6f21d45b0f58f52631a5c46fa3b4c9d jlabsolidapps-test.simg

compile solid_gemc inside container 
cd $SoLID_GEMC/source/$GEMC_VERSION/ 
scons OPT=1

questions

1. we have to use singularity because farm, supporting docker to help run at mac and windows. Is it worth it? please provide feedback for using docker on mac and windows

2. Is docker layers really safe for reprehensibility?

3. there are many application, using one container for all make it simple to use, but need more careful work to develop. Another way is one application one container. Is it better?