Solid container

From Hall A Wiki
Revision as of 16:30, 28 January 2020 by Zwzhao (Talk | contribs)

Jump to: navigation, search

introduction

'This page is outdated, see new info at https://github.com/JeffersonLab/solid_release/blob/master/howto'

jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.

SoLID docker images are made to run and develope SoLID software to maintain consistent result without need for installation.

The images are at docker store under account jlabsolid https://store.docker.com/profiles/jlabsolid

They are in 3 layers:

1. jlabsolidbase include a centos base images with some necessary rpms; 
2. jlabsolidapps add various applications above jlabsolidbase; 
3. jlabsolid add implementation of SoLID simulation,reconstruction,analysis above jlabsolidapps to complete solid software release

Read Note_about_container for any general things using container

for SoLID software user

get jlabsolid image
sudo docker pull jlabsolid/jlabsolid:test
location on ifarm, /group/solid/apps/jlabsolid-test.simg
location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolid-test.simg 
(jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")
load with singularity on ifarm and farm
ssh -X ifarm
module load singularity 
echo $DISPLAY                  (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
cd your_work_dir
singularity shell jlabsolid-test.simg 
(you are running as yourself in container, not as root)
(your_work_dir at host are mounted in container)
inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actual port with what you have)
inside container "xterm"   (test you can pass X11 application from inside container
load with singularity on linux host
cd your_work_dir
singularity shell jlabsolid-test.simg 
(you are running as yourself in container, not as root)
(your_work_dir at host are mounted in container)
load with docker on linux host by using host's Xwindows
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:$HOME --ipc=host jlabsolid/jlabsolid:test
(you are running as root in container)
(your home dir is mounted in container)
load with docker on linux host by using container's Xwindows
install x11docker
   wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
   sudo bash /tmp/x11docker --update
   rm /tmp/x11docker
install xorg-x11-server-Xephyr by yum or xserver-xephyr by apt-get
x11docker --desktop --homedir ${HOME} jlabsolid/jlabsolid:test fluxbox  
right click on the fluxbox window and open xterm
(you are running as yourself in container, not as root)
(your home dir dir is mounted in container)
run solid_gemc inside container
env -i tcsh
source /jlabsolidapps/JLAB_ROOT/setup
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC /solid_svn/solid_gemc2
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0                   (batch mode)
solid_gemc solid_SIDIS_He3_full.gcard                              (graphic mode)
exit or ctrl-d  (leave container)
How to use container to run jlab farm job
examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6

for SoLID software developer

checkout solid repo on host and load container 
cd your_work_dir
svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (just do it at the first time or run svn up to update it)
cd solid_svn
setenv solid_repo_FULL_PATH $PWD
load with docker
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host jlabsolid/jlabsolidapps:test 
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host eictool/jlab_software_2.2 (to run jlab_version=2.2)
or load with singularity
singularity shell --bind ${PWD}:/solid_repo jlabsolidapps-test.simg            (download http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg to run jlab_version=1.3)
singularity shell --bind ${PWD}:/solid_repo eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg   (download http://webhome.phy.duke.edu/~zz81/simg/eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg to run jlab_version=2.2)
compile and run solid_gemc inside container 
tcsh
set prompt = '[%n@%m #Container# %c]$ '     (do this only for singularity  to make it clear that you are inside container)
source /solid_repo/set_solid 1.3c       (use 1.3c or 2.2c for special setup inside container)
cd $SoLID_GEMC/source/$GEMC_VERSION    (just do it at the first time)
scons OPT=1 -j4                        (just do it at the first time)
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0       (batch mode)
solid_gemc solid_SIDIS_He3_full.gcard                  (graphic mode)
exit or ctrl-d  (leave container)

for SoLID container developer

work flow

at your local machine with docker installed,
get the right jlabsolidapps image, 
then add applications and publish next jlabsolidapps image, 
then add solid simulation,reconstruction,analysis code and publish next jlabsolid image
at your local machine use singularity to pull the docker images
The test images are made this way 
jlabsolidbase:test is auto built on docker cloud from this Dockerfile
jlabsolidapps:test is based on jlabsolidbase:test and build manually by adding jlab framework 1.3 [1]
jlabsolid:test is based on jlabsolidapps:test and contained solid svn checkout with compiled solid_gemc
get jlabsolidapps image
sudo docker pull jlabsolid/jlabsolidapps:test
location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg 
(jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")
load jlabsolidapps with singularity or docker
You should checkout svn on host, then load it via "--bind  $solid_repo_FULL_PATH:/solid_svn" for singularity, "-v $solid_repo_FULL_PATH:/solid_svn" for docker, "--sharedir $solid_repo_FULL_PATH" for x11docker

example

use jlabsolidapps to test application and publish next jlabsolidapps

use jlabsolidapps to test solid code on host, then push change to solid code repo, then and update /solid_svn, publish next jlabsolid

[user@localhost ~]$ cd $HOME
[user@localhost ~]$ svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn
[user@localhost ~]$ setenv solid_repo_FULL_PATH ${PWD}/solid_svn
[user@localhost ~]$ sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $solid_repo_FULL_PATH:/solid_svn --ipc=host jlabsolid/jlabsolidapps:test
[root@f18d5b96626c /]# tcsh
[root@f18d5b96626c /]# (install application or test solid code)
[root@f18d5b96626c /]# exit
[user@localhost ~]$ sudo docker commit f18d5b96626c
sha256:d55987e37b16b9a17e0795df19b2d6dbe368e0d24891e497805f79a1d733ec80
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
<none>                    <none>              d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidapps   test                a1b97160252c        11 hours ago        5.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker tag d55987e37b16 jlabsolid/jlabsolidapps:test
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
jlabsolid/jlabsolidapps   test                d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker login  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo docker push jlabsolid/jlabsolidapps:test  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo singularity pull docker://jlabsolid/jlabsolidapps:test
[user@localhost ~]$ md5sum jlabsolidapps-test.simg
e6f21d45b0f58f52631a5c46fa3b4c9d jlabsolidapps-test.simg
compile solid_gemc inside container 
cd $SoLID_GEMC/source/$GEMC_VERSION/ 
scons OPT=1

old

(testing as 2017/12)

jlab ifarm and farm support singularity container. A singularity container is made for the framework This could be a standard way to run simulation on any machine for consistent result without need for installation.

Read http://singularity.lbl.gov/faq to understand container and singularity and Refer to Note_about_singularity

singularity image
The image is built with singularity-2.3.1, centos7.3 with latest rpm on 20170811, and jlab_version_1.3 framework
location on ifarm for use, /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
location on web, http://www.phy.duke.edu/~zz81/simg/singularity_jlab_version_1.3_centos7_20170811.img.tgz, download it and do "tar zxf" to extract img file
How to use it on ifarm and farm with official svn checkout
singularity shell --bind /group/solid/solid_svn:/group/solid/solid_svn /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
source /apps/jlab_root/setup 
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC /group/solid/solid_svn/solid_gemc2
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard            (graphic mode, one windows has control and the other shows the detector)
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
exit   (leave container)
How to use it on any machine with your svn checkout
cd (your_choice_of_solid_repo_FULL_PATH)
svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (checkout the repo)
singularity shell --bind (your_choice_of_solid_repo_FULL_PATH):(your_choice_of_solid_repo_FULL_PATH) container_image_location/singularity_jlab_version_1.3_centos7_20170811.img (enter container. you home dir and your login script are shared between the host and container, sometimes you current dir will also)
source /apps/jlab_root/setup 
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC (your_choice_of_solid_repo_FULL_PATH)/solid_svn/solid_gemc2
cd $SoLID_GEMC/source/$GEMC_VERSION    (enter the source dir for solid_gemc)
scons OPT=1 -j4                        (compile solid_gemc) 
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard            (graphic mode, one windows has control and the other shows the detector)
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
exit   (leave container)
How to use it to run jlab farm job
examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6