Difference between revisions of "Solid container"

From Hall A Wiki
Jump to: navigation, search
(→‎for SoLID software developer)
(Link to new wiki)
 
(28 intermediate revisions by one other user not shown)
Line 1: Line 1:
−
(testing as 2018/03)
+
This page has been moved to [https://solid.jlab.org/wiki/index.php/container https://solid.jlab.org/wiki/index.php/container] in the new [https://solid.jlab.org/wiki SoLID Wiki].  Please do not edit this page.
 +
 
  
 
= introduction =  
 
= introduction =  
 +
 +
''''''This page is outdated, see new info at https://github.com/JeffersonLab/solid_release''''''
  
 
jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.
 
jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.
Line 21: Line 24:
 
  sudo docker pull jlabsolid/jlabsolid:test
 
  sudo docker pull jlabsolid/jlabsolid:test
 
  location on ifarm, /group/solid/apps/jlabsolid-test.simg
 
  location on ifarm, /group/solid/apps/jlabsolid-test.simg
−
  location on web, http://www.phy.duke.edu/~zz81/package/jlabsolid-test.simg  
+
  location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolid-test.simg  
 
  (jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
 
  (jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
 
  with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")
 
  with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")
  
−
  '''load jlabsolid with singularity on ifarm and farm'''
+
  '''load with singularity on ifarm and farm'''
−
  ssh -X ifarm1402
+
  ssh -X ifarm
−
ls /usr/local/singularity      (find and load the latest singularity installed)
+
  module load singularity  
−
  module load singularity-2.5.1
+
 
  echo $DISPLAY                  (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
 
  echo $DISPLAY                  (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
 
  cd your_work_dir
 
  cd your_work_dir
Line 34: Line 36:
 
  (you are running as yourself in container, not as root)
 
  (you are running as yourself in container, not as root)
 
  (your_work_dir at host are mounted in container)
 
  (your_work_dir at host are mounted in container)
−
  inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actaul port with what you have)
+
  inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actual port with what you have)
−
  inside container "xterm "  (test you can pass X11 application from inside container
+
  inside container "xterm"  (test you can pass X11 application from inside container
  
−
  '''load jlabsolid with singularity on your machine'''
+
  '''load with singularity on linux host'''
 
  cd your_work_dir
 
  cd your_work_dir
 
  singularity shell jlabsolid-test.simg  
 
  singularity shell jlabsolid-test.simg  
Line 43: Line 45:
 
  (your_work_dir at host are mounted in container)
 
  (your_work_dir at host are mounted in container)
  
−
  '''load jlabsolid with docker on your local machine by using your local machine's Xwindows'''
+
  '''load with docker on linux host by using host's Xwindows'''
 
  sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:$HOME --ipc=host jlabsolid/jlabsolid:test
 
  sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:$HOME --ipc=host jlabsolid/jlabsolid:test
 
  (you are running as root in container)
 
  (you are running as root in container)
 
  (your home dir is mounted in container)
 
  (your home dir is mounted in container)
  
−
  '''load it with docker on your local machine by container's Xwindows'''
+
  '''load with docker on linux host by using container's Xwindows'''
 
  install [https://github.com/mviereck/x11docker x11docker]
 
  install [https://github.com/mviereck/x11docker x11docker]
 
     wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
 
     wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
Line 59: Line 61:
 
  (your home dir dir is mounted in container)
 
  (your home dir dir is mounted in container)
  
−
  '''run solid_gemc inside container'''  
+
  '''run solid_gemc inside container'''
 
  env -i tcsh
 
  env -i tcsh
 
  source /jlabsolidapps/JLAB_ROOT/setup
 
  source /jlabsolidapps/JLAB_ROOT/setup
Line 75: Line 77:
 
= for SoLID software developer =  
 
= for SoLID software developer =  
  
−
  '''checkout solid svn on host, load jlabsolid/jlabsolidapps:test'''  
+
  '''checkout solid repo on host and load container'''  
 
  cd your_work_dir
 
  cd your_work_dir
−
  svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (just do it at the first time)
+
  svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (just do it at the first time or run svn up to update it)
 
  cd solid_svn
 
  cd solid_svn
−
  sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_svn --ipc=host jlabsolid/jlabsolidapps:test
+
setenv solid_repo_FULL_PATH $PWD
−
 
+
load with docker
 +
  sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host jlabsolid/jlabsolidapps:test  
 +
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host eictool/jlab_software_2.2 (to run jlab_version=2.2)
 +
or load with singularity
 +
singularity shell --bind ${PWD}:/solid_repo jlabsolidapps-test.simg            (download http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg to run jlab_version=1.3)
 +
singularity shell --bind ${PWD}:/solid_repo eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg  (download http://webhome.phy.duke.edu/~zz81/simg/eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg to run jlab_version=2.2)
  
 
  '''compile and run solid_gemc inside container'''  
 
  '''compile and run solid_gemc inside container'''  
−
  env -i tcsh
+
  tcsh
−
  source /jlabsolidapps/JLAB_ROOT/setup
+
  set prompt = '[%n@%m #Container# %c]$ '    (do this only for singularity to make it clear that you are inside container)
−
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
+
  source /solid_repo/set_solid 1.3c      (use 1.3c or 2.2c for special setup inside container)
−
  setenv SoLID_GEMC /solid_svn/solid_gemc2
+
−
  setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
+
 
  cd $SoLID_GEMC/source/$GEMC_VERSION    (just do it at the first time)
 
  cd $SoLID_GEMC/source/$GEMC_VERSION    (just do it at the first time)
−
  scons OPT=1                           (just do it at the first time)
+
  scons OPT=1 -j4                        (just do it at the first time)
 
  cd $SoLID_GEMC/script
 
  cd $SoLID_GEMC/script
 
  solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0      (batch mode)
 
  solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0      (batch mode)
Line 113: Line 118:
 
  sudo docker pull jlabsolid/jlabsolidapps:test
 
  sudo docker pull jlabsolid/jlabsolidapps:test
 
  location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
 
  location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
−
  location on web, http://www.phy.duke.edu/~zz81/package/jlabsolidapps-test.simg  
+
  location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg  
 
  (jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")
 
  (jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")
  
Line 157: Line 162:
 
  cd $SoLID_GEMC/source/$GEMC_VERSION/  
 
  cd $SoLID_GEMC/source/$GEMC_VERSION/  
 
  scons OPT=1
 
  scons OPT=1
−
 
−
= questions =
 
−
 
−
1. we have to use singularity because farm, supporting docker to help run at mac and windows. Is it worth it? please provide feedback for using docker on mac and windows
 
−
 
−
2. Is docker layers really safe for reprehensibility?
 
−
 
−
3. there are many application, using one container for all make it simple to use, but need more careful work to develop. Another way is one application one container. Is it better?
 
  
 
= old =
 
= old =
Line 178: Line 175:
 
  The image is built with singularity-2.3.1, centos7.3 with latest rpm on 20170811, and jlab_version_1.3 framework
 
  The image is built with singularity-2.3.1, centos7.3 with latest rpm on 20170811, and jlab_version_1.3 framework
 
  location on ifarm for use, /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
 
  location on ifarm for use, /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
−
  location on web, http://www.phy.duke.edu/~zz81/package/singularity_jlab_version_1.3_centos7_20170811.img.tgz, download it and do "tar zxf" to extract img file
+
  location on web, http://www.phy.duke.edu/~zz81/simg/singularity_jlab_version_1.3_centos7_20170811.img.tgz, download it and do "tar zxf" to extract img file
  
 
  '''How to use it on ifarm and farm with official svn checkout'''
 
  '''How to use it on ifarm and farm with official svn checkout'''

Latest revision as of 15:39, 7 December 2020

This page has been moved to https://solid.jlab.org/wiki/index.php/container in the new SoLID Wiki. Please do not edit this page.


introduction

'This page is outdated, see new info at https://github.com/JeffersonLab/solid_release'

jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.

SoLID docker images are made to run and develope SoLID software to maintain consistent result without need for installation.

The images are at docker store under account jlabsolid https://store.docker.com/profiles/jlabsolid

They are in 3 layers: 

1. jlabsolidbase include a centos base images with some necessary rpms; 
2. jlabsolidapps add various applications above jlabsolidbase; 
3. jlabsolid add implementation of SoLID simulation,reconstruction,analysis above jlabsolidapps to complete solid software release

Read Note_about_container for any general things using container

for SoLID software user

get jlabsolid image
sudo docker pull jlabsolid/jlabsolid:test
location on ifarm, /group/solid/apps/jlabsolid-test.simg
location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolid-test.simg 
(jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")

load with singularity on ifarm and farm
ssh -X ifarm
module load singularity 
echo $DISPLAY                  (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
cd your_work_dir
singularity shell jlabsolid-test.simg 
(you are running as yourself in container, not as root)
(your_work_dir at host are mounted in container)
inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actual port with what you have)
inside container "xterm"   (test you can pass X11 application from inside container

load with singularity on linux host
cd your_work_dir
singularity shell jlabsolid-test.simg 
(you are running as yourself in container, not as root)
(your_work_dir at host are mounted in container)

load with docker on linux host by using host's Xwindows
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:$HOME --ipc=host jlabsolid/jlabsolid:test
(you are running as root in container)
(your home dir is mounted in container)

load with docker on linux host by using container's Xwindows
install x11docker
   wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
   sudo bash /tmp/x11docker --update
   rm /tmp/x11docker
install xorg-x11-server-Xephyr by yum or xserver-xephyr by apt-get
x11docker --desktop --homedir ${HOME} jlabsolid/jlabsolid:test fluxbox  
right click on the fluxbox window and open xterm
(you are running as yourself in container, not as root)
(your home dir dir is mounted in container)

run solid_gemc inside container
env -i tcsh
source /jlabsolidapps/JLAB_ROOT/setup
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC /solid_svn/solid_gemc2
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0                   (batch mode)
solid_gemc solid_SIDIS_He3_full.gcard                              (graphic mode)
exit or ctrl-d  (leave container)

How to use container to run jlab farm job
examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6

for SoLID software developer

checkout solid repo on host and load container 
cd your_work_dir
svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (just do it at the first time or run svn up to update it)
cd solid_svn
setenv solid_repo_FULL_PATH $PWD
load with docker
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host jlabsolid/jlabsolidapps:test 
sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host eictool/jlab_software_2.2 (to run jlab_version=2.2)
or load with singularity
singularity shell --bind ${PWD}:/solid_repo jlabsolidapps-test.simg            (download http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg to run jlab_version=1.3)
singularity shell --bind ${PWD}:/solid_repo eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg   (download http://webhome.phy.duke.edu/~zz81/simg/eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg to run jlab_version=2.2)

compile and run solid_gemc inside container 
tcsh
set prompt = '[%n@%m #Container# %c]$ '     (do this only for singularity  to make it clear that you are inside container)
source /solid_repo/set_solid 1.3c       (use 1.3c or 2.2c for special setup inside container)
cd $SoLID_GEMC/source/$GEMC_VERSION    (just do it at the first time)
scons OPT=1 -j4                        (just do it at the first time)
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0       (batch mode)
solid_gemc solid_SIDIS_He3_full.gcard                  (graphic mode)
exit or ctrl-d  (leave container)

for SoLID container developer

work flow

at your local machine with docker installed,
get the right jlabsolidapps image, 
then add applications and publish next jlabsolidapps image, 
then add solid simulation,reconstruction,analysis code and publish next jlabsolid image
at your local machine use singularity to pull the docker images

The test images are made this way 
jlabsolidbase:test is auto built on docker cloud from this Dockerfile
jlabsolidapps:test is based on jlabsolidbase:test and build manually by adding jlab framework 1.3 [1]
jlabsolid:test is based on jlabsolidapps:test and contained solid svn checkout with compiled solid_gemc

get jlabsolidapps image
sudo docker pull jlabsolid/jlabsolidapps:test
location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg 
(jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")

load jlabsolidapps with singularity or docker
You should checkout svn on host, then load it via "--bind  $solid_repo_FULL_PATH:/solid_svn" for singularity, "-v $solid_repo_FULL_PATH:/solid_svn" for docker, "--sharedir $solid_repo_FULL_PATH" for x11docker

example

use jlabsolidapps to test application and publish next jlabsolidapps

use jlabsolidapps to test solid code on host, then push change to solid code repo, then and update /solid_svn, publish next jlabsolid 

[user@localhost ~]$ cd $HOME
[user@localhost ~]$ svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn
[user@localhost ~]$ setenv solid_repo_FULL_PATH ${PWD}/solid_svn
[user@localhost ~]$ sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $solid_repo_FULL_PATH:/solid_svn --ipc=host jlabsolid/jlabsolidapps:test
[root@f18d5b96626c /]# tcsh
[root@f18d5b96626c /]# (install application or test solid code)
[root@f18d5b96626c /]# exit
[user@localhost ~]$ sudo docker commit f18d5b96626c
sha256:d55987e37b16b9a17e0795df19b2d6dbe368e0d24891e497805f79a1d733ec80
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
<none>                    <none>              d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidapps   test                a1b97160252c        11 hours ago        5.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker tag d55987e37b16 jlabsolid/jlabsolidapps:test
[user@localhost ~]$ sudo docker image ls
REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
jlabsolid/jlabsolidapps   test                d55987e37b16        2 minutes ago       5.62GB
jlabsolid/jlabsolid       test                ff6d136eee4d        39 minutes ago      6.62GB
jlabsolid/jlabsolidbase   test                5408f7716e45        3 days ago          738MB
[user@localhost ~]$ sudo docker login  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo docker push jlabsolid/jlabsolidapps:test  (login as jlabsolid, ask password from manager)
[user@localhost ~]$ sudo singularity pull docker://jlabsolid/jlabsolidapps:test
[user@localhost ~]$ md5sum jlabsolidapps-test.simg
e6f21d45b0f58f52631a5c46fa3b4c9d jlabsolidapps-test.simg

compile solid_gemc inside container 
cd $SoLID_GEMC/source/$GEMC_VERSION/ 
scons OPT=1

old

(testing as 2017/12)

jlab ifarm and farm support singularity container. A singularity container is made for the framework This could be a standard way to run simulation on any machine for consistent result without need for installation.

Read http://singularity.lbl.gov/faq to understand container and singularity and Refer to Note_about_singularity 

singularity image
The image is built with singularity-2.3.1, centos7.3 with latest rpm on 20170811, and jlab_version_1.3 framework
location on ifarm for use, /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
location on web, http://www.phy.duke.edu/~zz81/simg/singularity_jlab_version_1.3_centos7_20170811.img.tgz, download it and do "tar zxf" to extract img file

How to use it on ifarm and farm with official svn checkout
singularity shell --bind /group/solid/solid_svn:/group/solid/solid_svn /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
source /apps/jlab_root/setup 
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC /group/solid/solid_svn/solid_gemc2
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard            (graphic mode, one windows has control and the other shows the detector)
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
exit   (leave container)

How to use it on any machine with your svn checkout
cd (your_choice_of_solid_repo_FULL_PATH)
svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn  (checkout the repo)
singularity shell --bind (your_choice_of_solid_repo_FULL_PATH):(your_choice_of_solid_repo_FULL_PATH) container_image_location/singularity_jlab_version_1.3_centos7_20170811.img (enter container. you home dir and your login script are shared between the host and container, sometimes you current dir will also)
source /apps/jlab_root/setup 
setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
setenv SoLID_GEMC (your_choice_of_solid_repo_FULL_PATH)/solid_svn/solid_gemc2
cd $SoLID_GEMC/source/$GEMC_VERSION    (enter the source dir for solid_gemc)
scons OPT=1 -j4                        (compile solid_gemc) 
setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
cd $SoLID_GEMC/script
solid_gemc solid_SIDIS_He3_full.gcard            (graphic mode, one windows has control and the other shows the detector)
solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
exit   (leave container)

How to use it to run jlab farm job
examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6
Retrieved from "https://hallaweb.jlab.org/wiki/index.php?title=Solid_container&oldid=40037"