Difference between revisions of "Off Line Analysis for transversity"

Revision as of 04:51, 24 April 2009

Quick Links:

This is temporary site of Off Line Analysis for transversity.

Analysis Software

Besides standard Hall A analyzer, Transversity analysis software contains

Database sets
BigBite Lib
RICH Lib
BB Norm Analysis Lib
replay scripts

The most up2date version is loaded in Hall A CVS server.

 cvs.jlab.org:/group/halla/analysis/cvs/transversity

CVS (Concurrent Versions System)

For those who is not very familiar with CVS. Here is a brief introduction:

From Wikipedia:CVS: In the field of software development, the Concurrent Versions System (CVS), also known as the Concurrent Versioning System, is a free software revision control system. Version control system software keeps track of all work and all changes in a set of files, and allows several developers (potentially widely separated in space and/or time) to collaborate. Dick Grune developed CVS in the 1980s. CVS has become popular in the open source software world and is released under the GNU General Public License.

Useful Reference Links

You don't need to know all. Just check what ever command you need when you need.

CVS for Transversity

We are in a stage that multiple people is working on development of same set of software. So CVS is introduced to keep files synchronize between people.

It always keep you using most up2date version of software
Distribute your updates to other people
Save a modification history

HowTo + Examples

Step 1: HowTo Start

It's suggested that everyone download a copy from CVS to your working computer (could be office computer or farm), and then work on your replay and development with this copy.

Notice:

A pre installed version of root and hall A analyzer is required.
- For farm replay, it's already set. To use the farm version, call source /w/work5602/transversity/analysis/SourceThisFirst
All the files are source code. So compile is needed after download (using script "makeall")

Example: Start a shell (tcsh)

# just to test your analyzer installation
[jinhuang@mitnig ~]$ analyzer   

# then set up the CVS environmental variables. You might want to consider save it to your ~/.tcshrc
[jinhuang@mitnig ~]$ setenv CVSROOT cvs.jlab.org:/group/halla/analysis/cvs
[jinhuang@mitnig ~]$ setenv CVS_RSH ssh
[jinhuang@mitnig ~]$ setenv CVSEDITOR emacs

# Download the newest version from cvs
[jinhuang@mitnig ~]$ cvs co transversity
[jinhuang@mitnig ~]$ cd transversity/

# run makeall script to setup symbolic links and do compiles 
[jinhuang@mitnig ~/transversity]$ makeall

# Test your new installation
[jinhuang@mitnig ~/transversity]$ cd replay/
[jinhuang@mitnig replay]$ analyzer 

rootlogon.C: Loading BigBite Library...
rootlogon.C: Loading BB Normalization Analysis Library...
rootlogon.C: Loading macro's:
         def.h
         HallA_style.cxx
         AnaShowRun.C
         libRICH.so
         replay_det_L.C
         replay_BB.C
         replay_det_BB.C
         get_bpm_pedestals.C
         replay_phys.C
         replay_scalar.C
         replay_asym.C

rootlogon.C: Done!

  ************************************************
  *                                              *
  *            W E L C O M E  to  the            *
  *       H A L L A   C++  A N A L Y Z E R       *
  *                                              *
  *        Release      1.5.9        Apr 22 2009 *
  *  Based on ROOT  5.22/00          Dec 17 2008 *
  *                                              *
  *            For information visit             *
  *        http://hallaweb.jlab.org/root/        *
  *                                              *
  ************************************************

CINT/ROOT C/C++ Interpreter version 5.16.29, Jan 08, 2008
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.

analyzer [0] replay_phys(4220,1000) 

# then it should begin replay

Step 2: HowTo Update your local copy

In case of anyone else updated the CVS, a email will be automatically distributed to tran_ana@jlab.org mail list. Then you should considering update your local copy of CVS.

[jinhuang@mitnig ~]$ cd transversity/  # go to your local directory containing the local copy
[jinhuang@mitnig ~/transversity]$ cvs update  # if the CVS environmental variable is not set, then repeat setting commands in step 1 
cvs server: Updating .
........
? bigbitelib/theoptics.d  <-------- if a file is on your local, but not exist in CVS, it's marked as "?"; these files are not touched during cvs update
P makeall                 <-------- if a file is updated, it's marked as "P". And it is automatically updated
cvs server: Updating BBNormAna
........
cvs server: Updating replay
M replay/rootlogon.C      <-------- if a file is modified by you, then it's marked as "M"; The modification is not be uploaded

Farm Replay

Structure

On farm file system, there are 3 disk:

disk1 : Our main script disk.
disk2 : Shared with E08007, mostly full
disk3 : 2TB disk for large root files

THe main analysis directory is /work/halla/transversity/disk1/analysis/, which includes various of components:

replay.sh = SourceMeFirst : script setting up environmental pathes. Configured for CSH
replay : replay directory holding replay scripts/configure files
farmreplay : specialized folder generating batch farm replay scripts
DB : database
analyzer : analyzer
bigbitelib : Bigbite library
RICHlib : rich lib
BBNormAna : Normalization analysis lib
onlineGUI : Online GUI

analysis directory could be copied to your own location to be customized.

HowTo do a quick test replay

One can directly use the common replay directory to do quick tests.

ssh ifarml5    # or any computer seeing working disk
cd /work/halla/transversity/disk1/analysis
source replay.sh
cd replay
analyzer
replay_phys(4029,100000);   //just like online replay

HowTo build your own batch farm replay directory

Build a directory for your own replay. You can change replay code and control the rootfile storage. And it will be capable of replaying on both ifarm and batch farm.

ssh ifarml5    # or any computer seeing working disk

cd /work/halla/transversity/disk1/<your name>

mkdir analysis    # this is the main directory for replay codes
cd analysis

cp -rv /work/halla/transversity/disk1/analysis/replay  ./
cp -rv /work/halla/transversity/disk1/analysis/farmreplay ./
cp -rv /work/halla/transversity/disk1/analysis/replay.sh ./

mkdir /work/halla/transversity/disk3/<your root file folder>
ln -svf /work/halla/transversity/disk3/<your root file folder> ROOTfiles

Now you have the basic structure:

To test replay in your new folder, follow Off Line Analysis for transversity#HowTo build your own replay directory (need to source replay.sh)
To do batch farm replay, follow example below:
1. enter directory farmreplay
2. edit construct.pl, modify following sections:
  - $neve : the event number to replay
  - @runnumber : list of run to replay
  - $ReplayType : name of replay script
  - $current : path of your analysis directory
3. run construct.pl; it will generate
  1. in_run# : the script for batch running for each run; double check it carefully
  2. sub_run# : Batch Job Command File for each run; double check it carefully
4. (Optional), test in_run# scripts by copying it to a tmp directory and run it.
5. jsub sub_run# will run the replay in batch farm

HowTO Customize Analyzer / Libs / Database

There are more folders in /work/halla/transversity/disk1/analysis/, which are various of components:

DB : database
analyzer : analyzer
bigbitelib : Bigbite library
RICHlib : rich lib
BBNormAna : Normalization analysis lib
onlineGUI : Online GUI

To use your own version of components, first copy its direcory to your own analysis folder, then

for database and analyzer, modify your replay.sh to set its path
for other libraries, just modify the *.so link in your replay directory to your own location

General Rule for Farm Disk Usage

All transversity farm disks are shared with other hall A experiment. The total space is limited. In the case that the disk is full, old file will be deleted by Hall A manager. Therefore, we need to avoid heavy disk usage by following proper rules:

On farm file system, there are 3 disks mounted at /work/halla/transversity/:

disk1, 800GB

Disk1 is our main script disk. Please make a directory with your user name.

It could be used to store small files. Please avoid large data. Please always keep a decent percentage of free space.

It's shared with GEN, Pi0.

disk2, 800GB

Shared with E08007, mostly full. Could be used to buffer test files.

disk3, 1-2TB

TB disk for large root files buffering. Quota is controlled by Dr. Robot Michaels. If the disk is full, old files will be deleted!

Please make a directory with your user name to separate your storage space
Please keep your root file as small as possible
Please delete unused root files as soon as possible
TmpROOTfiles which is linked to common replay directory will be periodically cleaned
ProductionROOTfiles is for farm production, performed and managed by collaboration.

HowTo Control your root file size

By controlling which variable to output and apply proper cut, you can not only limit the size of root file, but also speed up the replay.

Output define file: Controlling which variable to write. see description here
Cut File : Controlling which event to process and output. see description here