Difference between revisions of "Data handling scripts (to MSS and deletion)"

From Hall A Wiki
Jump to: navigation, search
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
On the ADAQ cluster, running from the adaq account, there are cron scripts which automatically copy the raw data to the MSS, and may also delete files from the local data disks if necessary.  This has been implemented for 20+ years for the HRS DAQ, the Parity DAQ, and the polarimeters Moller and Compton.  At times it was deployed for other DAQ systems too, like BDX and PEPPO.  If you are a maintainer of a DAQ systems please contact me (Bob Michaels) because there is some coordination and rules to follow, e.g. filenames must be unique, of limited size, and there should have agreed upon keywords like the experiment name in the filename.
+
Hall A:
  
−
For most users, the main rule is this : <font color="red">DON'T INTERFERE !  Please do not delete, move, or rename raw data files. Especially do not delete anything !</font> The risk is doing a "rm" command which wipes out weeks of precious beam time.  Well, that's very unlikely to happen because we copy promptly but the possibility exists.  Please have the self-discipline to not "rm" from a "data" disk.  Also, please do not put any files on a disk with the string "data" on the name of the disk  Those "data" disks are for raw data only. There are separate work disks and scratch disks for files that are not raw data, for example root output files.  Thank you for tolerating my paranoia.
+
Data are moved to MSS and deleted from the ADAQ disks using a cron job that runs on adaqeb1, adaqeb2, and adaqeb3.
 +
Login as the adaq account and type "crontab -l" to see what it is.
 +
/adaqfs/home/adaq/jmirror_scripts/adaqeb_sbs_jmirror
  
−
Briefly, the way the scripts work is as follows:
+
At the heart of the script is the JLAB-standard "jmirror" script.
  
−
1. <font color="blue">MSS copying: </font> All files on the data disks are checked to see if they are in the MSS already in the appropriate area, and if not we put them there using "jput".  There is a certain probability of order 1% that jput fails. These failures are logged.  Another script comes along and re-tries to copy any failures.  Essentially we try an infinite number of times to "jput", but in practice I think I've never seen two or more failures.  Now, there are various things that can go wrong, e.g. we logged that the file was copied but it actually was not, or we copied but there is no duplicate.  There are about 15 rare things that can go wrong.  For each of these, an email is sent to Bob Michaels, and I check the log files and fix the problem by hand.
+
Hall C:
  
−
2. <font color="blue">File deletion: </font>A cleanup script is run which first checks disk usage.  If the disks are getting full, it starts to delete files provided they are in the MSS, in duplicate, with the same name and same byte count as file on the local disk which is to be deleted.  We would rather not delete files because its useful to the experiment to have them locally. Therefore, the aggressiveness of the cleanup script is tuned to how full the disk is.  Specifically the amount of time the files can stay on disk is a function of the percentage of disk available.  Lately this ends up being about 3 weeks, typically.  If the disks get close to full and email is sent to Bob Michaels.
+
A similar jmirror-based script runs under cron on coda@cdaql5
 +
/home/coda/coda/jmirror_scripts/hallc_jmirror
 +
 
 +
 
 +
There are also scripts that check for anomolies in the process. You can find these also under cron.
 +
 
 +
[adaq@adaq1 ~]$ crontab -l
 +
 
 +
[coda@cdaql5 ~]$ crontab -l
 +
 
 +
 
 +
For most users, the main rule is this : <font color="red">DON'T INTERFERE !  Please do not delete, move, or rename raw data files.  Especially do not delete anything !</font> The risk is doing a "rm" command which wipes out weeks of precious beam time.  Well, that's very unlikely to happen because we copy promptly but the possibility exists.  Please have the self-discipline to not "rm" from a "data" disk.  Also, please do not put any files on a disk with the string "data" on the name of the disk partition, e.g. /adaq1/data1.  Those "data" disks are for raw data only. There are separate "work" disks and "scratch" disks for files that are not raw data, for example root output files. And those disks are not backed up.

Latest revision as of 13:36, 24 August 2023

Hall A:

Data are moved to MSS and deleted from the ADAQ disks using a cron job that runs on adaqeb1, adaqeb2, and adaqeb3. Login as the adaq account and type "crontab -l" to see what it is. /adaqfs/home/adaq/jmirror_scripts/adaqeb_sbs_jmirror

At the heart of the script is the JLAB-standard "jmirror" script.

Hall C:

A similar jmirror-based script runs under cron on coda@cdaql5 /home/coda/coda/jmirror_scripts/hallc_jmirror


There are also scripts that check for anomolies in the process. You can find these also under cron.

[adaq@adaq1 ~]$ crontab -l

[coda@cdaql5 ~]$ crontab -l


For most users, the main rule is this : DON'T INTERFERE ! Please do not delete, move, or rename raw data files. Especially do not delete anything ! The risk is doing a "rm" command which wipes out weeks of precious beam time. Well, that's very unlikely to happen because we copy promptly but the possibility exists. Please have the self-discipline to not "rm" from a "data" disk. Also, please do not put any files on a disk with the string "data" on the name of the disk partition, e.g. /adaq1/data1. Those "data" disks are for raw data only. There are separate "work" disks and "scratch" disks for files that are not raw data, for example root output files. And those disks are not backed up.

Retrieved from "https://hallaweb.jlab.org/wiki/index.php?title=Data_handling_scripts_(to_MSS_and_deletion)&oldid=44789"