PREV
Make New Entry,
Make Followup Entry
User name R. Michaels
Log entry time 05:05:27 on April15,2012
Entry number 374744
This entry is a followup to: 374732
keyword=diagnosis of duplicate file problem
First, some information about the files for the sequence
nearby where the duplicate file problem occurred.
For L-HRS DAQ
[a-onl@adaql2 ~]$ ls -la /adaql2/data*/*4423*
-rw-r--r-- 1 a-onl a-adaq 889487360 Apr 15 04:23 /adaql2/data3/g2p_4423.dat.0
-rw-r--r-- 1 a-onl a-adaq 841777152 Apr 15 04:32 /adaql2/data4/g2p_4423.dat.0
[a-onl@adaql2 ~]$ ls -la /adaql2/data*/*4424*
-rw-r--r-- 1 a-onl a-adaq 1738178560 Apr 15 04:46 /adaql2/data1/g2p_4424.dat.0
[a-onl@adaql2 ~]$ ls -la /adaql2/data*/*4425*
ls: No match.
[a-onl@adaql2 ~]$ ls -la /adaql2/data*/*4426*
-rw-r--r-- 1 a-onl a-adaq 361857024 Apr 15 04:55 (present run) /adaql2/data2/g2p_4426.dat.0
And for R-HRS DAQ
[a-onl@adaql2 ~]$ ls -la /adaql1/data*/*23377*
-rw-r--r-- 1 a-onl a-adaq 888668160 Apr 15 04:23 /adaql1/data4/g2p_23377.dat.0
[a-onl@adaql2 ~]$ ls -la /adaql1/data*/*23378*
-rw-r--r-- 1 a-onl a-adaq 895483904 Apr 15 04:32 /adaql1/data1/g2p_23378.dat.0
[a-onl@adaql2 ~]$ ls -la /adaql1/data*/*23379*
-rw-r--r-- 1 a-onl a-adaq 1699414016 Apr 15 04:45 /adaql1/data2/g2p_23379.dat.0
[a-onl@adaql2 ~]$ ls -la /adaql1/data*/*23380*
-rw-r--r-- 1 a-onl a-adaq 553451520 Apr 15 04:55 (present run) /adaql1/data3/g2p_23380.dat.0
----------------
What we conclude is :
The 2nd run 4423 (that ended at 4:32 am) was actually run 4424.
(This should be checked in the data file; the run number appears
in the datastream).
The file for "run 4424" is probably actually 4425.
The file for run 4425 does not exist.
The R-HRS did not get affected. It's run sequence was fine.
-------------------
What we did:
We did a "startcoda" after run 23379 and 4425 (corresponding to
the file called 4424). Runs 4426 and 23380 started normally and
were a repeat of the dummy target run.
I moved the 2nd "4423" file as follows
[a-onl@adaql2 ~]$ ls -la /adaql2/data*/*4423*
-rw-r--r-- 1 a-onl a-adaq 889487360 Apr 15 04:23 /adaql2/data3/g2p_4423.dat.0
-rw-r--r-- 1 a-onl a-adaq 841777152 Apr 15 04:32 /adaql2/data4/g2p_4423.dat.0
[a-onl@adaql2 ~]$ cd /adaql2/data4
[a-onl@adaql2 data4]$ mv g2p_4423.dat.0 g2p_4423_2nd.dat.0
Technically maybe I should have moved the 2nd 4423 to be 4424
and the 4424 to be 4425, but that's not how we've been recovering
so far. Note, for other incidents like this I have not been
on shift. The "2nd" file in MSS could be the earlier run or
the later run; it depends on which file got put into MSS first.
The analysis folks will need to discover the run number from
the CODA file, plus there's enough info in the halog to figure
it out as well (creation dates and such).
Would be better to fix CODA, of course.