The last run before I compiled the .crl file is run # 12220, which ended at about 22:26, Thursday, sep 10. The run looked fine.
After I recompiled the .crl, the coda did not download properly. It downloaded all the ROCs, but did not download TS1. Countless kcoda/startcoda, backing up on all the changes in the .crl file did not help.
I got help from Luis trying to resolve the problem, but we did not get very far.
Soon enough we contacted Kent, and I emailed/tried calling Bob since I could not reach him through his pager. Luis seemed to think that his pager had not been functioning properly lately.
Here is a list of things done over the course of the night:
- Kent checked the current version of .crl with the repository version. They were no differences! - Kent checked the database dump from sep3 (when HapNoSimple config was added), to today's database. Lots of differences!! I had opened the database on "cedit" to check for the autoled flags on the left arm. Maybe that was when I ruined the database. So, with Mark's help, we restored the database to the one from sep3. - Some more startcoda/kcoda, but the coda still hungup on download. The past startcoda calls were still in the background after multiple startcoda/kcoda calls. Had to force kill all the startcoda call multiple times. - Over the course of trying out numerous changes, we had commented/uncommented some autoled stuffs from happex7.boot (right arm) and hallavme14.boot (left arm). The left arm boot script was still loading autoled stuffs. - With Alexander's help, we realized that the crate no longer downloaded left arm properly. So we blanked out the autoled stuffs in the left arm boot script, and the coda downloaded fine.
Conclusion: We had a corrupt database, which has been fixed (I was the culprit). The left arm has not been able to handle the autoled routines in its current configurtaion, which will be resolved soon as well. For now, we are running coda fine, with all the autoled stuffs from the left arm commented out.
NOTE: When we started coda, the first two run numbers were 11938, 11939. Not sure if the past data files were actually overwritten. Need to check. We changed the run number on the coda to 12222, and coda has been running with the correct run number since. The coda has been in working condition since about 2:20 am, Friday sep 11.
A copy of this log entry has been emailed to: rom@jlab.org, mercado@jlab.org, paschke@jlab.org, dalton@jlab.org, souder@physics.syr.edu, kkumar@physics.umass.edu, rsholmes@physics.syr.edu, mcnulty@jlab.org, camsonne@jlab.org