Running CODA on Spectrometers DAQ Systems

Robert Michaels, rom@jlab.org, Jefferson Lab Hall A, updated July 8, 2005

This file: hallaweb.jlab.org/equipment/daq/guide2.html

I. NORMAL DAQ OPERATIONS

This assumes that runcontrol is running. If not, see Cold Start in section III.

Use the "a-onl" account on adaql2.

The run coordinator should know the passwords, which are also posted on the wall on a paper near the whiteboard.

Normally, runcontrol will be already running. If not, see section on Cold Start below. To start and stop runs, push the buttons "Start Run" and "End Run". Start Run is the same as the sequence prestart, go.

To change configurations, use the "Run Type" button. If you've been running, you will first have to push "Abort" button. Choose the configuration from the dialog box, then press "Download". The relevant configurations include:

TWOARM -- Two spectrometer arms, normal running.
pedrun -- Pedestal run for HRS (see below)

II. FREQUENTLY ASKED QUESTIONS

Where is the data ? To find run 1602, type "find_run 1602" from adaq computers on the adaq accounts. Another command: "lastrun" prints where the latest run is. The data are on /adaql2/dataN (N=1,2,3,..)

What is the deadtime ? The deadtime is displayed in the "datamon" window -- make sure you are using the one on same computer as runcontrol. Probably datamon is already running near the runcontrol window. If it is not running, type "datamon"

Prescale Factor ? Go to ~a-onl/prescale and edit file "prescale.dat"

Disk Handling ? There are several hundred Gbytes of disk for data taking. Automatic scripts choose the disk to write data, copy to MSS, delete the data when its safe, and avoid simultaneous read/write on a disk. Things to *NOT* do:

Do *NOT* erase or move data files (even if you think its junk).
Do *NOT* try to change run numbers.
Do *NOT* change names of run files.
Do *NOT* try to put data by hand into MSS.

It takes a couple hours for files to appear in MSS, and the files remain on adaq disk for typically 2 or 3 days.

Pedestal Runs The purpose of a pedestal run is to obtain pedestal files used by fastbus for pedestal suppression. If you've been running the normal spectrometer DAQ configuration, you'll need to press "Abort", then "RunTypes" and select the PEDRUN configuration. Download, StartRun. Run for about 5000 events, then EndRun. You may check the pedestal files in ~a-onl/ped (pedN.dat corresponding to ROC #N). See README there for more details. After pedestal run, change back to the running configuration. NOTE: DONT try to use PEDRUN for anything other than pedestal determination -- its confusing since the prescale factors are in a different file, etc. If you want a run with pedestal suppression turned off, see the README.

III. REBOOTING STUFF

HOW TO SHUTDOWN or REBOOT WORKSTATIONS

Rarely, the workstations don't function properly and the simplest way out is to reboot. Ole usually keeps some instructions posted near the PC terminal. One may hit Ctrl-Alt-F1 to go to console mode, then Ctrl-Alt-Del. If you have trouble starting X, type "startxfce4".

QUICK RESETS

Problems with CODA 2.5 can usually be solved with a simple reset or with a Cold Start. Call Bob Michaels for DAQ problems. Do NOT waste an hour stuck on resets.

If a ROC seems to be hung up, you can reboot it by going to the adaql2 workspace with all the small x-terms logged into the frontend computers and typing "reboot" at the vxWorks prompt (-> reboot). Wait 2 minutes and telnet back in to verify its alive. The name of the ROC computer is normally written in the name of the xterm (hallasfi1, hallasfi2, etc). You need to know what subset of these computers are used for your configuration, see the table in section V below. If the ROC seems really frozen, use the "Crate Resets" button in the magnet EPICS screen on hacsbc2 (controls computer). Note, both fastbus crates in one arm are reset with one button. To reset from the EPICS GUI, toggle the state of the button. Note, if a ROC cannot be reset via the GUI, they can only be reset by typing "reboot" at the vxWorks prompt. Another possibility is to type "ctrl-X" at the vxWorks prompt; this sometimes works when typing reboot does not work. Finally, if the above does not work one can gain access to the cpu and press the reset button (or power cycle the crate). If you don't know how, or don't know where the cpu is located, page Bob Michaels or his designate.

If you reboot the ROC, or if something on the workstation is hung up, try pushing the "Reset" button in runcontrol. Then Configure, Download, and StartRun as usual.

If a ``quick'' reset doesn't work, try a Cold Start (see below).

COLD START

To perform a cold start of the processes on the adaql2 computer, simply type "startcoda" on the a-onl account. This performs a "kcoda" for you, as well as starts all the components you need.

Before downloading, it is first a good idea to make sure the fastbus and VME crates (i.e. the ROCs) are running. One finds that when resetting, one can frequently avoid rebooting the frontend crates, and just restart runcontrol, but if you must: Press the reset buttons for the relevant subset of crates as explained in the Reset section above. A convenient way to reset (reboot) the ROCs is to go to the adaql2 workspace with x-terms logged into each frontend computer and enter "reboot" at the vxWorks prompt -> of each ROC. But one can also use the EPICS GUI as explained above. After a few minutes, telnet back in and verify they are up. Some of the crates take longer than others to reboot. Be patient. To start everything on the workstation, type "startcoda" as explained above. Eventually runcontrol will pop up. In runcontrol press the Connect button. After "connect", wait 10 seconds, then press "Run Type"; a dialog box pops up and you must choose the configuration you want, which is normally the first choice. Then press download and wait about 30 seconds. If download hangs up, press "Cancel", "Reset", and "Download" in runcontrol. After a successful download, you can "Start Run" to start a run.

If you ever logout of the DAQ computer adaql2 or have rebooted it, here is how to restart the preferred setup. Login as "a-onl". Emacs the prescale file in the background: "emacs ~a-onl/prescale/prescale.dat &" Also start datamon by typing "datamon". Next, in the "components" workspace, login to all the frontend computers by typing "telnet hallasfi1" where the name of the computer like hallasfi1 is in the name of the xterm window. Also in the name of these xterms is the portserver port where you can connect via RS232 (portserver instructions are at hallaweb.jlab.org/equipment/daq/portserver.html). So, an example name may be an xterm with title "ROC2--hallasfi2--hatsv3-port-8" meaning ROC2 is IP address hallasfi2 and on portserver hatsv3 at port 8. Note: If these xterm windows for components are not there, type "setupxterms" to bring them up.

V. Spectrometer DAQ Configuration for HAPPEX


        CODA Configrations 

        TWOARM             Two spectrometers
        pedrun             Pedestal run for TWOARM

DAQ crates (accurate as of July 8, 2005)
ROC	Description	IP Address	Portserver
ROC1	R-arm Lower Fastbus Crate	hallasfi1	hatsv3 port 2
ROC2	R-arm Upper Fastbus Crate	hallasfi2	hatsv3 port 8
ROC3	L-arm Lower Fastbus Crate	hallasfi3	hatsv4 port 3
ROC4	L-arm Upper Fastbus Crate	hallasfi4	hatsv4 port 2
TS0	R-arm VME/Scaler	hallavme2	hatsv3 port 3
TS1	L-arm VME/Scaler	hallavme4	hatsv4 port 14
ROC7	F1 TDC Dropped at present	hallavme17	hatsv4 port 10

Note, at the moment, F1 TDC crate is dropped. It might be added later during 2005; not needed for HAPPEX.

VI. THINGS THAT GO WRONG WITH DAQ or COMPUTERS

Lots of deadtime -- Please see hallaweb.jlab.org/equipment/daq/dtime_faq.html

Loss of network connectivity in the hall. Symptoms: Cannot run DAQ, cannot ping any VME computer, cannot ping the laptops on the detectors stack (ahut1, ahut2). cannot ping the network switch in the hall (129.57.192.2). Solution: Enter the hall, find the 192 switch in the 2nd or 3rd rack from entrance to hall, along the beamline (hopefully labelled; if not, look for little yellow cables with labels like "ROC4", "TS1", "moller", etc) and power cycle it.

Computer/Network problems: Try calling Bob Michaels (pager 584-7410). You may also call the computer-center expert on-call (cell phone 876-1794). For problems with the linux boxes or ROOT analyzer ask Ole Hansen, though Bob can probably help too. For problems with slow controls or hac ask Javier Gomez. Bodo Reitz and Dave Abbott are Bob's substitute when out of town.

How to Program the RAM of the ROCs. When fastbus loses power, it may be necessary to reload the VME RAM (it isn't as non-volatile as it should be.) Up to date info is kept at ~adev/doc/vmeram.doc Portserver instructions ~adaq/doc/portserver.doc which are also online.

If Bob is unavailable, some details about recovery are at jlabs1:/home/rom/doc/recovery.doc

VI. More info

More info: hallaweb.jlab.org/equipment/daq/guide.html and hallaweb.jlab.org/equipment/daq/daq_trig.html.

This page maintained by Robert Michaels rom@jlab.org