Running CODA on Spectrometers DAQ Systems

Robert Michaels, rom@jlab.org, Jefferson Lab Hall A, updated Feb 11, 2011 for CODA 2.6.1

paging Bob e-mail: rom@jlab.org

This file: hallaweb.jlab.org/equipment/daq/guide2.html

I. NORMAL DAQ OPERATIONS

This assumes that runcontrol is running. If not, see Cold Start in section III.

Run DAQ on adaql2 computer on the a-onl account. The run coordinator should know the passwords, which are also posted on the wall on a paper near the whiteboard.

Normally, runcontrol will be already running. If not, see section on Cold Start below (for the impatient: type "startcoda"). To start and stop runs, push the buttons "Start" (6th button from left, looks like >>) and "End" (5th button, looks like a box). Start is the same as the sequence prestart, go.

For Spring 2011 we're using the the THREEARM config, which is in the "Bigbite" session. (When we do (x.gt.2) this will change.) THREEARM is normally the one that will come up by default.

To change configurations, use the "Configurations" button and select COOL. If you've been running, you will first have to push "Reset" (<<) button. Choose the configuration from the dialog box, then press "Download" (looks like a disk drive). The relevant configurations include:
- THREEARM -- to run the standard coinc. config (no buffering)
- PEDRUN -- to do a pedestal run (see section II)..
Re: the buttons in rcgui: If you let the mouse hover over the button for a few seconds you can discover what the buttons mean because a little dialog box comes up and tells you, e.g. which button is "Start", etc.

II. FREQUENTLY ASKED QUESTIONS

Run number synch problem. With CODA 2.6.1 there seems to be a small probability that the run number reported by rcgui is different from the one in the data file. Firstly, the one that is correct is the one in the datafile name, which also appears as a record in the datafile, and which derives from MSQL. Secondly, fixing this goofy problem is sometimes tricky; you can try Options ... Set Run Number, but the GUI needs to be in a proper state. Or just report it to the expert and let him fix it.

Where is the data ? To find run 1602, type "find_run 1602" from adaq computers on the adaq accounts. Another command: "lastrun" prints where the latest run is. The data are on /adaqlM/dataN (M=1,2, N=1,2,3,..) as explained in the table above.

What is the deadtime ? Normally the deadtime is displayed in the "datamon" window -- make sure you are using the one on same computer as runcontrol. (Noted, Feb 11, we need to fix the deadtime monitoring for Bigbite ! Probably datamon is already running near the runcontrol window. If it is not running, type "datamon"

Prescale Factor ? Go to ~a-onl/prescale and edit file "prescale.dat"

Disk Handling ? There are several hundred Gbytes of disk for data taking. Automatic scripts choose the disk to write data, copy to MSS, delete the data when its safe, and avoid simultaneous read/write on a disk. Things to *NOT* do:

Do *NOT* erase or move data files (even if you think its junk).
Do *NOT* try to change run numbers.
Do *NOT* change names of run files.
Do *NOT* try to put data by hand into MSS.

It takes a couple hours for files to appear in MSS, and the files remain on adaq disk for typically 2 or 3 days.

Pedestal Runs The purpose of a pedestal run is to obtain pedestal files used by fastbus for pedestal suppression. If you've been running the normal spectrometer DAQ configuration, you'll need to press "Abort", then "RunTypes" and select the PEDRUN configuration. Download, StartRun. Run for about 5000 events, then EndRun. You may check the pedestal files in ~a-onl/ped (pedN.dat corresponding to ROC #N). See README there for more details. After pedestal run, change back to the running configuration. NOTE: DONT try to use PEDRUN for anything other than pedestal determination -- its confusing since the prescale factors are in a different file, etc. If you want a run with pedestal suppression turned off, see the README.

III. REBOOTING STUFF

HOW TO SHUTDOWN or REBOOT WORKSTATIONS

Rarely, the workstations don't function properly and the simplest way out is to reboot. Ole usually keeps some instructions posted near the PC terminal. One may hit Ctrl-Alt-F1 to go to console mode, then Ctrl-Alt-Del. If you have trouble starting X, type "startxfce4".

QUICK RESETS

Problems with CODA 2.6.1 can usually be solved with a simple Reset (<< in rcgui) or with a Cold Start. Call Larry Selvy, Bob Michaels, Vince Sulkosky, or Alexandre Camsonne for DAQ problems. Do NOT waste an hour stuck on resets.

If a ROC seems to be hung up, you can reboot it by going to the workspace with all the small x-terms logged into the frontend computers and typing "reboot" at the vxWorks prompt (-> reboot). Wait 2 minutes and telnet back in to verify its alive. The name of the ROC computer is normally written in the name of the xterm (hallasfi1, hallasfi2, etc). You need to know what subset of these computers are used for your configuration, see the table in section V below. If the ROC seems really frozen, use the "Crate Resets" button in the magnet EPICS screen on hacsbc2 (controls computer). Note, both fastbus crates in each HRS are reset with one button. (I'm not sure about Bigbite resets, sorry.) To reset from the EPICS GUI, toggle the state of the button. Note, if a ROC cannot be reset via the GUI, they can only be reset by typing "reboot" at the vxWorks prompt. Another possibility is to type "ctrl-X" at the vxWorks prompt; this sometimes works when typing reboot does not work. Finally, if the above does not work one can gain access to the cpu and press the reset button (or power cycle the crate). If you don't know how, or don't know where the cpu is located, page Larry Selvy first.

If you reboot the ROC, or if something on the workstation is hung up, try pushing the "Reset" button in runcontrol (rcgui). Thats the button that looks like "<<". Then Configure, Download, and Start a run as usual.

If a ``quick'' reset doesn't work, try a Cold Start (see below).

COLD START

To perform a cold start of the processes on the Linux computer (adaql2), simply type "startcoda" on the a-onl account. This performs a "kcoda" for you, as well as starts all the components you need.

Before downloading, it is first a good idea to make sure the fastbus and VME crates (i.e. the ROCs) are running. One finds that when resetting, one can frequently avoid rebooting the frontend crates, and just restart runcontrol, but if you must: Press the reset buttons for the relevant subset of crates as explained in the Reset section above. A convenient way to reset (reboot) the ROCs is to go to the adaql2 workspace with x-terms logged into each frontend computer and enter "reboot" at the vxWorks prompt -> of each ROC. But one can also use the EPICS GUI as explained above. After a few minutes, telnet back in and verify they are up. Some of the crates take longer than others to reboot. Be patient. To start everything on the workstation, type "startcoda" as explained above.

Eventually runcontrol (rcgui) will pop up. In runcontrol press the Platform button and then "connect". After "connect", verify you have the correct session (Bigbite) and configuration (THREEARM). Usually you will, but if not you'll need to select the session and config.

You may now do the "configure" by pressing the button with crossed screwdriver and wrench in the upper left corner. Be patient, configure takes awhile. Then you may download, which is the next button to the right, it looks like a tape drive. In general, the button names can be discovered by putting the mouse on it and letting it sit there for a few seconds; a small box appears with the name of the button.

To start a run, press the >> button.

To end a run press the square box that's sorta in the middle.

To reset, press the << button.

If you ever logout of the DAQ computer adaql2 or have rebooted it, here is how to restart the preferred setup. Login to the adaql2 console as "adaq" and then "ssh -l a-onl adaql2". Reason is that if you login directly as a-onl, some GUI windows like start-run don't pop up. Just do what I said. Start CODA by typing "startcoda" as explained above. Emacs the prescale file in the background: "emacs ~/prescale/prescale.dat &" Also start datamon by typing "datamon".

V. Spectrometer DAQ Configuration for Spring 2011


        CODA Configrations  -- on a-onl account on adaql2 computer 

        THREEARM  -- standard coincidence setup, unbuffered DAQ mode
        PEDRUN -- to do a pedestal run (see section II).

DAQ crates (accurate as of Feb, 2011)
ROC	Description	IP Address	Portserver
ROC1	R-arm Lower Fastbus Crate	hallasfi1	hatsv3 port 2
ROC2	R-arm Upper Fastbus Crate	hallasfi2	hatsv3 port 8
ROC3	L-arm Lower Fastbus Crate	hallasfi3	hatsv4 port 2
ROC4	L-arm Upper Fastbus Crate	hallasfi4	hatsv4 port 8
ROC5	Bigbite VME crate	bbvme1	bbps2 port 5
ROC8	Bigbite Fastbus Crate (HAND)	bbsfi1	bbps2 port 1
ROC9	Bigbite Fastbus crate	bbsfi2	bbps2 port 2
ROC10	Bigbite Fastbus crate	bbsfi3	bbps2 port 4
TS0	R-arm VME/Scaler	hallavme2	hatsv3 port 3
TS11	L-arm VME/Scaler	hallavme4	hatsv4 port 7
BBTS1 (ROC29)	Trigger Supervisor on Bigbite	bbts1	bbps2 port 4

VI. THINGS THAT GO WRONG WITH DAQ or COMPUTERS

Lots of deadtime -- Please see hallaweb.jlab.org/equipment/daq/dtime_faq.html

Computer/Network problems: Try calling Bob Michaels (pager 584-7410). You may also call the computer-center expert on-call (cell phone 876-1794). For problems with the linux boxes or ROOT analyzer ask Ole Hansen, though Bob can probably help too. For problems with slow controls or hac ask Javier Gomez. Alexandre Camsonne and Dave Abbott are Bob's substitute when out of town.

How to Program the RAM of the ROCs. When fastbus loses power, it may be necessary to reload the VME RAM (it isn't as non-volatile as it should be.) Up to date info is kept at ~a-onl/doc/vmeram.doc The Bigbite cpus are documented here : ~a-onl/boot_param_bb Portserver instructions ~adaq/doc/portserver.doc which are also online.

If Bob is unavailable, some details about recovery are at jlabs1:/home/rom/doc/recovery.doc

VI. More info

More info: hallaweb.jlab.org/equipment/daq/guide.html and hallaweb.jlab.org/equipment/daq/daq_trig.html.

This page maintained by Robert Michaels rom@jlab.org