HALL  A  RAW  DATA  STRUCTURE

R. Michaels,  Jefferson Lab,  e-mail:  rom@jlab.org,  updated Jan 28, 2003

•  OVERVIEW

The raw data structure recorded in the spectrometer DAQ datastream will be described.  This includes the following data:  1) Event data recorded with each accepted trigger  (fastbus and VME data from the two spectrometers and from the BPM and raster);  2) scalers;  3)  EPICS information;  and 4)  "special events" which are inserted once or infrequently during a run, such as prescale information and detector map.

Fortunately, there is already a large body of software which decodes and interprets these data, so in many cases the experimenter may use higher level variables, such as presented in ESPACE kumacs. The new ROOT/C++ analyzer also has a decoder.

I will not describe the various other datastreams in Hall A, such as Moller polarimeter,  e-P energy measurement,  Compton Polarimeter, etc.

For some notes about the year 2001 and a bit beyond see dstruct_year2001.html.     To find out about the data prior to the year 2001, see the URL:   dstruct_pre2001.html.

•  CODA  DATA  FORMAT

The best detailed reference on the CODA format was the old manual for CODA version 1.4, see appendix E.   I have saved this postscript file coda1.4.ps.   I will give a simplified summary here.  The data come in the form of an array of 32-bit words, in the case of event data, or character data, in the case of some of the special events.  In general, the  first part of the event structure is "header" information, indicating how long the event is, what the event type is, etc.  The most important header and other identifying data words are:

Identifying CODA Data Words

Location in Buffer	Content	Description
1st  word	event length	loop from J=1 to J=evbuffer(1)+1 in Fortran to access data = evbuffer(J)
ishft( evbuffer(2), -16 )	event  type	Types 1-12 are triggers that don't overlap in time.   Type 14 is any trigger overlap.  Other types correspond to prestart, go, end, etc. Types >128 are "special events".
evbuffer(4) in event type 17	run number	The run number, which is normally in the filename also.
evbuffer(5)	event number	The event number, uniquely assigned by CODA

After the header, the data are organized in banks corresponding to each ROC as of Sept 2002:   A ROC is a "readout controller", and it means a crate of fastbus or VME.   We have several ROCs in our spectrometer DAQ event stream:  1) ROC1 = fastbus on R-arm;  2) ROC2 = 2nd fastbus on R-arm scintillators; 3) ROC3 = fastbus on L-arm;   4) ROC4 = fastbus on L-arm (FPP crate). and being phased out is 5) ROC14 = VME crate for BPM and raster data.  The beginning of each ROC bank contains the length of the bank and an identifying tag to indicate which ROC number.  A bit of Fortran code which illustrates how to crawl through the banks is shown below.

          do 150 i = 1,MAXROC
c n1 = pointer to first word of ROC
            if(i.eq.1) then
               n1 = evbuffer(3) + 4
            else
               n1 = n1roc(irn(i-1))+lenroc(irn(i-1))+1
            endif
c irn = ROC number obtained from CODA
            irn(i) = ishft(iand(evbuffer(n1+1),'1f0000'x),-16)
            if(irn(i).le.0.or.irn(i).gt.maxroc) then
               write(6,*) 'fatal error in decoding'
               stop
            endif
c Store pointers and lengths
            if(i.eq.1) then
               n1roc(irn(1)) = n1
               lenroc(irn(1)) = evbuffer(n1)
               lentot = n1 + evbuffer(n1)
            else
               n1roc(irn(i)) = n1roc(irn(i-1))+lenroc(irn(i-1))+1
               lenroc(irn(i)) = evbuffer(n1roc(irn(i)))
               lentot = lentot + lenroc(irn(i)) + 1
            endif
            if(lentot.ge.len-1) goto 200
150     end do
200     numroc = i

Libraries exist for reading and writing CODA data format. The library is in the ESPACE distribution, in the /evio directory. A convenient example code for reading/writing CODA data (written in C) is here: rwcoda.c with a Makefile (one can link to CODA libraries or simply compile evio.c and link it instead).

If you want to be modern and hip, try my   CODA classes   (C++ classes for CODA interface). This and the hana decoder have been integrated into the new ROOT/C++ analyzer

•   DECODING  OF  DATA

The data are decoded by ESPACE in the routine event_decoder which is in the module coda.f.   This routine requires the detector map.   Normally the ``official detmap'' is kept in ~adaq/detmap directory on the ADAQ cluster.   This detector map gives the correspondence between the "raw channel number" and the "analysis detector channel".  By "raw channel number" I mean the combination of :  (ROC number, slot in ROC,  channel within the slot).  By "analysis detector channel"  I mean the channel within a detector as indexed by ESPACE.  The detector map is also stored in the CODA file as a special event and can be used by ESPACE by leaving out (or commenting out)  the line "set/file/detmap detmap.config" in the ESPACE kumac.  The absence of this line causes ESPACE to search the CODA file for the detmap.  The presence of this line causes ESPACE to take the detmap specified in the kumac, which then over-rides whatever is written in the CODA file.  The intention, of course, is that the detmap in the CODA file is the correct one at the time of the experiment.  There have been some reports that ESPACE occassionally (rarely) fails to find the detmap in a CODA file.  You can work around this problem by extracting it once and then renaming this file and using it as the the file pointed to in the kumac file.  The extracted detmap file is always called detector_map.lis.

•  EVENT   FLAGS

Each crate adds some ``event flag'' data, which are extra words that don't come from a module. The set of flags that exist has changed over time, and I will give a snapshot that is valid as of Tue Jan 28 12:12:52 EST 2003. These data consist of one or several words after a "header". A header is a unique flag which is impossible to misinterpret as data from a module.

In Fastbus:
   0xfabc0005  -- Header (hex format) denoting the start of event flags.
                  Because of the "f" this is impossible to confuse
                  with fastbus data.  The lowest 3 bits tell how
                  many flags follow, in this case 5.
   event count -- Event counter (locally counted in each CRL).
                  This increments to 20000 and is reset to 1.
   datascan    -- Tells the bit pattern of what fastbus crates
                  had a hit
   event time  -- The vxWorks event time.  This has 1/60 sec resolution.
   event type  -- This is the event type that the trigger supervisor
                  told to the ROC via a cable connection (hardware).
Then there is a flag showing the buffering mode
   0xfaaa0001  -- Header (0x means hex format)
   0xfabb0000  -- If you see this, data are unbuffered.
   0xfafbbf00+synchFlag -- If you see this the data are buffered.
                  If synchFlag !=0 then it was a synch event.

In ROC14:
   0xfadcb0b4 -- Header (0x means hex format)
   event time  -- The vxWorks event time.  This has 1/60 sec resolution.
   event type  -- This is the event type that the trigger supervisor
                  told to the ROC via a cable connection (hardware).

•   SCALERS

Scalers count raw hits on phototubes, as well as  important quantities like charge and triggers which are used to normalize the experiment.  The scalers are read online using the "xscaler" code, and the meaning of the channels is evident from the scaler configuration file which that code uses.  The scaler config file is normally located in the directory from which you must run "xscaler", namely ~adaq/$EXPERIMENT/right/scaler ~adaq/$EXPERIMENT/left/scaler where $EXPERIMENT is an environment variable like e94104.  

We attempt to keep the channel mapping the same. NOTE:   As of Sept 2003, the info about scalers is up-to-date. For older data, see also dstruct_year2001.html   and   dstruct_pre2001.html

The number of scaler channels in a bank is determined by the lowest 6 bits of the header. Some scaler have 16, others 32 channels. The helicity gated scalers are FIFO scalers run in G0 helicity mode, see also www.jlab.org/~rom/g0helicity.html.   Since there are two redundant scalers, there are two redundant copies of the helicity gated data.

Arrangement of  the Right   Spectrometer   Updated Jan, 2003

Bank of 16 or 32 Channels	Header	Usage
1st  Bank	0xceb00020	1-6=S1-Left, 7,8=blank, 9-13 and 15 =S2-Left, 14,16=blank, 17-22=S1-Right, 23,24=blank, 25-29 and 31 =S2-Right, 30,32=blank.
2nd  Bank	0xceb20010	Left.AND.Right PMTs: 1-6=S1, 7,8=blank, 9-13 and 15 =S2, 14,16=blank.
3rd  Bank	0xceb30010	1-10=Gas Cerenkov PMTs, 11=GasC-sum
4th  Bank	0xceb50010	EDTM Scaler
5th  Bank	0xceb60010	Spare channels
6th  Bank	0xceb70020	Plus Helicity gated normalization data. Channels 1-32 follow the same map as channels 1-32 of the ceb8 scaler.
7th  Bank	0xceb80020	Normalization Data not gated by helicity   channel 1=T1, 2=T2, 3 Upstream(x10)BCM, 4=MLU-Strobe, 5=Hel+ pulses, 6=Hel- pulses, 7=Upstream(x3)BCM, 8=1024Hz-clock, 9=Unser, 10=Downstream(x3)BCM, 11=Downstream(x10)BCM, 12=blank, 13=TS-accept(IF R-arm TS), 14=Upstream(x1)BCM, 15=EDT-Pulser, 16=Downstream(x1)BCM. 18 = helicity transitions, 19 = QRT, 20 = MPS, 21 = LNE, 22=T3 from L-arm, 23=T4 from L-arm.
8th  Bank	0xceb90020	Minus Helicity gated normalization data. Channels 1-32 follow the same map as channels 1-32 of the ceb8 scaler.
9th  Bank	0xceba0010	1st channel = 100 kHz clock. Other channels unused at present.
10th  Bank	0xcebb0020	This is the 2nd FIFO scaler for helicity plus gated data.
11th  Bank	0xcebc0020	This is the 2nd FIFO scaler for helicity minus gated data.

The scalers are cleared at the beginning of a run, and read out finally at the end of the run. The timing of the gates, which enables the scalers to count, is such that the end-of-run scaler readout is done after gates are turned off, and then after this final readout the gates are re-enabled, so that scalers can count in between runs. The end-of-run scaler data are written automatically to a file on ~a-onl/scaler/scaler_history.dat. Routines exist for reading this file and pulling out the particular scaler channel.

Scaler data are also inserted into the data stream as an event type 140 typically every 4 seconds, but not synchronized to any other event. The capability to read scalers at 30 Hz with zero deadtime is also supported, and these data appear as ROC10 and ROC11, see www.jlab.org/~rom/scaler_roc10.html for details. This allows us to run with the helicity delayed in G0 helicity mode. See also www.jlab.org/~rom/g0helicity.html.

•   BPM and Raster Data (ROC14 or ROC15)

The BPM/raster data are in fastbus. See the detector map in ~adaq/detmap/detmap.config.

One issue is that one must obtain good pedestals for BPMs. This is done by turning the beam off and asking MCC to set the BPMs to forced gain with gain setting zero as follows:

-- Open BPM window "BPM Diagnostics - SEE"
-- From there pull down the "Expert Screens" and open "Gain Control"
-- For IOCSE10 (this might change) use the pull down window
   and change from "auto gain" to "forced gain".   Then change the 
   Forced Gain values x and y to zero.
-- Do 10 min CODA run with ps8=4.
-- Remember to put back "autogain" when you are finished.  

•  EPICS   INFORMATION

Data from various EPICS databases are periodically inserted into the datastream, as well as placed in text files that are saved at the start-of-run and end-of-run.  These text files are stored as, for example, ~a-onl/epics/runfiles/Start_of_Run_1047.epics, and End_of_Run_1047.epics where 1047 is the run number. These data are also written to the electronics log book HALOG.   Each experiment is permitted to modify the script that controls this (ask me) but please let only ONE experienced, qualifed person modify the script (and it is probably good enough, too).  Incidentally, no other dealing with DAQ should ever be modified without asking me. I leave things open but I must be informed.

The data which are inserted periodically include the beam current, position monitors,  magnetic fields in our spectrometers, as well as some beamline elements, the beam energy, collimator positions, and several other things. Approximately every 30 seconds, a long list of information from the accelerator and from Hall A are inserted as event type 131.  Approximately every 5 seconds a shorter list of information, for which rapid updates are desirable, are inserted as event type 131.  The shorter list contain the BPMs near the target and the charge monitor data.  The synchronization of the data relative to our datastream is good to perhaps 1 second.  

Here is an fictitious example of the rapidly updated EPICS data which are inserted every approximately 5 seconds :

Tue Aug 27 12:59:43 EDT 2002
IPM1H04A.XPOS                  0.352823
IPM1H04A.YPOS                  0.430828
IPM1H04B.XPOS                  -0.130145
IPM1H04B.YPOS                  -0.48034
hac_bcm_average                5.29884
hac_bcm_dvm1_current           5.30776
hac_bcm_dvm2_current           5.28991
hac_unser_current              5.30134

There is some code in the C++ analyzer and ESPACE to pull out the data. However, since the EPICS "event" data are characters they are viewable simply with grep. From a Linux box you can type "grep -a IPM1H0 e01012_1455.dat.0" on the data file, and from SunOS you don't need the "-a".   grep is case sensitive unless you use "-i" option (so "grep -ai filename", etc).

•  SPECIAL   EVENTS

Event types less than 15 are physics triggers of various types.  There are also prestart, go, and end "events".  Event types greater than 127 are "special events", which are not really "events" and not particularly "special" either -- they are data inserted into the CODA datastream which are useful to record with the event data.  The following special events exist:

Specially Inserted Events

Event  Type	Format	Description
131	character	EPICS  Data  (see section above)
133	character	Prescale Factors  (decoded by ESPACE)
135	character	Detector Map (optionally used by ESPACE)
136	character	Trigger  Setup  (read it and see)
140	integer	Scaler Data (each spectrometer every few seconds)