Compton DAQ Dev

From Hall A Wiki
Jump to: navigation, search

Here are some notes on the project of developing the Compton counting mode DAQ. The original work was done mainly by Bryan Moffit and Ben Raydo in 2020, with input from others.

It is foreseen to be a precurser of a moller polarimeter DAQ as well as a counting-mode DAQ for the MOLLER project, etc.

See earlier work

Budget and Plan

          Budget and Plan

 Upgrade of Compton and Moller polarimeter DAQ in halls A and C.

 The 4 systems are approximately the same.  The FADC will be modified
 to support both counting mode and integrating mode.  

 Item      Price each      Qty       Line cost

 VXS crate   16.5 k        4         66 k
 VTP         10 k          4         40 k
 FADC (was not in budget)
 cpu (ATOM)   3 k          4         12 k
 SD           2 k          4          8 k
 scaler       4 k          4         16 k
 VETROC       2 k          8         16 k
 TI           2 k          4          8 k
 I/O          1.5 k        4          6 k
 Spares                              43 k
 TOTAL                              215 k$
It's essentially 43k$ per DAQ (times 5 since we need spares)
except we must scrounge the FADCs.  We'll buy the first system
for the hall C moller in FY2023.  This 43k$ is in the FY2023 budget.
After that we may need to refine the design.  Then in FY2024 we'll
purchase/build 3 more DAQs.  Finally, buy spares in FY2025.


VTP manual

Compton Trigger

Current Test Setup

updated R. Michaels Oct 12, 2022

Here is a picture

The basic configuration of the DAQ

VTP-Compton DAQ config:

ROC1 :


How to Run the Current Setup

This is a VXS crate located in the TEDF building cubicle 11. The crate is at the top of the rack. The crate was formerly installed in hall A during PREX/CREX.

Currently (Oct 12, 2022) using a 50 Hz front-panel pulser trigger on TI. Later want to modify this to use a trigger from VTP based on data from FADC and/or the VETROC.

updated Jun 8, 2023. Bryan Moffit

How to login to the two ROCs and start the ROC

ssh -X amoll1 -l sbs-onl
ssh -X havtp1 -l sbs-onl

How to start CODA on atedf3. Username adaq.


Readout Lists:

  • VTP
  • ROC1

Module configuration files:

  • VTP - Compton firmware
  • FADC250

VTP library update

Jun 8, 2023 - Bryan Moffit

The VTP library was updated to the current version in github.  Older version were moved to 

Updated the LINUXVME environment variables for havtp1 in

 export LINUXVME=${HOME}/comptonDaq/vtp

Built and installed vtp library
 cd ~/comptonDaq
 git clone -b devel-NPS
 cd vtp
 git rev-parse --short HEAD
 make install

Built and installed vtpFirmwareLoad program (used at boot)
  cd ~/comptonDaq/vtp/firmware
  make install

Built and installed DiagGui server 
  cd ~/comptonDaq/vtp/DiagGui
  make install

Confirmed vtpFirmwareLoad configuration file:

 havtp1            fe_vtp_hallb_z7.bin    fe_vtp_halla_v7_compton.bin 


Updated the VTP payload enables for modules missing from slot 15 and 16 (in ~/comptonDaq/havtp1.cnf):

 #        slot: 10 13  9 14  8 15  7 16  6 17  5 18  4 19  3 20
 #     payload:  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
 VTP_PAYLOAD_EN  0  1  0  1  0  0  0  0  0  0  0  0  0  0  1  0

 # Before update:
 VTP_PAYLOAD_EN  0  1  0  1  0  1  0  1  0  0  0  0  0  0  1  0

Ran the DAQ using Bob's instructions and was able to readout all modules, including VTP. The trigger rate appears to be 100Hz. 

Updated the VETROC configuration file (needs to use full hostname):
 amoll1:~/vetroc/ -> amoll1.cnf

Archive of Problems

Oct 12, 2022.

If I drop the VTP from the readout and remove tiRocEnable(2)
from amoll1:/home/sbs-onl/comptonDaq/rol/vtpCompton_list.c
then the DAQ runs fine and both the FADC and VETROC data
make sense.

Next I added the VTP.  Still a FP trigger.  It downloads
and prestarts. But here is where the problems start. 

Dec 3, 2022

For several weeks I had bus errors.  Chasing this down involved looking a lot at
the library (vtpLib), the readout list (CRL), the test codes, how the firmware
gets loaded, etc.  In a way, it was a good thing it happened because I learned
a lot.  In the end, it was a simple problem.  I was using a version of vtpLib
which was too modern for the firmware, so the library tried to access registers
that were not yet defined by that firmware.  The solution is get the correct branch
from github and pick a consistent set of firmware, library, test code, and readout
list.  These all have to "hang together" and be consistent.  Well, duh !

That was the main lesson.  And here are some smaller lessons.

One can find the address of the bus errors by logging into the serial interface
(uses minicom in our test setup).  
The address is printed in square brackets [0x43C02000], see vtpLib.h for what the address means,
this example is pointing to DMA transfer
and it means generally something is mismatched between the library vtpLib and the firmware, and
specifically in this case you are running the DMA test code in ./test directory but the 
DMA capability is not built into that version of the firmware.  

A "bus error" occurs when the program is compiled with a different (wrong) versions of the VTP library.

There are two firmwares for 2 chips on the VTP.  One is Z7 and the other is V7.  Z7 handles the CODA roc.
The coda_roc it can be a hardware roc (used by GEMs for high-speed readout) or a software roc (was used 
by Compton).  A 3rd development for Z7 is streaming readout (not used in hall A yet).  The other chip (V7) 
is used for Compton specific application and registers.

These firwmare are specified in 

It looks like our current problem is that we probably picked up the newest (or relatively new) library
but this is not compatible with the firmware for software roc on Z7 and maybe not for Compton on V7 chip.

There are various vtpLib available.

1. Compton version
2. hall B compton version
3. devel-NPS -- the latest as of today
4. probably others (?)

A goal is to have one library work for all firmware.  

Two possible solutions. 

1. Obtain the version of the Compton vtpLib library (instead of the newest lib)  
   This is what I did, and it worked.

2. (longer term) Try a "devel-NPS" version and compatible firmware -- Ben and Bryan will pursue this.

Even though solution 2 is preferred, solution 1 was a valuable learning point.

See Current Problems instead. The problem ended up being simple. For the time being I leave this here, although it's pretty much noise.

Specific problem with vtpLib. This affects both the Compton version and the HCAL version. I think if I fix this I will understand the rash of "Bus error"s and "Segmentation Fault"s when running CODA.

This code causes a Bus Error.  Actually lots of code causes Bus Error 
but let's focus on this one.  BTW, the Chip 0 FW type is 15

vtpGetFW_Type(int chip)
  int rval=0;

  if(chip == 1) {
      printf("here 1111\n");
      rval = vtp->clk.FW_Type;
// NOTE !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
// The following line never happens because of a Bus Error.
      printf("here 2222  chip %d   rval %d\n",chip,rval);
      rval = vtp->v7.clk.FW_Type;

  return rval;

Running vtpStatus causes a Bus Error when asking Chip 1 for the vtp->clk.FW_Type

[sbs-onl@havtp1 test]$ ./vtpStatus
 size = 131072
	spi mode:      0x0
	bits per word: 8
	max speed:     781250 Hz (781 KHz)
	 ---------- Checking VTP register map ---------- 
vtpLock: WARN: Previous owner of VTP (mutex) died unexpectedly
  Attempting to recover..
vtpInit: Setting up VTP PLL for 250MHz VXS reference
si5341_checkBasePart: ID=5341
si5341_readStatus: status = 00, OK
	SYSINCAL      0, OK
	LOSXAXB       0, OK
	LOSREF        0, OK
	LOL           0, OK

vtpV7PllReset: PLL successfully locked
vtpV7PllReset: PLL successfully locked
vtpV7PllLocked: PLL successfully locked
here 1111
Bus error

Comment re: status of vtpLib

There is not much difference between the Compton version and
the HCAL version.  Mainly printf statements and serdes checks.
In both the Compton version and the HCAL version, the vtpLib
was missing vtpEbBuildTestEvent(int len) and vtpEbReset().
Such as it was, the ./test codes would not build (undefined reference).
I found these subroutines elsewhere and put them in by hand.

Nov 9, 2022 (Nov 10 below)

Current status, globally, is that without the VTP the DAQ runs with a front panel trigger at 50 Hz and reads
the FADC and VETROC. Data looks ok. Adding the VTP causes problems.


Problems today (Nov 9, 2022)

1. There are complaints that I cannot vtpSetCompton_EnableScalerReadout
using firwmare 15.  Probably the least of my problems.

2. Something wrong with VTP Serdes links.  

It takes several seconds to time out.
Waiting on links: PP1 PP2 PP3 PP4 ... (etc)
vtpSerdesCheckLinks: ERROR - all serdes links not up!

3. What library to use ?

vtpEbResetFifo defined in /mnt/SBS/coda/3.10_arm/Linux-armv7l/lib/
but not in /home/sbs-onl/comptonDaq/vtp/
I'm told not to use /mnt/SBS/coda/3.10_arm/Linux-armv7l/lib/

Pulling the libvtp from the SBS HCAL setup, I see that it
is also quite different.  Which one to use ?

Generally, the fact that the readout code I got referred to routines that
are not in my libraries, I assume I must have a mismatch of readout code
and library from the Compton polarimeter.

4. Bus errors for (I think) any code with the string "Ti" in it.  

See "Bob1" in the readout list to show the lines that are commented out.

It looks like any code with the string "Ti" in it (presumably tries to talk 
to the TI) causes a bus error.   

5. To USE_DMA or not ?

Different behavior whether I do
#define USE_DMA
//#define USE_DMA

Whether I USE_DMA or not ...

With all the lines commented out that causes bus error, 
the DAQ goes into an active state but usually gets no events
i.e. rocTrigger() is not entered.

The DAQ does go through all the transitions without problems.
if those lines are commented out.
Again, see "Bob1" in the readout list.

Update Nov 10, 2022

I got the test code in 

to compile using, I think, the recommended vtpLib and header,
and no mixing of versions.  Aren't the codes supposed to just
work ?  Unfortunately, depending on the code, the either do a 
"Bus error", a "segmentation fault" or report an error like
vtpV7SetReset: ERROR: VTP not initialized

Chip #0 reports the firmware type is 15 (what we want, right ?)
Chip 0  FW type 15
But when I try to address Chip #1 it does a Bus error,
specifically on this line
rval = vtp->clk.FW_Type;
and never makes it to the next line.

I like the idea of these test codes.  You can learn a lot,
like how the configuration works (that seems to work).
I know everyone is busy with more important tasks, but it
does seem fair to ask for a hardware VTP to go along with a
working set of test codes.  However, maybe I screwed them 
up somehow.


Analysis Software

Nothing here yet. Work in progress. But there was software for the DAQ used during PREX/CREX.

Old Results