Difference between revisions of "DVCS how to"
|Line 34:||Line 34:|
* The shift leader or TO can help.
* The shift leader or TO can help.
= Beam Recovery Procedure =
= Beam Recovery Procedure =
Revision as of 07:59, 21 September 2010
Back to DVCS
(This page is obviously in progress -- please contribute to it)
- 1 ioc_portserver_reboot_locations
- 2 Troubleshooting
- 3 Shift-worker responsibilities
- 4 Beam Recovery Procedure
- 5 Data Acquisition
- 5.1 Basic Data Acquisition with CODA
- 5.2 Triggers, Scalers, and Dead Time
- 5.3 Cold Start Checklist (when beam has been off for several hours)
- 5.4 Starting CODA From Scratch
- 5.5 How to Take a Pedestal Run
- 6 Data analysis
- 7 Spectrometer detectors
- 7.1 How to change the momentum setting of HRS
- 7.2 Changing momentum setting of the septum magnets
- 7.3 How to reset the quadrupoles
- 7.4 How to reboot HRS magnet IOC
- 7.5 What/where to check VDC
- 7.6 How to set / reset phototube high voltage
- 7.7 How to reset the VDC high voltage
- 7.8 How to start xscaler (HRS) / view the scalers
- 8 Beam Line
- 9 Compton Operation
- 9.1 Running the Compton DAQ
- 9.2 Compton Stripchart
- 9.3 Checking the Offline Analysis
- 9.4 Compton Troubleshooting
- 10 Slow control
- 11 Miscellaneous
- Follow the runplan and communicate with RC or experts whenever problems happen that cannot be solved by shift workers.
- Maintain data taking quality and an efficient use of beam time.
- Follow the directives in the COO
- Log the following in a shift summary:
- run list (describing the goal of this run: eg production on LH2, BMC calibration ...)
- all major events including accesses.
- Watch the target, see Target Info
- Since being TO is easy, please help the shift leader and third person.
- Start and stop the DAQ.
- Do online replay of all production runs;
- Compare replay histograms with the sample ones and report to shift leader any unexplained differences. Halog them.
- The shift leader or TO can help.
Beam Recovery Procedure
Procedure for major beam down: more than 2 hours
Update: June 25, 2010.
This plan will need to be updated after beam is first established for APEX!
Check if the septum on !! (It gets turned off for access, sometimes people forget.)
Make sure we have APEX Tune. (If someone has turned off HRS magnets they may not come back in right state.)
- Empty target, establish correct beam position
- 5 uA empty target
- BPM4A (X,Y) = ( -6.5, 2.5) mm
- BPM4B (X,Y) = ( -6.5, 2.5) mm
- See white board for updates on these.
- Tolerances on these : +/- 0.3 mm
- Establish Raster (Sometimes gets turned off for access, sometimes people forget.)
- Still at empty target.
- All scint HV off (normally they are).
- Need to use T8 pulser, so use ps8 = 2 on L-HRS.
- spot_L check of raster, see 6x4 mm2 on target
- Put in BeO target, for visual check of spot (if camera still works)
- 10 uA rastered beam on BeO.
- May need to request MCC for rapid beam-on / beam-off to see spot.
- "Rapid" can be done by turning on/off laser at polarized source.
- Put in the Carbon hole target: the one labeled 'Carbon Hole' on the Hall A Cryotarget GUI.
- Run spot_L again, this time with ps8=65535 and ps1=?
- Observe the raster current pattern, see the carbon hole
- Flat means we're not hitting a frame !
- Gentle slope due in X is presumably form factor (don't worry)
- For reference, see halog 314132
Procedure for short beam down or target movement.
less than 2 hours of no beam, and no changes to beam
- IMPORTANT: Raster always on and 6x4 mm (observed on target) when Lead / Diamond target in beam !!
- If you are unsure of raster status, check it !!
- You can be sure raster is ok if
- You were just running and it was on.
- Check the scope in middle room (familiar TV pattern of raster)
- Look at raster EPICS screen.
- To check raster:
- Go to empty target
- Ask for 5 uA
- Do a spot_L run with ps8=2 as explained above.
- Put in Carbon Target:
- The one labeled 'Carbon' on the Hall A Cryotarget GUI.
- Ask for 3 uA. Check beam positions.
- BPM4A (X,Y) = ( -6.5, 2.5) mm
- BPM4B (X,Y) = ( -6.5, 2.5) mm
- See white board for updates on these.
- Tolerances on these : +/- 0.3 mm
- If you didn't do spot_L in the first step, do one a quick one now.
- Want 6x4 mm on target.
Basic Data Acquisition with CODA
This assumes runcontrol is already up. If not, go to Starting CODA From Scratch below.
Use the configs (these may change):
TWOARM == All Fastbus crates. This is the one you should normally use.
LeftHrs == Two Fastbus crates on L-HRS. ROC3 = lower crate, ROC4 = upper crate. RightHrs == Two Fastbus crates on R-HRS. ROC1 = lower crate, ROC2 = upper crate.
Note: Cannot run these simultaneously ! (Temporary setup)
pedrun == for generating the pedestal files for pedestal suppression.
Data appears in
/adaql2/data*/apex__NNN.dat.0 where NNN is the run number
You can find the last run with the command "lastrun"
Triggers, Scalers, and Dead Time
Setting prescale factor
The prescale factors are in
The comments at the top tell you which number corresponds to each trigger. Edit the string at the bottom of the file to modify the prescale settings.
Starting xscaler (HRS) and checking trigger rates
- Use hapc5 which is the PC on the top at the far right of counting room console. Normally xscaler will be running, but if not, log on as adaq, type "xscaler" and do what is commanded of you.
- If you need to start xscaler type: xscaler.
How to download and check the trigger
NOTE: I (Bob) am tempted to delete this section because for APEX all the trigger is done in NIM. The programmable CAMAC modules are not used. Therefore the instructions below are irrelevant for APEX.
To change the trigger:
- Logon to an adaq Linux box like "adaql1" or "adaql2" as user "atrig".
A self-explanatory graphical user interface pops up and you should do what it says and press "Download" and WAIT for it to finish and do not press Ctrl-C. The user should look for suspicious error messages in the window from which trigsetup was launched, e.g. to check if connection to the crate is ok.
- More details are described in Bob Michaels' page Loading and Checking the Hall A HRS Trigger.
How to check the dead time
If it is not already running, type datamon in a terminal on adaql2 in the a-onl account.
Definitions of DVCS triggers
- T1 = L-HRS S2m
- T2 = L-HRS S0
- T3 = R-HRS S2m
- T4 = coincidence of T1 and T3
- T5 = L-HRS singles from a coinc of S0 and GasCer.
- T6 = T4.and.GasCer on R-HRS (added later, see halog 329311).
- T8 = 1024 Hz pulser
Cold Start Checklist (when beam has been off for several hours)
- Confirm target is “empty target”
- Call for pulsed beam for tuning
- Tune beam to beam dump (straight through, no Compton chicane)
- Call for 10μA beam
- establish beam position
- check beam charge monitors
- check beam position monitors
- Confirm HRSs set to correct value
- Move target to THIN Carbon position for detector check out
- For detector checkout: current = 0.5μA
- Check out detectors at different currents
- Turn on raster
- Take short run with raster on
- Confirm raster size with spot++
- Resume runplan
Starting CODA From Scratch
You do not need to do this routinely - only if you are starting from a fresh login or after a computer crash. Log in a a-onl account on adaql2 ( check with Run Coordinator in case you don't know the password )
(Contact: Bob Michaels)
On one workspace you will see six xterms with names of the crates in
the title, e.g. "ROC4--hallasfi4--hatsv40-port6".
From each of the six xterms, you should telnet into the appropriate RS232 interface, e.g. for the above, "telnet hatsv40 2006" (if the prompt in the xterm is "
->", it is already connected). The "hallasfi4" is the internet name of the crate's cpu.
If the xterms are missing, type "setupxterms".
From these xterms, you reboot by typing "ctrl-X". Yes, you hold down the Ctrl key, then hit X. You can also type reboot but "ctrl-X" is better. You can also reboot from the crate resets GUI on HA Tools.
Open a terminal and type startcoda.
The runcontrol window will appear: File:Startcoda.png
Hit the "Connect" button, then "Configure" (choose a configuration e.g. LeftHrs), and "Start Run", or, to troubleshoot, "Download", "Prestart", and "Go".
How to Take a Pedestal Run
If you've been running the normal spectrometer DAQ configuration, you'll need to press "Abort", then "RunTypes" and select the PEDRUN configuration. Download, StartRun. Run for about 5000 events, then EndRun. You may check the pedestal files in ~/ped (pedN.dat corresponding to ROC #N). See README there for more details. After pedestal run, change back to the running configuration.
NOTE: DONT try to use PEDRUN for anything other than pedestal determination -- its confusing since the prescale factors are in a different file, etc. If you want a run with pedestal suppression turned off, see the README.
login to adaql4, adaql5, adaql6 or adaql7 with user adaq
> analyzer onlana.C
Once the analyzer has worked its way through the data, look at the data with the following commands:
- .x online.C+("detectorL", NNNN);
- .x online.C+("calo", NNNN);
- .x online.C+("physics", NNNN);
where NNNN is the run number.
Batch offline analysis
As soon as CODA files go to /mss, a batch job is submitted to the JLab farm to decode each file. Outputs are store under /mss/home/munoz/DVCS2/1stprod. A copy of the last 100 files is kept in the work disks under /work/halla/dvcs/disk1/1stprod.
Starting the Port Server Windows
On adaql2 type : start_monitors
this will open 6 xterms with in the title on which portserver to log :
so for example to log on hatsv4 port 3 type :
telnet hatsv4 2003
How to change the momentum setting of HRS
From the HAC window, there is a P0 SET field for each spectrometer in which you can enter the desired momentum setting. To change to a lower momentum from a higher previous setting, you just need to enter the new momentum value. To raise the momentum setting, you need to cycle the Q2 and Q3 magnets first and then enter the desired momentum setting. See the instructions for the cycling procedure on the instruction sheet to the right of the white board in the counting house or click here.
Changing momentum setting of the septum magnets
- On the Hall A Main Menu, click Magnet Controls and select BIGBOX Controls.
- Septum magnet is set the same as PREX, i.e. 729 Amps for 1.063 GeV/c or 685.7949 Amps/GeV/c. (n.b. Imax = 800 Amps).
- Enter Setpoint in Amps. The APEX production setting is 775.06 Amps.
- Click Power On if necessary.
- Note: Both septa are driven by the same power supply.
How to reset the quadrupoles
The reset for Q2 is simple, on the front of the power supply there are red lights on when the power supply is tripped. At the bottom of the board that has the red light there is a blue reset button. There may be more than one light on so there may be more than one blue button that needs to be pushed. Push blue buttons until all of the red lights are out. Once all of the lights are out, lower the large lever on the lower right side of the power supply and lift it back into position. this will reset the Q2 and Q3 power supplies locally. For Q1, it's simpler : there is a large sign in the back of the Q1 equipment rack on the power supply balcony that says the blue reset button is here. you press the button and the magnet resets.
How to reboot HRS magnet IOC
Go to the hall A main menu screen.
Click "IOCS". Note that the left HRS magnet IOC's (iocha14) are accessible at http://hareboot10 and the right HRS magnet IOC's(iocha16) are accessible at http://hareboot25 (Username and password for IOC reboot are available in the counting house). Close the "Hall A IOC Systems" window. To reboot:
- the left HRS magnet IOC's, go to http://hareboot10 and from the pull-up menu next to iocha14, select "immediate reboot".
- the right HRS magnet IOC's, go to http://hareboot25, click on control, select iocha16, from the pull-up menu select "immediate reboot".
After the IOC reboot, please make sure that the spectrometer-angle information is reentered into the Hall A General Tools GUI.
What/where to check VDC
VDC status can be checked on the HAC Main Control Window. High Voltages and Gas Flows are constantly being monitored.
How to set / reset phototube high voltage
From the Hall A Main Menu on hacsbc2, click on LeCroy HV. Choose the appropriate spectrometer, and then the appropriate detector. Usually the voltages are already plateaued, and you should only need to turn HV on. All voltage numbers are in Volts.
How to reset the VDC high voltage
Bring up the HAC control window per the above instructions. In the "Miscellaneous" section, click on the "Crate Resets" button. This will bring up the "Hall A Slow Controls" window. In the bottom left and right of this window are the reset buttons for the spectrometer subsystems. Click on the relevant "VDC high voltage" reset button. You then need to go back to the main HAC GUI and reset the tripped VDC to 4 kV by entering 4 in the VDC HV window. If the VDC trips again, reset the supply then try to ramp up the voltage slowly. If there are repeated trips, call an expert.
How to start xscaler (HRS) / view the scalers
start xscaler :
- Go to the hapc5 terminal in the middle of rack CH01A09. If it is not logged in, log in as user "adaq"
- Type ssh adaq@adaql4 and enter the password when prompted.
- Type goxscaler
- Type ./xscaler
to view the scalers :
- The simplest : login to adaq l1 as adaq
- Type xscaler
- Follow what it says
NB : you must be in the correct directory since there are 2 versions of xscaler : the new is ROOT GUI (the previous one is Calvin Howell's GUI)
- Make sure that the 12V box on the screen "HRS hadron systems FPP carbon doors" are always on (HallA mainmenu, FPP button, carbon doors).
How to Check the Raster (spot)
- Log onto "adaql2" as user "adaq"
- Type spot++ from any directory
- After checking the spot for a run make changes with MCC according to the needed spot size (note: MCC units are NOT in mm).
Running the Compton DAQ
- To start the compton DAQ, log onto compton@compton and type:
> coda reboot
- The correct CODA configuration is FADC_Prod
- To see the ROC window, open another terminal on compton@compton and type:
> telnet hatsv5 2005
- Start runs with the Start Run button, and stop them with the End Run button. Compton runs do not end automatically, and should last about one hour.
- At the end of the run, a grey box will appear. Type a short description of the run into the box (i.e. "production") and the hit the Submit button.
- It is important to monitor the Compton data online with a stripchart.
- To open the Compton stripchart, open a terminal on compton@compton and type:
> StripTool HappexIIICompton.stp
- The stripchart should look like:
- Purple line: Beam current (uA). In the figure you can see that it has tripped a few times.
- Yellow line: Vertical beam position. This should remain roughly constant, although it fluctuates during dithering cycles. A beam trip sends it to the bottom of the chart (position 0).
- Red line: Cavity power. This should be a square wave: the cavity is locked in the right polarization state, then turns off, then locks in the left polarization state, then turns off, then repeats. The powers for the right and left states appear different but are actually the same. The power should stay comfortably above 2kW. (Cavity may not lock during extended beam trips.) If the cavity is not locking, contact a Compton expert.
- Green line: Current in the photon detector PMT base. This should remain very constant during stable running. It will go to zero (the top of the chart) if the HV trips off. When this happens the HV needs to be turned back on.
- Pink line: Scattered photon rates not normalized to beam current.
- Blue line: Scattered photon rates normalized to beam current in the detector. As in the figure, these should track the cavity on/off states; if you can tell from the blue line alone whether the cavity is on or off, then you are probably getting good Compton data. If not, see the Compton Troubleshooting section
Checking the Offline Analysis
- The offline analysis takes about 45 minutes to run and begins after you hit the submit button.
- The analysis results appear in the Compton Logbook. The run number is on the far left, and the link to the html page brings up the analyzed plots.
- If the data is good production data, the plots should look somewhat like this.
The Compton rate (the blue line on the stripchart) flatlines or goes to zero;
This is a problem because it means that we are not getting good data
- Is there beam? If the beam has tripped, the rate should be zero and there is no problem.
- Is there a run going? If the run has ended the rate goes to zero. Start a new run.
- Has the high voltage tripped? If the green line on the stripchart has gone to zero, the high voltage needs to be turned back on.
- The Compton high voltage is in the beamline crate, card 11, channel 1. If this channel is at zero, and channels 7 and 8 are both on, channel 1 needs to be reset. This is done by turning the channel off and then back on again.
- If the entire crate has tripped off (all three channels in card 11) use the HV/OFF and then HV/ON buttons at the top of the HV GUI.
- If the high volgate needs to be reset make a HALOG entry, including if it was the crate or the channel which tripped off.
- If it's tripping over and over again, contact a Compton expert.
- Is the DAQ running?
- Sometimes when CODA crashes the photon rate flatlines. This means the rate does not change at all for quite some time (although the normalized rate will change during beam trips even if the DAQ has crashed).
- CODA needs to be rebooted. You must end the run submit the run. Then open a terminal on compton@compton and type coda reboot. To run CODA, hit the connect button, then the configure button. Choose the FADC_Prod configuration, download, and then you can go back to running like normal.
CODA is giving a ROC 6 error;
- If CODA is giving a ROC 6 error and the ROC 6 window is not responding, the ROC needs to be rebooted
- This is done by going to hareboot4 in a web browser. The username is hlauser and the password is written on the wall of the counting house. Go to outlet 6 and choose Immediate Reboot and then click the apply button on the bottom of the screen. Wait a few minutes until the ROC is rebooted and then re-download the FADC_Prod configuration in CODA.
The laser (the red line on the stripchart) is always zero;
This is a big problem because it means that there is no laser to Compton scatter off of, and we are therefore getting no data.
- This requires an expert; call a Compton expert.
If something is wrong, and you don't know how to fix it, contact a Compton expert!
How to bring up HAC control window (Hall A Tools)
The machine which runs the HAC GUI is hacsbc2. The monitor is located in the Hall A Counting House in rack CH01A02, just below the NMR dipole oscilloscopes. Click here to see the screen.
Open up a terminal (make sure you are logged in as hacuser, see the paper at the upper right corner of the whiteboard for password) and type NewTools at the prompt. This will open a window with one button: "EOS menu". Left-click on the button and select "EDM (HLA Main)". A separate window entitle "Hall A Main Menu" will appear. One of the entries is labeld "Tools". Clicking on the "Tools" button opens the "Hall A General Tools" screen, which is often referred to as the HAC window.
High Voltage controls
[We do not use the right arm]: For the right arm, login in an adaq computer as adev, and launch the program hvgui_vnc.
For the left arm and beamline, go in the Lecroy item in Hall A EPICS menu
For the DVCS calorimeter:
The HV control should be already running and you should be able to interact with it through a VNC server.
- connect as adev on any adaql machine,
- type "hvgui" to start the vnc window.
- at the top of the HV gui, select "maps" then DVCS for a human friendly display
In case the HV control is not running, you need to restart it through the VNC server wrapper:
- connect as adev on adaql10,
- type "start_hvgui".
For more information about the HV control for the DVCS calorimeter see HV_CALO
LED table control
The LED driver is located in the DVCS IOC. To access it:
- open a terminal, and type the command: "telnet iochacs"
- load the LED control library by typing: "ld < NewLedLib.o"
Then it is possible to control the led. Few commands:
- To switch ON the LED corresponding to a particular block: SetBlock(columnNbr, rowNbr)
- To scan several blocks of the calo with the LEDs: ScanCalo(MaxColumnNbr, MaxRowNumber, Number)
If no halog guis are up and running, log in as user "adaq" on any "adaq.." machine. Type halog a yellow and blue and gray window will appear Enter text in the large message area, your name in the User area, and a title in the Keywords area. Remember that experiments tend to suffer from too little documentation, not too much. Also, halog entries allow people to remotely check on the experiment progress without bothering you with phone calls. When complete, after pressing the "Make entry" button, the text in the message center to the lower right corner of the window should read 'Entry Complete', otherwise something is wrong. It often takes several minutes for entries to show up in halog. Entering graphics: Click 'Grab Screen' in the bottom button row of the halog window. The halog window will disappear, and the window the cursor is in will become the active window, its outline flashing. By moving the cursor you can select any window to be selected, by clicking on the left mouse button. You also can select an arbitrary part of the screen by holding down the left mouse button and dragging the cursor along. All the time a rectangle will flash that indicates the selected area. Releasing the mouse button will then cause this screen area to be included into the entry. It will show up as a thumbnail picture in the halog window, where you can discard it or accept it, and then make a second screen grab. The images are in GIF format.
How to bring up the Alarm Handler
From the "toolbox" icon in the taskbar, press the "Alarm Handler" label. Alarms are indicated by a beep and the alarm handler button flashing. The color red indicates values far out of range, while the color white indicates a communication failure. Clicking on the alarm handler button will bring up an alarm handler list. A detailed list is given in the left panel, and clicking on items here brings up a more detailed list in the right panel. The alarm can be acknowledged and silenced by clicking on the button beside the alarm indicator.
You can plot time dependencies of EPICS variables
Pressing the "toolbox" icon and launch "StripTool", which enables you to plot any EPICS variable(s) versus time. To find out the name of a particular EPICS variable, click on it with the middle mouse button on the MEDM GUI.
How to bring up the Beamtime Time Accounting Table
- Logon to "adaql1" as user "adaq"
- type "bta"
See http://hallaweb.jlab.org/equipment/daq/abu.html for more information.
What/where to check target temperature and pressure
The target temperature and pressure is displayed on the target GUIs running on cryotarg as well as on the camera monitors in the rack (CH01A06) left of the target computer. Look at the camera monitor on the top right of rack CH01A06. It displays the temperatures for the three loops on three white temperature controllers, with loop 1 at the top and loop 3 on the bottom. Ask the TO for help if you can't find the temperature or it doesn't make sense. The adjacent camera displays the JT valve controller readings and tachometers.