APEX Shift How-To |
|
Special Procedures for Modified Beamline
ioc_portserver_reboot_locations
Troubleshooting
Shift-worker responsibilities
Shift Leader
- Follow the runplan and communicate with RC or experts whenever problems happen that cannot be solved by shift workers.
- Maintain data taking quality and an efficient use of beam time.
- Follow the directives in the COO
- Log the following in a shift summary:
- run list (APEX DAQ runs).
- all major events including accesses.
Target Operator
- Watch the target, see Target Info
- Since being TO is easy, please help the shift leader and third person.
Third person
- Start and stop the DAQ.
- Do online replay of all production runs;
- Compare replay histograms with the sample ones and report to shift leader any unexplained differences. Halog them.
- The shift leader or TO can help.
Beam Recovery Procedure
Procedure for major beam down: more than 2 hours
Update: June 25, 2010.
This plan will need to be updated after beam is first established for APEX!
Check if the septum on !! (It gets turned off for access, sometimes people forget.)
Make sure we have APEX Tune. (If someone has turned off HRS magnets they may not come back in right state.)
- Empty target, establish correct beam position
- 5 uA empty target
- BPM4A (X,Y) = ( -6.5, 2.5) mm
- BPM4B (X,Y) = ( -6.5, 2.5) mm
- See white board for updates on these.
- Tolerances on these : +/- 0.3 mm
- Establish Raster (Sometimes gets turned off for access, sometimes people forget.)
- Still at empty target.
- All scint HV off (normally they are).
- Need to use T8 pulser, so use ps8 = 2 on L-HRS.
- spot_L check of raster, see 6x4 mm2 on target
- Put in BeO target, for visual check of spot (if camera still works)
- 10 uA rastered beam on BeO.
- May need to request MCC for rapid beam-on / beam-off to see spot.
- "Rapid" can be done by turning on/off laser at polarized source.
- Put in the Carbon hole target: the one labeled 'Carbon Hole' on the Hall A Cryotarget GUI.
- Run spot_L again, this time with ps8=65535 and ps1=?
- Observe the raster current pattern, see the carbon hole
- Flat means we're not hitting a frame !
- Gentle slope due in X is presumably form factor (don't worry)
- For reference, see halog 314132
Procedure for short beam down or target movement.
less than 2 hours of no beam, and no changes to beam
- IMPORTANT: Raster always on and 6x4 mm (observed on target) when Lead / Diamond target in beam !!
- If you are unsure of raster status, check it !!
- You can be sure raster is ok if
- You were just running and it was on.
- Check the scope in middle room (familiar TV pattern of raster)
- Look at raster EPICS screen.
- To check raster:
- Go to empty target
- Ask for 5 uA
- Do a spot_L run with ps8=2 as explained above.
- Put in Carbon Target:
- The one labeled 'Carbon' on the Hall A Cryotarget GUI.
- Ask for 3 uA. Check beam positions.
- BPM4A (X,Y) = ( -6.5, 2.5) mm
- BPM4B (X,Y) = ( -6.5, 2.5) mm
- See white board for updates on these.
- Tolerances on these : +/- 0.3 mm
- If you didn't do spot_L in the first step, do one a quick one now.
- Want 6x4 mm on target.
Data Acquisition
Basic Data Acquisition with CODA
This assumes runcontrol is already up. If not, go to Starting CODA From Scratch below.
Use the configs (these may change):
TWOARM == All Fastbus crates. This is the one you should normally use.
LeftHrs == Two Fastbus crates on L-HRS.
ROC3 = lower crate, ROC4 = upper crate.
RightHrs == Two Fastbus crates on R-HRS.
ROC1 = lower crate, ROC2 = upper crate.
Note: Cannot run these simultaneously ! (Temporary setup)
pedrun == for generating the pedestal files for
pedestal suppression.
Data appears in /adaql2/data*/apex__NNN.dat.0 where NNN is the run number
You can find the last run with the command "lastrun"
Triggers, Scalers, and Dead Time
Setting prescale factor
The prescale factors are in
/adaqfs/home/a-onl/prescale/prescale.dat
The comments at the top tell you which number corresponds to each trigger. Edit the string at the bottom of the file to modify the prescale settings.
Starting xscaler (HRS) and checking trigger rates
- Use hapc5 which is the PC on the top at the far right of counting room console. Normally xscaler will be running, but if not, log on as adaq, type "xscaler" and do what is commanded of you.
- If you need to start xscaler type: xscaler.
How to download and check the trigger
NOTE: I (Bob) am tempted to delete this section because for APEX all the trigger is done in NIM. The programmable CAMAC modules are not used. Therefore the instructions below are irrelevant for APEX.
To change the trigger:
- Logon to an adaq Linux box like "adaql1" or "adaql2" as user "atrig".
- Run
trigsetup
A self-explanatory graphical user interface pops up and you should do what it says and press "Download" and WAIT for it to finish and do not press Ctrl-C. The user should look for suspicious error messages in the window from which trigsetup was launched, e.g. to check if connection to the crate is ok.
- More details are described in Bob Michaels' page Loading and Checking the Hall A HRS Trigger.
How to check the dead time
If it is not already running, type datamon in a terminal on adaql2 in the a-onl account.
Definitions of APEX triggers
- T1 = L-HRS S2m
- T2 = L-HRS S0
- T3 = R-HRS S2m
- T4 = coincidence of T1 and T3
- T5 = L-HRS singles from a coinc of S0 and GasCer.
- T6 = T4.and.GasCer on R-HRS (added later, see halog 329311).
- T8 = 1024 Hz pulser
Cold Start Checklist (when beam has been off for several hours)
- Confirm target is "empty target"
- Confirm that septum is off
- Call for pulsed beam for tuning
- Tune beam to beam dump (straight through, no Compton chicane)
- Call for 10uA beam
- establish beam position
- check beam charge monitors
- check beam position monitors
- Turn on septum, confirm HRSs set to correct value
- Move target to THIN Carbon position for detector check out
- For detector checkout: current = 0.5uA
- Check out detectors at different currents
- Turn on raster
- Take short run with raster on
- Confirm raster size with spot++
- Resume runplan
Starting CODA From Scratch
You do not need to do this routinely - only if you are starting from a fresh login or after a computer crash. Log in a a-onl account on adaql2 ( check with Run Coordinator in case you don't know the password )
(Contact: Bob Michaels)
Reboot crates
On one workspace you will see six xterms with names of the crates in
the title, e.g. "ROC4--hallasfi4--hatsv40-port6".
From each of the six xterms, you should telnet into the appropriate RS232 interface, e.g. for the above, "telnet hatsv40 2006" (if the prompt in the xterm is "->", it is already connected). The "hallasfi4" is the internet name of the crate's cpu.
If the xterms are missing, type "setupxterms".
From these xterms, you reboot by typing "ctrl-X". Yes, you hold down the Ctrl key, then hit X. You can also type reboot but "ctrl-X" is better. You can also reboot from the crate resets GUI on HA Tools.
Start CODA
Open a terminal and type startcoda.
The runcontrol window will appear: File:Startcoda.png
Hit the "Connect" button, then "Configure" (choose a configuration e.g. LeftHrs), and "Start Run", or, to troubleshoot, "Download", "Prestart", and "Go".
How to Take a Pedestal Run
If you've been running the normal spectrometer DAQ configuration, you'll need to press "Abort", then "RunTypes" and select the PEDRUN configuration. Download, StartRun. Run for about 5000 events, then EndRun. You may check the pedestal files in ~/ped (pedN.dat corresponding to ROC #N). See README there for more details. After pedestal run, change back to the running configuration.
NOTE: DONT try to use PEDRUN for anything other than pedestal determination -- its confusing since the prescale factors are in a different file, etc. If you want a run with pedestal suppression turned off, see the README.
Data analysis
OnlineGUI
login to adaql4, adaql5, adaql6 or adaql7 with user adaq
> goonlana
> ./replayL run_number -------- Left HRS onlineGUI
> ./replayR run_number -------- Right HRS onlineGUI
Pedestal Scripts
Some of Sergey's scripts for pedestal and prl are also found on the adaql4,5,6,7 machines.
login to adaql4, adaql5, adaql6 or adaql7 with user adaq
> goolana
> cd ../sergey/scripts
> leadglass/left_fit1.C -------- prl1_adj
> leadglass/left_fit2.C -------- prl2_adj
> leadglass/left_fit.C --------- ped_left.txt & ped_right.txt
Starting the Port Server Windows
On adaql2 type : start_monitors
this will open 6 xterms with in the title on which portserver to log :
so for example to log on hatsv4 port 3 type :
telnet hatsv4 2003
Spectrometer detectors
How to change the momentum setting of HRS
From the HAC window, there is a P0 SET field for each spectrometer in which you can enter the desired momentum setting. To change to a lower momentum from a higher previous setting, you just need to enter the new momentum value. To raise the momentum setting, you need to cycle the Q2 and Q3 magnets first and then enter the desired momentum setting. See the instructions for the cycling procedure on the instruction sheet to the right of the white board in the counting house or click here.
Changing momentum setting of the septum magnets
* On the Hall A Main Menu, click Magnet Controls and select BIGBOX Controls. * Septum magnet is set the same as PREX, i.e. 729 Amps for 1.063 GeV/c or 685.7949 Amps/GeV/c. (n.b. Imax = 800 Amps). * Enter Setpoint in Amps. The APEX production setting is 775.06 Amps. * Click Power On if necessary. * Note: Both septa are driven by the same power supply.
How to reset the quadrupoles
The reset for Q2 is simple, on the front of the power supply there are red lights on when the power supply is tripped. At the bottom of the board that has the red light there is a blue reset button. There may be more than one light on so there may be more than one blue button that needs to be pushed. Push blue buttons until all of the red lights are out. Once all of the lights are out, lower the large lever on the lower right side of the power supply and lift it back into position. this will reset the Q2 and Q3 power supplies locally. For Q1, it's simpler : there is a large sign in the back of the Q1 equipment rack on the power supply balcony that says the blue reset button is here. you press the button and the magnet resets.
How to reboot HRS magnet IOC
Go to the hall A main menu screen.
Click "IOCS". Note that the left HRS magnet IOC's (iocha14) are accessible at http://hareboot10 and the right HRS magnet IOC's(iocha16) are accessible at http://hareboot25 (Username and password for IOC reboot are available in the counting house). Close the "Hall A IOC Systems" window. To reboot:
- the left HRS magnet IOC's, go to http://hareboot10 and from the pull-up menu next to iocha14, select "immediate reboot".
- the right HRS magnet IOC's, go to http://hareboot25, click on control, select iocha16, from the pull-up menu select "immediate reboot".
After the IOC reboot, please make sure that the spectrometer-angle information is reentered into the Hall A General Tools GUI.
What/where to check VDC
VDC status can be checked on the HAC Main Control Window. High Voltages and Gas Flows are constantly being monitored.
How to set / reset phototube high voltage
From the Hall A Main Menu on hacsbc2, click on LeCroy HV. Choose the appropriate spectrometer, and then the appropriate detector. Usually the voltages are already plateaued, and you should only need to turn HV on. All voltage numbers are in Volts.
For the right arm:
- Login as adev to adaql10.
- Type start_hvgui
- If you get a message that the gui is already running, type:hvgui
How to reset the VDC high voltage
Bring up the HAC control window per the above instructions. In the "Miscellaneous" section, click on the "Crate Resets" button. This will bring up the "Hall A Slow Controls" window. In the bottom left and right of this window are the reset buttons for the spectrometer subsystems. Click on the relevant "VDC high voltage" reset button. You then need to go back to the main HAC GUI and reset the tripped VDC to 4 kV by entering 4 in the VDC HV window. If the VDC trips again, reset the supply then try to ramp up the voltage slowly. If there are repeated trips, call an expert.
How to start xscaler (HRS) / view the scalers
start xscaler :
- Go to the hapc5 terminal in the middle of rack CH01A09. If it is not logged in, log in as user "adaq"
- Type ssh adaq@adaql4 and enter the password when prompted.
- Type goxscaler
- Type ./xscaler
to view the scalers :
- The simplest : login to adaq l1 as adaq
- Type xscaler
- Follow what it says
NB : you must be in the correct directory since there are 2 versions of xscaler : the new is ROOT GUI (the previous one is Calvin Howell's GUI)
- Make sure that the 12V box on the screen "HRS hadron systems FPP carbon doors" are always on (HallA mainmenu, FPP button, carbon doors).
Beam Line
How to Check the Raster (spot)
- Log onto "adaql2" as user "adaq"
- Type spot++ from any directory
- After checking the spot for a run make changes with MCC according to the needed spot size (note: MCC units are NOT in mm).
Slow control
How to bring up HAC control window (Hall A Tools)
The machine which runs the HAC GUI is hacsbc2. The monitor is located in the Hall A Counting House in rack CH01A02, just below the NMR dipole oscilloscopes. Click here to see the screen.
Open up a terminal (make sure you are logged in as hacuser, see the paper at the upper right corner of the whiteboard for password) and type NewTools at the prompt. This will open a window with one button: "EOS menu". Left-click on the button and select "EDM (HLA Main)". A separate window entitle "Hall A Main Menu" will appear. One of the entries is labeld "Tools". Clicking on the "Tools" button opens the "Hall A General Tools" screen, which is often referred to as the HAC window.
High Voltage controls
For the right arm, login in an adaq computer as adev, and launch the program hvgui_vnc.
For the left arm and beamline, go in the Lecroy item in Hall A EPICS menu
Miscellaneous
Starting HALOG
If no halog guis are up and running, log in as user "adaq" on any "adaq.." machine. Type halog a yellow and blue and gray window will appear Enter text in the large message area, your name in the User area, and a title in the Keywords area. Remember that experiments tend to suffer from too little documentation, not too much. Also, halog entries allow people to remotely check on the experiment progress without bothering you with phone calls. When complete, after pressing the "Make entry" button, the text in the message center to the lower right corner of the window should read 'Entry Complete', otherwise something is wrong. It often takes several minutes for entries to show up in halog. Entering graphics: Click 'Grab Screen' in the bottom button row of the halog window. The halog window will disappear, and the window the cursor is in will become the active window, its outline flashing. By moving the cursor you can select any window to be selected, by clicking on the left mouse button. You also can select an arbitrary part of the screen by holding down the left mouse button and dragging the cursor along. All the time a rectangle will flash that indicates the selected area. Releasing the mouse button will then cause this screen area to be included into the entry. It will show up as a thumbnail picture in the halog window, where you can discard it or accept it, and then make a second screen grab. The images are in GIF format.
How to bring up the Alarm Handler
From the "toolbox" icon in the taskbar, press the "Alarm Handler" label. Alarms are indicated by a beep and the alarm handler button flashing. The color red indicates values far out of range, while the color white indicates a communication failure. Clicking on the alarm handler button will bring up an alarm handler list. A detailed list is given in the left panel, and clicking on items here brings up a more detailed list in the right panel. The alarm can be acknowledged and silenced by clicking on the button beside the alarm indicator.
You can plot time dependencies of EPICS variables
Pressing the "toolbox" icon and launch "StripTool", which enables you to plot any EPICS variable(s) versus time. To find out the name of a particular EPICS variable, click on it with the middle mouse button on the MEDM GUI.
How to bring up the Beamtime Time Accounting Table
- Logon to "adaql1" as user "adaq"
- type "bta"
See http://hallaweb.jlab.org/equipment/daq/abu.html for more information.
What/where to check target temperature and pressure
The target temperature and pressure is displayed on the target GUIs running on cryotarg as well as on the camera monitors in the rack (CH01A06) left of the target computer. Look at the camera monitor on the top right of rack CH01A06. It displays the temperatures for the three loops on three white temperature controllers, with loop 1 at the top and loop 3 on the bottom. Ask the TO for help if you can't find the temperature or it doesn't make sense. The adjacent camera displays the JT valve controller readings and tachometers.