Software workshop at QMUL on 2-3 March 2005

Present: Bruce, Dave, Eric (part time), Florian, Gilles, Murrough, Norman, Stefan, Steve.
By phone (part time): Dimitri, Richard.

The meeting agenda consisted of a number of talks interspersed with much discussion. These minutes contain some brief notes from the discussion, mainly concentrating on any consensus (where it emerged) rather than blow by blow points. In the minutes I have followed the agenda order even though some talks were rearranged. Some comments in italics are my thoughts after the meeting. There are some actions at the end.

Bruce: Module Services Overview

Points arising from the discussion of this talk and some points common to all modules (but actually raised after other talks) were as follows:

• the talk mentioned a few shortfalls in module services, but it was felt this was basically the code we will use in 2007.
• regbuild classes and use of read/modify/write: alternative solution (with in memory bit manipulation and subsequent write) is already implemented and used in ppmServices. Consensus to deprecate the old code - this will only really happen if we force people to change by removing the old API at some date.
• VME block transfers: not supported in VME-- (though basic DMA is possible). Check whether existing HDMC block read/writes are actually being used. We could optimise by adding some faster block read/write methods at various layers in our code.
• present software is based on existence of a module in the database. For some purposes it would be nice to be able to use module services (eg crate scan, simple tests) and specify the module address without editing database or using it at all. Also nice to be able to clone a module. Would probably be a significant change. Not really practical.
• for scans of the crate and finding whats there, can we rely on the module ID everywhere? (Except non-L1Calo modules).
• module services (composites) parts files cannot be saved. How much of a problem is this? New PartManager should allow move to new parts file syntax (eg XML) - but not a priority given our other tasks.
• for all firmware programs it would be useful to have a command (ie pulse register) to reset the firmware to its default powerup state - a different operation from reloading the firmware into the device.
• do we need our own LTP services? Or try using the ROD crate DAQ one and gain experience with that style? We will probably be expected to use standard software for LTP, RODbusy and even TTCvi and will have to feed in our requirements and suggestions.
• could add methods to moduleServices API to support calibration sequences at pause/resume by specifically named methods, so only the run control needs to know the pause/resume is being abused for the purpose of multistep runs.
• do all modules correctly set the isPresent status flag? Appears not to be the case in the IGUI anyway. Run control should issue MRS message if module is not present!

Florian: PPM

The main points from the talk included:

• estimate 1 minute per PPM to download LUTs: need to check if the optimal existing HDMC Register and VMEBus methods are being used. If they are, some optimisation will be needed.
• suggested different XML files for different run types with separation into different calibration sources and other configuration choices. Can map onto other databases but still want lists of registers.
• we will need cabling/connectivity tool
• how to communicate with calorimeters during calibration runs? Later suggestion (calibration talk) is dont communicate, agree complete sequence in advance and describe in database
• not clear if Preprocessor ideas match this: Norman and Florian to have further discussions
• one possibility mentioned was: (a) find coarse timing (eg readout pointers etc) via PPM standalone program, then (b) use calibration sequences for finer timing and other calibrations.
• the GlinkKicker used for CPM readout tests is generic and could also be useful for PPM Glink readout tests at Heidelberg and/or at Mainz (needs DSSes and appropriate daughter cards).

Gilles: CPM

The following points were raised:

• two types of test: those entirely confined to CPM test crate and those with communication to other crates (eg glink test)
• so far sequences entirely controlled by the kicker, no multistep runs
• plan to use DAQ software for production testing: would like smoother way to run a succession of tests. Suggestion, if all choices were in IS, could successively set IS variables and start runs
• however some tests need special firmware: this is only changed via HDMC. The possibility to have both standard and test firmware preloaded in flash and switch between them is not yet used.
• an extended run type description would need to specify which firmware to use for a test.
• more checks are now made on the firmware bit files before loading them. Similar code to make checks is used by other modules. Can this be done in common?
• presently publishing histograms to OH and analysing them from there - there are changes in the OH API in the latest TDAQ software which will affect both publishing and displaying
• recently used masks of connected LVDS cable to disable unused CPchip channels - disabling serialiser inputs would need a firmware change.

Stefan: JEM

The main points mentioned were:

• new JemToolbox class: may have some common features with classes from the kickerBase package: Bruce and Stefan to check and try to converge
• although all HDMC part pointers should be set after custominit, jemServices checks all pointers everywhere: is this overkill? Answer: no, its good practice but not everyone is so careful.
• as with the CPM, no selection of test firmware from preloaded configurations is yet done - not expected to be a problem with the SystemACE
• suggestion to speed up firmware download by a kind of VME broadcast was not well received: "dont violate the VME standard" (or whats left of it in VME-- anyway!)
• SystemACE is used in JEM, CMM and ROD: a common HDMC part or at least common layout of registers in the programming model would be sensible
• presently writing ROOT histograms directly from JemKicker but thinking of moving to CPM style publishing in OH (NB forthcoming change in API noted above)
• suggestion to put sin and cos values used in Ex, Ey calculations into the trigger menu database

Norman: CMM

Points arising from the talk included:

• some problems (firmware resets) still to be fixed in cmmServices
• reorganisation of the memories into HDMC parts might help diagnostics
• next version of the CMM (CMM2 vs CMM1) will have a different memory map - does this need separate module services (some of us hope not) or can the old and new memory maps be made uniform (or old updated to look more like the new one?)
• would like SystemACE part: common with JEM and ROD?

Bruce: ROD

The main points from the talk were:

• can we use only neutral format with 6U RODs in future?
  Yes at least for JEM: the Glink format is going to change.
  No definite answer noted for CPM or CMM-CP
• should we change JEM format before starting to use 9U ROD?
  I didnt note down a conclusion on that one
• development session for new rodServices next week
• some joint work with Steve/Murrough may be useful once that gets going
• more database settings almost certainly needed

Norman: Installation and Commissioning

Discussion of this talk overlapped with the next one. Points specific to this talk were:

• we should only use the 9U ROD for installation and commissioning
• will USA15 cabling (and long cabling?) be well tested before we try tests with long cables connected to calorimeters?
• we still need to write down more detailed steps of what exactly we will do in the installation tests

Norman: Calibration

This talk and the previous one generated a lot of discussion (as expected). Some of the main conclusions were:

• although both calorimeters are currently using a single run with no state transitions to perform their calibrations, we would prefer to use state transitions for our internal calibrations and timing scans
• we will still have to cope with the calorimeters though
• all calibration sequences, with or without calorimeters, should be specified fully with a "plan" (or run type) in the database. This should specify what both L1Calo and the calorimeter(s) should do or expect at each step. There is no possibility of dynamic communication with the calorimeter calibration systems during a run.
• currently all our kickers make VME accesses at each step. Our goal, if we can do this with ROD Crate DAQ, would be to have VME accesses only by the run control process (actually the IOManager in ROD Crate DAQ) and not by any standalone programs.
• we will need more investigation of ROD Crate DAQ (and some manpower to implement it) before we go down this route.
• information read from VME (by existing kickers or ROD Crate DAQ readout thread) should be published using online services for analysis by a separate program.
• publishing mechanism may be IS, OH or the monitoring service, assuming we can suitably package up data which is not in the form of event fragments.
• the separate analysis program (or yet another step) would then store results in the database.
• any separate standalone programs ("kickers",analysers etc) should always (whatever the run type) be started by standard online infrastructure, not by the run controller. Such programs may exit(0) if they are not required to do anything for the current run type.
• we want to store all the information used to derive a new calibration (eg timing scan histograms) as well as the final timing calibration values themselves. This may mean we end up storing more data than we have previously advertised in response to the endless questionnaires about conditions database usage.
• for calibrations where we scan our system through a sequence of settings, the database software should provide module services with the new settings for it to reconfigure the module (if needed).
• the module should not need to know there is a scan going on (if that can be avoided) though there may be cases where optimisation requires that it should be able to know.
• NB with the current "writeable OKS" database there may be a problem with locking the database if calibration objects are changed during a sequence - to be investigated
   
  
• and finally, since the above represent a considerable change from our current usage, we need further discussions before deciding that is definitely our future course. This should be done by email, maybe with another phone meeting, hopefully concluding by the time of the Mainz meeting.
• in particular we need more step by step (bullet point or pseudo code) description of calibration procedures and details database items required

Production Software Discussion

We didnt really have time to discuss this as a separate issue. The first significant module in production will be the CMM. We will need for production what we want for installation and comissioning: automated test sequences.

Bruce: Standalone Programs

There was considerable discussion about the varieties of our standalone programs. Some of conclusions were:

• we will continue to have simple standalone test programs some of which will be run outside the run control environment for debugging, quick tests etc. Not all module services have these but some do and will continue to do so.
• the programs we refer to as "kickers" are of more than one type:
  • some (CPM/JEM/glink kickers) are presently reading data from VME, may do some analysis of it and publish some results. They are generally started by the run controller
  • some (eg ros kicker) are getting data via the monitoring system and analysing/comparing that
• outcome of this and calibration discussions is that we would like to move towards the second, with all VME access done in the context of the run control process (IOManager in RCD) with no VME access by other programs. Existing CPM/JEM kickers would evolve to be analysis only in this architecture.
• independent of that, all these standalone programs (including test programs) should inherit a common framework (eg Kicker, KickDb) for configuration and operation.
  Later thought: if no kickers will access modules by VME is this less of a requirement than if kickers will still do VME readout from modules during calibration scans?

Murrough: Run Control, ROD Crate DAQ, Run Types

The main points from the discussion were:

• we will have to move, we cannot stay with what we have
• using RCD for readout of data from non-ROD modules eg for timing scans etc should be investigated
• dont try to move starting of processes via our existing run controllers into RCD, use standard online way
• we did not have time for a detailed discussion of what we need to have in a new run type description - we will need to agree on details fairly soon though

Murrough: Databases

Time for discussion was limited. As a very brief summary:

• the presentation covered the new LCG tools and compared with our current entirely OKS based database
• there was a general worry about the amount of work in moving to LCG style database from what we have now
• priorities: calibration data and run type "plan"

Steve: Monitoring

We did not have time for Steve to give his talk. People can look at the slides. Steve just highlighted a couple of points:

• the new monitoring tools should appear in the next TDAQ release (it was intended they be in the latest release but there were problems and they didnt make it).
• several groups are thinking about common bytestream decoding in both online and offline worlds, especially when new eformat appears and is really used everywhere.

We had hoped to append a discussion on system wide status tools and GUIs to this discussion, but that also didnt happen.

Murrough: Other Issues

There was also no time for this talk. One or two slides were shown at other relevant points in the meeting.

Conclusions, Workplan, Manpower

The conclusions in terms of our goals or view of calibration procedure are mainly listed under the calibration talk. We need more discussion before finalising this though. Two days were enough to exhaust people but not enough to cover all the things we needed to talk about.

Workplan

The major items are:

• database: move to LCG tools and databases
• ROD Crate DAQ
• "Kicker" infrastructure
• Calibration procedures

Manpower

• Stefan is now full time on Physics (online software only at weekends!)
• Richard will be coming to RAL: may work on calibration and perhaps CMM software?

Actions

• Florian/Norman: look at details of PPM installation/calibration steps and procedures
• Stefan/Norman/Bruce: look into common HDMC part and/or common register layout for controlling the SystemACE
• Bruce/Stefan: compare features from JemToolbox and kickerBase classes
• Gilles/Stefan: compare code to check firmware bit files with a view to extracting common features (also useful in PPM?)
• Murrough/all: circulate the rough consensus on our aims for calibration sequence framework and discuss by email before Mainz
• Florian: check actual calls used in PPM LUT download
• Module service authors: move to in-memory bit manipulation style of using regbuild classes - deadline to be agreed