Summary of discussion on organising test vectors

This note is an attempt to summarise a discussion we had at Birmingham (on 4 Feb 2002) around the issues of how to organise the test vectors for the forthcoming tests.

This is my view/memory/understanding of the discussions. Please let me have any corrections before I try to convert it into a software note.

The discussion was focussed around an email exchange before the meeting. I have also used that to frame this summary.

Introduction: what is a test?

Generating the input test vectors

• various patterns and random data generated from a simple syntax in the Bill Stokes style.
• physics-like signals from offline trigger simulation
• analogue pulses from electronics simulation or measurements
• specific (non-physics like) data which tests all aspects of FPGA algorithms, trigger logic etc.

We will also want to cover the complete phase space of the system. Typically this can only be done very sparsely for most of the phase space, but we will need to make more detailed studies across significant boundaries.

Another aspect, which overlaps with the next section, concerns the connectivity of the system. We dont want to populate inputs which are not connected to readout.

Storing/describing the input test vectors

• although we could generate some input test data on the fly, we will almost certainly want to be able to store it and reuse it. Some data, eg measured analogue test pulses is only stored in files and wont be generated.
• the mechanism for storing test vectors needs to include the knowledge of the module(s) involved. A test vector might be applicable only to a module in a particular slot, or could be applied to any module in the crate, or might contain different sets of data for different slots, etc.
• the stored test vector specification may need to specify that some modules in the crate are to be loaded with test data while others (or parts of others) run normally - perhaps being passed test data loaded into other modules.
• the stored test vector may need to specify that special calibration and trigger menu files must be used with this test vector. In that case, the test vector specification may include pointers to already generated expected outputs.
• the stored test vector (for tests involving RODs) will need to specify the pattern of L1As and its relationship to the phase of the data in the playback memories.

Simulating/storing the predicted output data

• the simulation needs to have access to the input test vector and the complete loaded state of the modules, ie which calibrations, trigger menus, control register settings will be loaded into the real hardware for the test. We should probably assume that things like timing setup are done correctly, but BCID parameters and lookup tables in the PPMs will need to be known.
• the simulation also needs to know the connectivity of the system, ie which PPM is connected to which CPM/JEM, how the CMMs are interconnected, how the ROD inputs are outputs are assigned, etc.
• the simulation of readout data needs to know the pattern of L1A generation.
• it may be possible to do the simulation on the fly at the start of a run, but for repetitive tests it will be desirable to be able to store and reuse the expected test vector outputs.
• in general, except for very simple tests, the simulation needs to know the predicted outputs from one module in order to predict the outputs of modules to which it is connected. These modules may be in different crates, so it would be difficult to do the simulation in the distributed run controllers (which know the state of the module and which calibrations etc should be loaded into them). Instead the simulation will normally have to be done all in one place.
• but if the expected outputs are only required by monitoring programs and dont need to be loaded into hardware, it should be fairly easy to do the simulation at the start of the monitoring program.

Calibration and Trigger Menus

• These will generally be stored as files (or collections of files).
• Simple examples, with all channels the same, could be produced on the fly for some tests.
• For calibration runs, it will be useful if the calibration objects can be told to increment through their allowed range of values. These calibration runs will also require appropriate test vectors to be loaded.

View from module/submodule code

• the module and submodule objects in our Module Services package, and their equivalents in the Simulation package, will need:
  • the desired state of the (sub)module, ie register settings.
  • objects containing the calibration data and trigger menu.
  • the data stream (if any) to be loaded into playback memories.
• the calibration is foreseen to be structured hierarchically with containers for module and submodule calibration data of different types (timing, BCID, etc).
• the test vector data also needs some structure, ie ability to select the data for a particular channel of a module (or to be told that channel is not to be set into playback mode). This test vector data stream may contain data for a series of tests in a long sequence run.
• the hardware configuration, calibration and test vectors should, as far as possible, follow the same hierarchical organisation. Certainly as seen through the API, and where possible in the organisation in database and files.

Questions and suggested solutions

What can the database do?

A: Probably not. The OKS database allows you to create a top level file which reads in various other files. These other files could be different calibrations, different trigger menus, different input test datasets etc. This would allow all these to be reused in different ways in different tests.

Q: Could we stored calibrations (or larger collections of data) as a default version together with a set of updates?

A: This is how the future "Conditions Database" will probably work, but for the moment it will be easier to have separate whole files.

How do we specify the generation of L1As?

The TTCvi takes the ORBIT signal as an input, so we can make the full orbit a convenient multiple of 256 (eg 2560 which is close to the actual number of bunches per complete orbit). This will make it easier to maintain synchronisation between the module playback memories and the DSS L1A generation. Then the DSS could hold 12 complete orbits.

However even if we only set one L1A in a complete 32K slices, that still represents a minimum trigger rate of order 1kHz. To reduce the rate further would require a firmware modification in the DSS to allow a pause of N complete orbits after each M slices output from the DSS. The output buffer must also wraparound automatically (without software intervention) in order to keep in phase with the module playback memories.

How do we cope with BUSY?

• Tests without BUSY. Here we ensure that the system runs slowly enough that BUSY is never asserted. In this mode we can use DSS modules to automatically check the ROD data is correct.
• Tests with BUSY. If BUSY may be asserted we cant use the DSS. However, given a pattern of L1As and their predicted readout data, BUSY has the effect of suppressing one or more of the expected L1As. So a monitoring program can simply skip to the next actual event if the expected event is not present.

How long does the simulation take?

A few seconds is probably the psychological limit for on the fly simulation. If it takes longer, we will want to store the predicted outputs. However it would be acceptable for on the fly simulation to be used the first time, if the required combination of input data, calibrations etc has not yet been simulated. After that, the stored values can be reused.

How do we describe the connectivity?

The simulation includes the processor backplane, so the database only needs to contain information about cable links. It should include the following types of connection.

• LVDS cables. For convenience of setting up databases it should be possible to specify connections of groups of LVDS cables from one PPM to one CPM/JEM, but it must also be possible to specify the connection of individual cables.
• Glink cables to RODs.
• Slink cables to the ROS.
• Cables between CMMs. All possibilities must be permitted for test purposes.
• BUSY signals from RODs (and between BUSY modules).

Summary

• Tests will be specified by an overall database file tagged by name. The file should also contain a brief description.
• Any required test can be selected by name from the run control GUI.
• The top level database file should specify the names of the other files it needs, ie files of generated input data, calibration files, module settings, trigger menu, hardware configuration including all the cable connections and the files of simulated output data.
• It should be possible to run the simulation on the fly if the required output files dont yet exist.
• The run controller hierarchy must be suitably configured to ensure this happens (in one place) before any of the necessary files are required by other run controllers.
• The pattern of L1As will be generated using the DSS.
• The structure of both calibration and test vector files should follow the hierarchy of the hardware, ie division into crates, modules, submodules, etc.
• The API should present all the data required by a module or submodule via a single object.
• The calibration objects should support stepping through permitted values.

Outstanding points and plan of action

• The discussion should be written up and circulated [ML].
• The database proposals should be completed and documented [ML].
• List of data required in the database should be provided. Eg first draft for CMM [NG], check existing stuff for CPM [SJH].
• List of all required tests should be prepared!
• Mods to DSS firmware for L1A generated are required.