Release Notes for P01-09-01 (last updated 11/04/2003):

The latest revision of this document is located here.

The previous release notes is located here.

Important Notes:

Configuration snapshots and their impact on application script organization:

With this release the setting up of the configuration of the system is required to be done during the Setup transition. In the past, configuration changes were allowed during the StartRun transition. The change is necessary to allow configuration snapshots to be correctly taken. The requirement applies to the trigger system configuration as well. Application scripts are consequently also required to restore the state of the system to the Setup configuration by the end of the application's portion of the StopRun transition in order for a subsequent StartRun to proceed correctly. The example scripts in %ONLINE_ROOT%\RunControl\applications have been updated to reflect the current organization.

Including the trigger system state in the configuration snapshots required the handling of the trigger system to be moved into Run Control. Scripts should select their trigger setup during the Setup transition, using the same technique as in previous releases. At appropriate times, Run Control will take care of enabling and disabling the triggers.

In order to accommodate scripts that need to modify the trigger state during their operation, two new methods of the GGLT class, enableMask() and disableMask(), are provided. Users can force a snapshot to be taken using the takeSnapshot() method of rcTransitions. This should only be done when triggers are disabled.

Installation:

• Upgrading to LATTE 1.9:
  • As usual prior to installation the old LATTE directories should be renamed to Online_Pxx-xx-xx and VxWorks_Pxx-xx-xx where xx-xx-xx is the previous release number. This allows the user to go back to an earlier release if there is a problem with the latest release.
  • After installation runControl.cfg file needs to be carried over to the new installation. If you have created a runControl.cfg in the RunControl directory, this needs to be copied to the new Online\RunControl directory. If runControl.cfg exists in another directory and you are using the "runcontrolmain -c" option to specify a specific configuration file location, then no need to move the file.
  • There is a syntax change between 1.7 and earlier releases regarding the user list in the configuration file. If you look at the runControl.cfg.sample file in RunControl directory, you'll see that under the [users] section, the id is now before the equal sign, this change was required to support full names with spaces in them. Please modify configuration files accordingly.
  • Also RunControl\runId.cfg file from earlier releases needs to be copied over to the new installation directory. The machine ids needs to be updated to ensure uniqueness. runId.cfg.sample file contains comments indicating machine id assignments for each subsystem. This scheme allows for 100 teststands per subsystem and 999999 runs per teststand. If RunControl fails to access the runId.cfg after loading the application script it will throw an exception.
     
• Added Tools directory to the installation.
   
• cfitsio installation will now reflect the version of the C library that accompanies it. So if the cfitsio.dll version is 2.470 then the installation will show the same version. cfitsio.dll can be found in %WINDIR%\System32 directory.

VxWorks Related:

This release contains the "New Delhi" release of Flight Software interface libraries which are documented here.

CAL VxWorks GTEM library:

It was recently discovered that the low level VxWorks driver code for the GCxx registers wasn't correctly reporting the error summary bit back to applications. This has been fixed with this release and may have as a consequence that exceptions are now thrown when tests under previous releases may have completed successfully. To test this behavior, use the following recipe:

1. Bugger command payload parity with GLAT.parity_tem_cmd_payload
2. Write a GCRC/GCFE register -- RC latches parity error
3. Put command payload parity back to normal
4. Read a GCRC/GCFE register -- RC sets error summary bit in response and read fails with GCAL_ERR_SUMMARY, -3001.

This test should work similarly for GAxx and GTxx registers, but may currently
not due to the particular revisions of the ASICs in use.

Qt and PyQt Related:

• This release requires and uses Qt 3.2.0 and PyQt 3.8. All the required libraries can be found in the DLLs directory.
   
• pyuic.exe is also provided in the same directory and should be used to generate python code from the .ui files. The best way of doing this is to run setupRunControlMain.bat before running pyuic. This way an older version of pyuic won't be used by mistake.
   
• If sub-systems are doing any GUI development using the Qt Designer, they need to download and compile Qt 3.2.x on their systems.

Documentation:

• Added sources in Tools, GOSED, Browser, Logger and ScmCfg directories to the Doxygen documentation.

Hippodraw:

• This LATTE release contains HippoDraw version 1.3.2.
   
• Please refer to the following links for documentation on this release:
     -  HippoDraw (html)
     -  HippoDraw Python Interface API (html)

Functional blocks and registers:

• Added the following pseudo deadtime counter registers to GTIC:
  • sat_cal_lrs_ctr_0
  • sat_cal_lrs_ctr_1
  • sat_tkr_lrs_ctr_0
  • sat_tkr_lrs_ctr_1
  • sat_tkr_lrs_ctr_2
  • sat_tkr_lrs_ctr_3
  • sat_deadtime_lrs_ctr
     
• Marked the SAT counters as read only.
   
• Marked GLT dataless commands as write only.
   
• Changed cached value assignment behavior so that even if the status is non-zero the cached value is set to whatever is returned from VxWorks. If you need to read the value of a register after a "LATInterfaceException" use the following example as a starting point:

  try:
      x = gcrc.status
  except:
      print gcrc.getStatus()
      x = gcrc.regs['status'].getCachedValue(bypass=1)
  print x

Environmental Monitoring Display:

• Added PDU voltage and temperature monitors.

RunControl:

• Added the capability to export data to an external site (see Export Functionality below for more information).
   
• Added the ability to set the font and style. Note that the font and style change affects sub-system user interfaces as well. LATTE users should keep this in mind when designing their own screens.
   
• Under the "Global" tab "TEM Power Down" button has been added.
   
• Added "Env Monitoring On" watch item to the status panel.
   
• Implemented context menu for the status panel to copy item value to the clipboard. To copy the value of a watch item in the status panel, highlight it, right click the mouse button and choose "Copy".
   
• Added the following event statistics as watch items to status monitor:
  • Failed Events
  • Failed Event Sends
  • Valid Events
  • Discarded Events
  • Length Mismatched Events
  • Unsent Events

  These statistics are reset to zero at startRun and can be monitored in real time. See the GLAT documentation for an explanation of these statistic counters.
   

• Visual preferences (font and style) are now saved to the config file.
   
• Added user list maintenance tab in preferences. It is now possible to add or delete users through the preferences.
   
• Added support for dumping run report parameters as header keyword/value pairs in the FITS file. If run report parameter is a list or dictionary, separate binary tables are created and appended to the end of the FITS data as separate HDUs. See the following screenshots for examples:
  • HDU Structure (gif)
  • Primary HDU Headers (gif)
  • Version Data HDU (gif)
  • Modules that failed verification HDU (gif)
     
• Added snapshot enabled flag, snapshot path, event rate, average deadtime to the status panel.
   
• Exposed getFileTimeStamp method that returns the filename postfix string in the format: %y%m%d%H%M%S
   
• Exposed takeSnapshot, readConfig and applyConfig methods to scripts.
   
• Remove zero length badEvent and errorEvent files at the end of the run.
   
• While a snapshot or run report is being generated, log messages that show progress is printed on the console window.
   
• Added the ability to change server and logging level on the fly.
   
• Added average deadtime to the run report.
   
• Adjusted TEM voltages in TEMPowerUp.py based on Dave Nelson's input.
   
• Fixed a problem where the last item was being removed from preferences.
   
• Added preferences  and GLT to the DBN node.
   
• Trigger enable and disable behavior is now modified. rcTransition now uses enableMask and disableMask methods to commit the cached value of the trigger mask. This is needed to be able snapshot the GLT properly.
   
• In __stopRun() TACK value is now saved so that the user's setting is not lost, if a run is repeated without a reset.
   
• In RunControl/applications scripts added support for calculating the event rate correctly using the new GLT behavior.
   
• Moved application selection file dialog to its own class. Added checkbox to the dialog to be able to reuse the current schema file. The user scripts can call the reuseSchema() routine to skip the schema file open dialog and use the current schema. Based on the checkbox from the application selection dialog the schema file open dialog can be skipped.
   
• Under the Help menu, added menu option to go to the Electronic Logbook web site.
   
• To set up event data multicasting with Run Control, go to the Edit->Preferences menu and enter the desired multicast address/group under "Data Distribution Server Location", e.g. Machine ID. Also, ensure that the Disable Server check-box is not checked.

Command Client & Server:

• Fixed GGEMSC register access
   
• Added handling of saturating LRS counters for deadtime measurement.
   
• If you are running in local mode, make sure to turn the snapshots off. If your schema has defined too many components it will take a long time to snapshot everything even in local mode.

Event Client & Server:

In previous versions of evtSvr, there was a problem that allowed events to be truncated. This has been fixed in this release. The problem was due to there sometimes not being sufficient buffering space in the VxWorks network stack when event data was written to the socket faster than it was read by the workstation. Consequently, data that didn't fit was being dropped on the floor, causing evtCli to lose its place in the event data stream. Whenever this happened a counter called discardedEventSends (accessible from both VxWorks and the GLAT node) was incremented.

The fix that was applied retries sending the remainder of the event until the send succeeds. Stress testing has shown no corrupt events to be seen by evtCli. The problem with this fix is that it can induce deadtime because subsequent events don't get read out from the hardware while the tight retry loop is running. If the buffering in the hardware fills up before the events are read out, deadtime is asserted.

EBF:

EBF version 3.1.6 is included with this release. See release notes "EBFrelease.txt" in EBF directory.

The file %ONLINE_ROOT%\EBF\test\EBFdump.py has been updated to be capable of connecting to the event data multicast stream. To understand EBFdump, read from the bottom up. The function testRaw_Dist() serves as a simple example of how to create a Data Distribution client. The function takes one argument, the multicast group. Again, the machine ID can be used for this. If the server is running, doing python -i EBFdump.py and entering testRaw_Dist(<id>) at the python prompt (using the same <id> the server is using), should cause events to be dumped to the screen.

GOSED:

GOSED is the GLAST Online Single Event Display. It is based on the same structure as EBFdump.py. Currently, GOSED is set up to display only CAL and/or TKR data for one (selectable) tower. The data can come from either raw data (.arch) files, FITS event data files or from the event data multicast stream. Most features are pretty self explanatory. As noted above, to use GOSED in the multicast data reception mode, it must be listening at the same address on which the server (RunControl) is sending. There may be multiple instances of RunControl on a given subnet. For an instance of GOSED to distinguish between the data from each of the servers, a common address must be entered for both the server and the client. To enter this in GOSED, pull down the Tools->Server Preferences... menu. Under "Server Selection" enter the mutually agreed on server address, e.g. machine ID number. You can find it in Run Control by pulling down the Edit->Preferences menu and select the System tab. Use the value found under "Data Distribution Server".

• GOSED can now access FITS files.
   
• Server preferences are now saved to the gosed.cfg file

Testsuite:

• Modified test scripts to confirm to the new GGLT behavior.

Schema and Configuration:

• Divided up the envMonSchema.xml to its schema and configuration files (envMonSchema.xml and envMonConfig.xml). The config file gets included from the schema file now. If the user scripts need to access registers that require conversion constants, they can <include> envMonConfig.xml in their own schemas.
   
• Marked some of the GTIC registers as read only so that they will not be updated during an applyConfig of a snapshot.
   
• Allowed the configuration section to have its egus, constraints, rules and declarations.
   
• Added readConfig method, to be able to read separate config files into the existing lat.
   
• Only write non read-only registers as specified in gLAT.py.
   
• Continue to apply configuration even though some hardware components may be missing.
   
• Added optional configName and configVersion parameters to takeSnapshot.
   
• Snapshot file format is now compliant with the configuration file format.
   
• Added support for taking snapshot of the GLTtrigger mask and registers.
   
• Added checks to ignore certain register accesses when creating the snapshot.

Multicasting of Event Data:

Run Control supports the distribution of event data through multicasting. The Python classes that handle this are in a file called %ONLINE_ROOT%\Tools\DataDistributor.py. The reason multicasting is interesting for data distribution is that the amount of work the server CPU has to do is always the same, independent of the number of clients.

In order for multicasting to work, a multicast group must be chosen. A multicast group is a specialized IP address as described in http://www.iana.org/assignments/multicast-addresses. These IP addresses are not unique to a particular computer. From previous experience I chose to use values in the range 239.255.1.0 to 239.255.1.99.

At SLAC (and I'm guessing, by default), network routers are set to limit multicast traffic to a subnet. Since there may be multiple servers on a given subnet, a method is needed to distinguish between servers. We suggest to make the selection criterion based on the MachineId found in the runId.cfg file (also see %ONLINE_ROOT%\RunControl\runId.cfg.sample). MachineIds can run from 0 to 999, where the hundreds digit signifies the SubsystemId. So there are 10 possible subsystems, with 100 possible unique LATTE instances in each subsystem. (Sites and subsystems are expected to maintain this uniqueness themselves, to the extent that it is interesting. The MachineId is also used to distinguish which machine produced a data file, since Run numbers are tagged with this value.) The machine number is thus used to pick the specific multicast group to use.

Currently, the multicast group selection described above isn't enforced in any way, so if a server (e.g. RunControl, see Edit->Preferences->System) and a client (e.g. GOSED, see Tools->Server preferences) have the same multicast address, the system will work. RunControl's serving of the data can be turned off (and should be turned off if it's not going to be used) through a "disable" check-box on the preferences->system page.

We are expecting to multicast various types of data in the future, but for now only event data is supported. A client interested in the event data would instantiate one of two client classes: DataDistributorClient or DataDistributorPoll. Which one depends on whether the client is interested in all of the data, or only some of it. We currently include two clients with the LATTE release: the event display GOSED and EBFdump, see below. Since event displays are meant for human consumption, both uses DataDistributorPoll to pick an event out of the multicast stream only occasionally. The reason that Poll exists is to make sure that the socket stays drained, otherwise a bunch of data will be buffered up in the computer's network stack which is sequential, rather than being able to periodically take a sample of what's in the stream.

Note that multicasting is done using the UDP protocol. This means that there isn't a guaranteed delivery of the data or even a part of the data. Thus, an application can't depend on seeing all of the data and must also deal with being able to handle "corrupted" data, as in the case when the first packet of an event is lost but the second one shows up.

When creating a data distributor client, two types of information are needed: the type of data desired, and the address it is being served on. The event data is chosen with the name 'LAT_EBFdata'. The data type name is passed to the DataDistributorClient or DataDistributorClientPoll constructor, which uses it to translate it to an IP port number on which the data is to be served.

Since a constructor can't, in principle, fail, the setting up of the socket is left to the connect() method. connect() takes some form of address to open a socket. This address can take one of three forms:

• A unique number that identifies the server (both client and server must know this value). We suggest (and may at some point enforce) that the unique MachineId, discussed above, is used for this.
• <port>@<multicast address>, where <port> is a value that will override what is found from the data type name translation discussed above, and <multicast address> is some valid multicast IP address starting with the value 239. A name may be used for the <multicast address> value as well, as long as DNS can resolve it and it translates to a valid multicast IP address.
• <multicast address>, as discussed in the previous bullet. The port value is taken to be the one found from looking up the data type name in the port table.
   

The only reason why the latter two options might be useful is when the choice of port and/or multicast address chosen by the DataDistributorAddress class conflict with other applications running on the LAN subnet. Note that there is in principle no harm in having multiple servers multicasting on the same port/multicast address combination. The client will see the traffic from both sources. The client has access to the address information of the sender on each message received, so it can, in principle, filter on a particular sender, if it desires.

Corresponding to the connect() method is a disconnect() method. It takes no arguments and is used to gracefully shut down the client side of the socket.

To receive the data into a program, call the receive() method. It also doesn't take any arguments. It returns a list containing the data received and the sender's identification string.

The DataDistributorPoll class inherits from the DataDistributorClient class, so it is set up in the same way. Additional methods are start() and stop(), to start and stop the polling (the socket draining), respectively. To get the last polled data result, use the read() method.

There are no start up ordering issues between client and server. A client can be started before a server. The server can be restarted while the client continues to run. It will simply reconnect.

Export Functionality:

Data organization: All related files are now organized together in their own directory. The name of that directory is the timestamp of the run. For example, the data directory on the test lab machine contains the following directories, among others:
031009170702/ 031009171621/ 031009172246/ 031009173135/
031009173506/ 031009174312/ 031009175200/ 031009180208/
031009181107/ 031009181915/ 031009182633/ 031009183346/
031009184049/ 031009184737/ 031009185631/ 031009195134/
031009211847/ 031009213019/ 031009234742/

Each of these is the Run Control assigned timestamp of the run.
looking inside two of these:
031009170702:
    FullTower.xml
    MinitowerNoisyStrips-BadLadder.xml
    ebf031009170702.fits
    rcReport.out

031009172246:
    FullTower_diag.xml
    MinitowerNoisyStrips-BadLadder.xml
    ebf031009172246.fits
    rcReport.out

Each directory contains the schema (FullTower.xml), the additions to the schema (MinitowerNoisyStrips-BadLadder.xml), the FITS file and a copy of the relevant entry of the rcReport.out database. If the user script wishes to add additional files to this list, the programmer would use the addExportedFile() call, documented below.

Data export: The ability of Run Control to automatically export run related data to external storage has been implemented. In the 'Paths' tab of the preferences panel is a new text box to input the destination directory, and on the 'Options' tab is a radio box to enable/disable the exporting. If exporting is enabled, run control will attempt to copy the run directory to the destination directory at end of run. Note: The destination directory must be a fully qualified path.(e.g. V:\Data\fits)

New APIs. All are called from userApplication via self.<foo>

• addExportedFile(fn): takes 'fn', a fully qualified filename as input and appends it to a list of files to be copied to the destination directory defined above.
• getArchiveFile(), getBadEventsFile(), getErrorEventsFile(), getFITSfile(), getHSKfile(), getSchemaConfigFile(): Access to filenames used in Run Control. These names can be added to the run report. Not all of these are new, but are listed for completeness.
• getRunId(), evtCnt, getBadEvents(), getErrorEvents() Access to other run control information. Not all are new, but are listed for completeness.

Version Information:

(Retrieved by running versions.bat while in /Online directory)
The modules ending with _FW denote COMM board firmware and the modules ending with _HW denote COMM board hardware revisions.

Added cfitsio version. User scripts can retrieve it by executing cfitsio.__version__

Module	Version
EBF	3.1.6
GAEM	2.1.2.1
GAEM_FW	3.1
GAEM_HW	1.1
GGLT	1.3.6.0
GGLT_FW	AEM: 3.1, TEM: 2.2
GGLT_HW	1.1
GNAT	1.0.21.0
GTEM	libgtem.o1.3.4.0 libgtem_CALv4.o 1.3.4.1
GTEM_FW	2.2
GTEM_HW	1.1
PyQt	3.8
Python	0x20203f0
Qt	3.2.0
XBRD	0.0.3.0
XBRD_FW	1.2
XBRD_HW	1.1
cfitsio	2.470
cmdCli	1.38
cmdSvr	1.58
evtCli	1.60
evtSvr	1.63
gAttr	1.23
gConstraint	1.6
gEGU	1.6
gGroup	1.3
gLAT	1.39
gNode	1.24
gRule	1.3
gSchemaConfig	1.31
gutil	1.16
logging	0.4.7
sihippo	1.3.2
versions	1.27
xml	0.8.3
Component	Version
TEM
GCRC
GTRC
AEM
GARC
GAFE

Please contact Selim Tuvi, Ric Claus or Jim Panetta if you have problems or any questions (Contacts Page)
 

Back