Gleam System Tests

Introduction

Welcome! This system allows us to track the performance of our workhorse Gleam application as it evolves through its code releases. It allows us to run several configurations of Gleam (eg AllGamma, several energies of vertical gammas and muons, etc) and compare the output histograms to previous standard versions.

The portal to the system is at

http://www-glast.slac.stanford.edu/software/SysTests/TestInfo.asp

JAS is used as the web plot display tool, with asp and visual basic providing the connection to the underlying database.

Usage

Having selected a version of GlastRelease (which sets the Gleam version), you will be presented with a table of tests for that version. It has the following features:

lists the CPU time and memory used in the process generating the test output
a link to view the histograms
a button showing all the metadata associated with the test (currently, mean, sigma, # entries and KS test value for each histogram)
buttons showing the KS test results above and below threshold, and a summary count of those two conditions.

If you are not familiar with KS testing for comparisons of distributions, see this link.

Once you select a particular test's histograms to browse, you'll see a pull-down menu of histogram titles available in the Root file and a button to enable overlays.

Caveats

April 20, 2003

Not all the tests are working, so avoid those with cpu times set to zero. The test names should be self-explanatory. The metadata consists of histogram means, sigmas and number of entries, plus a KS test if a comparison has been done to a ‘standard’ histogram. KS test thresholds have been set to 0.5 for now and you can see all histograms above and below the threshold.
The Histos link allows you to select individual histograms and optionally overlay them on the standard, using JAS. When we understand the KS tests, hopefully they will point us at the histos that have changed; for now it is a bit tedious to march through all the histos. We’re talking to the JAS folks about a bulk plotting option.
Hopefully the histogram names are reasonably self explanatory (on our list to document), but you can look at the test code in the systests package in cvs:

http://glast.stanford.edu/cgi-bin/cvsweb/systests/src/GRtests/?cvsroot=CVS_SLAC

R.Dubois Last Modified: 2003-04-20 15:54:36 -0700