Testing

Testing will span the range from

• unit tests - individual component tests
• system tests - collections of units working together, producing statistics and plots representing the performance of entire applications
• instrument performance tests - periodic reassessment of the instrument performance as characterized by the simulation and reconstruction tools
• end-to-end tests - "Mock Data Challenges" which will exercise all the machinery, tools and ability to perform science analysis

Unit tests

These are tests that individual code packages run on themselves in a stand-alone fashion. Each package should have a test/ directory with applications having known outcomes. These would be run automatically by the Release Manager whenever the packages are tagged and when a Release is declared. Failure of the test is reported to the package owner.

[example unit tests]

System Tests

These tests are performed when a Release is declared. The entire application is run as a unit and diagnostics generated. See the Release Manager for a fuller description of this process. The diagnostics are generated with releases, tracked from release to release and compared to standards, with variances flagged.

Instrument Performance Tests

These tests will evaluate on an ongoing basis  the basic performance parameters of the instrument and show they meet the relevant LAT Performance Specifications.  In particular, we must examine (after all background rejection and resolution cuts)

• the PSF as a function of energy and angle, for Front and Back sections of the TKR;
• the energy resolution, on-axis and at >60 degrees incidence angle, as a function of energy;
• the effective area as a function of energy and angle (and, hence, the FOV);
• the residual background as a fraction of the accepted high-latitude diffuse flux as a function of energy.

Performing these tests regularly (perhaps annually) and tracking the input and output will allow us to understand the evolution of the code and to verify that no change has taken us far from understood performance. This is really a set of regular tests for the software system itself, as well as yielding a current understanding of the expected instrument performance.

Such a performance test is underway now for the Instrument PDR.

[set of plots from Tracy here?]

End to End Testing - Mock Data Challenges

These are large-scale tests of the entire system: from bulk processing of simulated source raw data, through Level 1 processing followed by Level 2 analysis. In the HEP tradition, the underlying physics put in to the simulations [examples relevant here] are not revealed to those doing analysis: their job is to find it. The MDC's are large scale efforts involving a good deal of the collaboration, and certainly the Science Working Groups.

It is anticipated that there will be 2-3 such MDCs prior to launch: one in mid 2002 (input to CDR), another towards the end of 2005, and perhaps one in the middle.

R.Dubois Last Modified: 07/15/2001 18:52