Release Management Facility

The primary function of this facility is to provide a suite of system-wide diagnostics that tells LAT developers when a release is performing as desired. The diagnostic suite shall provide the following functionality:

allow developers to build, run and test a complete package, of which their software is a component
allow package maintainers to determine if a package is suitable for release
allow LAT to track variation in performance and behavior of different software releases
allow easy viewing and tracking of diagnostic results, e.g. via a Web interface

In addition the facility shall provide the following automatically performed functions:

check for new release versions (tagged versions) of all major SAS packages (release packages)
attempt to build, in all supported operating systems, each newly tagged package found
attempt to run tests, in all supported operating systems, for each newly tagged package that builds successfully
perform above steps for all packages that newly tagged release packages depend on
log a summary of results of above functions to a database
attempt a daily build, in all supported operating systems, of the development version (head) of each package
inform package maintainers and/or designated contact of the results of above functions

The science analysis software (SAS) for LAT consists of a large number of interdependent, generally independently maintained software packages. Packages that provide major functionality (e.g. performing Monte Carlo simulations of detector response) are referred to as release packages and consist of sets of packages maintained both by various LAT collaborators and developers external to LAT.

The method of tracking changes and maintaining proper synchronization for this large set of interacting packages, sometimes referred to as software release management, is primarily determined by the version and configuration management systems that LAT has chosen to use (Concurrent Versions System - CVS, and Configuration Management Tool - CMT). At the lowest level, via the use of CVS, a code repository is maintained (currently containing on the order of 100 independently maintained packages) that holds the current code base and tracks all changes to a package since it was placed in the repository. CMT allows users to define and maintain dependencies between packages and various versions of packages. When it is determined that a set of packages (including their versions) that perform some function (e.g. Monte Carlo simulations of detector response) is ready for release, the set of packages is collected into a single package called a release package and "tagged" with a unique label identifying that particular release. Individual packages are also tagged when it is determined that they are ready for general use; it is in fact these tags that are used in the specification of a release package in terms of its components.

Given that LAT code developers work a variety of platforms it is often difficult for individual developers to track the effects of their development in all supported environments in which the release packages are expected to run. It is primarily for this reason that an automated release management facility is useful; it helps to track potential problems and inform developers as the problems occur.

In addition to attempting to build dependent packages and run developer designed tests for all dependent packages in a release, the Release Manager will be designed to build and run tests for development code (referred to as in the head of the repository) as well. This can provide useful diagnostics to developers before it is decided to release a package.

Finally the Release Manager will log the results of the build and test stages for release packages in a database, as well as informing developers and/or designated contacts of build and test results for both release and development packages.

A current prototype of the Release Manager has been designed and written in Perl. Perl is a reasonable choice for the Release Manager due to its well tested libraries for network support and ease of extensibility. Current functionality includes checking the repository for new tags of release packages, checking out and attempting to build newly tagged packages, and informing designated contacts, via email, of the results. The Release Manager currently only works in Unix; current plans are to design it to run the build and test steps on a remote Windows server as well.

Testing

Currently the testing facility provided by the Release Manager simply attempts to locate (using a determined name convention and location) and run, if found, a user supplied test function for each package, and inform the developer and/or designated contact(s) of the results. This should suffice for post-release testing, as package maintainers are the most qualified for determining proper diagnostics for their package. Some general tests that are to a large degree package independent shall be added to the Release Manager. These will provide both system-wide and unit test capability and generally fall into standard software testing categories. They include:

regression testing - i.e. whether the current release generates different results than the previous release
conformance testing - i.e. whether the code satisfies various conventions outlined by the collaboration
performance testing
various other tests, e.g.: comparison of various histogram statistics, chi squared tests, Kolmogorov-Smirnov tests

Another reason for using Perl is the availability of the Perl facility Test.pm which provides useful extensions for software testing and would be a convenient way to implement any additional testing functionality for the Release Manager.

Notification and Logging

Package contacts are currently notified by email of build and test failures for release packages, with a summary describing the failure. Current plans are to log this summary to a database as well as generate more extensive summary information to email developers regarding build and test failures on various operating systems for development code.

K.Young Last Modified: 07/16/2001 12:55