The Standard Environment for the Analysis of LAT Data

v10 (Monday, Sept 16, 2002)

 

1         Introduction

1.1      Purpose

This document describes the environment in which data from the LAT instrument on GLAST will be analyzed, the process by which this environment was defined, and the methods by which it will be developed.  The contents of this document were developed by the SSC-LAT Software Working Group.  Because the feasibility of implementing various analysis algorithms will be the subject of trade studies and computer technology will advance in the years until the launch of GLAST, the design of the analysis environment will evolve.  Nonetheless, here we outline our understanding of the tools necessary to analyze LAT data and our concept of how the user community will be served.

 

This document was prepared with the assumption that the reader is familiar with standard GLAST mission and LAT instrument terminology.  Appendix A defines some of the most important terms.

 

1.2      The User Communities

Two communities will analyze LAT data, each with its own needs.  LAT team members are clustered at a number of institutions in different countries.  Many in this community will presumably be heavy users of the LAT data, and thus LAT team members will be able to turn to colleagues both at their own institutions and on the instrument team for assistance in analyzing the LAT data and using the analysis tools.  The LAT IOC will provide the LAT team with LAT Level 1 data.  LAT team members may use the analysis tools on the same servers on which the databases are located, or the databases may be cross-mounted on the analysis computers; computational, storage, and data retrieval bandwidth limitations may not be as severe as for individual guest investigators.

 

We envisage the typical member of the general astrophysical community to be an investigator analyzing LAT data independent of the LAT instrument team, using the desktop computer generally available at the time of the launch of GLAST.  Note that by Moore’s Law, at the beginning of Phase 2 the average desktop computer will be ~10 times more powerful than current computers!  Most of these investigators will have had experience analyzing high energy astrophysics data.  The GLAST Science Support Center (SSC) at Goddard will make available both data and analysis tools to this community.  The Level 1 data for this community will reside on the SSC’s servers.  The user will extract data from the SSC’s databases, and export the resulting data files back to his/her home institution over the internet.  Thus the distinction between the central server and the user’s computer is relevant.

 

Because the SSC (and the LAT instrument team) will not have the resources to assist users individually, the analysis tools and their interface must be robust enough to withstand the use of relatively unsophisticated users.  The documentation must be sufficient for an investigator to learn quickly and easily how to perform standard types of analysis.  GLAST will be most successful if investigators who do not specialize in LAT analysis will feel that the LAT data are accessible.

 

1.3      The Challenges of Analyzing LAT Data

Analyzing LAT data will be challenging.  The point spread function (PSF) at high energies will be fairly small (~0.15°) but at low energy (where most of the photons will be detected) the PSF will be quite large (~3.5°).  Because of the LAT’s greater sensitivity relative to EGRET, many more weak sources will be detected.  At low energy it will be difficult to disentangle the different sources.  Therefore, the analysis of even a point source is inherently multi-dimensional:  spatial, spectral and temporal.  The EGRET data were analyzed as a series of two dimensional maps in different energy bands accumulated over a time range; we hope to perform true multi-dimensional analysis.

 

Almost all the data will be accumulated while GLAST surveys the sky; even during “pointed observations” the field-of-view (FOV) will drift.  Consequently, each photon will be detected at a different orientation between the source and the detector normal; a different instrument response applies to each photon.  Even with the LAT’s greater effective area relative to EGRET, the data space (e.g., apparent energy, apparent origin on the sky, orientation to the LAT) will be large and sparsely populated with detected photons.

 

The LAT’s FOV will be extremely large (>2 sr), and GLAST will usually scan the sky.  Therefore exposure will be built up in ~30 minute chunks every orbit, not continuously.  Exposure to a given source will build up more slowly in this survey mode than for pointed observations.  Even with a large effective area, the photon detection rate of the LAT will make the detection of pulsars known at other wavelengths very sensitive to the pulsar ephemeris.  The exposure will be accumulated at a variety of orientations of the LAT relative to sources of interest, resulting in a variable effective area; GLAST’s observing modes will introduce a window function that will complicate the analysis of periodic sources such as pulsars.

 

Consequently, the science tools must implement sophisticated data analysis techniques (e.g., likelihood methods for sparsely-populated, multidimensional data spaces) if the LAT’s full potential is to be realized.  The development of these tools will entail trade studies to determine what methods will be feasible given the computational resources available to the average investigator.

 

1.4      Responsibilities of the LAT Instrument Team and the SSC For the Definition and Development of the Standard Analysis Environment

 

The Announcement of Opportunity (AO) for GLAST’s instruments stated that the instrument teams are responsible for the “production of data analysis software” and the SSC for the “generation of high-level data analysis tools” (AO, p. 10).  While LAT scientists will actively develop analysis tools, the team does not have sufficient resources to create a robust analysis environment for the general community.  Conversely, SSC scientists are intellectually stimulated by the complexity of the LAT data analysis, will be engaged in the science that will result, and must be intimately familiar with the analysis tools provided to the community the SSC will support; consequently SSC scientists are willing and eager to participate in the development of these tools.  Since the SSC is responsible for making possible the analysis of GLAST data by the general scientific community, the SSC has a keen interest in the completeness of the suite of analysis tools available at launch.

 

Consequently it was agreed that the LAT team and the SSC will jointly define the science tools and the environment in which they will be used, the LAT team will be responsible for managing the development of the science tools, and SSC scientists and programmers will participate in the development effort.  There is a formal understanding that the tools have to serve the general scientific community; for example, this dictates the use of FITS format files.

 

To implement this division of labor and responsibilities the SSC-LAT Software Working Group was established.  Meeting weekly by VRVS teleconferencing, the working group consists of Toby Burnett, Seth Digel (co-chair), and Richard Dubois from the LAT team and David Band (co-chair), Jay Norris and Bob Schaefer from the SSC.  The working group has been concerned with the definition of the tools, the architecture of the relevant databases (particularly the event database), the software environment and the Level 1 pipeline.  Once software is developed this working group will oversee the review the resulting tools to determine whether they satisfy the requirements for the analysis environment.

 

A detailed management plan is presented in §5.1.

 

1.5      The Process of Defining the Standard Analysis Environment

The definition of the standard analysis environment (SAE) is based on experience with EGRET and other high energy astrophysics missions, the scientific goals for the LAT described in the GLAST Science Requirements Document, developments in software technology, and analysis of the challenges posed by LAT data.  Identification of the analysis tools needed to support LAT data analysis began in earnest in January, 2000, with a LAT software workshop at SLAC.  The definition of the SAE presented here is an extension of that initial work, more complete and considerably refined.  Once the strong role of the SSC in the development of the high-level analysis software for the LAT was clarified (§1.4), the definition of the analysis environment described here was undertaken as a joint effort overseen by the SSC-LAT Software Working Group. 

 

The analysis environment was defined with the needs and requirements of the LAT team and guest investigator communities in mind.  Many of these requirements are stated in the GLAST Mission System Specification (MSS).  On the most basic level, this environment will "provide validated science data to the GLAST user community." (MSS 3.1.2.6)  It will provide a "single higher-level analysis software environment for use by the community and by the instrument teams" (MSS 3.5.1.4) and will "respect standards that ensure software portability, independence of vendor, and compatibility with existing multi-mission high-energy astrophysics tools." (MSS 3.5.1.5)  The system is constructed to "interface with the High Energy Astrophysics Science Archive Research Center (HEASARC) in support of multi-wavelength studies." (MSS 3.5.3.4).  A key aspect of the compatibility with the existing high energy astrophysics infrastructure is the use of FITS format files, with HEASARC-defined keywords.  Finally, the analysis environment will "interface with GLAST investigators via the commercial internet" and will "provide access to GLAST data that is made public via the commercial internet."  (MSS 3.5.3.5)

 

The definition process has been iterative and driven by use cases that describe specific analysis tasks and how they could be performed.  Therefore, the tools on the SAE are well matched to the scientific goals specified in the GLAST Science Requirements Document, including:  determine the positions and fluxes of blazars (SRD §2.1), analyze the spectral and temporal properties of GRBs (SRD §2.3), study small extended sources like molecular clouds (SRD §2.5), analyze the timing and spectral properties of gamma-ray pulsars (SRD §2.6), and perform periodicity searches and search for counterparts of unidentified sources (SRD §2.7). Contributions have come from within the SSC-LAT working group, the SSC and the LAT and GBM instrument teams.  The SAE presented here was the topic of a 3-day workshop at SLAC in June 2002 with broad representation from the LAT collaboration and the SSC (approximately 30 attendees in total).  The workshop included consideration of the definitions of the analysis tools as well as the software development environment, the management plan, and considerations for defining the development schedule.

 

In defining the SAE, we have primarily considered the requirements for each of the components, but we have also tried to identify areas where implementation considerations are the greatest.  For many of the components of the analysis environment, the approach to implementation is clear, either because the algorithm is well known from other high-energy astronomy mission analysis systems, or from the limited scope of the tool.  For some components, in particular databases and the core interactive analysis tools, the best approach is not as clear.  We have identified potential options for their implementation and included time in the development schedule for prototyping them.

 

1.6      Scope of the analysis environment

The purpose of the analysis environment defined here is to enable the general scientific community to extract astrophysical results from the LAT Level 1 data.  Thus the environment assumes the existence of lists of “photons” from the LAT, and does not concern itself with the analysis of tracks in the LAT (event reconstruction and classification) that results in the parameters of the photons.

 

In addition, the SAE does not include the specialized tools that would be required to address some of the scientific goals for the LAT, for example tools to construct the interstellar emission model, derive the point source catalog, detect transients (both topics assigned to the LAT team), search for particle annihilation lines from the Galactic Center, or study solar flares or cosmic rays.  For these topics and no doubt others that we have not yet imagined, the LAT team and guest investigators will have to write their own software.  Because the SAE does have a wide scope and flexible tools for model definition (e.g., some aspects of solar flares could be studied with the GRB tools), our expectation is that only a small fraction of the community will need additional tools.  We plan to make contributed software available to the community after proper vetting.

 

2         Components of the analysis environment

In this section we introduce the SAE, with emphasis on the components of the environment and their interrelations.  Complete specifications of the tool and databases that constitute the SAE are provided in Appendix C.  In what follows, the components are identified by their designations (D for database, U for utility, A for analysis tool, O for observation simulation, and UI for user interface, followed by number that we have assigned) as they are discussed.  The distinction in designation between analysis tools and utilities is somewhat subjective; the intention is that tools designated as utilities produce output needed by analysis tools, including providing interfaces to the databases.

2.1      Event data, pointing & livetime history, and response functions

As stated in §1, the SAE is intended for astrophysical research with LAT data.  The principal inputs are already-processed (Level 1) event data, LAT pointing and livetime information, and the instrument response functions that provide the high-level characterization of the LAT (Fig. 2.1).  The event data are assumed to have charged particle backgrounds (events recorded in the LAT due to the passage of cosmic rays) eliminated and gamma rays characterized by event class, energy, time, and direction in celestial coordinates (D1).  Event class can be considered a set of flag values that indicate the quality of the energy and direction determinations.  The pointing and livetime history (D2) is needed to calculate exposures, and ultimately calibrated fluxes.  The response functions (D3) are used in most high-level analyses, to permit comparison of models for gamma-ray distributions with observations.

Figure 2.1

 

D1 and D2 will have implementations that are specific to the LAT (for example, the event data in D1 may be stored as spatially indexed database tables in a relational database system).  For long-term maintenance of the analysis tools in the post-mission era, and in general to make the analysis tools independent of the implementation of the database system, D1 and D2 will be accessed only by utilities U1 and U3.  These will format the requests, e.g., construct database queries, as well as translate the output to FITS format, if it is not already.  By agreement with the HEASARC the analysis tools will read and write FITS format files compatible with the OGIP (Office of Guest Investigator Programs) standard.  Also by agreement with the HEASARC, D3 will be implemented in CALDB, the HEASARC-defined system of indexed FITS files for representing instrument response functions.  (See the detailed description of D3 in Appendix C).  As no translation of D3 for the post-mission era of maintenance by HEASARC is foreseen, we do not define a special intermediary utility for accessing D3.

 

2.2      Event display

The SAE does include a facility for viewing the low-level information – the pattern of hits in the tracker, energy depositions in the calorimeter logs, and hit tiles in the anticoincidence detector, along with the assignments of hits to tracks and the parameters derived for event classification – for an event.  The provision of an event display tool (UI1, Fig. 2.2) is not to provide an analysis step that relates to other parts of the SAE, but instead transparency, literally a window into the lower-level processing that results from the SAE will be built on.  In many instances, for examples for GRBs detected by the LAT, new results will hinge on one or a few gamma rays, and the event display will permit investigators to judge quality for themselves.

Figure 2.2

2.3      Exposure

The calculation of exposure (U4, Fig. 2.3) requires the pointing and livetime history, as mentioned above, as well as the instrument response functions (D3).  These represent the high-level response of the LAT, in terms of its effective area, energy resolution, and point-spread function as functions of the gamma-ray arrival direction (in instrument coordinates), its energy, and other parameters.  The exact set is still to be determined but may include plane (or tower) of conversion and event class (a kind of quality flag for energy and direction determinations and an indication of which background rejection cuts the event passes).  If the response of the LAT changes on orbit, e.g., in response to a hardware failure, the instrument response functions will need to be updated, and will therefore become time dependent.  A utility for extracting and displaying response functions is provided in the SAE (U10).

 

Figure 2.3

 

In Figure 2.3, the connection between the IRFs (D3) and the exposure calculation is indicated as a dotted line because U4 may actually be implemented to factor out the instrument response (effective area) from the livetime accumulation.  Although the effective area is certainly part of the exposure, factoring the calculation in this way may have several implementation advantages.  See the discussion in the Open Issues section of the requirements summary for U4 in Appendix C for more information.

 

2.4      Likelihood analysis

The standard analysis of LAT data will be via model fitting, owing to the limited numbers of gamma rays, the relatively poor angular resolution of pair conversion telescopes, and the pervasiveness of structured diffuse gamma-ray emission from the Milky Way.  (Approximately 60% of the gamma rays that EGRET detected were diffuse emission from the Milky Way.)  The SAE includes an interstellar emission model (U5, Fig. 2.4) that describes the spectral distribution of the emission across the sky.  The model will likely have some adjustable parameters, perhaps related to the Galactic cosmic-ray distribution, that will be optimized in the model fitting analysis.  (U5 will be updated at least once based on analysis of LAT data, and the adjustable parameters may either be fixed or have recommended values specified as a result.)

 

Figure 2.4 (To reduce the complexity of the diagram, tools that require the same set of inputs as each other are drawn showing the inputs connecting to a common vertical bar, and the tools individually connect to the vertical bar.)

 

The workhorse analysis tool in the SAE is the model fitting tool (A1, Fig. 2.4).  As for other high-energy gamma-ray astronomy missions, for the LAT we expect that the model fitting tool will implement maximum likelihood analysis.  For the SAE we envision a flexible tool (U7) for defining models to be analyzed with A1.  The likelihood analysis (described more fully in the use cases in §4) can be used to detect gamma-ray point sources and characterize their positions, fluxes, and spectra.  U7 will allow specification of extended sources as well, small angular size diffuse sources that the LAT may resolve.  As defined for the SAE, A1 analyzes data for finite intervals of time, fitting models that are static (for the time interval analyzed).  Analyses of data for successive time intervals can be used to monitor for flaring gamma-ray sources.

2.5      Point source catalog, astronomical catalogs, and source identification

Supporting the high-level analysis of point sources will be tools to access the LAT point source catalog (D5, the initial version of which will be based on the LAT sky survey data; Fig. 2.5), and catalogs of astronomical sources useful for counterpart identifications (D6, e.g., flat spectrum radio sources or X-ray sources).  These will be available both to the source model definition tool (U7) and to a source identification tool (A2), which will evaluate probabilities of chance coincidences between gamma-ray sources and those in other catalogs.  A2 will rely on properties of the catalogs – flux distributions, coverages, etc., in deriving the probabilities.

Figure 2.5

 

2.6      Pulsar analyses

All of the routine analysis of point sources can be carried out for pulsars as well, with optional selection of data based on pulsar phase.  During the mission, radio timing observations of known and candidate gamma-ray pulsars will be undertaken (by a GLAST project-supported effort) to produce a database of pulsar ephemerides (D4, Fig. 2.6).  The utilities U11, U10, and U12 will extract timing information for the desired pulsar, correct the gamma-ray arrival times to the solar system barycenter, and then calculate pulsar phases.  This information will be appended to the gamma-ray event data extracted from D1.  The phase assignment is separate from the arrival time correction to facilitate export of pulsar data from the SAE.  The likelihood analysis tool (A1) and the map generation utility (U6, discussed below) will optionally accept specification of pulsar phase ranges for their analyses and map generation and will scale the exposures appropriately.

Figure 2.6

 

Searches for periodic emission using the ephemeris of a known pulsar will be possible with U2 and A3.  U2 applies an energy-dependent selection cut (narrower at greater energies) to the event data around the position of the pulsar to maximize the pulsed signal.  Then A3 is used to phase fold the resulting gamma rays and apply statistical tests for periodicity.  It can also search narrow ranges of frequency, in case the radio ephemeris is not quite accurate.

 

For bright unidentified sources, with many gamma rays, the pulsar period search tool (A4) can search for periodic gamma-ray emission.  The process is very computation intensive, but standard algorithms are well understood.  Indeed the Geminga class of gamma-ray pulsars that have no periodic radio emission is expected to grow greatly (as a result of successful searches for gamma-ray periodicity) with the increased effective area and angular resolution of the LAT.

 

2.7      Map generation

The map generation utility (U6, Fig. 2.7) will generate gamma-ray, exposure, and intensity maps for user-specified regions of the sky, time ranges, energy ranges, and other sets of parameters, such as event class and zenith angle cuts [mention with exposure calculation].  The likelihood analysis itself will not be map based, or at least not based on single maps of the sky, owing to the scanning mode of operation of the LAT together with its large field of view and significant variation of IRFs across the field of view.  Binning the data into maps on the sky (as U6 will do) will be useful for visual inspection, and export, of the data, but will generally result in the loss of useful information for quantitative analysis with the likelihood tool.

 

Figure 2.7

 

2.8      Gamma-ray bursts

A comprehensive set of GRB analysis tools is included in the SAE (Fig. 2.8).  Most of the tools will have multi-instrument (e.g., LAT, GBM, Swift, and Integral) data analysis capabilities.  The analysis tools can be used for either LAT or GBM data, although here we present only the instrument response-related tools for the LAT; similar tools for the GBM will be treated elsewhere.  Bursts will be localized by the instrument teams in their standard response to the detection of a burst, although users will be able to localize bursts using the standard likelihood analysis tool A1.  The tools that specialize in GRB analysis take advantage of the bright, transient nature of bursts for study of their temporal profiles and spectra in the regime where celestial backgrounds can be neglected for the LAT data.

 

The SAE includes an interactive visualization tool (U13) for display of binned data as counts spectra or time profiles, or unbinned data to provide visual assessment of bursts, their pulses and spectral evolution.  The analysis tools will provide similar graphical displays to assist the user.

 

Figure 2.8

 

Binned spectral analysis will suffice for bright GRBs for which many LAT photons are detected.  For this analysis the data are binned in time and apparent energy.  The analysis path for these bursts proceeds via binning the data (A5; A6 rebins data that are already binned), generating the DRMs (Detector Response Matrices, versions of the instrument response functions for one dimensional spectra) appropriate to the spectra's time and energy bins (U14), then fitting spectral models via an XSPEC-like tool (A8).  We anticipate that joint fits to the LAT and GBM data will be standard. Alternatively, the temporal profiles can be analyzed using A7, which is also planned to be able to utilize unbinned event data (selected for a the time range of a GRB and a suitably large region of the sky).

 

For GRBs with fewer photons unbinned spectral analysis will be required.  For this analysis the data are binned in time but not in apparent energy.  Although this tool (A9) will be based on likelihood methods similar to the general likelihood tool A1, the spectra will be unbinned in only one dimension and will involve only a few events; consequently the GRB tool will use computational techniques that cannot be applied in the A1 tool, e.g., using analytic derivatives of the spectral models to maximize the likelihood.  Since the spectral models will be simple and the region around the burst will not be modeled spatially, the standard model definition utility is unnecessary.  The instrument response required for this analysis will be generated by the standard exposure tool (U4).

 

Spectral-temporal modeling of GRBs will be accomplished via a third likelihood analysis tool that is implemented to model the full time and spectral dependence of bursts using detailed physical models (e.g., for the gamma-ray production from colliding relativistic shells) with adjustable parameters (A10). A10 is to be implemented with well defined interfaces for additional physical models as they are developed or contributed.

 

2.9      Alternative data sources

The SAE includes two tools for observation simulation (Fig. 2.9).  OS1 generates simulated livetime and pointing history records based on a user-define observation strategy and orbit parameters.  In the SAE it can substitute for D2 and U3.  OS2 generates simulated Level 1 data consistent with a specified source model (U7) and pointing & livetime history; it can substitute for D1 and U1.  The observation simulation tools will be essential for verifying the performance of the analysis tools.  As an aid to scientific analysis and the preparation of guest investigator proposals, these tools will also enable users to assess the detectability of a source of a given strength at a given position.  OS2 will have well-defined interfaces to modules for specific kinds of gamma-ray sources.  We anticipate having modules to simulate transient emissions from, e.g., blazars, periodic emission from pulsars, GRBs, and small extended sources.

 

Figure 2.9

 

For the purposes of subsetting already-extracted event data, without generating a query for the database, the SAE includes U8 to interactively apply additional cuts to the locally-stored Level 1 data.  This is also depicted as an alternative data source for the high-level analysis.

 

2.10User interface

The user interface components of the SAE relate to the interactions between users and analysis tools in general, such as common scripting or graphical user interfaces, or for specific applications, such as the event display UI1 discussed above.  The general user interface components are not explicitly included in the dataflow diagrams described in the preceding sections.  They are listed and briefly described in Table 2.5.

 

2.11Summary of the standard analysis environment

The final diagram of the SAE (Fig. 2.10) has data flow generally progressing from left to right, from lowest-level to highest-level data products.  The components of the analysis environment shown in the diagram are also listed in Tables 2.1-2.5 [which may introduce data formats and include links to their relevant tools.]

 

Fig. 2.10  The overall data flow in the standard analysis environment.

 

In most cases, the software and database components of the SAE will be installed on client computers.  The principal exception is the Level 1 database (D1), which we anticipate will run on specialized, dedicated central servers in order to provide adequate data retrieval capability.  The instrument response functions (D3), depending on the ultimate size of their implementation in CALDB, might also be made available for remote access from servers at LAT sites and the SSC.

 

 

ID	Name	Description	Size after 1 year
D1	Event Summary (level 1)	Summary information (i.e., everything needed for post-Level 1 analyses) for photons and cosmic rays, possibly segregated by event classification	200 Gb (only 10% photons)
D2	Pointing, livetime, and mode history	Timeline of location, orientation, mode, and accumulated livetime	70 Mb
D3	Instrument response functions	PSF, effective area, and energy redistribution functions in CALDB format	TBD
D4	Pulsar ephemerides	Timing information (maintained during the mission) for a set of radio pulsars identified as potential LAT sources	1 Mb
D5	LAT Point source catalog	Summary information for LAT-detected point sources, from which the LAT point source catalog is extracted	10 Mb
D6	Other high-level databases	Astronomical catalogs useful for counterpart searches for LAT sources	TBD

Table 2.1 Databases

 

ID	Name	Description
U1	Event data extractor	Basic front end to the event summary database D1
U2	User-level data extraction environment	Allows the user to make further cuts on the photons extracted from D1, including energy-dependent cuts on angular distance from a pulsar position
U3	Pointing/livetime history extractor	Basic front end to the pointing, livetime, and mode history database D2
U4	Exposure calculator	Uses IRFs (D3) and pointing, livetime and mode history (D2) to calculate the exposure quantities for the likelihood analysis (A1)
U5	Interstellar emission model	Not software per se, although the model itself will likely have parameters (or more complicated adjustments) that need to be optimized once and for all, in concert with an all sky analysis for point sources.
U6	Map generator	Binning of photons, regridding of exposure, coordinate reprojection, calculation of intensity
U7	Source model definition	Interactive construction of a model for the region of the sky by specifying components such as the interstellar emission, point sources, and extended sources, along with their parameter values; the resulting models can be used in the likelihood analysis (A1) and the observation simulator (O2)
U8	IRF visualization	Extracting PSF, effective areas, and energy distributions from the CALDB (D3) for visualization
U9	Catalog access	Extracting sources from catalogs, LAT and otherwise (D5 & D6)
U10	Gamma-ray arrival time barycenter correction	For pulsar analysis
U11	Pulsar ephemeris extractor	Interface to the pulsar ephemeris database (D4)
U12	Pulsar phase assignment	Using ephemeris information from U11 and barycenter time corrected gamma rays from U10, assigns phases to gamma rays
U13	GRB visualization	Displays various types of GRB lightcurves and spectra
U14	LAT GRB DRM generator	Generates the detector response matrix appropriate for fitting binned LAT GRB spectral data.

Table 2.2 Utilities

 

 

ID	Name	Description
A1	Likelihood analysis	Point source detection, characterization (position, flux, spectrum), generalized models for multiple/extended sources
A2	Source identification	Quantitative evaluation of the probability of association of LAT sources with counterparts in other catalogs
A3	Pulsar profiles	Phase folding of gamma-ray arrival times (after phases are assigned by A3) and periodicity tests
A4	Pulsar period search	Search for periodic emission from a point source
A5	GRB event binning	Bins events into time and energy bins
A6	GRB rebinning	Rebins binned GRB data into larger time and energy bins
A7	GRB temporal analysis	Analyzes GRB lightcurves using various techniques such as FFTs, wavelets, etc.
A8	GRB binned spectral analysis	Fits binned spectra; may be XSPEC
A9	GRB unbinned spectral analysis	Fits event data with spectral models
A10	GRB spectral-temporal physical modelling	Fits a GRB physical model to LAT or LAT+GBM data

Table 2.3 Analysis tools

 

ID	Name	Description
O1	Livetime/pointing simulator	Generates simulated pointing, livetime, and mode histories (analogue of D2) for a specified time range and observing strategy.  This should be available as a tool, but probably most generally useful will be precomputed simulated D2 databases, which can be used subsequently for any study with O2.
O2	High-level observation simulator	Generates gamma rays according to the instrument response functions (i.e., after detection, reconstruction, and background rejection), bypassing the instrument simulation step

Table 2.4 Observation simulation tools

 

ID	Name	Description
UI1	Event display	Displays the tracks of an individual event, requires access to Level 0.5 data
UI2	Image/Plot display	General facility for image and plot display
UI3	User Support	Defines the scope of user support at the level of the components of the standard analysis environment
UI4	Command-line interface and scripting	Requirements for command line and scripting interfaces to the components of the standard analysis environment
UI5	GUI interface and Web access	GUI interface to the standard analysis environment.  For some tools, especially U1 and U4, which have databases stored on a remote server, Web access seems most practical.

Table 2.5 User interface

 

3         User environment

3.1      User interface

The suite of analysis tools and utilities constitute the standard analysis environment that will enable users to analyze the LAT data.  The users will access these tools through two different interfaces according to their temperament, experience and needs. The first is a command line interface that will be familiar to UNIX users and to X-ray astrophysicists experienced in running FTOOLS.  In the second the tools will be run through an overarching GUI-based environment, and the tools will interact with the user through GUIs.  Through the use of I/O libraries both interfaces can be implemented.  From the viewpoint of the software development, the command-level interface may be more fundamental since the GUI-based interface binds together the tools that are separate entities in the command-line interface, but from the viewpoint of most users, the GUI-based interface may predominate.  Of course, some users may mix the two interfaces, using the GUI-based interface to prepare the data for a particular analysis, and then running a particular tool, or group of tools, with user-defined scripts that perform or automate the analysis (e.g., fitting series of spectra in a particular way).

 

3.1.1      The command line interface

 

The tools can be divided into two categories.  Some tools will perform single, specific tasks, and after they are entered, they will require very little, if any, further input from the user.  Following Unix tradition, some information may be passed as optional arguments, or as with HEASARC's FTOOLS or the Chandra data analysis system's CIAO tools, there may be "parameter files" containing default values for various options that can be readily modified by the user.  Examples of LAT command line tools include the pulsar phase assignment program (U12) and the LAT GRB response matrix generator (U14).  Several existing generic HEASARC FTOOLS, such as fselect and fcopy, will also be useful for manipulating LAT data.

 

Several of the LAT analysis tools will require a substantial amount of interactive input from the user.  The primary example is the likelihood analysis tool (A1) that will require the user to adjust the gamma-ray sky model by adding sources, changing source positions, freezing and/or thawing model parameters, etc., in order to facilitate the fitting process.  Other examples of interactive programs include the source model definition Tool (U7) and the GRB spectral modeling tools (A8-A10).

 

A strength of the command line interface is that users can easily create new tools through scripting.  The scripts may be embedded in the tool; a possible prototype is the HEASARC's X-ray spectral analysis program XSPEC.  Programs such as the likelihood tool (A1) will have a command-line interface, and a scripting language could be embedded in a fashion similar to the use of Tcl (Tool Command Language) in XSPEC.  Alternatively, the tools may be linked together with scripts.  The integration of the IRAF command language as an extension of the increasingly popular Python scripting language may serve as a model.  In this approach, the scripting language interpreter provides the analysis environment; and functions for manipulating and analyzing the LAT data are added by implementing them as loadable extension modules.

 

In either case, we reap the benefits of adding scripting: the scripting language adds a "macro" capability.  Repetitive tasks, such as fitting a number of different datasets with the same source model, can be readily automated using the control structures of the scripting language.  Complex sequences of analysis tasks can be put into a single script.  This will enable users to keep track of their analysis sequences, and their scripts will augment the functionality the likelihood tool (A1), while still allowing the set of basic tasks to be well-defined and relatively small in number.  In addition, the scripted routines can be readily modified to suit the individual needs of each user.  Another benefit of having a scripting capability is that it greatly facilitates the implementation of a graphical interface to our tools.  Scripting languages such as Tcl, Perl, or Python have programming support for the popular Tk widget libraries as well as other graphical interfaces, and these languages provide more efficient GUI programming than do the corresponding C++ APIs.

 

3.1.2      The GUI-based interface

 

In the GUI-based interface the user will invoke the tools through GUIs and then will interact with the tools through the GUIs.  Since the GUIs present the user with standard analysis options, this interface will make it particularly easy for new and occasional users to analyze LAT data.  We will endeavor to make most types of analysis possible through the GUI interface, but in some cases an advanced user will require the added control afforded by the command-line interface and the scripting capability.

 

The GUIs will be designed to give the interface a unified look and feel.  The same terminology will be used throughout, and standard buttons (e.g., for help or exiting a tool) will be in the same place.  Pop-up windows to input or output files will start in logical default directories, and GUIs to modify parameter values will present the current sent of parameters as the default.

 

The BATSE GRB analysis tools RMFIT (developed at MSFC) and SOAR (developed at UCSD) are two examples of the integration of different analysis functionalities into a single environment with the use of GUIs and graphics.  In particular, RMFIT implements interactive graphics.  Unfortunately, both of these burst analysis tools are written in IDL, a commercial data analysis environment, and therefore cannot be distributed as official tools

 

3.2      Distribution plans

3.2.1      Data Distribution Plans

The LAT IOC will transfer post-Level 0 data to the SSC, and to one or more mirror sites for use by LAT team members in other countries.  (The MOC will deliver the Level 0 data directly to the SSC for archiving, as described in the report of the Data Products Working Group.  The data available at the SSC will also be mirrored at the ASI Science Data Center.)  Of primary interest for the SAE are the data required for the databases listed in Table 2.1.  These data sets will most likely be transferred from the LIOC to the SSC using the Data Transfer System (DTS), a set of Perl scripts written for transferring data between centers for the XMM mission; Swift will also use DTS.  During Phase 1 access to the Level 1 data is restricted to the LAT team for validation and verification and to a small number of guest investigators.  Level 1 data for transients (GRBs, bright blazar flares, and solar flares) will be supplied to the SSC as soon as the Level 1 processing is complete for immediate release.

 

Once the Level 1 data become publicly available in Phase 2, investigators from the general scientific community will extract data from the SSC’s databases.  The data extraction utilities U1 and U3 (see §2) will be run on the SSC’s servers through the SSC’s website.  The extracted data will be formatted into FITS files that the investigators will transfer (e.g., by ftp) back to their home institutions.  In a limited number of cases where network distribution is impractical, the SSC will distribute it by CD-ROM or DVD.  To obtain LAT data from the SSC a scientist must become a “qualified investigator” by identifying him/herself to the SSC as a legitimate investigator.  Respecting the GLAST mission’s data policies, the ASDC will supply data to Italian astrophysicists.  Note that in Phase 2 there are no proprietary data, only a 90 day proprietary period for specific investigations after all the necessary data become available for the research proposed by a successful guest investigator.

 

3.2.2      Software Distribution Plans

Some of the data extraction tools will be run on the database servers, however most of the science analysis tools will be downloaded to the investigator’s computer.  As discussed below, we currently plan to support the Windows and Linux operating system; other flavors of Unix will be considered based on their popularity in the astrophysics community.  As far as possible, we will adhere to portable coding practices, including the isolation of system-dependencies.  The LAT science tools will be distributed over the internet as  compressed tars files or RPMs for Linux users and as zip files for Windows users.  The general scientific community will download these files through the SSC’s website.  The distributed files will contain executable binaries for all the software, a README file, release notes, source code, and documentation covering the installation and usage of the software.  Note that both the executable binaries and the source code will be provided, as is the standard practice for high energy astrophysics missions.  Further documentation on the LAT science tools will be posted on the SSC website.  Only in rare cases will the GLAST science analysis software be available on other media—CD-ROM, DVD, or tape—and distribution by this method will be considered only on a case by case basis.

 

3.3      Documentation and help

User Guides and Reference Manuals for all of the LAT analysis software will be available on-line, both as standard web pages and as appropriately formatted down-loadable PDF files.  These documents will describe the use of the analysis tools as well as the algorithms underlying them and the circumstances that are appropriate for applying each tool.  Following the example of CIAO, we will have web pages of "analysis threads".  These are essentially detailed use cases that show the precise sequence of commands, along with appropriate explanations, to do a particular analysis.  We will also maintain a web archive of useful analysis scripts that have been produced internally or submitted by users.  It is expected that the content of the analysis threads web page and script archive will evolve and grow as the mission matures.  Guidelines for preparing the scripts will be provided, and user-submitted scripts will be vetted and tested before being made part of the archive.

 

For the command-line tools as well as for commands executed from within the interactive tools, standard reference manual documentation will be provided in a format similar to Unix man pages.  The content of the man pages will be accessible from a searchable web page that has hypertext links for all supported concepts.  This will be similar to the web-based ahelp documentation provided by CIAO.

 

Finally, we will have a "help-desk" service to which users can make inquiries regarding LAT data analysis, the software itself, or bug reports.  Submissions can be made either to the help-desk email address or through a web interface.  In order to facilitate interaction between the users and the help-desk service, specific guidelines for reporting bugs will be provided.  All inquiries and responses will be logged; and as user questions or bug-reports accumulate, we will archive the more common inquiries and their resolutions on a FA web page.

 

4         Use cases

4.1      Determine the Average Spectrum of 3C 273 For a Particular One-Month Period

 

1. Extract the gamma-ray data from the event summary database (D1) for this period of time for a region of sky centered on 3C 273.  The minimum size of the region is dependent on the minimum energy desired for the spectral analysis; U1 will provide guidance.  (The rule of thumb is that the region of the sky has to be large enough so that the analysis of 3C 273 is not affected by point sources that are outside the region selected.)  At this stage, additional cuts on the gamma rays can be imposed, such as on zenith angle or inclination angle.  Again, U1 will provide guidance about standard cuts.  Specification of the class(es) of gamma rays may also be made at this time.  The classes are presently TBD; for example, distinct classes of gamma rays with good energy measurement or direction determination might be defined.

 

2. Calculate the corresponding exposure using U4.  U4 can automatically generate the exposure matrices with the same cuts as used for a gamma-ray dataset produced by U1.

 

3. Next, define a model for the gamma-ray emission in the region of the sky using the source model definition tool U7.  This should be largely automatic, at least for models containing only the interstellar emission, an isotropic component, and point sources.  U7 understands the parameters of the interstellar emission model (U5) and will have access to the current catalog of LAT point sources (D5), perhaps even for the particular time range of interest.  The catalog will include position, flux, and (most likely) spectral index estimates for point sources in the region of the sky.  The output of U7 is the definition of the model (interstellar emission terms, isotropic emission spectrum, and point source positions, fluxes, and spectral indices) readable by the likelihood analysis tool A1.

 

4. At this point, it would be a good idea to visually inspect the data for the region to be analyzed by generating maps of intensity both for the observations (via U6) and for the model (an optional output of U7).  The maps can be displayed using the display utility U2.

 

5. Then use the likelihood analysis tool A1 to first optimize the positions and fluxes assumed for the point sources, and any adjustable parameters for the diffuse emission model, in the region analyzed using the source model and the gamma-ray and exposure data.  (The positions of identified sources should be constrained to remain fixed at their catalog values during this optimization.) 

 

6. Have A1 write an updated model definition, then use this model definition in A1 to search the region of interest for additional point sources that might have missed inclusion in the general catalog.  The source identification tool (A2) may be used at this time to determine if counterparts for these sources have been seen at other wavelengths.  It may also be desirable at this point to remove any point sources that are not actually detected in this time range.  In support of the analysis, A1 can display likelihood test statistic maps, the model intensity, the (observed minus modelled) excess map, and the locations of point sources in the model, using the display utility U2.  After a final optimization pass for the source parameters is performed, have A1 write the final model for the point sources.

 

7. Two approaches for deriving the spectrum of 3C 273 are possible at this point.  The first approach is to modify the model to use different spectral models for 3C 273 (e.g., a power-law with a spectral break), and reoptimize the parameters to see which spectral model best represents the data.  In this case, the results are represented by the values of  spectral parameters adopted for the model of 3C 273.  The second is to re-run A1 using the final model and narrower energy ranges (~factor of 2 wide, collectively spanning the energy range of interest), to derive fluxes and confidence intervals.  This process can be automated using the scripting capability of A1, and template scripts should be available.  The result should be a flux measurement (or upper limit) for each energy range and a corresponding plot.

 

4.2      Determine the Light Curve of 3C 273 For a Particular One-Month Period

 

Steps 1–5 for this analysis are the same as for the spectral analysis described above.  After these steps, the model for the diffuse emission and the point sources in the field should be established.  Note that for short time intervals, such as will be analyzed here, the spectral model for 3C 273 need not be particularly complex.  A power-law will suffice, and it may even be worth evaluating whether holding the spectral shape fixed at some representative form increases the flux sensitivity enough (by reducing the number of degrees of freedom in the likelihood function) that fluxes can be measured on shorter time intervals.

 

6. Based on the detection significance of 3C 273 in the whole month interval, decide what time interval will be used for evaluating the flux.  Most likely this will be days.

 

7. Extract gamma-ray data sets (U1 to query D1 or U8 to subset a previously-extracted set of gamma-ray events), generate corresponding exposures (U4) for each of these intervals, and reoptimize the model parameters for each time interval (A1) to determine the flux and confidence interval (or upper limit).  U8 can be used to subselect already-extracted data by time range (The fluxes and positions of the other point sources in the region probably can be held fixed for this analysis.)  This can be done for various energy ranges subselected from the gamma-ray and exposure datasets if desired.  Note that this process of extracting data, generating exposure, and applying a common model in the likelihood analysis, can be automated.

 

8. If desired, time intervals can be lengthened or shortened based on the findings in step 7, and the analysis re-run.

 

4.3      Study Gamma-Ray Emission From a Pulsar

 

Study of pulsars with LAT data can follow two paths.  The first is evaluation whether periodic emission is evident in the data and the second is characterization of the pulsar as a point source using phase selected data in the likelihood analysis tool A1.  Both approaches start with extracting gamma-ray data from the vicinity of the pulsar, using U1 as in use case #1:

 

1. Extract the gamma-ray data (U1) for a region of the sky around the pulsar large enough for point-source analysis, for the time and energy ranges of interest.

 

2. Extract pulsar ephemeris (U11) for the pulsar, if it is known in D4, by specifying the name of the pulsar or a region of the sky, and the time of observation.

 

3. Calculate barycentric photon arrival times (U10) by specifying a provisional or known pulsar position.

 

4. To enhance the detectability of periodicity in the LAT data, the gamma-ray data may need to be passed through U2, which selects the gamma rays within a certain (energy and inclination angle dependent) radius of the position of the pulsar.  This cut discriminates against the diffuse gamma-ray emission.  The position of the pulsar may be specified either via a pulsar ephemeris extracted from D4 in step 2 or explicitly.

 

Periodicity searches and periodicity significance tests:

 

5. If the pulse frequency at the time of observation is completely unknown, search for pulsations in the gamma-ray data in a wide range of timing parameters (A4).  A4 will generate a plot of significance of periodicity in the data as a function of trial frequencies and frequency derivatives.  Once a pulsation is found, periodicity tests can be applied at pulse frequencies close to the one determined in this step (A3), in a manner described in the next step.

 

6. If the pulse frequency at the time of observation is approximately known, search for a pulsation in the gamma-ray data at pulse frequencies close to the expected one (A3) by specifying either a pulsar ephemeris extracted from D4 in step 2 or the timing parameters explicitly, with a range of the search.  A3 will apply periodicity tests to the data and generate a plot of significance of periodicity in the data as a function of trial frequencies.

 

Phase-binned analysis:

7. Assign phases to these gamma-rays (U12) by specifying either a pulsar ephemeris extracted from D4 in step 2, the timing parameters determined by step 5 or 6, or the timing parameters explicitly.

 

8. Use U4 to calculate the exposure corresponding to the gamma rays extracted by U1.  U4 can automatically generate the exposure matrices with the same cuts as used for a dataset produced by U1.

 

9. The rest of the analysis proceeds as in steps 3-6 of use case #1, with the exception that A1 can be told what phase range to include. For the likelihood analysis, it selects only the gamma rays in the specified phase range and scales the exposure by the fraction of the phase range selected.  In this way, phase resolved fluxes, spectra, and even source position maps can be derived.  The scriptability of A1 will allow this process to be automated once the desired phase ranges are found.  In particular, an initial analysis of all of the data could be used to fix the parameters of the interstellar emission model and of the other point sources in the region, and then the automated analysis could be scripted to use these values while optimizing the pulsar flux and spectrum for each phase range.

 

4.4      Binned GRB Spectral Analysis

 

1. The GBM and LAT teams localize the burst.  Notices have been sent out by GCN.

2. LAT data are extracted for a region with a radius of ~1 PSF at the lowest energies (U1 applied to D1; further cuts may be applied by U2).  Special cuts may be applied to increase the number of photons at the expense of increasing the PSF.

3. The pointing history for the time of the burst is extracted for the time of the burst (U3 applied to D2).  The time sampling may need to be increased relative to the default, at the very least for an accurate deadtime estimate.  For short bursts a single pointing direction may suffice.

4.GBM burst data is extracted (or provided by the GBM team).  Event data will be available from just before the trigger for a certain time after the trigger.  Additional binned background data will also be available.

5. Event data from other missions (e.g., Swift, Integral) may also be available.  These data will be accompanied by the appropriate detector response matrices (DRMs).  DRMs relate the incoming burst spectrum to the observed event spectrum.  The events are often called “counts.”

6. The light curves at different energies can be studied using the visualization tool (U13).  This will give the analyst a feel for the structure of the burst, its duration, and its spectral evolution.  If the analyst chooses to use his/her own time binning for the spectroscopy, this is where he/she will choose the time bins.

7. The event data for the different detectors are read into the GRB Event Binning tool (A5).  The analyst will input the desired energy bins.  He/she will choose among a number of time binning schemes.  Currently planned are:  user input; constant time width; signal-to-noise ratio (S/N) bins; and Bayesian Blocks.  For the S/N and Bayesian Blocks binning the analyst will input the desired detector data and energy range that should be used; these methods may have other parameters.  The binning tool (A5) will then bin the event data into energy and temporal bins.  The output will be PHA2 files readable by XPSEC.  The XSPEC family of files does not include the livetime or energy bins in the PHA files, but puts this information in the ARF and RMF files used for the DRMs.  A5 outputs an ARF file with the livetime equal to the width of the time bin and unit effective area, and an RMF file with the energy channels defined and a perfect energy redistribution matrix. 

8. Where relevant, A5 will also produce background files (in PHA2 format) using background models generated elsewhere. Few if any background LAT photons are expected in the region around the burst during the burst’s short duration; with an expected photon rate of a few Hz spread over a large FOV, very few non-burst photons will fall within the extraction region (which will be only ~0.03 sr).  Therefore no background calculation is necessary for LAT data.

9. DRM generating tools are called for the different detectors.  For the LAT this tool is U14.  U14 integrates the LAT’s IRF over the region the LAT photons are accumulated; this integration accounts for the photons that fall outside this region.  The DRM generating tools take in the incomplete ARF and RMF files produced by A5, and add the effective area to the ARF file and the energy redistribution matrix to the RMF file.  In addition, U14 will apply a livetime fraction to the width of the time bins in the ARF file.  A5 will produce a series of spectra.  If the spectra have different width time bins, an ARF file is necessary for each spectrum.  Different RMF files are necessary only if the angle between the burst and the LAT normal changes significantly during the burst.

10. If the GBM data are already binned, they can be rebinned in time using A6 to accommodate the temporal binning of the other data.  Note that the non-burst background is necessary for GBM data; the relevant tool will be developed outside of the LAT science tool effort.

11. The result of the above steps is the necessary input to XSPEC, which we plan to use as A8.  Standard burst spectral models, such as the “GRB” model, are in XSPEC’s library.  XSPEC can perform joint fits to different data sets.  While XSPEC can read in PHA2 files that contain more than one spectrum per file, fitting series of spectra requires a script run within XSPEC.

12. The resulting fits can be compared to the data using the visualization tool (U13).  A8 outputs a file with the spectral fits that U13 will use to compare the data and the model.

4.5      Binned GRB Temporal Analysis

 

Apply steps 1-10 of Case 4.4 to get the burst data from the different datasets binned temporally.  For most analysis methods constant time width bins should be used.  The desired energy bins can be used ab initio, or the temporal analysis tool can sum the energy bins as needed.

 

11. Input the PHA2 files with the binned data into the temporal analysis tool (A7).

 

12. Choose the desired options in A7.  For methods such as FFT, wavelet and pulse decomposition, a single time series needs to be identified (detector and energy band).  For cross-correlations two time series need to be identified.

4.6      Unbinned Burst Spectral Analysis

 

Apply steps 1-3 of Case 4.4 to extract photons from a region around the burst for a time period of interest, and to extract the pointing history for this period.  One spectrum at a time is fit in this method; thus the burst data are binned in time and unbinned in apparent energy.

 

4. Use U14 to calculate the “exposure” for the burst photons.  The precise form of this exposure will depend on the formulation of the likelihood.

 

5. Use A9 to perform the unbinned spectral analysis.  A9 uses the likelihood for the detected LAT photons; standard spectral models (e.g., power law, “GRB”) are built into A9.

 

6.  The results can be visualized using U13.

 

 

4.7      Make a Sky Map Image

 

1. Extract the gamma-ray data for the desired time range, energy range, region of the sky, event type, etc. using U1.

 

2. Generate the corresponding exposure using U4 as in use case 1 above. 

 

3. Use the map generator (U6) to bin and grid the gamma rays in the desired projection, either for the energy range and region of the sky, etc. of the initial selection (step 1) or for sub-selections of these ranges.  In general, some parameters (like inclination and azimuth) will be integrated over (or 'marginalized') in this step.  Intensity maps for energy ranges broader than the energy resolution will be constructed as the sum of intensity maps for narrower ranges, so that spectral variations will not skew the weighting of the energy ranges with respect to each other.

 

4. Do the same with the exposure using U6.  In this case, no binning is required, of course.  (The exposure is tabulated or perhaps parameterized.)  However, the same marginalizations that were applied to the gamma-ray counts must also be applied in this step.  Also, a gamma-ray spectrum must be assumed at this step, so that the energy dependence of the effective area is properly weighted.  It is likely that a standard value for the assumed spectral index (like -2.1, which was used for EGRET) is sufficient.  It is probably not worth the trouble to try to derive the spectrum (which in any case would depend on position) from the gamma-ray and exposure data themselves.

 

5. Display the resulting counts, exposure, and intensity (counts/exposure) maps using the image display tool U2.  It is assumed here that U2 understands how to make map projections and can also adjust, e.g., the color table. 

 

6. U2 and the catalog access tool U9 can be used to plot overlays of sources extracted from astronomical catalogs if desired.

 

 

4.8      Multimission Point Source Analysis

 

Perform steps 1-5 of the standard point source analysis (i.e., as in use case 1) to derive a model for the region surrounding the point source of interest.

 

6.   For the point source of interest extract count spectra, background spectra and IRFs from the non-LAT detectors.  Read this data into the likelihood tool, linking this data to the spectral parameters for the point source of interest.  If needed, define intercalibration parameters to accommodate the errors in the normalizations of the different detectors.

 

7. Perform a joint fit between the LAT and other detectors’ data.  In general, the parameters corresponding to the surrounding region will be held fixed while the parameters describing the source will be fit.

 

8. The fit to the count spectra from the other detectors can be displayed.  Various goodness-of-fit measures can be inspected.

 

4.9      Observation Simlations for Evaluating Prospective Observations

 

For proposals of pointed observations as well as for assessing the feasibility of coordinated multiwavelength campaigns, the LAT observation simulation software can be used.

 

1.   Determine the time range for the proposed observations, based on constraints such as telescope availability, source visibility, etc.

 

2.   Run the livetime/pointing simulator O1 for the desired time range.  One will generally use standard orbit parameters as input, but it will probably be desirable to consider different observation types (survey mode, pointed, or pointed scan) and observing strategies.

 

3.   Use the source model definition tool (U7) to specify the point sources and interstellar emission model to be used.  One may wish  to map out the likely parameter space of the sources by varying their spectral and temporal properties, thereby creating several different models to test.

 

4.   The livetime/pointing output of O1 and the source model(s) of U7 are then fed to the high-level observation simulator O2.  O2 creates photon event lists that are identical in format to the output of the photon database extractor U1.

 

5.   The output of O1 and O2 can then be analyzed as in the previous use cases to determine what sort of constraints can be placed on the sources given the prospective observations.

 

5         Development concept

5.1      Management

We recognize several challenges to the development of the SAE:  widely-dispersed contributors who represent a broad range of experience in astronomy and software development and work for different institutions.  We also recognize that scope of the development effort is broad, and development needs to proceed in parallel on several fronts.  Some of the components that are vital to the SAE, including the Level 1 database system and the core Likelihood Analysis tool, have stringent performance requirements and owing to the nature of the LAT and the its scanning observing mode, will not directly inherit from existing analysis systems.  Below the management approaches that are being implemented to mitigate each of these challenges are described.

 

We are already taking advantage of Internet-based tools, such as the VRVS conferencing system and instant messaging software, to keep in regular contact across the science tools definition effort.  Weekly meetings to discuss issues related to the definition of the analysis environment are already taking place (http://www-glast.slac.stanford.edu/ScienceTools/meetings) and these will transition to meetings about development progress and issues.  The SSC-LAT Working Group also meets weekly via VRVS (http://www-glast.slac.stanford.edu/ScienceTools/slwg/meetings).  Also, although the developers of the SAE will be widely distributed geographically, the code management will be via a central cvs repository at SLAC.  This has been implemented and used successfully for the development of GLEAM, the LAT instrument simulation tool.  The central repository facilitates code versioning, code sharing, and routine build testing.

 

Regarding the organization of the such a broad development effort, the tools and databases may be grouped fairly naturally by function into a few areas:  databases and related utilities for accessing them, analysis tools, observation simulation, and user interface.  Analysis tools further divide into source detection, source identification, pulsar studies, and GRB studies.  Rather than having one person (or the SSC-LAT Working Group members collectively) attempt to oversee development of all of the components and the interfaces between them, the development effort within each of the 7 areas (Table 5.1) will have an assigned manager.  The managers will be named by the SSC-LAT working group for their experience in the corresponding subject and their willingness to devote the required time to the oversight work. These managers will be responsible for monitoring implementation plans (development of prototypes), progress with respect to the schedule, component-level testing (see §6.1), and the interfaces with the other development areas.  This will permit close oversight of the development effort and clearly-defined communications between development areas.  The managers will report to the SSC-LAT working group, and completed modules will be delivered to the working group.  The latter will oversee functional testing, code reviews, and documentation reviews before acceptance.  (Note that testing and reviews like these are planned throughout the development process; see §6.  The SSC-LAT working group will oversee acceptance testing and review.)  Releases of the analysis environment will be coordinated with the planned Mock Data Challenges (described in §6.3).

 

ID	Component
	Databases and related utilities
D1	Event summary (gamma rays and all events)
U1	Event data extractor
U2	Subselections on already-extracted event data
D2	Pointing/livetime/mode history
U3	Pointing/livetime/mode extractor
D6	Other high-level (incl. GRB, transient summaries, astron. catalogs)
U9	Catalog access tool
	Source detection (likelihood analysis)
D3	CALDB for IRFs
U4	Exposure calculator
U7	Source model definition
U8	Response function visualization (for D3)
A1	Likelihood analysis
	Pulsars
D4	Pulsar ephemerides
U10	Barycenter arrival time corrector
U11	Pulsar ephemeris extractor
U12	Phase assignment
A3	Pulsar profiles, periodicity tests
A4	Period search
	GRBs
U13	GRB visualization
A5	GRB event binning
A6	GRB data rebinning
A7	GRB temporal analysis
U14	LAT GRB DRM generator
A8	GRB binned spectral analysis
A9	GRB unbinned spectral analysis
A10	GRB spectral-temporal physical modeling
	Catalog analysis
D5	LAT point source catalog
U5	Interstellar emission model
U6	Map generator
U9	Catalog access (for D5)
A2	Source identification
	Observation simulation
O1	Livetime/pointing simulator
O2	High-level observation simulator (which will contain modules for the various source classes)
	User interface
UI1	Event display
UI2	Image/plot display
UI3	User support
UI4	Command line interface and scripting
UI5	GUI interface & Web access

Table 5.1 The 7 development areas for the SAE.  The associated tools and databases are indicated.

 

At the module level, development responsibility will be assigned by the SSC-LAT working group, again based on interest and experience.  The working group reserves the right to replace managers and tool developers (in consultation with the managers) in the event that development or management effort doesn’t meet expectations.  Ultimate responsibility for the analysis environment lies with the instrument team, and in the unexpected circumstance that the overall organization of the development effort is found to be deficient, fundamental changes will be made as required.

 

For the LAT-specific core components of the SAE, we have endeavored to identify the open issues, i.e., options for implementation, that need to be evaluated.  The schedule and resource assignments (§7) have been constructed to allow time for the development decisions to be made rationally, and with fairly conservative completion dates (tied to the Mock Data Challenges) that allow time if needed for late changes of direction or other implementation challenges with the core components.

5.2      Languages

The main language we will use for implementation of the science tools is C++, making use of the object oriented design capability and superior performance (as compared with e.g., Java).  If the need arises, contributions to the code for specific algorithms can be accepted in other languages (e.g. FORTRAN) provided robust compilers are available for that language for the operating systems we will support, and the resulting compiled code can be wrapped into C++ classes. 

 

We will be using coding standards developed by the LAT team developing the low-level simulation of the LAT and the track reconstruction code

(http://www-glast.slac.stanford.edu/software/CodeHowTo/codeStandards.html).  Standards for in-line code documentation have also been adopted from LAT procedures (http://wwwglast.slac.stanford.edu/software/core/documentation/doc/codeRecommendations.pdf).

 

We are planning to provide a scripting capability to users.  The goal is to provide the user with the ability to: 1) easily build scripts to perform sets of analyses, and 2) to provide an easy interface to customize the analysis.  The idea is to contain the core functionality of the analysis tools in shareable C++ libraries.  We will then build wrappers for the C++ libraries as modules in the appropriate scripting language.  The scripting language of choice has not been finalized, but contenders are perl, python, and ruby. Note that open source tools exist (like SWIG http://www.swig.org/) which can easily build such wrappers in all three scripting languages from C++ source code.

 

Operating systems we intend to support are Windows and Linux.

 

5.3      Development Environment

Development will take place on both Windows and Linux platforms, so some of the development tools will be platform specific, and some will work on both platforms.

 

The common tools for version tracking (CVS - http://www.cvshome.org), for configuring software builds (CMT - http://www.cmtsite.org), and for code documentation (doxygen - http://www.stack.nl/~dimitri/doxygen) are good, stable, and have been successfully used by the LAT team for years now.  CVS is a version tracking program that allows remote access to the code repository.  The same repository will be used for both Unix and Windows code, as is done currently by the LAT team.  CMT allows all aspects of the build configuration to be controlled for any given package with a single “requirements“ file.  CMT uses this to automatically generate makefiles and executes them to build the binaries. The doxygen tool generates documentation in HTML and Latex, so it can be either posted for web searching (most commonly used mode) or compiled into a postscript document for printing.  The documentation is inserted in specially formatted comments in the code. 

 

The code writing/debugging environment is different for the two platforms (unix and Windows, Table 5.1)

 

Tool	Linux	Windows
C++ compiler	gcc	VisualC++
C++ debugger	ddd/gdb	VisualC++
Java compiler	javac	VisualJ++
Java debugger	jdb /jde/xemacs	VisualJ++

Table 5.2  Compilers and debuggers for the supported operating systems

The Java tools are included to cover the possibility that the graphical interface will be done with Java.

 

The intention is make the tools compatible with the HEASARC FTOOLS.  At this point it is not clear whether this means adopting the whole HEAdas package or using part of HEASARC software, such as CCFITS for reading FITS files in C++.   

 

The handling of bug reports from users is described in UI3 in Appendix C.  The SSC-LAT working group will oversee tracking of bug fixes.  During development of the SAE a bug reporting e-mail distribution list will be implemented and bug tracking software, e.g. Bugzilla (http://www.bugzilla.org) will be investigated.

5.4      Code Package Development and Hierarchy

The CMT code management tool (http://www.cmtsite.org/), which we will use for the SAE, encourages the development of modular code into ‘packages.’  Each package could contain  a high-level analysis tool or utility (the A and U components of the 'SAE), or a base utility used by more than one tool.  In addition CMT incorporates the concept of bundling constituent packages into applications.  An application can be either an executable that loads various tools into memory as needed, or a standalone executable for a single tool.  The required versions of the constituent packages that are needed for a tool can be specified in CMT.  The use of well-defined class libraries and shareables allows construction of well-separated code units (avoiding “spaghetti code”) with a hierarchy spanning basic utilities at the bottom to the analysis tools set at the top.  This encourages re-use of code through the common basic utilities.

 

A code infrastructure (sometimes referred to as an archictecture, see http://proj-gaudi.web.cern.ch/proj-gaudi/GDG/v2/Output/GDG_Architecture.html#1010951) will be defined for the SAE.  It will enforce uniform coding practices and define the general layout of the code itself.  The important common functions among the analysis tools (e.g., for i/o or manipulations of times or coordinates) will be defined by base classes and interfaces used by the packages’ algorithms.

 

Our planned use of C++ standard library components, which are widely supported across compilers, allows common functionality across platforms.

 

 

 

6         Testing methodology

We plan to base much of the testing methodology on the already existing LAT ground software test plan, described in LAT-TD-00875.  This plan describes the testing elements that will be used to ensure the quality of the software throughout the development and integration of software elements of the SAE:

·              Package standalone tests:  These consist of test programs with scope restricted to just that functionality of the package being tested.  A package can have many unit tests (i.e., tests of individual components of the package), but must have at least one that can be run by the test server that demonstrates fulfillment of the requirements for the package.

·              Application system tests:  Suites of tests applied at the analysis tool level.  For example, these could be the individual elements used in a particular analysis chain, such as data extractor, exposure calculation, model definition, likelihood analysis, and the user interface display tool.  These tests can include information specific to constituent packages and be dependent on the interaction of packages with each other as well as on the overall performance of the tools.

·              Nightly package builds:  Attempts to build and run a unit test for the package HEADs (development version of the code in the CVS repository) each night.  Packages that lack unit tests will be flagged.

·              Code reviews:  Periodic reviews of code at the package level, examining adherence to coding and documentation rules, with an eye to visual inspection of algorithms.

·              Performance reviews at the analysis tools level

 

The Release Manager server described in LAT-TD-00875 will automatically run the unit and system tests on the supported operating systems daily. The results will be stored in a database and compared to expectations.  Failures of any of the automated tests will be reported to the package owner and Release Manager.

 

6.1      Component-level testing

Each component, or software package, must be demonstrated to meet the requirements specified for it in the appropriate requirements document.  This demonstration must include a code walk-through that ensures:

·        Consistency of style

·        Clarity of comments

·        Clarity of coding

·        Use of standard programming practice

·        Use of standard language syntax.

 

In addition, packages must perform the function(s) required, which will be demonstrated by running them in a standalone mode with the test server.  This compliance will be recorded in a database and be available for review.  In the event that delivered packages do not meet the specifications, the developers will be notified for additional work and testing.

 

6.2      Suite of tests

As mentioned in §5.4, the constituent packages for an analysis tool are defined by CMT ‘checkout packages’ which fully specify the version numbers.  Major checkout packages, generally individual analysis tools, will have test programs. They may have multiple tests, since several input configurations may be needed to verify performance.  It is expected that the testing software run by the Release Manager will create datasets (or reference precomputed datasets) for the various test configurations and then run the tests to generate diagnostics.

 

Diagnostics standards will be maintained for comparison against the current tests. The results of these tests will be recorded by the Release Manager and compared against thresholds for flagging suspect results.  The standards will be updated as deemed necessary to track the current understanding of the code performance.

 

Each requirement shall be included in at least one test of the component required to perform it as well as in the overall test of the software.  The analysis tools will have tests that exercise each of the requirements placed on it and its components.  Entire analysis tools must demonstrate that they continue to perform the requirements of each component package as well as requirements placed upon the tool as a whole.

 

Because requirements for the tools generally include the specifications of performance in terms of time or resources, the test suites will also record benchmarks of the software performance.

6.3      End-to-end Tests and Mock Data Challenges

End-to end testing of the SAE will start with running a simulated delivery of data through the Level 1 processing pipeline and into databases available for high-level analysis.  Analysis of this simulated Level 1 data will be referred to as a Mock Data Challenge.

Mock data challenges will exercise the analysis software in a real-life simulation where the tester will be required to infer, using the analysis tools of the SAE, the scenarios contained in the simulated data.  We expect that the data challenges will deliberately insert less-than-perfect information content into the mix.

 

It is anticipated that Mock Data Challenges will expose performance bottlenecks and logical errors  that are only triggered by the sequential and interdependent exercise of software elements that relate to each other in ways that may not have been tested when the software elements were tested independently.   The data challenges will also allow judgment to be made of the overall robustness of the system and will identify operational weaknesses or gaps in requirements.

 

As described in §7, the development schedule for the SAE includes 2 Mock Data Challenges, the first occurring before the SAE is still just partially functional.

7         Schedule for development

7.1      Milestones

In terms of the definition of the SAE, the next formal milestone will be the LAT Critical Design Review (CDR) in April, 2003.  As part of the preparation for that review, the LAT team will conduct an internal review of all aspects of the ground software, including the SAE, in March, 2003.  (Regarding the SAE, the internal review will be coordinated with the SSC.)

 

In terms of the development of the SAE, we plan to tie the development schedule to the staging of 2 Mock Data Challenges (MDCs).  The first is planned for June-October, 2004 and the second for June-December 2005.  As described in §6.3, the MDCs are effectively end-to-end tests of the SAE.  (The core tools of the SAE will be ready for the first MDC, although the SAE overall will not be complete.)  For generation and analysis of the simulated data, the MDCs will involve the participation of many LAT team and SSC members who are not science tools developers.  The MDCs will therefore be rigorous tests of the usability of the SAE, including, e.g., the documentation of the tools.

 

In part owing to their great value for testing the SAE, the MDCs will clearly be major distractions to the developers, and therefore we are scheduling only 2.  The nominal duration of each challenge is fairly long, to allow 1-1.5 months for defining the datasets, 1 month for the analysis and 1-2 months for reconciliation of the results.  For the first MDC, only some of the GRB tools and none of the pulsar tools will be ready.  The second MDC is planned to be a test of the full SAE.

 

7.2      Schedule overview

The overall schedule for the development of the SAE is presented in Appendix B.  The schedule identifies the dependencies between the tools – this information is used for sequencing the development activities.  For components (i.e., tools or databases) that we recognize have important open issues regarding the implementation (e.g., the Level 1 database D1 and the likelihood analysis A1), specific time for prototyping is allotted in the schedule.  These tools are also scheduled to begin development sooner rather than later, to mitigate risk from unanticipated difficulties.

 

The estimated full-time equivalent (FTE) labor required to develop each component shown in the schedule is an educated guess.  Our intent is that they should be conservative estimates, and they are assumed to include the overhead for preparation of testing modules, the support of code reviews, and preparation of documentation (including user manuals).  The pulsar analysis tools have well-known algorithms, and most of the GRB tools have fairly similar counterparts in analysis systems for other GRB instruments (e.g., BATSE and GBM).  These considerations are reflected in the labor estimates.

 

We are more confident in the relative estimates of required FTEs, which are based on our understanding of the relative complexities of the tools and databases, than in the absolute estimates of FTEs required.  We note that the schedule has the entire development effort complete by the end of the reconciliation period for the second MDC, i.e., at the end of 2005, nine months before launch.  The remainder of the time before launch is planned to be a period of maintenance and refinement of the tools, but can also be considered as development time held in reserve in case the development time estimates prove to be too optimistic.  Again, the schedule also has the core tools of the SAE developed first, so any need to use this ‘cushion’ period should be apparent relatively early.  (The great majority of the labor for the development of the SAE will be contributed from across the collaboration and so does not represent a direct cost to NASA or the DOE; risk in the development schedule for the SAE represents therefore represents relatively limited cost risk in terms of NASA or DOE funds.)

 

7.3      Resources required vs. available (over time)

Table 7.1 summarizes the distribution of labor for the development of the SAE, broken down by quarter and divided into development areas.  Of the overall total, about 40 person-years, approximately 25% is assigned to the core analysis tools (labelled ‘Likelihood analysis’; see Table 5.1), and 20% to the databases and related utilities.  Pulsar and GRB-related tools have about 10% of the development effort each, a reflection on the availability of algorithms and even prototype tools from other missions.

 

	2002	2003				2004				2005				Total
Development area	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3	Q4
Databases and related utilities	6.3	6.3	6.3	3.3	3.3	2	2	0	1.2	1.2	1.2	1.2	1.2	35.5
Analysis tools groups
Likelihood analysis	6.7	6.7	7.7	6.7	5.4	4	4	0	0	0	0	0	0	39.2
Pulsar analysis	0	0	0	0	0	0	0	3	3	7	5	0	0	18
GRBs	1.9	1.9	2.8	2.8	1.3	0.9	1.6	0.8	0.8	1	1	0.25	0.25	17.3
Catalog analysis	0	0.5	0.5	0.5	0.5	3.3	3.3	3.8	1.8	1.8	0.5	0	0	16.5
Observation simulation	3	3	3	3	2	0	0	0	0	0	0	0	0	14
User interface	1.7	3	3.8	3.1	2.5	2.5	2.5	1	1	1	1	1	1	25.1
Total required	19.6	21.4	24.1	19.4	15	12.7	13.4	8.6	7.8	12	8.7	2.45	2.45	167.6
Est. FTEs available1	11	14	14	15	16	17	17	17	17	17	17	17	17	206

Table 7.1  Distribution of FTEs by development area and time.  The entries are in FTE-quarters; for FTE-years, divide by 4.  1 Approximate totals, from survey by institution among participants at the June 2002 Science Tools Workshop at SLAC.

 

Table 7.1 also compares the distribution of labor needed with what is presently estimated to be available from the LAT collaboration and the SSC.  The estimates of available labor come from a survey taken at the June 2002 Science Tools Software Workshop and include the SSC, SU-HEPL, SU-SLAC, CEA Saclay, INFN, IN2P3, and Ruhr-Universität Bochum.  The estimates of available labor are just projections, but they are the best estimates available.  The table indicates a significant shortfall in the available labor early in the development schedule, but ample labor available over time to develop the SAE.  The near term shortfall is certainly a concern, but may not impact the overall development schedule.  The available labor is projected to amply exceed the requirements well before the first MDC.  The labor projections in Table 7.1 are derived under the assumption that the effort estimated to be required to develop each tool is expended uniformly over its development time defined in Appendix B.  We anticipate that the distribution of labor can be shifted so that more can be assigned to later quarters leading up to the first MDC without affecting readiness for the MDC.  The ample surplus of available labor in subsequent years is not at all certain, on consideration of the years of extrapolation from the present.  We will continually reassess the projected requirements for labor, and will find a suitable way to address any surplus in the happy circumstance that one arises.

 

Additional labor to support MDCs – for analysis of simulated data, for example – is expected to be contributed by LAT team and SSC members not affiliated with the science tools development effort.  Therefore, accounting for their time is not included in the estimated development times for the various components of the SAE.

 

Appendix A—Glossary

Base Class – A class upon which other classes are derived.  Base classes can define a core set of functionality that is generally useful to other more specific classes.

 

CALDB—CALibration Data Base, the HEASARC’s directory system for storing the calibration files used in producing IRFs.

 

CMT—Code Management Tool, a utility to enable package maintenance and code building, supporting multiple platforms.

 

CVS – Concurrent versions system, for management of collaborative software development (http://www.cvshome.org/)

 

EGRET—Energetic Gamma-Ray Experiment Telescope, the LAT’s forerunner on CGRO.

 

Event—A particle or photon that triggered the LAT, survived the on-board cuts, and whose data was sent to the ground.

 

FITS—The standard data format in high energy astrophysics.  The files consist of ASCII headers with standard keywords, and binary data tables.

 

FOV—Field of View, the area of the sky that an instrument views.

 

GBM—GLAST Burst Monitor, GLAST’s secondary instrument to detect and localize bursts, and extend the spectral coverage of GRBs to lower energies.

 

HEAD-Most current versions of modules currently under development and stored in CVS

 

HEAdas – Software library containing a mission-independent set of FITS utilities (HEAtools) developed by the HEASARC.  These include tools to access CALDB.  HEAdas is also meant to contain mission-specific packages developed by outside groups.

 

HEAD

 

HEASARC—The organization within GSFC’s LHEA that archives high energy astrophysics data.

 

Interface – defines a related set of methods.  An interface defines the inputs and outputs of the methods, but not their implementation.

 

IOC—Instrument Operations Center, the instrument team’s organization that ingests data from the instrument, monitoring housekeeping information and processing the science data.

 

IRF—Instrument Response Function, the mapping between the incoming photon flux and the observables reported by the instrument

 

LAT—Large Area Telescope, GLAST’s primary instrument

 

Level 0 data—The raw data from the spacecraft after the telemetry packets have been time-ordered and repeated packets have been removed.  The data streams from the spacecraft and the instruments are separated.  The processing that produces Level 0 data is called Level 0 processing.

 

Level 1 data—Data from which many of the instrumental artifacts have been removed and that are ready for astrophysical data analysis.  The processing that produces Level 1 data is called Level 1 processing.  LAT Level 1 data consist of reconstructing a description of an event from the tracks within the LAT.

 

LIOC—LAT IOC

 

Package—A unit of code modules contributing to a single function. These are defined in the CMT environment.

 

PHA, PHA2

 

Phase 0—The first 30-60 days after launch during which the spacecraft and instruments are turned on and checked out.

 

Phase 1—The first year of scientific operations after launch and checkout during which the spacecraft will survey the sky and the instrument teams will validate the data.  During Phase 1 the LAT Level 1 data will be accessible only to the LAT instrument team and a few guest investigators.

 

Phase 2—The mission after the conclusion of Phase 1 until the spacecraft leaves orbit or is turned off.  During Phase 2 the LAT Level 1 data will be accessible to the general community as soon as they are processed.

 

Photon—A LAT event identified by the Level 1 processing as a photon.  In X-ray astronomy this would be called a “count.”

 

PSF—Point Spread Function, the characteristic radius of the circle on the sky within which the photons from a source at the center fall

 

Release Manager (RM)—Server process which performs and tracks release and nightly code builds, and system tests.

 

Appendix B—Schedule for development of the Standard Analysis Environment

 

Component		Depends on for start of devel.	Depends  on for testing or completion	Which MDC? (*)	Start proto-typing	Finish proto-typing	Finish implement-ation & testing	Total FTE-years	Notes w/FTEs/unit time
Databases and Related Utilities
Event summary access
D1	Event summary (gamma rays and all events)	none	O2	1	02 Q4	03 Q1	03 Q4	2.5	2
U1	Event data extractor	D1		1	02 Q4		03 Q2	1	4/3
U2	Subselections on already-extracted event data	U1		1
Pointing information
D2	Pointing/livetime/mode history	none	O1	1	02 Q4	02 Q4	03 Q2	1.5	2
U3	Pointing/livetime/mode extractor	D2		1	02 Q4		03 Q2	0.75	1
Other databases and access
D6	Other high-level (incl. GRB, transient summaries, astron. catalogs)	none		2+	04 Q4		05 Q4	1.5	6/5
U9	Catalog access tool	D6		2+	04 Q1		04 Q2	0.5	1
Analysis Tools
Likelihood analysis
D3	CALDB for IRFs			1	03 Q2		03 Q3	1	1, depends on getting the IRFs
U4	Exposure calculator	O1, D3		1	02 Q4	03 Q1	03 Q4	3	2.4
U7	Source model definition	UI2	O2, D3,U6	1+	02 Q4		03 Q2	1	4/3
U8	Response function visualization (for D3)	D3		1	04 Q1		04 Q2	0.5	1
A1	Likelihood analysis	O2, U4	UI2	1+	02 Q4	03 Q3	04 Q2	5	~3
Pulsar analysis
D4	Pulsar ephemerides	none		2	04 Q3		05 Q1	0.5	2/3
U10	Barycenter arrival time corrector	U1		2	04 Q3		05 Q1	0.5	2/3
U11	Pulsar ephemeris extractor	D4		2	04 Q3		05 Q1	0.5	2/3
U12	Phase assignment	U10, O2	U11	2	05 Q1		05 Q2	0.5	1
A3	Pulsar profiles, periodicity tests	UI2, A3		2	05 Q1		05 Q2	1	2
A4	Period search	UI2, O2		2	05 Q1		05 Q2	1	2
GRBs
U13	GRB visualization	UI2		1+	02 Q4	03 Q2	04 Q1	0.6	0.4
A5	GRB event binning	none	U1	1	02 Q4	03 Q1	03 Q3	0.5	0.5, prototyping deadline set by Swift
A6	GRB data rebinning	none	A5	1	03 Q2		03 Q4	0.1	0.4/3
A7	GRB temporal analysis	U14		1	05 Q1	05 Q2	05 Q4	0.25	0.25
U14	LAT GRB DRM generator		D3	1	02 Q4	03 Q1	03 Q3	0.5	0.5
A8	GRB binned spectral analysis	U14		1	02 Q4	03 Q1	03 Q3	0.5	0.5, prototyping deadline set by Swift
A9	GRB unbinned spectral analysis	D3	U1, U4	2	04 Q2	04 Q4	05 Q2	1	0.8
A10	GRB spectral-temporal physical modeling	none		1+	03 Q2	03 Q4	04 Q2	1	0.8
Catalog analysis
D5	LAT point source catalog	none	none	2+	04 Q1	04 Q3	05 Q1	1	0.8
U5	Interstellar emission model	none	none	1+	03 Q1	04 Q1	05 Q1	1	0.5, & a lot of contrib. outside of SAS
U6	Map generator			1	03 Q2		04 Q2	1	0.8
U9	Catalog access (for D5)	none	D5	2+	04 Q3	04 Q4	05 Q2	0.5	0.5
A2	Source identification	none	other catalogs	2	04 Q1		04 Q3	1.5	2
Observation Simulation
O1	Livetime/pointing simulator	none		1	02 Q4		03 Q3	1	1
O2	High-level observation simulator (which will contain modules for the various source classes)	none	O1	1+ (pulsar modules later)	02 Q4	03 Q1	03 Q4 (core)	2.5	2, 05 Q1 for complete set of source models
User Interface
UI1	Event display	none	none	2			03 Q4?	0	0, tool will be inherited
UI2	Image/plot display			1	02 Q4	03 Q1	03 Q2	0.5	2/3, tool will be selected
UI3	User support			2	02 Q4		05 Q4	3.25	1
UI4	Command line interface and scripting			1	03 Q1		03 Q3	1	4/3
UI5	GUI interface & Web access			1+	03 Q4		04 Q2	0.5	2/3

 

 

Appendix C—Detailed Tool Descriptions

Databases

 

Date:    13 September 2002 improved some comments in the Existing counterparts section; 9 Sept 2002 (draft v3,MH removed barycenter offset; 28 August 2002 SD removed barycenter arrival time of gamma ray, corrected event rate from 25 to 30 Hz, record size from 250 to 200 bytes; 7 May 2002, now includes sun and moon angles)

Function

This database will house all the reconstructed events from the LAT and carry all the information necessary for higher analysis. It is not directly accessed by the analysis tools.  It receives queries from, and passes data to, the Data Extractor (U3).

Inputs (for gamma rays)

See inputs for the Data Extractor (U3). 

Databases required

NA

Outputs

The provisional contents of the database, defined in the report of the Data Products Working Group as LS-002 (and updated to include the angles of the sun and moon – see U4), are as follows:

 

	Column Name	Units
1	Energy of event	GeV
2	1-sigma uncertainty of energy	GeV
3	RA, J2000	deg
4	Dec, J2000	deg
5	Localization uncertainty, est. 1-sigma radius	deg
6	Time (Mission Elapsed Time)	s
7	Conversion layer	dimensionless
8	Number of SSD hits	dimensionless
9	Number of tracker hits NOT reconstructed	dimensionless
10	Conversion point, (x,y,z)	m
11	Reconstruction trajectory-event (dir. cosine)	dimensionless
12	Reconstruction trajectory-secondary 1 (dir. cosine)	dimensionless
13	Reconstruction trajectory secondary 2 (dir. cosine)	dimensionless
14	Secondary energies	GeV
15	ACD tiles hit (bit flags)	dimensionless
16	Quality_Parm--quality parameter for fitted trajectory	dimensionless
17	Data_Quality--Overall status of event	dimensionless
18	Deadtime accumulated since start of mission	s
19	Instrument mode (slew, diagnostic, ...)	dimensionless
20	TKR, CAL-only flag--2bit for CAL, TKR or Both	dimensionless
21	Zenith angle	deg
22	Earth azimuth angle (from north to east)	deg
23	S/C position from earth center, (x,y,z)*	km
24	Angle from sun	deg
25	Angle from moon	deg
26	Ground point--lat.	deg
27	Ground point--lon.	deg
28	McIlwain B	gauss
29	McIlwain L	Earth radii
30	Geomagnetic Lat.	deg
31	Geomagnetic Long.	deg
32	Recon. Version	dimensionless
33	Calib. table versions	dimensionless
34	Multiple event reconstruction	dimensionless
35	Reconstruction number within event	dimensionless
36	Original Event ID	dimensionless

 

* x-axis is direction RA,DEC = 0,0, z-axis is north, y-axis defined for earth-centered right-hand coordinate system, J2000

            

Also from the DPWG report:

The reconstructed trajectory of the photon (item 13) is in instrument coordinates, and so specifies the inclination angle and azimuth of the photon.                                                                             

Instrument modes (item 19) should probably be defined in a second extension.  

Perhaps multi-gamma events should just have a 'primary' gamma ray defined here, plus a flag set to indicate that special processing should be done.                                                                     

Note that >1 photon can be claimed by Reconstruction (may translate into multiple Event Summary entries, with number_of_photons entry)                                                                      

Quality flags above are intended to be representative of the background rejection/PSF enhancement cut parameters      

 

Performance requirements

See LAT Event Summary Database Requirements Document

Other modules required

None.

Host environment

Database server system

Existing counterparts

Nothing directly applicable.  Spatial indexing methods (hierarchical tessellations of the sky) for facilitating access to data by region of the sky have been designed for other astronomical applications.  Two widely-used indexing schemes (HTM and HEALpix) are well documented and some software is available for efficient coordinate-spatial index conversion is available.  Owing to the scanning mode of operation of the LAT, which implies that data for a given region of the sky are spread out in arrival time, and the nature of the Level 1 database (relatively static, with most access being read-only), sequential searches of flat files offer performance competitive to relational database searches.

 

Each entry requires roughly 200 bytes of storage, and with a telemetry-limited event rate of 30 Hz (including cosmic rays), this yields about 200 Gb of data a year.

 

Data retrieval times from 2-dimensional spatial queries are the most relevant performance statistic.  We are currently benchmarking performance to determine these and other performance measures for potential database architectures.  One enhancement we are likely to make is to separate the gamma rays into a their own database since they will be the primary focus of the analysis.  (Cosmic-ray events in the telemetry will primarily be used by the LAT team for low-level calibration.)  Rapid retrieval of data from a 20 Gb/year photon database should not be problematic.  Other issues being considered are ease of maintenance, ease of updates (due to reprocessing), and ease of backup.  We expect to choose the database architecture by the end of 2002.

Open issues for definition or implementation

 

1. Some input parameters for the Data Extractor (U3) might not be passed on to the Event summary database, depending on whether the Data Extractor does any ‘post processing’ on retrieved data and on the implementation of U2.  This is probably neither here nor there as far as the requirements summaries go, however.

 

Date:    13 September 2002 (draft v7 fixed reference to Pointing extractor); 7 May 2002 (draft v2, added sun and moon positions, v7, added)

Function

This is the database of pointing and observation history that is needed to calculate exposures.  It contains information about the orientation, location, and operation of the LAT for regular time intervals, ~30 s.  It is not directly accessed by the analysis tools.  Instead, it receives queries from, and passes data to, the Pointing/livetime history extractor (U3).  The positions of the sun and moon are included here solely to facilitate cuts on their positions in the generation of exposure.  They are both gamma-ray sources (the sun impulsively) and both of course shadow sources they pass in front of.

Inputs

See inputs for the Pointing/livetime history extractor (U3). 

Databases required

NA

Outputs

The provisional contents of the database, defined in the report of the Data Products Working Group as LS-005 and augmented here to include the SAA flag and positions of the sun and moon, are as follows:

 

	Contents	Units
1	starting time of interval (Mission Elapsed Time)	s
2	ending time of interval (Mission Elapsed Time)	s
3	position of S/C at start of interval (x,y,z inertial coordinates)	km
4	viewing direction at start (LAT +z axis), 2 angles	dimensionless
5	orientation at start (LAT +x axis), 2 angles	dimensionless
6	zenith direction at start, 2 angles	dimensionless
7	LAT operation mode	dimensionless
8	livetime	s
9	SAA flag	dimensionless
10	S/C longitude	deg
11	S/C longitude	deg
12	S/C altitude	km
13	direction of the sun, 2 angles	deg
14	direction of the moon, 2 angles	deg

Performance requirements

Performance requirements will be considered in detail in the full requirements document.  None of the requirements is likely to be challenging in terms of implementation of the database.  See Science Tools Database Requirements Document

Other modules required

None.

Host environment

Database server system

Existing counterparts

Nothing directly applicable.

Open issues for definition or implementation

1. Should latitude, longitude, and altitude be translated into geomagnetic coordinates in anticipation of any need to generate exposure maps for different ranges of geomagnetic conditions?  (This might be useful, e.g., if the background rejection is not extremely good.)

 

 

Date:    6 Sept 2002 (draft v2, updated 15 Sept 2002)

 

Function

The GLAST CALDB is the directory and indexing structure that stores the calibration files required for producing the instrument response functions (IRFs) for a given analysis; the analysis tools will extract the relevant data from the CALDB structure.  IRFs are the high-level specifications of the performance of the LAT.  As is customary, the response will be decomposed into the point-spread, effective area, and energy redistribution functions.  These response functions depend on photon energy, direction of incidence, plane of conversion in the LAT tracker, and event quality flags.  They may also be expected to be functions of time, in the event that the instrumental response changes significantly, e.g., as a result of partial failure of one subsystem.

 

At a minimum the CALDB structure stores those FITS-format calibration files that are directly necessary for calculating the IRFs.  Thus CALDB may include files with the PSF as a function of energy and inclination angle, energy redistribution matrices, etc.  Different versions of the calibration files are retained, permitting users to replicate previous analyses that used earlier versions of the calibration files. 

 

Thus the functions of the CALDB system are:

1. To store and archive calibration files.

2. To provide a naming convention and header structure for all calibration files.

3. To index the calibration files for software access based on FITS header keywords.

4. To update the calibration data independent of software updates, while maintaining configuration control.

5. To provide a traceable history of the database’s calibration files by maintaining and identifying different versions.

6. To provide an interface between calibration files in a FITS-format understood by the IRF-generating software and IRF files with recognized FITS-formats (e.g., PHA files understood by XSPEC).

Inputs

The exact set of instrumental parameters to be used for tabulating the response is still being evaluated.  For example, the scientific return for tabulating the variation of energy resolution plane-by-plane within the tracker vs. tabulating it just for the front and back sections of the tracker, needs to be determined via analysis of simulations.  (It should be noted that response functions for ‘CAL-only’ events where the gamma ray does not convert until it enters the calorimeter likely also will be needed.)  The best choice of event quality flags will be established by study of the behavior of event reconstructions.  It must be decided whether the response functions should be represented as parameterized functions or matrices of numbers.

 

Databases required

 

NA

Outputs

If the functions are to be stored as matrices, the grid points must be spaced closely enough to allow linear interpolation.  Here we present some rough estimates of the storage required, subject to revision.  The photon energy values will necessarily be unequally spaced, perhaps equally spaced in log(energy).  Approximately 200 energy values should be adequate.  About 50 values of polar angle of incidence should suffice to cover the entire field of view.  Equal spacing in the cosine of the angle may be the most useful choice.  It is expected that there will be little variation in the response functions with azimuthal angle of incidence, but this must be checked with simulations.  If azimuth is relevant, then at most 16 values should be needed.  The number of event types will probably not exceed 10.  Combining these sizes leads to an upper limit of 200 * 50 * 16 * 10 = 1.6 million floating point numbers for the effective area matrix.  This is a very modest amount of storage, even if different tabulations are maintained for each plane of conversion.

 

The other response functions will require more storage.  The energy redistribution function will have another index for the measured energy, with perhaps 50 values needed.  This leads to a matrix size of 80 million numbers.  If the point spread function is radially symmetric in the estimated angle, then it will require the same storage.  It is likely that there will be a significant “fisheye” distortion at large inclinations, so a further factor of up to 40 may be needed.  That would lead to an upper-limit storage requirement of 3.2 billion numbers per plane of conversion tabulated.  Such a large object may become unwieldy, so it will be necessary to seek ways to factor the problem further.

 

The CALDB system defined by HEASARC can provide a workable implementation of the response functions, and that is the intended approach.  Some of the standard CALDB file formats are suitable for the matrices described above.  The HEASARC provides utilities for both remote access and local installation of CALDB files.

Performance requirements

Performance requirements will be considered in detail in the full requirements document.  None of the requirements is likely to be challenging in terms of implementation of the database.

Other modules required

None.

Host environment

At the SSC the GLAST CALDB will be incorporated within the HEASARC’s CALDB system, but other sites will have stand-alone CALDB structures modeled after the HEASARC’s system.  User may download a subset of the CALDB containing only the desired version of the calibration files, or may access the CALDB remotely via the internet.

Existing counterparts

The basic structure and use of the CALDB has been specified by the HEASARC. Documentation for setting up and managing a HEASARC-compatible CALDB exists online at http://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_doc.html, wherein numerous documents are available.

Open issues for definition or implementation

1. There is an area in which the needs of the LAT don’t match up very well with CALDB.  Most of the existing CALDB formats prefer that the measured energy be expressed in PHA channels.  The energy of each LAT photon is estimated by combining information from several calorimeter PHAs and from many tracker hits.  Variations in detector gains and in the list of dead tracker strips will be absorbed in the algorithm.  Of course, this will require a new category of calibration data objects for this information.

 

 

Date:    13 September 2002 (draft v8, added link to req. document); 9 September 2002 (draft v4, edited by M. Hirayama)

Function

This is the pulsar timing information to be maintained during the LAT mission for assigning pulsar phases to gamma rays.  A user can access it by using the Pulsar Ephemeris extractor (U11) or the Catalog Access tool (U9).  The pulsar timing information mainly originates from radio timing observations, but this database will contain radio-quiet pulsars (e.g. Geminga pulsar).

Inputs

See inputs for the Pulsar Ephemeris extractor (U11) and the Catalog Access tool (U9).

Databases required

NA

Outputs

See outputs for the Pulsar Ephemeris extractor (U11) and the Catalog Access tool (U9).

Contents

The contents of the database are itemized below.  They are defined based on the pulsar timing files used for EGRET for familiarity to the user community.  For generality and consistency with format provided by pulsar observers, times in this file should be specified in MJD rather than Mission Elapsed Time. The second table below contains the additional information required for binary pulsars.

 

Spin parameters

Item	Contents	Units
1	Pulsar name	dimensionless
2	Right Ascension (J2000)	deg
3	Declination (J2000)	deg
4	Start of interval of validity for timing info (MJD)	days
5	End of interval of validity (MJD)	days
6	Infinite-frequency geocentric UTC arrival time of a pulse (MJD)	days
7	Pulsar rotation frequency (*)	Hz
8	First derivative of pulsar frequency (*)	Hz^2
9	Second derivative of pulsar frequency (*)	Hz^3
10	Root-mean-square radio timing residual (periods)	dimensionless
11	Source of timing information	dimensionless
12	Flag for binary pulsars	dimensionless

(*) The integer part of item 6 is the barycentric (TDB) epoch of items 7-10.  The integer part and the fraction part should be stored separately, because standard double precision is not sufficient to keep the required precision for the sum.

 

Orbital parameters: for binary pulsars only

Item	Contents	Units
13	Orbital period	s
14	Projected semi-major axis	s (light travel time)
15	Orbital eccentricity	dimensionless
16	Barycentric time (TDB scale) of periastron (MJD)	days
17	Longitude of periastron	deg
18	First derivative of longitude of periastron	deg per Julian year
19	Time-dilation and gravitational redshift parameter	s
20	First derivative of orbital period	dimensionless
21	Source of orbital parameters	dimensionless

 

Performance requirements

Performance requirements will be considered in detail in the full requirements document.  This database is not expected to be particularly large or to have particularly demanding access requirements.  See the Science Tools Database Requirements Document.

Other modules required

None.

Host environment

Database server system if required (see open issues below).  This database should be accessible through a web interface.  A user can download this database to his/her server for faster access.

Existing counterparts

Nothing directly applicable.  The corresponding files for the EGRET analysis system are plain ASCII files.

Open issues for definition or implementation

There are a couple of ways to organize this database.  Two configuration of database are considered feasible at the moment: a single flat table and four structured tables.  These options are explained below.

 

1. a) Single flat table

The database consists of single table with one row for each pulsar entry that contains all the items in the table above.  The structure is very simple, but multiple entries for a single pulsar must be allowed, and multiple names for a pulsar (e.g., Crab and PSR0531+21) are not allowed in this scheme.

 

b) Four structured tables

The database consists of four data tables: a pulsar name table, a spin parameter table, an orbital parameter table, and an originator code table.  Each table contains the following items.  The pulsar name table is separate from other tables to support multiple names for a pulsar such as the Crab pulsar.

 

In the tables below, pulsar ID, spin parameter ID, orbital parameter ID, and originator ID are a sequential number uniquely assigned in each table.

 

Pulsar name table

Primary key:  pulsar name

Data item:  pulsar ID

 

Spin parameter table

Primary key:  spin parameter ID

Data items:  pulsar ID, items 2-12 in the table above.  Item 12 is a primary key in the originator table.

 

Orbital parameter table

Primary key:  orbital parameter ID

Data items:  pulsar ID, items 13-21 in the table above.  Item 12 is a primary key in the originator table.

 

Originator table

Primary key:  originator ID

Data items:  name of originator, description of originator, name of contact person, contact information (e.g. postal address, phone number, and e-mail address)

 

Three options to implement this database have been identified:

 

1. Single file

The simplest implementation of this database is to have a single file (preferably in ASCII format or FITS).  This allows a user to download the entire database onto his/her local machine without any database software installed.  However, update, maintenance, and transaction control must be done manually.

Note that either of the two configuration above (flat-table configuration and four-table) can be put in a single file, for example, by using FITS extensions.

 

2. Conventional database system

This database can fit into any relational database available.  This allows easy maintenance, update, and transaction control for the database maintainer.  However, in order to download this database to a user’s local machine, he/she has to install the same database software as the GLAST team has.

 

Combined solution: database system at the server, single file at a client

This database can be implemented in a relational database system at the server machine for easy maintenance, while the downloaded database can be in a single file so that a user can access it without any database software.  This solves all the issues mentioned above, but it requires the Pulsar Ephemeris Extractor tool (U11) to be capable to handle two different forms of ephemeris database.

 

The database must be updated by the person(s) that are designated by the GLAST collaboration/operation team.  The update procedure is not determined yet.

 

 

Date:    13 September 2002 (draft v7 – updated link to requirements document); 25 April 2002 (draft v1)

 

The LAT team will produced and maintain a catalog of detected sources in the LAT data. The catalog will be valuable both for further study itself and as a resource for definition of models in U7.

Function

This is the online form of the point source catalog (to be constructed by the LAT team).  It is not directly accessed by the analysis tools, but instead receives queries from, and passes data to, the Catalog Access tool (U9).

Inputs

See inputs for the Catalog Access tool (U9). 

Databases required

NA

Outputs

The provisional contents of the database, defined in the report of the Data Products Working Group as LS-008, are as follows:

 

	Contents	Units
1	source name (“telephone number”)	dimensionless
2	RA	deg
3	Dec	deg
4	th68 semimajor, semiminor axis, and position angle	deg
5	th95 semimajor, semiminor axis, and position angle	deg
6	flux (>100 MeV, avg. for the time interval of the catalog)	cm-2 s-1
7	flux uncertainty, 1 sigma (as above)	cm-2 s-1
8	photon spectral index (avg)	dimensionless
9	variability index	dimensionless
10	significance (avg)	dimensionless
11	significance (peak)	dimensionless
12	peak flux (for time interval above?)	cm-2 s-1
13	peak flux uncertainty	cm-2 s-1
14	time of peak flux (wrt reference date)	s
15	interval of time	s
16	flux history	cm-2 s-1
17	flux uncertainty, 1 sigma (as above)	cm-2 s-1
18	start times of flux history entries	s
19	end times of flux history entries	s
20	candidate counterparts	dimensionless
21	degrees of confidence for the counterparts	dimensionless
22	flags (confusion, low latitude,…)	dimensionless

 

Performance requirements

Performance requirements will be considered in detail in the full requirements document.  This database is not expected to be particularly large or to have particularly demanding access requirements.  See the Science Tools Database Requirements Document.

Other modules required

None.

Host environment

Database server system

Existing counterparts

Nothing directly applicable.  Spatial indexing methods (hierarchical tessellations of the sky) like those developed for the Level 1 database (D1) might be carried over directly to this application, although the size of the database will be so much smaller that spatial indexing probably will not be required for adequate performance.

Open issues for definition or implementation

1. Are there any other quantities that should be included in the catalog entry for each point source?  Note that typical sources, i.e., sources not much brighter than the detection limit, will have poorly-determined spectra, and so in general detailed spectral information probably does not belong in the catalog.

2. Should multiple versions of the catalog be defined?  In the analysis process, a large catalog of all detections of sources (say, on a daily, weekly, or longer timescale) could be compiled, then that catalog cross-correlated with itself to determine which sources had (probably) been seen more than once.  This process would also be useful for searching for transient sources.

 

 

Date:    10 September 2002 (draft v5, edited by R. Schaefer)

Function

 

D6 is a catch-all designation for all other catalogs used by the tools.  These catalogs include the GRB and LAT transient source catalogs, which are both largely subsets of the LAT point source catalog.  It is also possible that the grand catalogue just compiled for the INTEGRAL mission may be used.  This catalog contains information about sources observed by other instruments (e.g., the EGRET point source catalog, and x-ray point source catalogs).   D6 will likely be implemented as tables in a conventional database system, such as Oracle and MySQL, or left as ASCII tables to be browsed by U9.

Inputs

See inputs for the Catalog Access Tool  (U9).

Databases required

NA

Outputs

See outputs for the Catalog Access tool (U9).

 

The database will contain at least LAT derived burst and transient catalogs.  Queries to the database will return the following provisional rows of information based on the Data Products Working Group report.

 

LAT Transient Catalog:

The output of queries to the LAT Transient Catalog as defined in the report of the Data Products Working Group as LS-007

 

Column Number	Column Name	Units
1	source name (“telephone number”)	dimensionless
2	RA (best position, e.g., from when transient was brightest)	deg
3	Dec	deg
4	th68 semimajor, semiminor axis, and position angle	deg
5	th95 semimajor, semiminor axis, and position angle	deg
6	flux (>100 MeV, avg. for the time interval of the detection)	cm-2 s-1
7	flux uncertainty, 1 sigma (as above)	cm-2 s-1
8	photon spectral index or hardness ratio (avg)	dimensionless
9	variability index	dimensionless
10	significance (avg)	dimensionless
11	significance (peak)	dimensionless
12	peak flux (for time interval above?)	cm-2 s-1
13	peak flux uncertainty	cm-2 s-1
14	time of peak flux (wrt MJDREF)	s
15	interval of time	s
16	flux history	cm-2 s-1
17	flux uncertainty, 1 sigma (as above)	cm-2 s-1
18	start times of flux history entries	s
19	end times of flux history entries	s
20	candidate counterparts	dimensionless
21	degrees of confidence for the counterparts	dimensionless
22	flags (confusion, low latitude,…)	dimensionless
	Note that source name (column 1) should clearly indicate that source is provisional or a transient, e.g., ‘GL TR ####+####’

 

 

 

LAT Burst Catalog:

The output of queries to the LAT Burst Catalog as defined in the report of the Data Products Working Group as LS-009;

 

Column Number

Column Name