GLAST Standard Analysis Environment (SAE) Evaluation Questionnaire
Checkup 1
October 12, 2004 - October 29, 2004

Reviewer:
Dirk Petry
Software Component:
Likelihood and related tools mentioned in the Likelihood tutorial
Summary of Tests Performed:
This is part 1 of my evaluation in which I went though the whole
Likelihood tutorial and tried to reproduce the results.

User Interface

1. What, if anything, about the choice and meaning of parameters
could be improved? Are there new parameters which need to be
added? Are the current parameters sensible?

1*** CEL is not a good abbreviation for "equatorial"
since also the galactic coordinate system is a "celestial" coordinate system
(and there are others). A good name would be "EQU" or "FK5" if this is
what is meant.

There is also a typo in the prompt for this parameter.

2*** "Longitude" and "Latitude" are unconventional and misleading.
Ideally, one would choose between "GAL" and "EQU" first and then one
would be asked for "Gal. Lon." and "Gal. Lat." or
"RA" and "DEC" respectively.

3*** generally "0" is not a good value to indicate "no cut applied"
*** if 0 is actually a valid cut

4*** For input file names, it would be good to check for the existance
of the file right after the user hits return for the specific
parameter.

6*** In the source model xml file, the tag name source_library and
title "source library" are misleading. It will take people a long
time to understand that this file is not a library from which individual
sources are taken to construct a model, but that this file _is_ the model
itself. The tag name "source_model" would be better.

7*** Just looking at the xml file, it is not clear what units the parameters are
in. Adding some comment lines using <!-- comment --> would be helpful.



2. Are there any prompts you would suggest rewording?

see point (1)

and there are typos in the prompts for gamma-ray probability

3. Are there any hidden parameters which you think should be
prompted for or vice versa?

4. Would you like the prompts to appear in a different order?

see point (1)

5. Would you suggest changes to the default values of any
parameters? Changes to min, max fields?


Tool Behavior

6. In what way, if any, did the tool perform unexpectedly?

1*** dataSubselector gives a cryptic error message when not able to find
the input file:

Caught N3tip12TipExceptionE at the top level:
Unable to create file named
"!ptsrcs_events_filtered.fits(ptsrcs_events_0000.fits)"

*** instead of saying that it can't find the input file, it says
that it can't create the output file.

7. Were there any aspects of the tool's output which seem
scientifically suspicious to you? (e.g. fluxes too large,
number of events selected too small, etc.)

1*** The exposure map generated by expMap looked significantly different from
that in the tutorial even though I used the same (?) input data as in
the tutorial.

8. Was there information (i.e. output) the tool did not provide
which you think it should?

1*** Since the execution times of expMap and TsMap can be many hours,
it would be nice to have the application make a crude estimation
of the execution time and display this to the user before
starting the calculation. The user can then decide how long
a coffee break he can have ;-)

*** In any case, the program should time itself and tell the user
at the end how long it took to do the calculation. The user can
then plan for future program runs.

2*** In the output of Likelihood, it would be nice to have units and
energy band boundaries.
Instead of
3C 273:
Prefactor: 4.89543 1.47879
Index: -2.85491 0.307341
Scale: 100
Npred: 46.4724
TS value: 24.3121

have:
3C 273:
Flux (cm^(-2)s^(-1)MeV(-1)) =
( 4.89543 1.47879 ) x (E/100 MeV)^( -2.85491 0.307341 )
(Emin = 30 MeV, Emax = 200000 MeV)
Npred: 46.4724
TS value: 24.3121

etc.

3*** the exposure maps produced by expMap seem to depend on whether
they are made using an exposure cube from makeExposureCube
or not (see attachments).

4*** the exposure maps produced by expMap differed from those
shown in the tutorial even though the input was the same(?)


9. Was there information (i.e. output) the tool provided which
you think it should not?


Future Development

10. List new features you would like to see implemented, in
descending order of importance. Please flag any which you
think are critical for DC2.

*** see point 8 above

11. Are there any aspects of this tool you would change in order
for it to work more smoothly with other parts of the SAE?

1 *** expMap should call makeExposureCube internally if it is not given
*** an exposure cube file

2*** It will be useful to have Likelihood produce machine readable output
(simple ASCII table) such that other programs can easily be run
based on Likelihood results (especially useful when scripting things
like source monitors or catalog analysis).

The format for the table could be something like

Source Prefactor Error Index Error Scale Npred TS
"3C 279" 11.7346 1.87847 -2.2288 0.10648 100 142.781 246.293
"Extragalactic Diffuse" 1.45 0 -2.1 0 100 821.137 -1
"Galactic Diffuse" 11 0 -2.1 0 100 512.301 -1

etc.
Alternatively, a FITS file could be used.


JIRA Summary

Please list the identifiers of any/all JIRA issues which were filed
in connection with this evaluation.

No submissions yet. Would like to hear from Jim Chiang and James Peachey
concerning which of the points they would like to see in JIRA.


General Notes

The scitools_setup script is made for csh and doesn't work under bash.
Bash, however, is quite common and one can't expect people to change
over to csh or tcsh just because they want to run the scitools.
=> Please provide also scitools_setup.sh


Attachments

checkout.txt - my minutes of the evaluation
expMap1.fits - exposure map generated by using first makeExposureCube, then expMap
expMap.fits - exposure map generated by using using only expMap
TsMap.fits - TS map generated as described in the minutes