DC1 Timing considerations

Processing Strategy

The processing is in three steps

1. Create a pointing history record. Currently this is determined from the launch time in astro::EarthOrbit. In v0r4 it is
    
   astro::JulianDate JD_missionStart =astro::JulianDate(2005, 1, 1,0.0);
   astro::JulianDate JDStart = astro::JulianDate(2005.,7,18,0.0);

   Time is (temporarily)  measured from launch at JDStart. (It should be mission start time.)  The history of course includes a pointing strategy, rocking by 35 degrees. See the TDS class Event::Exposure for a definition of the quantities.

   • This is currently written to an ascii file by ExposureAlg.
2. Choose a source of incoming particles, and generate 86400 separate jobs, with run numbers from 0 to 86399, each generating a mcRootFile output file with the McEvent header and a McParticle with the incoming particle parameters. They will be named incoming_<run>.root, where <run> is the run number. The CPU time for this is minor, 20 sec for such a job on a 2 GHz machine. (So a factor of  20: 20 CPU-days total for this phase.)
3.  Each of the orbit-second files is then input for a full simulation. The simulation uses the run and sequence numbers found in the incoming file to set the random number seed. To significantly reduce CPU time and the size of output files, we run OnboardFilter in a filter mode.
   
   Output is to four files, for now: the merit_xxx.root n-tuple, and  mc_xxx.root, digi_xxx.root, and recon_xxx.root.
   
   Combined with the pointing history file created in phase 1, there is enough information in the merit tuple to create FT1 files for science analysis. However, it would be convenient to have an algorithm write out the FT1 directly.

Timing

The package DataChallenge is used to implement the above strategy. It "uses" Gleam, and contains the basic job options files for each of the three steps. The test_DataChallenge application is run automatically by the release manager, and is set to process step #3 on a file (src/test/incoming_99.9.root) containing 0.1 orbit-sec of incoming protons, from CrProtonMix at t=99.9 sec. I summarize the timing and results from running  v0r1 (GR LATEST 1.167) on a Linux and a Windows 2000 server, both 2 GHz processors (I think for the former).  There are 2737 incoming protons, producing  656 triggers. (So, 6.6 kHz.)

   87026  digi_99.9.root
 1430787  mc_99.9.root
   51829  merit_99.9.root
  653717  recon_99.9.root

   95,670 digi_99.9.root
1,527,178 mc_99.9.root
   53,519 merit_99.9.root
  725,145 recon_99.9.root

Observations:

• G4 on Linux seems to be rather slower than on Windows. The latter is compiled with optimization, former the G4 default installation.
• The extrapolated  orbit-day processing time is 7080 CPU days =1000 CPU weeks on a machine like glast02
• The 2.2 MB in the output scales to about 2 TB. But if we limit to the merit tuple, this is 4.6 GB.

Notes:

• This is a low estimate, since it does not yet include CrGamma, as that breaks the simulation for now. (And no "real" photons yet,  but ~10 Hz will not add much to the above 400 Hz.)
• It is only for one orbit position, which is perhaps a bit high. It will be averaged with points sampled from a full orbit in the future.
•  Richard has estimated the CPU time at 1250 CPU weeks, and 6.2 TB of triggered data.

Last updated: 08/31/2003 13:43:39 -0700

Toby Burnett