Core Minutes 4/24/2007

• GR status, 55 day run: (Richard) There is a new HEAD build incorporating changes to the DOCA calculation. Tom is about to launch a muon run. If that looks ok, gamma runs will be done at UW and then billions of background events. Proposed improvements to AcdDigi and ACD geometry will not happen this round.
• LAT/Spacecraft alignment: (Toby, Richard) Job options parameters exist to introduce misalignment. Gleam can use known objects like the Crab to detect it. Then what do we do with it? It's a sort of calibration.
• SAA modeling: (Richard) Tom is not seeing drop-out in the SAA as we should. This needs to be sorted out before the background runs. [As of Wednesday, Toby has found and fixed this.]

  Shape and position of the SAA are hard-wired into astro, but in reality they change with time. Framework used to handle calibrations could serve to keep track of this. For both this and LAT/Spacecraft alignment, will start out with an ideal placeholder.

• RM, Installer: (Navid) There were problems last week with RM jobs failing immediately after being submitted, dying with non-zero return codes (which should never happen for RM jobs). By the time he had added some diagnostics, the problem went away.

  He set LATEST builds to use CMT v1r18 with disastrous results, so switched back to v1r16. The problem is almost impossible to duplicate outside of the RM environment. He would like to try again. There is no other way to track this down. [Update: The same sort of symptoms came back when Navid re-instituted v1r18. It looks like Windows builds are ok, but Linux RHEL3 and RHEL4 fail badly. For many packages RM is not able to detect that they have libraries and test programs.]

  (Richard) Installer made some incomplete builds. This has happened before. Can there be some sort of checksum to look for completeness? (Navid) Problems with Windows builds were due to Samba not writing everything to nfs; the builds look fine from the Windows side. (Richard) A new version of Samba was installed recently; previous one had been very old. Perhaps this will help. (Navid) As for a checksum, it would be difficult to know when to make the comparison; that is, to distinguish between "bad build" and "not done".

  Lock file problem (a lock used to prevent multiple instances of RM from creating the same LATEST build) was due to change of accounts (glast --> glastrm); glastrm didn't have necessary permissions. Navid still hasn't changed Windows over to the new account. He was planning to do it later today [Tuesday] but at Richard's request will hold off for a few days.

• Lyon Pipeline: [Claudia kindly provided these notes since EVO would not allow her to speak.]
  • Java code customization for Lyon batch system (BQS) was finished a couple of weeks ago.
  • A service machine has been configured by CCIN2P3 teams to be able to submit jobs to the farm and to access NFS file systems where log files will be provisionally stored, before going to SLAC.
  • I have installed customized codes in this service machine and launched the batch submission deamon. It is now running and listening.
  • I made several tests using some adapted versions of JobControl Demo package, first from 'normal' interactive CCIN2P3 machines as Client, then from SLAC. Those very first tests went fine: Job submission, job removal.
  • Tony Johnson suggested launching backgndSC-GR-v10r1 simulation, as a first real test. For this purpose, I installed and compiled GR-v10r1 and installed GPLtools python package, both in an NFS file system in Lyon, specially dedicated to PipelineII.
  • Tony has now included the customized codes in the whole package and attempted to compile all the codes together. Some problems related to different Job ID return type coming from the two systems (LSF and BQS) are still pending to be able to fully compile and use them. I should be able to solve this problem very quickly, so that backgndSC-GR-v10r1 should hopefully be launched in Lyon by the end of this week.
  • Some issues are still pending:
    • Full set up of rsync procedure for log files transfer
    • Definition of output files storage system (HPSS or NFS), following the definition of what will be done with these files in the future

  We will have a VRVS meeting maybe next Thursday or at the beginning of next week to discuss the remaining issues.

• Pointer skimmers Here is a report from Heather, who couldn't be at the meeting.

  Last week there was a EVO meeting with: Richard, Warren, Anders, Eric, Tracy, David C. and myself. We discussed at length Tracy suggestion to utilize a relational database to store run and event numbers and pointers to files containing those events. Tracy explained the advantages which included providing a general utility to filter the data using a core set of bits to extract interesting events, the ability to reuse some of Joanne's code for accessing and managing data in a relational DB, etc. Concerns about this approach included: overlap with the data catalog, the need to maintain a large database, the possibility that the kitchen sink would end up in this DB to allow skimming of events. We decided to put aside the discussion for now and instead work on the remaining two pieces of the problem as neatly presented by Tracy:

  1. Given a list of run numbers, provide the full paths to the ROOT files containing those runs
  2. Retrieve the actual events, given a run and event list.

  Right now, Tracy is working on updating RootIo to centralize the file access calls into RootIoSvc...ultimately, such file access will probably live in a non-Gaudi package that is generally usable by other pieces of software such as the data skimmer that David is working on.

  Reference: https://confluence.slac.stanford.edu/display/SAS/Pointer+Skims

• ROOT and ... (Richard reporting for Heather) ROOT 5.14D and Python 2.5 on Windows is a bust. (Toby) will ask Gordon Watts about it.

  (David C.) is working on validating ROOT 5.14 with the Skimmer.

• RHEL4 (Navid, Joanne) As of LATEST1.3076, with new tags for a couple of recently-fixed packages, the only packages not building in RHEL4 are ldfReader, LdfConverter, userAlg, and Gleam. ldfReader has a bug which is causing it to fail on all platforms and its failure is causing the other packages to fail because they can't find ldfReader.so.
• Announcements (Richard) There is an ISOC workshop next week Monday–Wednesday; consequently there will be no Core meeting.
• Progress on updating the system tests (added by Leon 4/25/07) I couldn't get into the meeting, so Richard asked me to add a short report to the minutes. There's been fair progress since my report to C&A on Monday (which I also never gave because vrvs wasn't working). I've found the right code, and now all but three of the histograms in the AllGamma test agree with those from the current test (all use the same cut, which is giving a different result, don't know why). Tracy has fixed most of the memory leak. I've checked the new code into cvs and tagged it.

previous

minutes index

next

J. Bogart, Last Modified: 01-Jun-2010 15:48:39 -0700