Core Minutes 11/02/2004  

Present:    Joanne Bogart,  Toby Burnett,  Pol D'Avezac, James Chiang, Seth Digel, Richard Dubois,  Riccardo Giannitrapani,  Navid Golpayegani,  Heather Kelly, Michael Kuss, Matt Langston, Chuck Patterson, James Peachey, Dirk Petry, Robert Schaefer, Tom Stephens

• Installer: (Navid) He has made a new version which fixes the design flaw described in this message to the Core Software List.
• Memory leak plugged: (Navid) He found and fixed a leak in Onboard Filter, perhaps the leak Julie had seen. (Richard) Would be good to verify by running the particular System Test for which the leak was easily detectable. Julie would know the details, but is currently travelling.
• Pipeline: (Richard for Dan) Pipeline was broken last week.  The problem was traced to a field too small to accommodate long names I&T was using.  Dan increased the field size in the database, but originally neglected to make necessary changes to all scripts referencing the field.  After this fix, it seemed that 20-30% of Pipeline jobs were failing.  In fact the jobs were not failing, but the Pipeline couldn't always detect the true outcome.

  Background: LSF (SLAC batch system) has a blocking mode under which submitting process is supposed to be blocked until the job is done.  But "done" doesn't always quite mean, well, done, and the lag between "LSF done" and done was often long enough that Dan's ad hoc attempts to wait it out weren't always successful.

  Discussions are now underway with SCS to get this fixed at their end; meanwhile Dan is trying out alternate work-arounds . (Richard) In addition to I&T use of Pipeline, would like to try out all-gamma and DC1-style runs.

• Pipeline configuration interface: (Matt) Read about about the XML files used to configure a pipeline instance. The write-up includes an example file and the XML schema it must conform to. If you're within the SLAC firewall (won't be necessary ultimately. We have done our part; still waiting for SCS to fix this.) you can see the page which allows you to upload such an XML file.
• New RM status: (Joanne for Alex, Navid) The new RM, which has considerably more flexibility and extensibility than the existing facility and will ultimately handle Windows as well as Linux builds, is necessarily a complicated beast with many components. It needs to be implemented and tested incrementally. Almost all the bits are there to exercise it with a representative part of Linux-only builds, in parallel with production RM. Pieces include a two-part database (one generic part to supervise and initiate stages of activity and one RM-specific part), Perl scripts to do the actual checkouts, builds, etc., a piece to handle LSF job submission, and so forth. Before this test can take place the Perl scripts which do RM application functions on Linux have to be modified to communicate with the RM-specific part of the database. After the Linux-only design has been validated with this test, incorporating the full functionality of the current RM on Linux should be relatively straightforward, though far from instantaneous.

  The Windows implementation has additional wrinkles since there is no batch system and builds are normally done interactively. Riccardo and Marco have already provided a solution for the second problem in MRcmt, a script which may be invoked from the command line to do the kinds of things (recursive checkout, configure, build) needed by RM. (Richard) In fact LSF is advertised to run under Windows. SCS hasn't yet figured out how to configure it properly for SLAC, but this is in the works. If it succeeds the new RM could use it. [Later addition from Alex: SCS tried to get LSF to work was tried some time ago and failed; seems to have been languishing ever since.  Probably the project doesn't have much priority. Still later addition from Richard: it's not entirely moribund. Randy Melen of SCS reports there has been some progress recently.]
  

• ScienceTools checkout complete: (Seth)  Issues to be addressed include prompting, error messages, and conformance of FITS files to standards. (James) It's recently been demonstrated that discrepancies between FITS files created and standards are small to non-existent. Seth will enlarge on this summary at a later date.
• GSSC software testing: (Dirk) See this page for plans and status. (Richard) The Missions Ground Systems manager (breaking a tie) has decided that FASTCopy will be used to move files from MOC to SLAC and from SLAC to SSC.
• DC2 Coordinating Committee: (Richard) Julie will chair it. Proposal is for next steering meeting to take place following next week's Core meeting, which should preferable be kept to an hour or less.
• InstallArea fully disabled: (Toby) Earlier changes didn't quite eliminate all of it [in particular, not lib subdirectory. ed.] but the latest tag of GlastPolicy does.

∫ Flight(w, ..) dw

• EM status: (Heather)  New tag  v3r0406p0, made yesterday, includes, most notably, the new CalibGenCal. Will make another tag shortly to pick up new EM2. She would like to get System Tests going, but Julie is on the road. We need a sub!
• ROOT upgrade: (Heather) ..is perhaps lagging somewhat relative to Xerces work; if so, Xerces changes should go in first. Richard suggested contacting James discuss possible impact of upgrade on ScienceTools, which Heather cheerfully agreed to do. Only tenuously related to this were a couple remarks about PyROOT: Toby says it now can be used on Windows; Richard reports that Berrie is turning into a fan (using it on Linux, of course).
• Xerces upgrade: (Joanne) ..is much less sure that Xerces work will be ready before ROOT, but progress has been made since last week. Jim will take care of ScienceTools-only packages needing conversion, which he says should be very quick. Joanne hopes to convert the remaining 2 or 3 packages in a small number of days. However she has run into a possible snag providing a suitable binary distribution of Xerces for Windows.  The Xerces project only provides Release binaries, built with VS 6.0.  Sample programs which are part of this distribution run properly on a XP 2003 .Net system; she hasn't yet tried linking our code against the Xerces library. But even if it seems ok, since it's Release rather than Debug we would be running a risk: mixing Release and Debug can cause problems with heap memory.  So far her attempts to build Debug binaries from the source distribution have failed, but this is likely to be operator error.
• LDF file validation: (Richard) We need it! At Heather's request, Tony Waite has retrieved a program which does more or less the right set of things for balloon data. The hope is that someone from ISOC could use this as a guide to write something appropriate for LDF files, though certain kinds of validation will have to be done by SAS.
• CAL: (Zach) New CalibGenCal has been released and draft documentation (Doxygen and comprehensive guide) is available.
• TKR: (Leon) Of restructuring tasks, pattern recognition still needs to be addressed. He, Tracy and Bill  had a planning meeting and have their marching orders.  Work is underway.

Coming Soon

• Monthly Status Review: ..takes place tomorrow. We're towards the end of the agenda.
• Richard remote:  He'll be working at home for approximately two weeks, starting Thursday.

previous

minutes index

next

J. Bogart Last Modified: 01-Jun-2010 15:45:57 -0700