Main Page | Namespace List | Class Hierarchy | Alphabetical List | Class List | Directories | File List | Namespace Members | Class Members | File Members | Related Pages

hippo Python extension module.

The hippo Python extension module is designed to be used interactively or via Python scripts.

Thus the interface is somewhat different from the C++ interface to the HippoDraw library. The hippo module documentation is generated by Python's pydoc module and is the same as the documentation available from the help command in the Python shell.

Using HippoDraw interactively can be as simple as two lines of Python code. Below is an example session.

> python
Python 2.4 (#2, Apr 15 2005, 17:09:59)
[GCC 3.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hippo
>>> app = hippo.HDApp()
>>>

Obviously, even typing these two lines for every session can get boring. Instead one can put the commands in a file and use that as initialization step for the Python session. For example, the file, canvas.py, in the testsuite directory contains

import hippo
app = hippo.HDApp()
canvas = app.canvas()

where we also show how to get a handle on the current canvas window. One can run this script from a UNIX shell or Windows command prompt like this

> python -i canvas.py
>>>

This launches the complete HippoDraw application in a separate thread with the ability to interact with it from the Python shell.

Getting help and documentation

Python has interactive help system. The get all help for the hippo module, just do the following...

> python
Python 2.4 (#2, Apr 15 2005, 17:09:59)
[GCC 3.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hippo
>>> help ( hippo )

This gives you all the built-in documentation in a pager like the UNIX more or less command which is not always convienent. However, one can get the documentation on one class by ...

> python
Python 2.4 (#2, Apr 15 2005, 17:09:59)
[GCC 3.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hippo
>>> help ( hippo.HDApp )

Or even one member function like this ...

> python
Python 2.4 (#2, Apr 15 2005, 17:09:59)
[GCC 3.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hippo
>>> help ( hippo.HDApp.canvas )

Another way to access the same information is to use the pydoc program that came with your Python installation (but under Windows).

> pydoc hippo

This also gives you all the built-in help in a pager. Probably the most convienent method is to generate html version of the documentation. You do this by typing ...

> pydoc -w hippo

and a hippo.html file is created in your working directory. Here is what it looks like. Not very pretty, but it is the standard ouput from pydoc. You can use this link as your documentation for HippoDraw. However, it will get updated for each release and you may be using an older version.

Creating the NTuple in Python

One might generate some data in Python that you want to display with HippoDraw. For example, you could generate a Python list with random Gaussian distribution like this

>>> import random
>>> x = []
>>> for i in range ( 10000 ) :
...     x.append ( random.gauss ( 45, 10 ) )
...
>>>

To display the data as a histogram, one can then type

>>> from hippo import Display
>>> hist = Display ( 'Histogram', ( x, ), ('Gaussian', ) )
>>> canvas.addDisplay ( hist )
>>>

The first argument to the Display function specifies the type of display to create. The second is a Python tuple of Python list objects that will be used by the display. The third argument is a Python tuple of string labels for the lists.

You can now modify the plot, for example, changing the width of the bins in two ways. From the Python shell, one can invoke a member function of the histogram object like this...

>>> hist.setBinWidth ( 'x', 2 )
>>>

But it is much easier to use Axis inspector and change it with the slider or text field.

The function created a DataSource called a ListTuple. It holds references to the Python list objects as columns. The list is not copied, just referenced. It also holds the labels of each column. Displays don't redraw themselves unless they know there's been a change, like changing the bin width. But should the contents of your Python list change, the Display wouldn't know about it. But you can force the display to redraw like this...

>>> hist.update()
>>>

The Python tuple of strings provide the data source labels, but they also giving the bindings of the displays to the data source. Some displays have binding that are optional. For the example, an "XY Plot" display had binding for the X and Y axis, and optionally, for an error on X or Y. To say which optional bindings not to use the "nil" column label is used. The we can do the following

"""
   Demonstrates making simple XY plot.  

   author Paul F. Kunz <Paul_Kunz@slac.stanford.edu>
"""
# Gets the HippoDraw application.
# import setPath
from setPath import app, canvas
from hippo import Display

# Create list of data
energy = [90.74, 91.06, 91.43, 91.50, 92.16, 92.22, 92.96, 89.24, 89.98, 90.35]
sigma  = [ 29.0, 30.0,  28.40, 28.80, 21.95, 22.90, 13.50,  4.50, 10.80, 24.20]
errors = [  5.9,  3.15,  3.0,   5.8,  7.9,   3.1,   4.6,    3.5,   4.6,   3.6]

# make a plot to test it.
xy = Display ( "XY Plot", [ energy, sigma, errors ],
               ['Energy', 'Sigma', 'nil', 'error' ] )

canvas.addDisplay ( xy )

xy.setTitle ( 'Mark II Z0 scan' )

print "An XY plot is now displayed.   You can use the Inspector dialog"
print "to modify the appearance or fit a function to it."

The "nil" string can also be use by the Data inspector as well. Note in this example, we used list of lists instead of tuple of lists. Either can be used.

Speaking of the Data Inspector, sometimes it is more convenient to give HippoDraw all the data you might want to use for displays, and use the Data Inspector to create them. To do this, one creates a DataSource manually. There are three kinds supported: ListTuple, NTuple, and NumArrayTuple. They share a common interface and differ on how they store the column data. As we've seen, the ListTuple stores references to Python list objects. The NTuple makes copies of Python list objects and stores it internally as a C++ vector of doubles. The NumArrayTuple stores references to rank 1 numarray objects. The NTuple has the feature that you can add and replace rows or columns.

Creating displays with the DataInspector doesn't preclude one from also creating them with Python. The interface is similar to what we've already seen. For example

>>> energy = [90.74, 91.06, 91.43, 91.5, 92.16, 92.22, 92.96, 89.24, 89.98 ]
>>> sigma  = [ 29.0, 30.0,  28.40, 28.8, 21.95, 22.9,  13.5,   4.5,  10.8 ]
>>> errors = [  5.9,  3.15,  3.0,   5.8,  7.9,   3.1,   4.6,   3.5,   4.6,]
>>> ntuple = NTuple () # an empty NTuple
>>> ntc = NTupleController.instance ()
>>> ntc.registerNTuple ( ntuple )
>>> ntuple.addColumn ( 'Energy', energy )
>>> ntuple.addColumn ( 'Sigma', sigma )
>>> ntuple.addColumn ( 'error', errors )

>>> xy = Display ( "XY Plot", ntuple  ('Energy', 'Sigma', 'nil', 'error' ) )
>>> canvas.addDisplay ( xy )
>>>

Registering the ntuple with the NTupleController is necessary in order for the Data Inspector to know of their existence.

Getting data from a file

An NTuple data source can also be created by reading a plain text file. See ASCII file format for NTuple for the details. The example file, histogram.py, in the testsuite directory shows how to read a file and create displays from Python. It contains ...

After reading a HippoDraw compatible DataSource file, this Python script creates two displays. It sets the range on the first and the bin width on the second. The results of running this script are shown below.

hist_2.png

Result of using histogram.py

The Display class is actually a small wrapper around the internal HippoDraw C++ library class. It is needed because Qt is running in a separate thread from Python. Since creating a display and perhaps modifying it requires interaction with Qt's event loop, the application must be locked before calling a member function of the actual HippoDraw class and then unlocked when returning.

Using the hippoplotter interface

Making use of Python's default parameter value feature in calling functions, Jim Chiang has extended the HippoDraw interface with his The hippoplotter.py module.

The file, pl_exp_test.py, in the testsuite directory shows an example of using this module.

The above script leads to the canvas shown below

hist_exp.png

Results of pl_exp_test.py script

Extracting data from a display

The interaction with HippoDraw from Python is not just one direction. Once can extract data from the displays and use them in Python. The file function_ntuple.py illustrates this...

00001 """ -*- python -*-
00002 
00003    This script adding functions and fitting.  It also demonstrates
00004    retreiving an ntuple from the histogram to do something with its
00005    contents.
00006 
00007 Author: Paul_Kunz@slac.stanford.edu
00008 
00009 $Id: function_ntuple.py,v 1.3 2004/07/12 22:02:55 pfkeb Exp $nt
00010 
00011 """
00012 from setPath import app, canvas
00013 
00014 # Create NTuple with its controller so Inspector can see it.
00015 from hippo import NTupleController
00016 ntc = NTupleController.instance()
00017 nt1 = ntc.createNTuple ( 'aptuple.tnt' )
00018 
00019 from hippo import Display
00020 
00021 hist = Display ( "Histogram", nt1, ("Cost", ) )
00022 canvas.addDisplay( hist )
00023 
00024 # Get the data representation so we can add function to it.
00025 datarep1 = hist.getDataRep()
00026 from hippo import Function
00027 gauss = Function ( "Gaussian", datarep1 )
00028 gauss.addTo ( hist )
00029 
00030 # Get the function parameters and display them.
00031 print "Before fitting"
00032 parmnames = gauss.parmNames ( )
00033 print parmnames
00034 
00035 parms = gauss.parameters ( )
00036 print parms
00037 
00038 # Now do the fitting.
00039 gauss.fit ( )
00040 
00041 print "After fitting"
00042 parms = gauss.parameters ( )
00043 print parms
00044 
00045 # Add another function.
00046 gauss1 = Function ( "Gaussian", datarep1 )
00047 gauss1.addTo ( hist )
00048 
00049 # Do another fit, should fit to linear sum
00050 gauss1.fit ()
00051 
00052 
00053 # Add Chi-squared per d.f. display
00054 canvas.addTextRep ( hist, 'Chi-squared' )
00055 
00056 # Create an NTuple from the histogram.
00057 # Calculate the residuals
00058 
00059 result = hist.createNTuple ()
00060 ntc.registerNTuple ( result )
00061 
00062 coords = result.getColumn ( 'Cost' )
00063 values = result.getColumn ( 'Density' )
00064 res = []
00065 for i in range ( result.rows ) :
00066     x = coords[i]
00067     diff = values[i] - gauss1.valueAt ( x )
00068     res.append ( diff )
00069 
00070 # Add a column and display it.
00071 result.addColumn ( 'residuals', res )
00072 resplot=Display ( "XY Plot", result, ( 'Cost', 'residuals', 'nil', 'Error' ) )
00073 canvas.addDisplay ( resplot )
00074 
00075 print "The histogram was fitted to the sum of two gaussians."
00076 print "Then histogram bins were retrieved to calculate "
00077 print "the residuals.  These were then plotted as an XY Plot."
00078 print 'One could have used the "Create residuals display" button on the'
00079 print "Inspector, but that wouldn't have demonstrated anything."
00080 
00081 

Like the previous script, it fits two functions to a histogram. It also shows how to extract the function parameter names and their values. Near the end of the script, one extracts the contents of the histogram bins in the form of an NTuple. In the for loop at the end, one uses the NTuple to calculate the residuals between the function and the bin contents and put them in a Python list. The the list is added as a column to the NTuple. Finally, one creates an XYPlot to display them and adds it to the canvas. The result looks like this...

hist_resid.png

Results of function_ntuple.py

However, one didn't have to write this script to plot the residuals, as the is a control in the Function inspector that does it for you.

Using a FITS file

A FITS file can be used as input to HippoDraw. Here's how one can it to view an image of the EGRET All-Sky survey from such a file

The resulting canvas is shown below

canvas_egret.png

The EGRET All-Sky survey.

The FITS data format is a standard astronomical data and mandated by NASA for some projects. It supports images as well as binary or ASCII tables. A FITS table is essentially a NTuple with added information in the form of keyword-value pairs. James Chiang also wrote the following Python function to convert a FITS table to a HippoDraw NTuple.

00001 #!/usr/bin/env python
00002 """
00003 Read in a series of FITS table files and make them accessible as
00004 numarrays, optionally creating a HippoDraw NTuple.
00005 
00006 @author J. Chiang <jchiang@slac.stanford.edu>
00007 """
00008 #
00009 # $Id: FitsNTuple.py,v 1.8 2004/07/17 00:24:07 jchiang Exp $
00010 #
00011 
00012 class FitsNTuple:
00013     def __init__(self, fitsfiles, extension=1):
00014         import sys, numarray, pyfits
00015         cat = numarray.concatenate
00016         #
00017         # If fitsfile is not a list or tuple of file names, assume
00018         # it's a single file name and put it into a single element
00019         # tuple.
00020         #
00021         if type(fitsfiles) != type([]) and type(fitsfiles) != type(()):
00022             fitsfiles = (fitsfiles, )
00023         #
00024         # Process each file named in the list or tuple.
00025         #
00026         columnData = {}
00027         for i, file in zip(xrange(sys.maxint), fitsfiles):
00028             print "adding", file
00029             table = pyfits.open(file.strip(" "))
00030             if i == 0: self.names = table[extension].columns.names
00031             for name in self.names:
00032                 if i == 0:
00033                     columnData[name] = table[extension].data.field(name)
00034                 else:
00035                     columnData[name] = cat((columnData[name],
00036                                             table[extension].data.field(name)))
00037         #
00038         # Add these columns to the internal dictionary.
00039         #
00040         self.__dict__.update(columnData)
00041         
00042     def makeNTuple(self, name=None, useNumArray=1):
00043         import hippo, sys, numarray
00044         if useNumArray:
00045             nt = hippo.NumArrayTuple()
00046         else:
00047             nt = hippo.NTuple()
00048         if name != None:
00049             nt.setTitle(name)
00050         ntc = hippo.NTupleController.instance()
00051         ntc.registerNTuple(nt)
00052         for name in self.names:
00053             if type(self.__dict__[name][0]) == numarray.NumArray:
00054                 columns = self.__dict__[name]
00055                 columns.transpose()
00056                 for i, col in zip(xrange(sys.maxint), columns):
00057                     colname = "%s%i" % (name, i)
00058                     nt.addColumn(colname, col)
00059             else:
00060                 nt.addColumn(name, self.__dict__[name])
00061         return nt

Using ROOT files

Another example is reading a ROOT file that has the form of an ntuple as define in RootNTuple class. The Python code might look like this...

This script not only uses ROOT, but it also uses numarray. It converts a ROOT brach into a numarray array so it can do vector calculations. The ROOT C++ macro to do the equivalent of the above Python script would be considerable more complex.

Limitations.

With this release, not all of HippoDraw's C++ library is exposed to Python. Although this could be done, it is thought to be not necessary. Rather, selected higher level components are exposed in one of two ways. Some classes are exposed directly with a one to one relationship between the C++ member functions and Python member functions. An example is the NTuple class.

One can view the reference documentation for the hippo extension module with Python's online help command, One can also use the pydoc program to view it or generated HTML file with the command "pydoc -w hippo".

In order to be able to have an interactive Python session that interacts with the HippoDraw canvas items and at the same time have interaction with the same items from the Inspector, it was necessary to run the HippoDraw application object in a separate thread. Threading conflicts could then occur. Thus some of HippoDraw's C++ classes are exposed to Python via a thin wrapper class which locks the Qt application object before invoking an action and unlocks it when done.

One good thing about Python is that what ever you do, Python never crashes. Thus, what ever you do with the HippoDraw extension module should not crash Python. An interactive user, however, can easily mis-type an argument to a function. For example, he could try to create a display with "ContourPlot" instead of "Contour Plot". For such errors, the C++ library throws a C++ exception. The HippoDraw extension module catches them and translates them to a Python exception. Thus, when the Python user makes an error, he will receive a message instead of crashing his session.

Another reason the wrapper classes exist is to try to present a more Python "interactive friendly" interface to the user than the raw C++ interface which was designed for the application writer. With this release, it is not clear what a more "friendly" interface should look like. Maybe the Python extension module should be closer to the C++ interface and provide Python classes to wrap them in a more friendly way like James Chiang has done. Feed back on this topic would be very welcome.

Bug:
@ hsimple.py and pl_exp_test.py, don't do anything under run_test_scripts.py.
(This isn't a bug. run_test_scripts.py launches scripts using the import command. Since hsimple.py and pl_exp_test.py are launched by SciPy_demo.py before they are launched by run_test_scripts.py, Python's import mechanism prevents the scripts from being re-run.)

Bug:
@ useBinner.py needs a CR to do anything under run_test_scripts.py.

Bug:
@@ Need numarray installed to use hippoplotter.
Can we separate out the parts that need numarray.
Generated for HippoDraw-1.14.8.5 by doxygen 1.4.3