| # Gaudi Framework, version v22r2 |

Home |  | Generated: Tue May 10 2011

Classes | Functions | Variables

# GaudiTest Namespace Reference

## Classes
class | TemporaryEnvironment
| Utility Classes. More...
class | TempDir
class | TempFile
class | CMT
class | BasicOutputValidator
| Output Validation Classes. More...
class | FilePreprocessor
class | FilePreprocessorSequence
class | LineSkipper
class | BlockSkipper
class | RegexpReplacer
class | LineSorter
| Special preprocessor sorting the list of strings (whitespace separated) that follow a signature on a single line. More...
class | ReferenceFileValidator
class | GaudiFilterExecutable
class | GaudiExeTest
class | HTMLResultStream
## Functions
def | which
| Locates an executable in the executables path ($PATH) and returns the full path to it.
def | rationalizepath
def | findReferenceBlock
def | countErrorLines
def | _parseTTreeSummary
def | findTTreeSummaries
def | cmpTreesDicts
def | getCmpFailingValues
def | parseHistosSummary
def | findHistosSummaries
## Variables
string | __author__ = 'Marco Clemencic CERN/PH-LBC'
| File: GaudiTest.py Author: Marco Clemencic CERN/PH-LBC.
tuple | maskPointers = RegexpReplacer("0x[0-9a-fA-F]{4,16}","0x########")
tuple | normalizeDate
tuple | normalizeEOL = FilePreprocessor()
tuple | skipEmptyLines = FilePreprocessor()
| normalizeExamples = maskPointers+normalizeDate
tuple | h_count_re = re.compile(r"^(.*)SUCCESS\s+Booked (\d+) Histogram\(s\) :\s+(.*)")

## Function Documentation

def GaudiTest::_parseTTreeSummary | ( |  | lines,
|  |  | pos
| ) |  | `[private]`

    Parse the TTree summary table in lines, starting from pos.
    Returns a tuple with the dictionary with the digested informations and the
    position of the first line after the summary.

Definition at line 616 of file GaudiTest.py.

    00617                                   :
    00618     """
    00619     Parse the TTree summary table in lines, starting from pos.
    00620     Returns a tuple with the dictionary with the digested informations and the
    00621     position of the first line after the summary.
    00622     """
    00623     result = {}
    00624     i = pos + 1 # first line is a sequence of '*'
    00625     count = len(lines)
    00626
    00627     splitcols = lambda l: [ f.strip() for f in l.strip("*\n").split(':',2) ]
    00628     def parseblock(ll):
    00629         r = {}
    00630         cols = splitcols(ll[0])
    00631         r["Name"], r["Title"] = cols[1:]
    00632
    00633         cols = splitcols(ll[1])
    00634         r["Entries"] = int(cols[1])
    00635
    00636         sizes = cols[2].split()
    00637         r["Total size"] = int(sizes[2])
    00638         if sizes[-1] == "memory":
    00639             r["File size"] = 0
    00640         else:
    00641             r["File size"] = int(sizes[-1])
    00642
    00643         cols = splitcols(ll[2])
    00644         sizes = cols[2].split()
    00645         if cols[0] == "Baskets":
    00646             r["Baskets"] = int(cols[1])
    00647             r["Basket size"] = int(sizes[2])
    00648         r["Compression"] = float(sizes[-1])
    00649         return r
    00650
    00651     if i < (count - 3) and lines[i].startswith("*Tree"):
    00652         result = parseblock(lines[i:i+3])
    00653         result["Branches"] = {}
    00654         i += 4
    00655         while i < (count - 3) and lines[i].startswith("*Br"):
    00656             branch = parseblock(lines[i:i+3])
    00657             result["Branches"][branch["Name"]] = branch
    00658             i += 4
    00659
    00660     return (result, i)

def GaudiTest::cmpTreesDicts | ( |  | reference,
|  |  | to_check,
|  |  | ignore = `None`
| ) |  |

    Check that all the keys in reference are in to_check too, with the same value.
    If the value is a dict, the function is called recursively. to_check can
    contain more keys than reference, that will not be tested.
    The function returns at the first difference found.

Definition at line 682 of file GaudiTest.py.

    00683                                                      :
    00684     """
    00685     Check that all the keys in reference are in to_check too, with the same value.
    00686     If the value is a dict, the function is called recursively. to_check can
    00687     contain more keys than reference, that will not be tested.
    00688     The function returns at the first difference found.
    00689     """
    00690     fail_keys = []
    00691     # filter the keys in the reference dictionary
    00692     if ignore:
    00693         ignore_re = re.compile(ignore)
    00694         keys = [ key for key in reference if not ignore_re.match(key) ]
    00695     else:
    00696         keys = reference.keys()
    00697     # loop over the keys (not ignored) in the reference dictionary
    00698     for k in keys:
    00699         if k in to_check: # the key must be in the dictionary to_check
    00700             if (type(reference[k]) is dict) and (type(to_check[k]) is dict):
    00701                 # if both reference and to_check values are dictionaries, recurse
    00702                 failed = fail_keys = cmpTreesDicts(reference[k], to_check[k], ignore)
    00703             else:
    00704                 # compare the two values
    00705                 failed = to_check[k] != reference[k]
    00706         else: # handle missing keys in the dictionary to check (i.e. failure)
    00707             to_check[k] = None
    00708             failed = True
    00709         if failed:
    00710             fail_keys.insert(0, k)
    00711             break # exit from the loop at the first failure
    00712     return fail_keys # return the list of keys bringing to the different values

def GaudiTest::countErrorLines | ( |  | expected = `{'ERROR':0`,
|  |  | FATAL
| ) |  |

Definition at line 580 of file GaudiTest.py.

    00580                                {'ERROR':0, 'FATAL':0}, **kwargs):
    00581     """
    00582     Count the number of messages with required severity (by default ERROR and FATAL)
    00583     and check if their numbers match the expected ones (0 by default).
    00584     The dictionary "expected" can be used to tune the number of errors and fatals
    00585     allowed, or to limit the number of expected warnings etc.
    00586     """
    00587     stdout = kwargs["stdout"]
    00588     result = kwargs["result"]
    00589     causes = kwargs["causes"]
    00590
    00591     # prepare the dictionary to record the extracted lines
    00592     errors = {}
    00593     for sev in expected:
    00594         errors[sev] = []
    00595
    00596     outlines = stdout.splitlines()
    00597     from math import log10
    00598     fmt = "%%%dd - %%s" % (int(log10(len(outlines))+1))
    00599
    00600     linecount = 0
    00601     for l in outlines:
    00602         linecount += 1
    00603         words = l.split()
    00604         if len(words) >= 2 and words[1] in errors:
    00605             errors[words[1]].append(fmt%(linecount,l.rstrip()))
    00606
    00607     for e in errors:
    00608         if len(errors[e]) != expected[e]:
    00609             causes.append('%s(%d)'%(e,len(errors[e])))
    00610             result["GaudiTest.lines.%s"%e] = result.Quote('\n'.join(errors[e]))
    00611             result["GaudiTest.lines.%s.expected#"%e] = result.Quote(str(expected[e]))
    00612
    00613     return causes
    00614
    00615

def GaudiTest::findHistosSummaries | ( |  | stdout ) |

    Scan stdout to find ROOT TTree summaries and digest them.

Definition at line 789 of file GaudiTest.py.

    00790                                :
    00791     """
    00792     Scan stdout to find ROOT TTree summaries and digest them.
    00793     """
    00794     outlines = stdout.splitlines()
    00795     nlines = len(outlines) - 1
    00796     summaries = {}
    00797     global h_count_re
    00798
    00799     pos = 0
    00800     while pos < nlines:
    00801         summ = {}
    00802         # find first line of block:
    00803         match = h_count_re.search(outlines[pos])
    00804         while pos < nlines and not match:
    00805             pos += 1
    00806             match = h_count_re.search(outlines[pos])
    00807         if match:
    00808             summ, pos = parseHistosSummary(outlines, pos)
    00809         summaries.update(summ)
    00810     return summaries

def GaudiTest::findReferenceBlock | ( |  | reference,
|  |  | stdout,
|  |  | result,
|  |  | causes,
|  |  | signature_offset = `0`,
|  |  | signature = `None`,
|  |  | id = `None`
| ) |  |

    Given a block of text, tries to find it in the output.
    The block had to be identified by a signature line. By default, the first
    line is used as signature, or the line pointed to by signature_offset. If
    signature_offset points outside the block, a signature line can be passed as
    signature argument. Note: if 'signature' is None (the default), a negative
    signature_offset is interpreted as index in a list (e.g. -1 means the last
    line), otherwise the it is interpreted as the number of lines before the
    first one of the block the signature must appear.
    The parameter 'id' allow to distinguish between different calls to this
    function in the same validation code.

Definition at line 533 of file GaudiTest.py.

    00535                                  :
    00536     """
    00537     Given a block of text, tries to find it in the output.
    00538     The block had to be identified by a signature line. By default, the first
    00539     line is used as signature, or the line pointed to by signature_offset. If
    00540     signature_offset points outside the block, a signature line can be passed as
    00541     signature argument. Note: if 'signature' is None (the default), a negative
    00542     signature_offset is interpreted as index in a list (e.g. -1 means the last
    00543     line), otherwise the it is interpreted as the number of lines before the
    00544     first one of the block the signature must appear.
    00545     The parameter 'id' allow to distinguish between different calls to this
    00546     function in the same validation code.
    00547     """
    00548     # split reference file, sanitize EOLs and remove empty lines
    00549     reflines = filter(None,map(lambda s: s.rstrip(), reference.splitlines()))
    00550     if not reflines:
    00551         raise RuntimeError("Empty (or null) reference")
    00552     # the same on standard output
    00553     outlines = filter(None,map(lambda s: s.rstrip(), stdout.splitlines()))
    00554
    00555     res_field = "GaudiTest.RefBlock"
    00556     if id:
    00557         res_field += "_%s" % id
    00558
    00559     if signature is None:
    00560         if signature_offset < 0:
    00561             signature_offset = len(reference)+signature_offset
    00562         signature = reflines[signature_offset]
    00563     # find the reference block in the output file
    00564     try:
    00565         pos = outlines.index(signature)
    00566         outlines = outlines[pos-signature_offset:pos+len(reflines)-signature_offset]
    00567         if reflines != outlines:
    00568             msg = "standard output"
    00569             # I do not want 2 messages in causes if teh function is called twice
    00570             if not msg in causes:
    00571                 causes.append(msg)
    00572             result[res_field + ".observed"] = result.Quote("\n".join(outlines))
    00573     except ValueError:
    00574         causes.append("missing signature")
    00575     result[res_field + ".signature"] = result.Quote(signature)
    00576     if len(reflines) > 1 or signature != reflines[0]:
    00577         result[res_field + ".expected"] = result.Quote("\n".join(reflines))
    00578
    00579     return causes

def GaudiTest::findTTreeSummaries | ( |  | stdout ) |

    Scan stdout to find ROOT TTree summaries and digest them.

Definition at line 661 of file GaudiTest.py.

    00662                               :
    00663     """
    00664     Scan stdout to find ROOT TTree summaries and digest them.
    00665     """
    00666     stars = re.compile(r"^\*+$")
    00667     outlines = stdout.splitlines()
    00668     nlines = len(outlines)
    00669     trees = {}
    00670
    00671     i = 0
    00672     while i < nlines: #loop over the output
    00673         # look for
    00674         while i < nlines and not stars.match(outlines[i]):
    00675             i += 1
    00676         if i < nlines:
    00677             tree, i = _parseTTreeSummary(outlines, i)
    00678             if tree:
    00679                 trees[tree["Name"]] = tree
    00680
    00681     return trees

def GaudiTest::getCmpFailingValues | ( |  | reference,
|  |  | to_check,
|  |  | fail_path
| ) |  |

Definition at line 713 of file GaudiTest.py.

    00714                                                        :
    00715     c = to_check
    00716     r = reference
    00717     for k in fail_path:
    00718         c = c.get(k,None)
    00719         r = r.get(k,None)
    00720         if c is None or r is None:
    00721             break # one of the dictionaries is not deep enough
    00722     return (fail_path, r, c)
    00723
    # signature of the print-out of the histograms

def GaudiTest::parseHistosSummary | ( |  | lines,
|  |  | pos
| ) |  |

    Extract the histograms infos from the lines starting at pos.
    Returns the position of the first line after the summary block.

Definition at line 726 of file GaudiTest.py.

    00727                                   :
    00728     """
    00729     Extract the histograms infos from the lines starting at pos.
    00730     Returns the position of the first line after the summary block.
    00731     """
    00732     global h_count_re
    00733     h_table_head = re.compile(r'SUCCESS\s+List of booked (1D|2D|3D|1D profile|2D profile) histograms in directory\s+"(\w*)"')
    00734     h_short_summ = re.compile(r"ID=([^\"]+)\s+\"([^\"]+)\"\s+(.*)")
    00735
    00736     nlines = len(lines)
    00737
    00738     # decode header
    00739     m = h_count_re.search(lines[pos])
    00740     name = m.group(1).strip()
    00741     total = int(m.group(2))
    00742     header = {}
    00743     for k, v in [ x.split("=") for x in  m.group(3).split() ]:
    00744         header[k] = int(v)
    00745     pos += 1
    00746     header["Total"] = total
    00747
    00748     summ = {}
    00749     while pos < nlines:
    00750         m = h_table_head.search(lines[pos])
    00751         if m:
    00752             t, d = m.groups(1) # type and directory
    00753             t = t.replace(" profile", "Prof")
    00754             pos += 1
    00755             if pos < nlines:
    00756                 l = lines[pos]
    00757             else:
    00758                 l = ""
    00759             cont = {}
    00760             if l.startswith(" | ID"):
    00761                 # table format
    00762                 titles = [ x.strip() for x in l.split("|")][1:]
    00763                 pos += 1
    00764                 while pos < nlines and lines[pos].startswith(" |"):
    00765                     l = lines[pos]
    00766                     values = [ x.strip() for x in l.split("|")][1:]
    00767                     hcont = {}
    00768                     for i in range(len(titles)):
    00769                         hcont[titles[i]] = values[i]
    00770                     cont[hcont["ID"]] = hcont
    00771                     pos += 1
    00772             elif l.startswith(" ID="):
    00773                 while pos < nlines and lines[pos].startswith(" ID="):
    00774                     values = [ x.strip() for x in  h_short_summ.search(lines[pos]).groups() ]
    00775                     cont[values[0]] = values
    00776                     pos += 1
    00777             else: # not interpreted
    00778                 raise RuntimeError("Cannot understand line %d: '%s'" % (pos, l))
    00779             if not d in summ:
    00780                 summ[d] = {}
    00781             summ[d][t] = cont
    00782             summ[d]["header"] = header
    00783         else:
    00784             break
    00785     if not summ:
    00786         # If the full table is not present, we use only the header
    00787         summ[name] = {"header": header}
    00788     return summ, pos

def GaudiTest::rationalizepath | ( |  | p ) |

Definition at line 272 of file GaudiTest.py.

    00273                       :
    00274     p = os.path.normpath(os.path.expandvars(p))
    00275     if os.path.exists(p):
    00276         p = os.path.realpath(p)
    00277     return p

def GaudiTest::which | ( |  | executable ) |

Locates an executable in the executables path ($PATH) and returns the full path to it.

If the executable cannot be found, None is returned

Definition at line 263 of file GaudiTest.py.

    00264                      :
    00265     if os.path.isabs(executable):
    00266         return executable
    00267     for d in os.environ.get("PATH").split(os.pathsep):
    00268         fullpath = os.path.join(d,executable)
    00269         if os.path.exists(fullpath):
    00270             return fullpath
    00271     return None

## Variable Documentation

string GaudiTest::__author__ = 'Marco Clemencic CERN/PH-LBC'

File: GaudiTest.py Author: Marco Clemencic CERN/PH-LBC.

Definition at line 5 of file GaudiTest.py.

tuple GaudiTest::h_count_re = re.compile(r"^(.*)SUCCESS\s+Booked (\d+) Histogram\(s\) :\s+(.*)")

Definition at line 724 of file GaudiTest.py.

tuple GaudiTest::maskPointers = RegexpReplacer("0x[0-9a-fA-F]{4,16}","0x########")

Definition at line 407 of file GaudiTest.py.

tuple GaudiTest::normalizeDate

Initial value:

    00001 RegexpReplacer("[0-2]?[0-9]:[0-5][0-9]:[0-5][0-9] [0-9]{4}[-/][01][0-9][-/][0-3][0-9] *(CES?T)?",
    00002                                "00:00:00 1970-01-01")

Definition at line 408 of file GaudiTest.py.

tuple GaudiTest::normalizeEOL = FilePreprocessor()

Definition at line 410 of file GaudiTest.py.

tuple GaudiTest::normalizeExamples = maskPointers+normalizeDate

Definition at line 433 of file GaudiTest.py.

tuple GaudiTest::skipEmptyLines = FilePreprocessor()

Definition at line 413 of file GaudiTest.py.

All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Defines