All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
Public Member Functions | Public Attributes | List of all members
GaudiMP.pTools.Syncer Class Reference
Inheritance diagram for GaudiMP.pTools.Syncer:
Inheritance graph
[legend]
Collaboration diagram for GaudiMP.pTools.Syncer:
Collaboration graph
[legend]

Public Member Functions

def __init__
 
def syncAll
 
def syncAllRolling
 
def processHang
 
def checkAll
 
def checkLastEvents
 

Public Attributes

 limit
 
 step
 
 d
 
 manyEvents
 
 limitFirst
 
 keys
 
 nWorkers
 
 log
 

Detailed Description

Definition at line 567 of file pTools.py.

Constructor & Destructor Documentation

def GaudiMP.pTools.Syncer.__init__ (   self,
  nWorkers,
  log,
  manyEvents = False,
  limit = None,
  step = None,
  firstEvent = None 
)

Definition at line 569 of file pTools.py.

569 
570  limit=None, step=None, firstEvent=None ) :
571  # Class to help synchronise the sub-processes
572  self.limit = limit
573  self.step = step
574  self.d = {}
575  self.manyEvents = manyEvents
576 
577  for i in xrange(-2, nWorkers) :
578  self.d[ i ] = SyncMini( Event(), lastEvent=Event() )
579  if self.manyEvents :
580  self.limitFirst = firstEvent
581 
582  self.keys = self.d.keys()
583  self.nWorkers = nWorkers
584  self.log = log

Member Function Documentation

def GaudiMP.pTools.Syncer.checkAll (   self)

Definition at line 674 of file pTools.py.

674 
675  def checkAll( self ) :
676  # Check the status of each Sync object
677  # return True or False
678  currentStatus = [ mini.check() for mini in self.d.values() ]
679  return all( currentStatus )
def GaudiMP.pTools.Syncer.checkLastEvents (   self)

Definition at line 680 of file pTools.py.

680 
681  def checkLastEvents( self ) :
682  # check if all of the lastEvents are set to true in self.d[k][1]
683  stat = [ sMini.checkLast() for sMini in self.d.values() ]
684  return all(stat)
685 
686 # =========================== Methods =========================================
def GaudiMP.pTools.Syncer.processHang (   self)

Definition at line 668 of file pTools.py.

668 
669  def processHang( self ) :
670  self.log.critical('Some proc is hanging during Event processing!')
671  for k in self.keys :
672  self.log.critical( "Proc/Stat : %i / %s"%(k,self.d[k].check()) )
673  return
validate_patch.check
def check
Definition: validate_patch.py:152
def GaudiMP.pTools.Syncer.syncAll (   self,
  step = "Not specified" 
)

Definition at line 585 of file pTools.py.

585 
586  def syncAll( self, step="Not specified" ) :
587  # is it this method, or is it the rolling version needed?
588  # if so, drop through...
589 
590  if self.manyEvents :
591  sc = self.syncAllRolling( )
592  return sc
593 
594  # Regular version ----------------------------
595  for i in xrange( 0, self.limit, self.step ) :
596  if self.checkAll( ) :
597  self.log.info('%s : All procs done @ %i s'%(step,i))
598  break
599  else :
600  time.sleep(self.step)
601 
602  # Now the time limit is up... check the status one final time
603  if self.checkAll() :
604  self.log.info("All processes : %s ok."%(step))
605  return SUCCESS
606  else :
607  self.log.critical('Some process is hanging on : %s'%(step))
608  for k in self.keys :
609  hangString= "%s : Proc/Stat : %i/%s"%(step,k,self.d[k].check())
610  self.log.critical( hangString )
611  return FAILURE
validate_patch.check
def check
Definition: validate_patch.py:152
def GaudiMP.pTools.Syncer.syncAllRolling (   self)

Definition at line 612 of file pTools.py.

612 
613  def syncAllRolling( self ) :
614  # Keep track of the progress of Event processing
615  # Each process syncs after each event, so keep clearing
616  # the sync Event, and re-checking
617  # Note the time between True checks too, if the time
618  # between events exceeds singleEvent, this is considered a hang
619 
620  # set the initial time
621  begin = time.time()
622  firstEv = {}
623  timers = {}
624  for k in self.keys :
625  self.d[k].reset()
626  firstEv[k] = False
627  timers[k] = 0.0
628 
629  active = self.keys
630  while True :
631  # check the status of each sync object
632  for k in active :
633  sMini = self.d[k]
634 
635  if sMini.check() or sMini.checkLast():
636  if sMini.checkLast() and sMini.check() :
637  # if last Event set,then event loop finished
638  active.remove( k )
639  alive = time.time()-begin
640  self.log.info( "Audit : Node %i alive for %5.2f"\
641  %(k,alive) )
642  else :
643  sMini.reset()
644  else :
645  # the event still has not been checked, how long is that?
646  # is it the first Event?
647  wait = time.time()-sMini.getTime()
648  cond = wait > self.limit
649  if not firstEv[k] :
650  cond = wait > self.limitFirst
651  firstEv[k] = True
652  if cond :
653  # It is hanging!
654  self.log.critical('Single event wait : %5.2f'%(wait))
655  self.processHang()
656  return FAILURE
657 
658  # Termination Criteria : if all procs have been removed, we're done
659  if self.checkLastEvents() :
660  self.log.info('TC met for event loop')
661  break
662  else :
663  # sleep, loop again
664  time.sleep(self.step)
665 
666  self.log.info("All processes Completed all Events ok")
667  return SUCCESS

Member Data Documentation

GaudiMP.pTools.Syncer.d

Definition at line 573 of file pTools.py.

GaudiMP.pTools.Syncer.keys

Definition at line 581 of file pTools.py.

GaudiMP.pTools.Syncer.limit

Definition at line 571 of file pTools.py.

GaudiMP.pTools.Syncer.limitFirst

Definition at line 579 of file pTools.py.

GaudiMP.pTools.Syncer.log

Definition at line 583 of file pTools.py.

GaudiMP.pTools.Syncer.manyEvents

Definition at line 574 of file pTools.py.

GaudiMP.pTools.Syncer.nWorkers

Definition at line 582 of file pTools.py.

GaudiMP.pTools.Syncer.step

Definition at line 572 of file pTools.py.

The documentation for this class was generated from the following file: