The Gaudi Framework  v32r1 (f65d50dc)
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions | List of all members
GaudiHive.precedence.CruncherSequence Class Reference
Inheritance diagram for GaudiHive.precedence.CruncherSequence:
Inheritance graph
[legend]
Collaboration diagram for GaudiHive.precedence.CruncherSequence:
Collaboration graph
[legend]

Public Member Functions

def __init__ (self, timeValue, IOboolValue, sleepFraction, cfgPath, dfgPath, topSequencer, showStat=False, timeline=False, outputLevel=INFO)
 
def get (self)
 

Public Attributes

 timeValue
 
 IOboolValue
 
 sleepFraction
 
 cfg
 
 dfg
 
 enableTimeline
 
 outputLevel
 
 sequencer
 

Static Public Attributes

list unique_sequencers = []
 
dictionary dupl_seqs = {}
 
list OR_sequencers = []
 
list unique_algos = []
 
dictionary dupl_algos = {}
 
list unique_data_objects = []
 

Private Member Functions

def _declare_data_deps (self, algo_name, algo)
 
def _generate_sequence (self, name, seq=None)
 

Detailed Description

Constructs the sequence tree of CPUCrunchers with provided control flow and data flow precedence rules.

Definition at line 135 of file precedence.py.

Constructor & Destructor Documentation

◆ __init__()

def GaudiHive.precedence.CruncherSequence.__init__ (   self,
  timeValue,
  IOboolValue,
  sleepFraction,
  cfgPath,
  dfgPath,
  topSequencer,
  showStat = False,
  timeline = False,
  outputLevel = INFO 
) 
Keyword arguments:
timeValue -- timeValue object to set algorithm execution time
IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
showStat -- print out statistics on precedence graph

Definition at line 146 of file precedence.py.

146  def __init__(self,
147  timeValue,
148  IOboolValue,
149  sleepFraction,
150  cfgPath,
151  dfgPath,
152  topSequencer,
153  showStat=False,
154  timeline=False,
155  outputLevel=INFO):
156  """
157  Keyword arguments:
158  timeValue -- timeValue object to set algorithm execution time
159  IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
160  cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
161  dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
162  showStat -- print out statistics on precedence graph
163  """
164 
165  self.timeValue = timeValue
166  self.IOboolValue = IOboolValue
167  self.sleepFraction = sleepFraction
168 
169  self.cfg = nx.read_graphml(_buildFilePath(cfgPath))
170  self.dfg = nx.read_graphml(_buildFilePath(dfgPath))
171 
172  self.enableTimeline = timeline
173 
174  self.outputLevel = outputLevel
175 
176  # Generate control flow part
177  self.sequencer = self._generate_sequence(topSequencer)
178 
179  if showStat:
180  import pprint
181 
182  print "\n===== Statistics on Algorithms ====="
183  print "Total number of algorithm nodes: ", len(
184  self.unique_algos) + sum(
185  [self.dupl_algos[i] - 1 for i in self.dupl_algos])
186  print "Number of unique algorithms: ", len(self.unique_algos)
187  print " -->", len(
188  self.dupl_algos
189  ), "of them being re-used with the following distribution: ", [
190  self.dupl_algos[i] for i in self.dupl_algos
191  ]
192  # pprint.pprint(dupl_algos)
193 
194  print "\n===== Statistics on Sequencers ====="
195  print "Total number of sequencers: ", len(
196  self.unique_sequencers) + sum(
197  [self.dupl_seqs[i] - 1 for i in self.dupl_seqs])
198  print "Number of unique sequencers: ", len(self.unique_sequencers)
199  print " -->", len(
200  self.dupl_seqs
201  ), "of them being re-used with the following distribution: ", [
202  self.dupl_seqs[i] for i in self.dupl_seqs
203  ]
204  # pprint.pprint(dupl_seqs)
205  print "Number of OR-sequencers: ", len(self.OR_sequencers)
206 
207  print "\n===== Statistics on DataObjects ====="
208  print "Number of unique DataObjects: ", len(
209  self.unique_data_objects)
210  # pprint.pprint(self.unique_data_objects)
211  print
212 
GaudiHive.precedence._buildFilePath
def _buildFilePath(filePath)
Definition: precedence.py:12

Member Function Documentation

◆ _declare_data_deps()

def GaudiHive.precedence.CruncherSequence._declare_data_deps (   self,
  algo_name,
  algo 
)
private 
Declare data inputs and outputs for a given algorithm.

Definition at line 217 of file precedence.py.

217  def _declare_data_deps(self, algo_name, algo):
218  """ Declare data inputs and outputs for a given algorithm. """
219 
220  # Declare data inputs
221  for inNode, outNode in self.dfg.in_edges(algo_name):
222  dataName = inNode
223  if dataName not in self.unique_data_objects:
224  self.unique_data_objects.append(dataName)
225 
226  if dataName not in algo.inpKeys:
227  algo.inpKeys.append(dataName)
228 
229  # Declare data outputs
230  for inNode, outNode in self.dfg.out_edges(algo_name):
231  dataName = outNode
232  if dataName not in self.unique_data_objects:
233  self.unique_data_objects.append(dataName)
234 
235  if dataName not in algo.outKeys:
236  algo.outKeys.append(dataName)
237 

◆ _generate_sequence()

def GaudiHive.precedence.CruncherSequence._generate_sequence (   self,
  name,
  seq = None 
)
private 
Assemble the tree of sequencers.

Definition at line 238 of file precedence.py.

238  def _generate_sequence(self, name, seq=None):
239  """ Assemble the tree of sequencers. """
240 
241  if not seq:
242  seq = GaudiSequencer(name, ShortCircuit=False)
243 
244  for n in self.cfg[name]:
245  # extract entity name and type
246  algo_name = n.split('/')[1] if '/' in n else n
247 
248  if self.cfg.node[n].has_key('type'):
249  # first rely on explicit type, if given
250  algo_type = self.cfg.node[n].get('type')
251  else:
252  # if the type is not given explicitly, try to extract it from entity name,
253  # and, if unsuccessful, assume it is an algorithm
254  algo_type = n.split('/')[0] if '/' in n else 'Algorithm'
255 
256  if algo_type in ['GaudiSequencer', 'AthSequencer', 'ProcessPhase']:
257  if algo_name in ['RecoITSeq', 'RecoOTSeq', 'RecoTTSeq']:
258  continue
259 
260  if n not in self.unique_sequencers:
261  self.unique_sequencers.append(n)
262  else:
263  if n not in self.dupl_seqs:
264  self.dupl_seqs[n] = 2
265  else:
266  self.dupl_seqs[n] += 1
267 
268  seq_daughter = GaudiSequencer(algo_name, OutputLevel=INFO)
269  if self.cfg.node[n].get('ModeOR') == 'True':
270  self.OR_sequencers.append(n)
271  seq_daughter.ModeOR = True
272  # if self.cfg.node[n].get('Lazy') == 'False':
273  # print "Non-Lazy - ", n
274  seq_daughter.ShortCircuit = False
275  if seq_daughter not in seq.Members:
276  seq.Members += [seq_daughter]
277  # iterate deeper
278  self._generate_sequence(n, seq_daughter)
279  else:
280  #rndname = ''.join(random.choice(string.lowercase) for i in range(5))
281  #if algo_name in unique_algos: algo_name = algo_name + "-" + rndname
282  if n not in self.unique_algos:
283  self.unique_algos.append(n)
284  else:
285  if n not in self.dupl_algos:
286  self.dupl_algos[n] = 2
287  else:
288  self.dupl_algos[n] += 1
289 
290  avgRuntime, varRuntime = self.timeValue.get(algo_name)
291 
292  algo_daughter = CPUCruncher(
293  algo_name,
294  OutputLevel=self.outputLevel,
295  varRuntime=varRuntime,
296  avgRuntime=avgRuntime,
297  SleepFraction=self.sleepFraction
298  if self.IOboolValue.get() else 0.,
299  Timeline=self.enableTimeline)
300 
301  self._declare_data_deps(algo_name, algo_daughter)
302 
303  if algo_daughter not in seq.Members:
304  seq.Members += [algo_daughter]
305 
306  return seq
CPUCruncher
A class that implements a search for prime numbers.
Definition: CPUCruncher.h:19
GaudiSequencer
Sequencer for executing several algorithms, stopping when one is faulty.
Definition: GaudiSequencer.h:28
Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:332

◆ get()

def GaudiHive.precedence.CruncherSequence.get (   self)

Definition at line 213 of file precedence.py.

213  def get(self):
214 
215  return self.sequencer
216 
Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:332

Member Data Documentation

◆ cfg

GaudiHive.precedence.CruncherSequence.cfg

Definition at line 160 of file precedence.py.

◆ dfg

GaudiHive.precedence.CruncherSequence.dfg

Definition at line 161 of file precedence.py.

◆ dupl_algos

dictionary GaudiHive.precedence.CruncherSequence.dupl_algos = {}
static

Definition at line 142 of file precedence.py.

◆ dupl_seqs

dictionary GaudiHive.precedence.CruncherSequence.dupl_seqs = {}
static

Definition at line 139 of file precedence.py.

◆ enableTimeline

GaudiHive.precedence.CruncherSequence.enableTimeline

Definition at line 163 of file precedence.py.

◆ IOboolValue

GaudiHive.precedence.CruncherSequence.IOboolValue

Definition at line 157 of file precedence.py.

◆ OR_sequencers

list GaudiHive.precedence.CruncherSequence.OR_sequencers = []
static

Definition at line 140 of file precedence.py.

◆ outputLevel

GaudiHive.precedence.CruncherSequence.outputLevel

Definition at line 165 of file precedence.py.

◆ sequencer

GaudiHive.precedence.CruncherSequence.sequencer

Definition at line 168 of file precedence.py.

◆ sleepFraction

GaudiHive.precedence.CruncherSequence.sleepFraction

Definition at line 158 of file precedence.py.

◆ timeValue

GaudiHive.precedence.CruncherSequence.timeValue

Definition at line 156 of file precedence.py.

◆ unique_algos

list GaudiHive.precedence.CruncherSequence.unique_algos = []
static

Definition at line 141 of file precedence.py.

◆ unique_data_objects

list GaudiHive.precedence.CruncherSequence.unique_data_objects = []
static

Definition at line 144 of file precedence.py.

◆ unique_sequencers

list GaudiHive.precedence.CruncherSequence.unique_sequencers = []
static

Definition at line 138 of file precedence.py.

The documentation for this class was generated from the following file: