The Gaudi Framework  v33r0 (d5ea422b)
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions | List of all members
GaudiHive.precedence.CruncherSequence Class Reference
Inheritance diagram for GaudiHive.precedence.CruncherSequence:
Inheritance graph
[legend]
Collaboration diagram for GaudiHive.precedence.CruncherSequence:
Collaboration graph
[legend]

Public Member Functions

def __init__ (self, timeValue, IOboolValue, sleepFraction, cfgPath, dfgPath, topSequencer, showStat=False, timeline=False, outputLevel=INFO)
 
def get (self)
 

Public Attributes

 timeValue
 
 IOboolValue
 
 sleepFraction
 
 cfg
 
 dfg
 
 enableTimeline
 
 outputLevel
 
 sequencer
 

Static Public Attributes

list unique_sequencers = []
 
dictionary dupl_seqs = {}
 
list OR_sequencers = []
 
list unique_algos = []
 
dictionary dupl_algos = {}
 
list unique_data_objects = []
 

Private Member Functions

def _declare_data_deps (self, algo_name, algo)
 
def _generate_sequence (self, name, seq=None)
 

Detailed Description

Constructs the sequence tree of CPUCrunchers with provided control flow and data flow precedence rules.

Definition at line 149 of file precedence.py.

Constructor & Destructor Documentation

◆ __init__()

def GaudiHive.precedence.CruncherSequence.__init__ (   self,
  timeValue,
  IOboolValue,
  sleepFraction,
  cfgPath,
  dfgPath,
  topSequencer,
  showStat = False,
  timeline = False,
  outputLevel = INFO 
) 
Keyword arguments:
timeValue -- timeValue object to set algorithm execution time
IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
showStat -- print out statistics on precedence graph

Definition at line 160 of file precedence.py.

160  def __init__(self,
161  timeValue,
162  IOboolValue,
163  sleepFraction,
164  cfgPath,
165  dfgPath,
166  topSequencer,
167  showStat=False,
168  timeline=False,
169  outputLevel=INFO):
170  """
171  Keyword arguments:
172  timeValue -- timeValue object to set algorithm execution time
173  IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
174  cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
175  dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
176  showStat -- print out statistics on precedence graph
177  """
178 
179  self.timeValue = timeValue
180  self.IOboolValue = IOboolValue
181  self.sleepFraction = sleepFraction
182 
183  self.cfg = nx.read_graphml(_buildFilePath(cfgPath))
184  self.dfg = nx.read_graphml(_buildFilePath(dfgPath))
185 
186  self.enableTimeline = timeline
187 
188  self.outputLevel = outputLevel
189 
190  # Generate control flow part
191  self.sequencer = self._generate_sequence(topSequencer)
192 
193  if showStat:
194  import pprint
195 
196  print("\n===== Statistics on Algorithms =====")
197  print("Total number of algorithm nodes: ", len(self.unique_algos) +
198  sum([self.dupl_algos[i] - 1 for i in self.dupl_algos]))
199  print("Number of unique algorithms: ", len(self.unique_algos))
200  print(" -->", len(self.dupl_algos),
201  "of them being re-used with the following distribution: ",
202  [self.dupl_algos[i] for i in self.dupl_algos])
203  # pprint.pprint(dupl_algos)
204 
205  print("\n===== Statistics on Sequencers =====")
206  print("Total number of sequencers: ", len(self.unique_sequencers) +
207  sum([self.dupl_seqs[i] - 1 for i in self.dupl_seqs]))
208  print("Number of unique sequencers: ", len(self.unique_sequencers))
209  print(" -->", len(self.dupl_seqs),
210  "of them being re-used with the following distribution: ",
211  [self.dupl_seqs[i] for i in self.dupl_seqs])
212  # pprint.pprint(dupl_seqs)
213  print("Number of OR-sequencers: ", len(self.OR_sequencers))
214 
215  print("\n===== Statistics on DataObjects =====")
216  print("Number of unique DataObjects: ",
217  len(self.unique_data_objects))
218  # pprint.pprint(self.unique_data_objects)
219  print()
220 
GaudiHive.precedence._buildFilePath
def _buildFilePath(filePath)
Definition: precedence.py:23

Member Function Documentation

◆ _declare_data_deps()

def GaudiHive.precedence.CruncherSequence._declare_data_deps (   self,
  algo_name,
  algo 
)
private 
Declare data inputs and outputs for a given algorithm.

Definition at line 225 of file precedence.py.

225  def _declare_data_deps(self, algo_name, algo):
226  """ Declare data inputs and outputs for a given algorithm. """
227 
228  # Declare data inputs
229  for inNode, outNode in self.dfg.in_edges(algo_name):
230  dataName = inNode
231  if dataName not in self.unique_data_objects:
232  self.unique_data_objects.append(dataName)
233 
234  if dataName not in algo.inpKeys:
235  algo.inpKeys.append(dataName)
236 
237  # Declare data outputs
238  for inNode, outNode in self.dfg.out_edges(algo_name):
239  dataName = outNode
240  if dataName not in self.unique_data_objects:
241  self.unique_data_objects.append(dataName)
242 
243  if dataName not in algo.outKeys:
244  algo.outKeys.append(dataName)
245 

◆ _generate_sequence()

def GaudiHive.precedence.CruncherSequence._generate_sequence (   self,
  name,
  seq = None 
)
private 
Assemble the tree of sequencers.

Definition at line 246 of file precedence.py.

246  def _generate_sequence(self, name, seq=None):
247  """ Assemble the tree of sequencers. """
248 
249  if not seq:
250  seq = GaudiSequencer(name, ShortCircuit=False)
251 
252  for n in self.cfg[name]:
253  # extract entity name and type
254  algo_name = n.split('/')[1] if '/' in n else n
255 
256  if 'type' in self.cfg.node[n]:
257  # first rely on explicit type, if given
258  algo_type = self.cfg.node[n].get('type')
259  else:
260  # if the type is not given explicitly, try to extract it from entity name,
261  # and, if unsuccessful, assume it is an algorithm
262  algo_type = n.split('/')[0] if '/' in n else 'Algorithm'
263 
264  if algo_type in ['GaudiSequencer', 'AthSequencer', 'ProcessPhase']:
265  if algo_name in ['RecoITSeq', 'RecoOTSeq', 'RecoTTSeq']:
266  continue
267 
268  if n not in self.unique_sequencers:
269  self.unique_sequencers.append(n)
270  else:
271  if n not in self.dupl_seqs:
272  self.dupl_seqs[n] = 2
273  else:
274  self.dupl_seqs[n] += 1
275 
276  seq_daughter = GaudiSequencer(algo_name, OutputLevel=INFO)
277  if self.cfg.node[n].get('ModeOR') == 'True':
278  self.OR_sequencers.append(n)
279  seq_daughter.ModeOR = True
280  # if self.cfg.node[n].get('Lazy') == 'False':
281  # print "Non-Lazy - ", n
282  seq_daughter.ShortCircuit = False
283  if seq_daughter not in seq.Members:
284  seq.Members += [seq_daughter]
285  # iterate deeper
286  self._generate_sequence(n, seq_daughter)
287  else:
288  #rndname = ''.join(random.choice(string.lowercase) for i in range(5))
289  #if algo_name in unique_algos: algo_name = algo_name + "-" + rndname
290  if n not in self.unique_algos:
291  self.unique_algos.append(n)
292  else:
293  if n not in self.dupl_algos:
294  self.dupl_algos[n] = 2
295  else:
296  self.dupl_algos[n] += 1
297 
298  avgRuntime, varRuntime = self.timeValue.get(algo_name)
299 
300  algo_daughter = CPUCruncher(
301  algo_name,
302  OutputLevel=self.outputLevel,
303  varRuntime=varRuntime,
304  avgRuntime=avgRuntime,
305  SleepFraction=self.sleepFraction
306  if self.IOboolValue.get() else 0.,
307  Timeline=self.enableTimeline)
308 
309  self._declare_data_deps(algo_name, algo_daughter)
310 
311  if algo_daughter not in seq.Members:
312  seq.Members += [algo_daughter]
313 
314  return seq
CPUCruncher
A class that implements a search for prime numbers.
Definition: CPUCruncher.h:29
GaudiSequencer
Sequencer for executing several algorithms, stopping when one is faulty.
Definition: GaudiSequencer.h:38
ranges::Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:358

◆ get()

def GaudiHive.precedence.CruncherSequence.get (   self)

Definition at line 221 of file precedence.py.

221  def get(self):
222 
223  return self.sequencer
224 
ranges::Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:358

Member Data Documentation

◆ cfg

GaudiHive.precedence.CruncherSequence.cfg

Definition at line 174 of file precedence.py.

◆ dfg

GaudiHive.precedence.CruncherSequence.dfg

Definition at line 175 of file precedence.py.

◆ dupl_algos

dictionary GaudiHive.precedence.CruncherSequence.dupl_algos = {}
static

Definition at line 156 of file precedence.py.

◆ dupl_seqs

dictionary GaudiHive.precedence.CruncherSequence.dupl_seqs = {}
static

Definition at line 153 of file precedence.py.

◆ enableTimeline

GaudiHive.precedence.CruncherSequence.enableTimeline

Definition at line 177 of file precedence.py.

◆ IOboolValue

GaudiHive.precedence.CruncherSequence.IOboolValue

Definition at line 171 of file precedence.py.

◆ OR_sequencers

list GaudiHive.precedence.CruncherSequence.OR_sequencers = []
static

Definition at line 154 of file precedence.py.

◆ outputLevel

GaudiHive.precedence.CruncherSequence.outputLevel

Definition at line 179 of file precedence.py.

◆ sequencer

GaudiHive.precedence.CruncherSequence.sequencer

Definition at line 182 of file precedence.py.

◆ sleepFraction

GaudiHive.precedence.CruncherSequence.sleepFraction

Definition at line 172 of file precedence.py.

◆ timeValue

GaudiHive.precedence.CruncherSequence.timeValue

Definition at line 170 of file precedence.py.

◆ unique_algos

list GaudiHive.precedence.CruncherSequence.unique_algos = []
static

Definition at line 155 of file precedence.py.

◆ unique_data_objects

list GaudiHive.precedence.CruncherSequence.unique_data_objects = []
static

Definition at line 158 of file precedence.py.

◆ unique_sequencers

list GaudiHive.precedence.CruncherSequence.unique_sequencers = []
static

Definition at line 152 of file precedence.py.

The documentation for this class was generated from the following file: