The Gaudi Framework  v32r2 (46d42edc)
Public Member Functions | Public Attributes | Static Public Attributes | Private Member Functions | List of all members
GaudiHive.precedence.CruncherSequence Class Reference
Inheritance diagram for GaudiHive.precedence.CruncherSequence:
Inheritance graph
[legend]
Collaboration diagram for GaudiHive.precedence.CruncherSequence:
Collaboration graph
[legend]

Public Member Functions

def __init__ (self, timeValue, IOboolValue, sleepFraction, cfgPath, dfgPath, topSequencer, showStat=False, timeline=False, outputLevel=INFO)
 
def get (self)
 

Public Attributes

 timeValue
 
 IOboolValue
 
 sleepFraction
 
 cfg
 
 dfg
 
 enableTimeline
 
 outputLevel
 
 sequencer
 

Static Public Attributes

list unique_sequencers = []
 
dictionary dupl_seqs = {}
 
list OR_sequencers = []
 
list unique_algos = []
 
dictionary dupl_algos = {}
 
list unique_data_objects = []
 

Private Member Functions

def _declare_data_deps (self, algo_name, algo)
 
def _generate_sequence (self, name, seq=None)
 

Detailed Description

Constructs the sequence tree of CPUCrunchers with provided control flow and data flow precedence rules.

Definition at line 139 of file precedence.py.

Constructor & Destructor Documentation

◆ __init__()

def GaudiHive.precedence.CruncherSequence.__init__ (   self,
  timeValue,
  IOboolValue,
  sleepFraction,
  cfgPath,
  dfgPath,
  topSequencer,
  showStat = False,
  timeline = False,
  outputLevel = INFO 
) 
Keyword arguments:
timeValue -- timeValue object to set algorithm execution time
IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
showStat -- print out statistics on precedence graph

Definition at line 150 of file precedence.py.

150  def __init__(self,
151  timeValue,
152  IOboolValue,
153  sleepFraction,
154  cfgPath,
155  dfgPath,
156  topSequencer,
157  showStat=False,
158  timeline=False,
159  outputLevel=INFO):
160  """
161  Keyword arguments:
162  timeValue -- timeValue object to set algorithm execution time
163  IOboolValue -- *BooleanValue object to set whether an algorithm has to experience IO-bound execution
164  cfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with control flow dependencies
165  dfgPath -- relative to $GAUDIHIVEROOT/data path to GRAPHML file with data flow dependencies
166  showStat -- print out statistics on precedence graph
167  """
168 
169  self.timeValue = timeValue
170  self.IOboolValue = IOboolValue
171  self.sleepFraction = sleepFraction
172 
173  self.cfg = nx.read_graphml(_buildFilePath(cfgPath))
174  self.dfg = nx.read_graphml(_buildFilePath(dfgPath))
175 
176  self.enableTimeline = timeline
177 
178  self.outputLevel = outputLevel
179 
180  # Generate control flow part
181  self.sequencer = self._generate_sequence(topSequencer)
182 
183  if showStat:
184  import pprint
185 
186  print("\n===== Statistics on Algorithms =====")
187  print("Total number of algorithm nodes: ", len(self.unique_algos) +
188  sum([self.dupl_algos[i] - 1 for i in self.dupl_algos]))
189  print("Number of unique algorithms: ", len(self.unique_algos))
190  print(" -->", len(self.dupl_algos),
191  "of them being re-used with the following distribution: ",
192  [self.dupl_algos[i] for i in self.dupl_algos])
193  # pprint.pprint(dupl_algos)
194 
195  print("\n===== Statistics on Sequencers =====")
196  print("Total number of sequencers: ", len(self.unique_sequencers) +
197  sum([self.dupl_seqs[i] - 1 for i in self.dupl_seqs]))
198  print("Number of unique sequencers: ", len(self.unique_sequencers))
199  print(" -->", len(self.dupl_seqs),
200  "of them being re-used with the following distribution: ",
201  [self.dupl_seqs[i] for i in self.dupl_seqs])
202  # pprint.pprint(dupl_seqs)
203  print("Number of OR-sequencers: ", len(self.OR_sequencers))
204 
205  print("\n===== Statistics on DataObjects =====")
206  print("Number of unique DataObjects: ",
207  len(self.unique_data_objects))
208  # pprint.pprint(self.unique_data_objects)
209  print()
210 
GaudiHive.precedence._buildFilePath
def _buildFilePath(filePath)
Definition: precedence.py:13

Member Function Documentation

◆ _declare_data_deps()

def GaudiHive.precedence.CruncherSequence._declare_data_deps (   self,
  algo_name,
  algo 
)
private 
Declare data inputs and outputs for a given algorithm.

Definition at line 215 of file precedence.py.

215  def _declare_data_deps(self, algo_name, algo):
216  """ Declare data inputs and outputs for a given algorithm. """
217 
218  # Declare data inputs
219  for inNode, outNode in self.dfg.in_edges(algo_name):
220  dataName = inNode
221  if dataName not in self.unique_data_objects:
222  self.unique_data_objects.append(dataName)
223 
224  if dataName not in algo.inpKeys:
225  algo.inpKeys.append(dataName)
226 
227  # Declare data outputs
228  for inNode, outNode in self.dfg.out_edges(algo_name):
229  dataName = outNode
230  if dataName not in self.unique_data_objects:
231  self.unique_data_objects.append(dataName)
232 
233  if dataName not in algo.outKeys:
234  algo.outKeys.append(dataName)
235 

◆ _generate_sequence()

def GaudiHive.precedence.CruncherSequence._generate_sequence (   self,
  name,
  seq = None 
)
private 
Assemble the tree of sequencers.

Definition at line 236 of file precedence.py.

236  def _generate_sequence(self, name, seq=None):
237  """ Assemble the tree of sequencers. """
238 
239  if not seq:
240  seq = GaudiSequencer(name, ShortCircuit=False)
241 
242  for n in self.cfg[name]:
243  # extract entity name and type
244  algo_name = n.split('/')[1] if '/' in n else n
245 
246  if 'type' in self.cfg.node[n]:
247  # first rely on explicit type, if given
248  algo_type = self.cfg.node[n].get('type')
249  else:
250  # if the type is not given explicitly, try to extract it from entity name,
251  # and, if unsuccessful, assume it is an algorithm
252  algo_type = n.split('/')[0] if '/' in n else 'Algorithm'
253 
254  if algo_type in ['GaudiSequencer', 'AthSequencer', 'ProcessPhase']:
255  if algo_name in ['RecoITSeq', 'RecoOTSeq', 'RecoTTSeq']:
256  continue
257 
258  if n not in self.unique_sequencers:
259  self.unique_sequencers.append(n)
260  else:
261  if n not in self.dupl_seqs:
262  self.dupl_seqs[n] = 2
263  else:
264  self.dupl_seqs[n] += 1
265 
266  seq_daughter = GaudiSequencer(algo_name, OutputLevel=INFO)
267  if self.cfg.node[n].get('ModeOR') == 'True':
268  self.OR_sequencers.append(n)
269  seq_daughter.ModeOR = True
270  # if self.cfg.node[n].get('Lazy') == 'False':
271  # print "Non-Lazy - ", n
272  seq_daughter.ShortCircuit = False
273  if seq_daughter not in seq.Members:
274  seq.Members += [seq_daughter]
275  # iterate deeper
276  self._generate_sequence(n, seq_daughter)
277  else:
278  #rndname = ''.join(random.choice(string.lowercase) for i in range(5))
279  #if algo_name in unique_algos: algo_name = algo_name + "-" + rndname
280  if n not in self.unique_algos:
281  self.unique_algos.append(n)
282  else:
283  if n not in self.dupl_algos:
284  self.dupl_algos[n] = 2
285  else:
286  self.dupl_algos[n] += 1
287 
288  avgRuntime, varRuntime = self.timeValue.get(algo_name)
289 
290  algo_daughter = CPUCruncher(
291  algo_name,
292  OutputLevel=self.outputLevel,
293  varRuntime=varRuntime,
294  avgRuntime=avgRuntime,
295  SleepFraction=self.sleepFraction
296  if self.IOboolValue.get() else 0.,
297  Timeline=self.enableTimeline)
298 
299  self._declare_data_deps(algo_name, algo_daughter)
300 
301  if algo_daughter not in seq.Members:
302  seq.Members += [algo_daughter]
303 
304  return seq
CPUCruncher
A class that implements a search for prime numbers.
Definition: CPUCruncher.h:19
GaudiSequencer
Sequencer for executing several algorithms, stopping when one is faulty.
Definition: GaudiSequencer.h:28
ranges::Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:348

◆ get()

def GaudiHive.precedence.CruncherSequence.get (   self)

Definition at line 211 of file precedence.py.

211  def get(self):
212 
213  return self.sequencer
214 
ranges::Gaudi::Functional::details::get
auto get(const Handle &handle, const Algo &, const EventContext &) -> decltype(details::deref(handle.get()))
Definition: FunctionalDetails.h:348

Member Data Documentation

◆ cfg

GaudiHive.precedence.CruncherSequence.cfg

Definition at line 164 of file precedence.py.

◆ dfg

GaudiHive.precedence.CruncherSequence.dfg

Definition at line 165 of file precedence.py.

◆ dupl_algos

dictionary GaudiHive.precedence.CruncherSequence.dupl_algos = {}
static

Definition at line 146 of file precedence.py.

◆ dupl_seqs

dictionary GaudiHive.precedence.CruncherSequence.dupl_seqs = {}
static

Definition at line 143 of file precedence.py.

◆ enableTimeline

GaudiHive.precedence.CruncherSequence.enableTimeline

Definition at line 167 of file precedence.py.

◆ IOboolValue

GaudiHive.precedence.CruncherSequence.IOboolValue

Definition at line 161 of file precedence.py.

◆ OR_sequencers

list GaudiHive.precedence.CruncherSequence.OR_sequencers = []
static

Definition at line 144 of file precedence.py.

◆ outputLevel

GaudiHive.precedence.CruncherSequence.outputLevel

Definition at line 169 of file precedence.py.

◆ sequencer

GaudiHive.precedence.CruncherSequence.sequencer

Definition at line 172 of file precedence.py.

◆ sleepFraction

GaudiHive.precedence.CruncherSequence.sleepFraction

Definition at line 162 of file precedence.py.

◆ timeValue

GaudiHive.precedence.CruncherSequence.timeValue

Definition at line 160 of file precedence.py.

◆ unique_algos

list GaudiHive.precedence.CruncherSequence.unique_algos = []
static

Definition at line 145 of file precedence.py.

◆ unique_data_objects

list GaudiHive.precedence.CruncherSequence.unique_data_objects = []
static

Definition at line 148 of file precedence.py.

◆ unique_sequencers

list GaudiHive.precedence.CruncherSequence.unique_sequencers = []
static

Definition at line 142 of file precedence.py.

The documentation for this class was generated from the following file: