GaudiProfiling/doc/jemallocprofiler.md
Go to the documentation of this file.
1 Profiling Gaudi jobs with Jemalloc
2 ===============================================================================
3 
4 Integration of Jemalloc within gaudi jobs.
5 Uses the Jemalloc library from http://www.canonware.com/jemalloc/
6 
7 It is possible to profile the memory used by Gaudi jobs, using the jemalloc library
8 as documented in https://github.com/jemalloc/jemalloc/wiki
9 
10 To run the profiler, it is necessary to:
11  * preload libjemalloc.so, using the LD_PRELOAD environment variable
12  * set the MALLOC_CONF environment variable to enable profiling
13 
14 gaudirun.py has been updated to set the environment accordingly (a prerequisite is however
15 that libjemalloc.so has to be available in the library path).
16 
17 A Gaudi algorithm has also been developped to perform memory heap dumps at various event,
18 and is configured using the StartFromEventN, StopAtEventN and DumpPeriod, as described
19 in the example below.
20 
21 Run
22 --------------------------------------------------------------------------------
23 
24 ### Change Options File
25 
26 Simple example of using the JemallocProfile algorithm in a Gaudi configurable:
27 
28 ~~~~~~~~{.py}
29 #!/usr/bin/env gaudirun.py
30 from Configurables import JemallocProfile
31 #...
32 jp = JemallocProfile()
33 jp.StartFromEventN = 49
34 jp.StopAtEventN = 99
35 jp.DumpPeriod = 10
36 
37 GaudiSequencer("PhysicsSeq").Members += [ jp ]
38 ~~~~~~~~
39 
40 ### Run the job
41 ~~~~~~~~{.sh}
42 $> gaudirun.py --profilerName=jemalloc --run-info-file=runinfo.json myoptions.py
43 ~~~~~~~~
44 
45 Please note the the --profilerName=jemalloc to enbale the profiling, and the run-info-file that produces
46 a file containing information useful to interpret the results (process id of the Gaudi job, and the absolute path
47 of the executable, necessary to run the pprof analysis tool).
48 
49 Analyze
50 --------------------------------------------------------------------------------
51 ### With text output
52 
53 The pprof analysis tool from the Google performances tools (http://goog-perftools.sourceforge.net/)
54 is necessary to analyze the heap files.
55 
56 It can be used to comapre the memory in two heap files in the following way:
57 
58 ~~~~~~~~{.sh}
59 $> pprof -text --base=<firstheap>.heap <executable name> <comparewith>.heap
60 ~~~~~~~~
61 
62 ### To produce a postscript file
63 
64 ~~~~~~~~{.sh}
65 $> pprof -gv --base=<firstheap>.heap <executable name> <comparewith>.heap
66 ~~~~~~~~