Auditor that uses NVTX to annotate ranges and incidents for profiling with NVIDIA tools. More...

Inheritance diagram for NVTXAuditor:

Collaboration diagram for NVTXAuditor:

Private Member Functions
StatusCode	initialize () override
	Initialize the auditor and register for all available incidents if the Incident service exists.
StatusCode	finalize () override
	Finalize the auditor, releasing the NVTX domain.
void	handle (const Incident &incident) override
	Handle an incident by annotating it with an NVTX marker.
std::string	make_key (std::string const &event, std::string const &caller, EventContext const &ctx) const
	Helper to create an identifier key for storing an associated range in the map.
std::string	make_label (std::string const &event, std::string const &caller) const
	Helper to create a label for the audited event.
nvtxRangeId_t	start_range (std::string const &label, EventContext const &ctx)
	Start an NVTX range for the audited event.
void	before (std::string const &event, std::string const &caller, EventContext const &ctx) override
	Handle 'before' audit for given event by starting an associated NVTX range.
void	after (std::string const &event, std::string const &caller, EventContext const &ctx, StatusCode const &=StatusCode::SUCCESS) override
	Handle 'after' audit for given event by ending the associated NVTX range.

Private Attributes
tbb::concurrent_hash_map< std::string, nvtxRangeId_t >	m_ranges

Static Private Attributes
static nvtxDomainHandle_t	s_domain { nullptr }
static uint32_t	s_numCategories { 0 }

Additional Inherited Members
Public Types inherited from extends< Gaudi::Auditor, IIncidentListener >
using	base_class
	Typedef to this class.
using	extend_interfaces_base
	Typedef to the base of this class.
Public Types inherited from Gaudi::Auditor
using	Factory = Gaudi::PluginService::Factory<IAuditor( const std::string&, ISvcLocator )>
Public Types inherited from PropertyHolder< CommonMessaging< implements< IAuditor, IProperty > > >
using	PropertyHolderImpl
	Typedef used to refer to this class from derived classes, as in.
Public Types inherited from CommonMessaging< implements< IAuditor, IProperty > >
using	base_class
Public Types inherited from extend_interfaces< Interfaces... >
using	ext_iids
	take union of the ext_iids of all Interfaces...
Public Member Functions inherited from extends< Gaudi::Auditor, IIncidentListener >
void const *	i_cast (const InterfaceID &tid) const override
	Implementation of IInterface::i_cast.
StatusCode	queryInterface (const InterfaceID &ti, void **pp) override
	Implementation of IInterface::queryInterface.
std::vector< std::string >	getInterfaceNames () const override
	Implementation of IInterface::getInterfaceNames.
Public Member Functions inherited from Gaudi::Auditor
	Auditor (std::string name, ISvcLocator *svcloc)
	Auditor (const Auditor &a)=delete
Auditor &	operator= (const Auditor &rhs)=delete
StatusCode	sysInitialize () override
StatusCode	sysFinalize () override
void	before (std::string const &, std::string const &, EventContext const &) override
void	after (std::string const &, std::string const &, EventContext const &, StatusCode const &=StatusCode::SUCCESS) override
const std::string &	name () const override
bool	isEnabled () const override
SmartIF< ISvcLocator > &	serviceLocator () const override
	The standard service locator.
template<class T = IService>
SmartIF< T >	service (std::string_view name, bool createIf=false) const
	Access a service by name, creating it if it doesn't already exist.
Public Member Functions inherited from PropertyHolder< CommonMessaging< implements< IAuditor, IProperty > > >
StatusCode	setProperty (const Gaudi::Details::PropertyBase &p)
	Set the property from a property.
StatusCode	setProperty (const std::string &name, const char *v)
	Special case for string literals.
StatusCode	setProperty (const std::string &name, const std::string &v)
	Special case for std::string.
StatusCode	setProperty (const std::string &name, const TYPE &value)
	set the property form the value
	PropertyHolder ()=default
Gaudi::Details::PropertyBase &	declareProperty (Gaudi::Details::PropertyBase &prop)
	Declare a property.
Gaudi::Details::PropertyBase *	declareRemoteProperty (const std::string &name, IProperty *rsvc, const std::string &rname="")
	Declare a remote property.
StatusCode	setPropertyRepr (const std::string &n, const std::string &r) override
	set the property from name and value string representation
StatusCode	getProperty (Gaudi::Details::PropertyBase *p) const override
	get the property
const std::vector< Gaudi::Details::PropertyBase * > &	getProperties () const override
	get all properties
bool	hasProperty (std::string_view name) const override
	Return true if we have a property with the given name.
Gaudi::Details::PropertyBase *	property (std::string_view name) const
	\fixme property and bindPropertiesTo should be protected
void	bindPropertiesTo (Gaudi::Interfaces::IOptionsSvc &optsSvc)
PropertyHolder &	operator= (const PropertyHolder &)=delete
Public Member Functions inherited from CommonMessagingBase
virtual	~CommonMessagingBase ()=default
	Virtual destructor.
const SmartIF< IMessageSvc > &	msgSvc () const
	The standard message service.
MsgStream &	msgStream () const
	Return an uninitialized MsgStream.
MsgStream &	msgStream (const MSG::Level level) const
	Predefined configurable message stream for the efficient printouts.
MsgStream &	always () const
	shortcut for the method msgStream(MSG::ALWAYS)
MsgStream &	fatal () const
	shortcut for the method msgStream(MSG::FATAL)
MsgStream &	err () const
	shortcut for the method msgStream(MSG::ERROR)
MsgStream &	error () const
	shortcut for the method msgStream(MSG::ERROR)
MsgStream &	warning () const
	shortcut for the method msgStream(MSG::WARNING)
MsgStream &	info () const
	shortcut for the method msgStream(MSG::INFO)
MsgStream &	debug () const
	shortcut for the method msgStream(MSG::DEBUG)
MsgStream &	verbose () const
	shortcut for the method msgStream(MSG::VERBOSE)
MsgStream &	msg () const
	shortcut for the method msgStream(MSG::INFO)
MSG::Level	msgLevel () const
	get the cached level (originally extracted from the embedded MsgStream)
bool	msgLevel (MSG::Level lvl) const
	get the output level from the embedded MsgStream
Protected Member Functions inherited from CommonMessaging< implements< IAuditor, IProperty > >
MSG::Level	setUpMessaging () const override final
	Set up local caches.
MSG::Level	resetMessaging ()
	Reinitialize internal states.
void	updateMsgStreamOutputLevel (int level)
	Update the output level of the cached MsgStream.

Detailed Description

Auditor that uses NVTX to annotate ranges and incidents for profiling with NVIDIA tools.

This auditor creates NVTX ranges around audited events and marks incidents with NVTX markers. The ranges and markers are created in a custom "Gaudi" NVTX domain. Ranges and markers related to events are annotated with event number in the payload, colored based on an event number and put in categories based on event slot. Ranges and markers not related to events are put into the default category with no payload and default color.

Author: Mateusz Jakub Fila

Definition at line 54 of file NVTXAuditor.cpp.

Member Function Documentation

◆ after()

void NVTXAuditor::after	(	std::string const &	event,
		std::string const &	caller,
		EventContext const &	ctx,
		StatusCode const &	= StatusCode::SUCCESS )

inlineoverrideprivate

Handle 'after' audit for given event by ending the associated NVTX range.

Parameters

event	The name of the audited event.
caller	The name of the caller of the audited event.
ctx	The EventContext associated with the audited event.
status	The status code of the audited event.

Note: In case of a suspension event, the range for the execution is created to indicate resumption rather than creating a range for suspension.

Definition at line 258 of file NVTXAuditor.cpp.

                                                             {
    // Suspension has ended, so resume the execution by creating a new
    // execution range. Do not create a range for Suspend itself.
    if ( IAuditor::Suspension == event ) {
      auto execute_key = make_key( IAuditor::Execute, caller, ctx );
      auto label       = make_label( IAuditor::Execute, caller );
      auto id          = start_range( label, ctx );
      auto ret         = m_ranges.emplace( execute_key, id );
      if ( !ret ) {
        error() << std::format( "Execution range for {} already exists while resuming", execute_key ) << endmsg;
        nvtxDomainRangeEnd( s_domain, id ); // the newly started range would otherwise never be closed
      } else {
        debug() << std::format( "Resumed execution range for {}", execute_key ) << endmsg;
      }
      return;
    }
 
    auto key = make_key( event, caller, ctx );
    auto acc = decltype( m_ranges )::accessor{};
    if ( m_ranges.find( acc, key ) ) {
      nvtxDomainRangeEnd( s_domain, acc->second );
      m_ranges.erase( acc );
    } else {
      error() << std::format( "Range for {} does not exist", key ) << endmsg;
    }
    debug() << std::format( "Finished range for {}", key ) << endmsg;
  }

◆ before()

void NVTXAuditor::before	(	std::string const &	event,
		std::string const &	caller,
		EventContext const &	ctx )

inlineoverrideprivate

Handle 'before' audit for given event by starting an associated NVTX range.

Parameters

event	The name of the audited event.
caller	The name of the caller of the audited event.
ctx	The EventContext associated with the audited event.

Note: In case of a suspension event, the range for the execution is closed instead of creating a range for suspension.

Definition at line 219 of file NVTXAuditor.cpp.

                                                                                                   {
    // Suspension pauses the execution, so close the currently active
    // execution range instead of creating a range for the suspension.
    if ( IAuditor::Suspension == event ) {
      auto execute_key = make_key( IAuditor::Execute, caller, ctx );
      auto acc         = decltype( m_ranges )::accessor{};
      if ( !m_ranges.find( acc, execute_key ) ) {
        error() << std::format( "Execution range for {} does not exist while suspending", execute_key ) << endmsg;
      } else {
        nvtxDomainRangeEnd( s_domain, acc->second );
        m_ranges.erase( acc );
        debug() << std::format( "Suspended execution range for {}", execute_key ) << endmsg;
      }
      return;
    }
 
    auto key   = make_key( event, caller, ctx );
    auto label = make_label( event, caller );
    auto id    = start_range( label, ctx );
    auto ret   = m_ranges.emplace( key, id );
    if ( !ret ) {
      error() << std::format( "Range for {} already exists", key ) << endmsg;
      nvtxDomainRangeEnd( s_domain, id ); // the newly started range would otherwise never be closed
    } else {
      debug() << std::format( "Started range for {}", key ) << endmsg;
    }
  }

◆ finalize()

StatusCode NVTXAuditor::finalize ( )

inlineoverrideprivatevirtual

Finalize the auditor, releasing the NVTX domain.

Returns: StatusCode indicating success or failure.

Reimplemented from Gaudi::Auditor.

Definition at line 117 of file NVTXAuditor.cpp.

                                 {
    if ( s_domain ) {
      nvtxDomainDestroy( s_domain );
      s_domain = nullptr;
    }
    return base_class::finalize();
  }

◆ handle()

void NVTXAuditor::handle ( const Incident & incident )

inlineoverrideprivate

Handle an incident by annotating it with an NVTX marker.

Parameters

incident The incident to handle.

Annotates the given incident with an NVTX marker. If the incident has a valid event context, the marker is annotated with event number in the payload, colored based on event number and put in category based on event slot. If the incident does not have a valid event context, the marker is created without payload, default color and default category.

Definition at line 134 of file NVTXAuditor.cpp.

                                                   {
    auto eventAttrib          = nvtxEventAttributes_t{};
    eventAttrib.version       = NVTX_VERSION;
    eventAttrib.size          = NVTX_EVENT_ATTRIB_STRUCT_SIZE;
    eventAttrib.messageType   = NVTX_MESSAGE_TYPE_ASCII;
    eventAttrib.message.ascii = incident.type().c_str();
 
    if ( !incident.context().valid() ) {
      nvtxDomainMarkEx( s_domain, &eventAttrib );
      debug() << std::format( "Marked incident {} from {}", incident.type(), incident.source() ) << endmsg;
      return;
    }
 
    eventAttrib.colorType        = NVTX_COLOR_ARGB;
    eventAttrib.color            = ::cycle_color( incident.context().evt() );
    eventAttrib.category         = incident.context().slot() + 1;
    eventAttrib.payloadType      = NVTX_PAYLOAD_TYPE_UNSIGNED_INT64;
    eventAttrib.payload.ullValue = incident.context().evt();
 
    nvtxDomainMarkEx( s_domain, &eventAttrib );
    debug() << std::format( "Marked incident {} from {} with context {}", incident.type(), incident.source(),
                            incident.context() )
            << endmsg;
  }

◆ initialize()

StatusCode NVTXAuditor::initialize ( )

inlineoverrideprivatevirtual

Initialize the auditor and register for all available incidents if the Incident service exists.

Returns: StatusCode indicating success or failure.

Note: Creates the NVTX domain and categories, which must be done only once.

Name a category for each slot in the HiveWhiteBoard

name a category for a single slot if no HiveWhiteBoard

Reimplemented from Gaudi::Auditor.

Definition at line 77 of file NVTXAuditor.cpp.

                                   {
 
    return base_class::initialize()
        .andThen( [this]() {
          auto incidentSvc = service<IIncidentSvc>( "IncidentSvc" );
          if ( incidentSvc.isValid() ) {
            incidentSvc->addListener( this, "ALL" );
          } else {
            info() << "No IncidentSvc, NVTXAuditor will not be able to annotate incidents." << endmsg;
          }
        } )
        .andThen( [this]() {
          if ( s_numCategories > 0 ) {
            error() << "Attempting to initialize NVTX categories more than once." << endmsg;
            return StatusCode::FAILURE;
          }
 
          s_domain = nvtxDomainCreateA( "Gaudi" );
 
          auto hiveWhiteBoard = service<IHiveWhiteBoard>( "EventDataSvc" );
          if ( hiveWhiteBoard.isValid() ) {
            s_numCategories = hiveWhiteBoard->getNumberOfStores();
            for ( auto slot = 0u; slot < s_numCategories; ++slot ) {
              // slot + 1 for 1-indexing of categories in NVTX
              nvtxDomainNameCategoryA( s_domain, slot + 1, std::format( "Slot {}", slot ).c_str() );
            }
          } else {
            s_numCategories = 1;
            nvtxDomainNameCategoryA( s_domain, 1, "Slot 0" );
          }
          return StatusCode::SUCCESS;
        } );
  }

◆ make_key()

std::string NVTXAuditor::make_key	(	std::string const &	event,
		std::string const &	caller,
		EventContext const &	ctx ) const

inlineprivate

Helper to create an identifier key for storing an associated range in the map.

Parameters

event	The name of the audited event.
caller	The name of the caller of the audited event.
ctx	The event context associated with the audited event.

Returns: A key for the event.

Definition at line 166 of file NVTXAuditor.cpp.

                                                                                                       {
    if ( ctx.valid() ) { return std::format( "{}:{}:{}", caller, event, ctx.evt() ); }
    return std::format( "{}:{}", caller, event );
  }

◆ make_label()

std::string NVTXAuditor::make_label	(	std::string const &	event,
		std::string const &	caller ) const

inlineprivate

Helper to create a label for the audited event.

Definition at line 174 of file NVTXAuditor.cpp.

                                                                                {
    return std::format( "{}:{}", caller, event );
  }

◆ start_range()

nvtxRangeId_t NVTXAuditor::start_range	(	std::string const &	label,
		EventContext const &	ctx )

inlineprivate

Start an NVTX range for the audited event.

Parameters

label	The label to attach to the range.
ctx	The event context associated with the audited event.

Returns: The opaque handle for the started NVTX range.

Starts an NVTX range for the audited event. If the context is valid, the range is annotated with event number in the payload, colored based on event number and put in category based on event slot. If the context is not valid, the range is created without payload, default color and default category.

Definition at line 187 of file NVTXAuditor.cpp.

                                                                               {
    auto eventAttrib          = nvtxEventAttributes_t{};
    eventAttrib.version       = NVTX_VERSION;
    eventAttrib.size          = NVTX_EVENT_ATTRIB_STRUCT_SIZE;
    eventAttrib.messageType   = NVTX_MESSAGE_TYPE_ASCII;
    eventAttrib.message.ascii = label.c_str();
 
    if ( !ctx.valid() ) {
      debug() << std::format( "Starting range for {}", label ) << endmsg;
      return nvtxDomainRangeStartEx( s_domain, &eventAttrib );
    }
 
    eventAttrib.colorType        = NVTX_COLOR_ARGB;
    eventAttrib.color            = ::cycle_color( ctx.evt() );
    eventAttrib.category         = ctx.slot() + 1;
    eventAttrib.payloadType      = NVTX_PAYLOAD_TYPE_UNSIGNED_INT64;
    eventAttrib.payload.ullValue = ctx.evt();
 
    debug() << std::format( "Starting range for {} with context {}", label, ctx ) << endmsg;
    return nvtxDomainRangeStartEx( s_domain, &eventAttrib );
  }

Member Data Documentation

◆ m_ranges

tbb::concurrent_hash_map<std::string, nvtxRangeId_t> NVTXAuditor::m_ranges

private

Definition at line 69 of file NVTXAuditor.cpp.

◆ s_domain

nvtxDomainHandle_t NVTXAuditor::s_domain { nullptr }

inlinestaticprivate

Definition at line 60 of file NVTXAuditor.cpp.

60{ nullptr };

◆ s_numCategories

uint32_t NVTXAuditor::s_numCategories { 0 }

inlinestaticprivate

Definition at line 64 of file NVTXAuditor.cpp.

64{ 0 };

The documentation for this class was generated from the following file:

GaudiCUDA/src/NVTXAuditor.cpp

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ after()

◆ before()

◆ finalize()

◆ handle()

◆ initialize()

◆ make_key()

◆ make_label()

◆ start_range()

Member Data Documentation

◆ m_ranges

◆ s_domain

◆ s_numCategories