FiberManager Class Reference

The FiberManager manages a pool of threads used to run boost::fiber fibers. More...

#include </builds/gaudi/Gaudi/GaudiHive/src/FiberManager.h>

Collaboration diagram for FiberManager:

Public Member Functions
	FiberManager (int n_threads, std::size_t stackSize=16ULL *1024 *1024)
	FiberManager constructor.
	~FiberManager ()
template<typename F>
void	schedule (F &&func)
	Schedule work to run on the asynchronous pool.

Private Types
using	StackAlloc = boost::fibers::fixedsize_stack
	Stack allocator used for fibers.
using	SchedAlgo = boost::fibers::algo::shared_work

Private Attributes
boost::fibers::condition_variable	m_shuttingDown_cv {}
boost::fibers::mutex	m_shuttingDown_mtx {}
std::vector< std::jthread >	m_threads {}
std::vector< SchedAlgo * >	m_schedAlgoList {}
std::size_t	m_stackSize
	Stack size for fibers – virtual memory so this is not consume.

Detailed Description

The FiberManager manages a pool of threads used to run boost::fiber fibers.

These fibers can be suspended while waiting for GPU operations (with CUDA and HIP support built-in), remote I/O operations, and the like, providing a base for asynchronous algorithm support.

Author

Beojan Stanislaus

Version

1.0

Date

November 2023

Definition at line 33 of file FiberManager.h.

Member Typedef Documentation

SchedAlgo

using FiberManager::SchedAlgo = boost::fibers::algo::shared_work

private

Definition at line 75 of file FiberManager.h.

StackAlloc

using FiberManager::StackAlloc = boost::fibers::fixedsize_stack

private

Stack allocator used for fibers.

Definition at line 74 of file FiberManager.h.

Constructor & Destructor Documentation

FiberManager()

FiberManager::FiberManager	(	int	n_threads,
		std::size_t	stackSize = 16ULL * 1024 * 1024 )

FiberManager constructor.

Parameters

n_threads

Number of threads for CPU portion of asynchronous algorithms. These are in addition to the TBB worker threads used for CPU algorithms.

Note

The calling thread is initialized with fiber scheduler and starts participating in sharing the global work queue of fibers, the current context becomes the main fiber of the calling thread. The main fiber should not be suspended, preemptive blocking is allowed. After construction, the calling thread must not be assigned different fiber scheduling algorithm.

Definition at line 18 of file FiberManager.cpp.

                                                               : m_stackSize( stackSize ) {
  // Start a fiber scheduler with a shared work queue on the current thread.
  // The fiber scheduler is configured not to sleep in case there are no ready fibers, but this situation should never
  // happen as the main fiber of this thread should never be suspended so it's always active and scheduler will never
  // try to pick-up another fiber.
  boost::fibers::use_scheduling_algorithm<SchedAlgo>();
 
  // Create a pool of threads running fiber schedulers with shared work queue.
  // The scheduler in each thread is configured to sleep while no fibers are ready and needs to be manual
  // notified if it's asleep when a new work item is added.
  const auto sleep_while_idle = true;
  m_schedAlgoList.resize( n_threads );
  m_threads.reserve( n_threads );
  for ( int i = 0; i < n_threads; ++i ) {
    m_threads.emplace_back( [this, i]() {
      // Save handle to the scheduler of this thread so it can be manually notified.
      auto* algo_ptr          = new SchedAlgo( sleep_while_idle );
      m_schedAlgoList.at( i ) = algo_ptr;
#if ( BOOST_VERSION >= 108400 )
      boost::fibers::initialize_thread( algo_ptr,
                                        boost::fibers::make_stack_allocator_wrapper<StackAlloc>( m_stackSize ) );
#else
      boost::fibers::context::active()->get_scheduler()->set_algo( algo_ptr );
#endif
      // Suspend the main fiber of this worker so it doesn't exit prematurely.
      std::unique_lock lck{ m_shuttingDown_mtx };
      m_shuttingDown_cv.wait( lck );
    } );
  }
}

~FiberManager()

FiberManager::~FiberManager

(

)

Definition at line 49 of file FiberManager.cpp.

                            {
  // Notify the main fibers of the worker threads to exit.
  m_shuttingDown_cv.notify_all();
}

Member Function Documentation

schedule()

template<typename F>

void FiberManager::schedule ( F && func )

inline

Schedule work to run on the asynchronous pool.

AsynchronousAlgorithms should use a relatively small amount of CPU time, and should use boost::fiber functionality to yield while waiting for offloaded work to complete.

Parameters

func	The AlgTask, when used in AvalancheSchedulerSvc

Note

This function should only be called from a thread participating in the global work sharing of fibers, e.g. the thread that constructed the FiberManager.

Definition at line 63 of file FiberManager.h.

                            {
    // Add a new fiber invoking the function to the shared work queue, then notify the workers in case they were
    // sleeping.
    boost::fibers::fiber( boost::fibers::launch::post, std::allocator_arg, StackAlloc( m_stackSize ),
                          std::forward<F>( func ) )
        .detach();
    for ( auto* p : m_schedAlgoList ) { p->notify(); }
  }

Member Data Documentation

m_schedAlgoList

std::vector<SchedAlgo*> FiberManager::m_schedAlgoList {}

private

Definition at line 79 of file FiberManager.h.

{};

m_shuttingDown_cv

boost::fibers::condition_variable FiberManager::m_shuttingDown_cv {}

private

Definition at line 76 of file FiberManager.h.

{};

m_shuttingDown_mtx

boost::fibers::mutex FiberManager::m_shuttingDown_mtx {}

private

Definition at line 77 of file FiberManager.h.

{};

m_stackSize

std::size_t FiberManager::m_stackSize

private

Stack size for fibers – virtual memory so this is not consume.

Definition at line 82 of file FiberManager.h.

m_threads

std::vector<std::jthread> FiberManager::m_threads {}

private

Definition at line 78 of file FiberManager.h.

{};

---

The documentation for this class was generated from the following files:

• GaudiHive/src/FiberManager.h
• GaudiHive/src/FiberManager.cpp