The Gaudi Framework  v32r2 (46d42edc)
Gaudi::IODataManager Class Reference

#include <src/component/IODataManager.h>

Inheritance diagram for Gaudi::IODataManager:
Collaboration diagram for Gaudi::IODataManager:

Classes

struct  Entry
 

Public Member Functions

StatusCode initialize () override
 IService implementation: initialize the service. More...
 
StatusCode finalize () override
 IService implementation: finalize the service. More...
 
StatusCode connectRead (bool keep_open, Connection *ioDesc) override
 Open data stream in read mode. More...
 
StatusCode connectWrite (Connection *con, IoType mode=Connection::CREATE, CSTR doctype="UNKNOWN") override
 Open data stream in write mode. More...
 
StatusCode disconnect (Connection *ioDesc) override
 Release data stream. More...
 
Connection * connection (const std::string &dsn) const override
 Retrieve known connection. More...
 
Connections connections (const IInterface *owner) const override
 Get connection by owner instance (0=ALL) More...
 
StatusCode read (Connection *ioDesc, void *const data, size_t len) override
 Read raw byte buffer from input stream. More...
 
StatusCode write (Connection *con, const void *data, int len) override
 Write raw byte buffer to output stream. More...
 
long long int seek (Connection *ioDesc, long long int where, int origin) override
 Seek on the file described by ioDesc. Arguments as in ::seek() More...
 
- Public Member Functions inherited from extends< Service, IIODataManager >
void * i_cast (const InterfaceID &tid) const override
 Implementation of IInterface::i_cast. More...
 
StatusCode queryInterface (const InterfaceID &ti, void **pp) override
 Implementation of IInterface::queryInterface. More...
 
std::vector< std::stringgetInterfaceNames () const override
 Implementation of IInterface::getInterfaceNames. More...
 
- Public Member Functions inherited from Service
const std::stringname () const override
 Retrieve name of the service. More...
 
StatusCode configure () override
 
StatusCode initialize () override
 
StatusCode start () override
 
StatusCode stop () override
 
StatusCode finalize () override
 
StatusCode terminate () override
 
Gaudi::StateMachine::State FSMState () const override
 
Gaudi::StateMachine::State targetFSMState () const override
 
StatusCode reinitialize () override
 
StatusCode restart () override
 
StatusCode sysInitialize () override
 Initialize Service. More...
 
StatusCode sysStart () override
 Initialize Service. More...
 
StatusCode sysStop () override
 Initialize Service. More...
 
StatusCode sysFinalize () override
 Finalize Service. More...
 
StatusCode sysReinitialize () override
 Re-initialize the Service. More...
 
StatusCode sysRestart () override
 Re-initialize the Service. More...
 
 Service (std::string name, ISvcLocator *svcloc)
 Standard Constructor. More...
 
SmartIF< ISvcLocator > & serviceLocator () const override
 Retrieve pointer to service locator. More...
 
StatusCode setProperties ()
 Method for setting declared properties to the values specified for the job. More...
 
template<class T >
StatusCode service (const std::string &name, const T *&psvc, bool createIf=true) const
 Access a service by name, creating it if it doesn't already exist. More...
 
template<class T >
StatusCode service (const std::string &name, T *&psvc, bool createIf=true) const
 
template<typename IFace = IService>
SmartIF< IFace > service (const std::string &name, bool createIf=true) const
 
template<class T >
StatusCode service (const std::string &svcType, const std::string &svcName, T *&psvc) const
 Access a service by name and type, creating it if it doesn't already exist. More...
 
template<class T >
StatusCode declareTool (ToolHandle< T > &handle, std::string toolTypeAndName, bool createIf=true)
 Declare used tool. More...
 
SmartIF< IAuditorSvc > & auditorSvc () const
 The standard auditor service.May not be invoked before sysInitialize() has been invoked. More...
 
- Public Member Functions inherited from PropertyHolder< CommonMessaging< implements< IService, IProperty, IStateful > > >
 PropertyHolder ()=default
 
Gaudi::Details::PropertyBasedeclareProperty (Gaudi::Details::PropertyBase &prop)
 Declare a property. More...
 
Gaudi::Details::PropertyBasedeclareProperty (const std::string &name, TYPE &value, const std::string &doc="none")
 Helper to wrap a regular data member and use it as a regular property. More...
 
Gaudi::Details::PropertyBasedeclareProperty (const std::string &name, Gaudi::Property< TYPE, VERIFIER, HANDLERS > &prop, const std::string &doc="none")
 Declare a PropertyBase instance setting name and documentation. More...
 
Gaudi::Details::PropertyBasedeclareRemoteProperty (const std::string &name, IProperty *rsvc, const std::string &rname="")
 Declare a remote property. More...
 
StatusCode setProperty (const Gaudi::Details::PropertyBase &p) override
 set the property form another property More...
 
StatusCode setProperty (const std::string &s) override
 set the property from the formatted string More...
 
StatusCode setProperty (const std::string &n, const std::string &v) override
 set the property from name and the value More...
 
StatusCode setProperty (const std::string &name, const TYPE &value)
 set the property form the value More...
 
StatusCode getProperty (Gaudi::Details::PropertyBase *p) const override
 get the property More...
 
const Gaudi::Details::PropertyBasegetProperty (const std::string &name) const override
 get the property by name More...
 
StatusCode getProperty (const std::string &n, std::string &v) const override
 convert the property to the string More...
 
const std::vector< Gaudi::Details::PropertyBase * > & getProperties () const override
 get all properties More...
 
bool hasProperty (const std::string &name) const override
 Return true if we have a property with the given name. More...
 
 PropertyHolder (const PropertyHolder &)=delete
 
PropertyHolderoperator= (const PropertyHolder &)=delete
 
- Public Member Functions inherited from CommonMessaging< implements< IService, IProperty, IStateful > >
MSG::Level msgLevel () const
 get the cached level (originally extracted from the embedded MsgStream) More...
 
bool msgLevel (MSG::Level lvl) const
 get the output level from the embedded MsgStream More...
 
MSG::Level outputLevel () const
 Backward compatibility function for getting the output level. More...
 
- Public Member Functions inherited from CommonMessagingBase
virtual ~CommonMessagingBase ()=default
 Virtual destructor. More...
 
const SmartIF< IMessageSvc > & msgSvc () const
 The standard message service. More...
 
MsgStreammsgStream () const
 Return an uninitialized MsgStream. More...
 
MsgStreammsgStream (const MSG::Level level) const
 Predefined configurable message stream for the efficient printouts. More...
 
MsgStreamalways () const
 shortcut for the method msgStream(MSG::ALWAYS) More...
 
MsgStreamfatal () const
 shortcut for the method msgStream(MSG::FATAL) More...
 
MsgStreamerr () const
 shortcut for the method msgStream(MSG::ERROR) More...
 
MsgStreamerror () const
 shortcut for the method msgStream(MSG::ERROR) More...
 
MsgStreamwarning () const
 shortcut for the method msgStream(MSG::WARNING) More...
 
MsgStreaminfo () const
 shortcut for the method msgStream(MSG::INFO) More...
 
MsgStreamdebug () const
 shortcut for the method msgStream(MSG::DEBUG) More...
 
MsgStreamverbose () const
 shortcut for the method msgStream(MSG::VERBOSE) More...
 
MsgStreammsg () const
 shortcut for the method msgStream(MSG::INFO) More...
 

Protected Types

typedef const std::stringCSTR
 
typedef std::map< std::string, Entry * > ConnectionMap
 
typedef std::map< std::string, std::stringFidMap
 

Protected Member Functions

StatusCode connectDataIO (int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
 
StatusCode reconnect (Entry *e)
 
StatusCode error (CSTR msg, bool rethrow)
 Small routine to issue exceptions. More...
 
StatusCode establishConnection (Connection *con)
 
- Protected Member Functions inherited from Service
 ~Service () override
 Standard Destructor. More...
 
int outputLevel () const
 get the Service's output level More...
 
- Protected Member Functions inherited from PropertyHolder< CommonMessaging< implements< IService, IProperty, IStateful > > >
Gaudi::Details::PropertyBaseproperty (const std::string &name) const
 
- Protected Member Functions inherited from CommonMessaging< implements< IService, IProperty, IStateful > >
MSG::Level setUpMessaging () const
 Set up local caches. More...
 
MSG::Level resetMessaging ()
 Reinitialize internal states. More...
 
void updateMsgStreamOutputLevel (int level)
 Update the output level of the cached MsgStream. More...
 

Protected Attributes

Gaudi::Property< std::stringm_catalogSvcName
 
Gaudi::Property< bool > m_useGFAL {this, "UseGFAL", true, "flag for auto gfal data access"}
 
Gaudi::Property< bool > m_quarantine
 
Gaudi::Property< int > m_ageLimit {this, "AgeLimit", 2, "age limit"}
 
Gaudi::Property< bool > m_disablePFNWarning
 
ConnectionMap m_connectionMap
 Map with I/O descriptors. More...
 
SmartIF< IFileCatalogm_catalog
 Reference to file catalog. More...
 
FidMap m_fidMap
 Map of FID to PFN. More...
 
SmartIF< IIncidentSvcm_incSvc
 
- Protected Attributes inherited from Service
Gaudi::StateMachine::State m_state = Gaudi::StateMachine::OFFLINE
 Service state. More...
 
Gaudi::StateMachine::State m_targetState = Gaudi::StateMachine::OFFLINE
 Service state. More...
 
Gaudi::Property< int > m_outputLevel {this, "OutputLevel", MSG::NIL, "output level"}
 
Gaudi::Property< bool > m_auditInit {this, "AuditServices", false, " unused"}
 
Gaudi::Property< bool > m_auditorInitialize {this, "AuditInitialize", false, "trigger auditor on initialize()"}
 
Gaudi::Property< bool > m_auditorStart {this, "AuditStart", false, "trigger auditor on start()"}
 
Gaudi::Property< bool > m_auditorStop {this, "AuditStop", false, "trigger auditor on stop()"}
 
Gaudi::Property< bool > m_auditorFinalize {this, "AuditFinalize", false, "trigger auditor on finalize()"}
 
Gaudi::Property< bool > m_auditorReinitialize {this, "AuditReinitialize", false, "trigger auditor on reinitialize()"}
 
Gaudi::Property< bool > m_auditorRestart {this, "AuditRestart", false, "trigger auditor on restart()"}
 
SmartIF< IAuditorSvcm_pAuditorSvc
 Auditor Service. More...
 

Additional Inherited Members

- Public Types inherited from extends< Service, IIODataManager >
using base_class = extends
 Typedef to this class. More...
 
using extend_interfaces_base = extend_interfaces< Interfaces... >
 Typedef to the base of this class. More...
 
- Public Types inherited from Service
using Factory = Gaudi::PluginService::Factory< IService *(const std::string &, ISvcLocator *)>
 
- Public Types inherited from PropertyHolder< CommonMessaging< implements< IService, IProperty, IStateful > > >
using PropertyHolderImpl = PropertyHolder< CommonMessaging< implements< IService, IProperty, IStateful > > >
 Typedef used to refer to this class from derived classes, as in. More...
 
- Public Types inherited from CommonMessaging< implements< IService, IProperty, IStateful > >
using base_class = CommonMessaging
 
- Public Types inherited from extend_interfaces< Interfaces... >
using ext_iids = typename Gaudi::interface_list_cat< typename Interfaces::ext_iids... >::type
 take union of the ext_iids of all Interfaces... More...
 

Detailed Description

Definition at line 29 of file IODataManager.h.

Member Typedef Documentation

◆ ConnectionMap

typedef std::map<std::string, Entry*> Gaudi::IODataManager::ConnectionMap
protected

Definition at line 40 of file IODataManager.h.

◆ CSTR

typedef const std::string& Gaudi::IODataManager::CSTR
protected

Definition at line 31 of file IODataManager.h.

◆ FidMap

typedef std::map<std::string, std::string> Gaudi::IODataManager::FidMap
protected

Definition at line 41 of file IODataManager.h.

Member Function Documentation

◆ connectDataIO()

StatusCode IODataManager::connectDataIO ( int  typ,
IoType  rw,
CSTR  fn,
CSTR  technology,
bool  keep,
Connection *  con 
)
protected

Definition at line 219 of file IODataManager.cpp.

220  {
221  MsgStream log( msgSvc(), name() );
222  std::string dsn = dataset;
223  try {
224  StatusCode sc( StatusCode::SUCCESS, true );
225  if ( ::strncasecmp( dsn.c_str(), "FID:", 4 ) == 0 )
226  dsn = dataset.substr( 4 ), typ = FID;
227  else if ( ::strncasecmp( dsn.c_str(), "LFN:", 4 ) == 0 )
228  dsn = dataset.substr( 4 ), typ = LFN;
229  else if ( ::strncasecmp( dsn.c_str(), "PFN:", 4 ) == 0 )
230  dsn = dataset.substr( 4 ), typ = PFN;
231  else if ( typ == UNKNOWN )
232  return connectDataIO( PFN, rw, dsn, technology, keep_open, connection );
233 
234  if ( std::find( s_badFiles.begin(), s_badFiles.end(), dsn ) != s_badFiles.end() ) {
235  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
237  }
238  if ( typ == FID ) {
239  auto fi = m_connectionMap.find( dsn );
240  if ( fi == m_connectionMap.end() ) {
241  IFileCatalog::Files files;
242  m_catalog->getPFN( dsn, files );
243  if ( files.empty() ) {
244  if ( !m_useGFAL ) {
245  if ( m_quarantine ) s_badFiles.insert( dsn );
246  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
247  error( "connectDataIO> failed to resolve FID:" + dsn, false ).ignore();
249  } else if ( dsn.length() == 36 && dsn[8] == '-' && dsn[13] == '-' ) {
250  std::string gfal_name = "gfal:guid:" + dsn;
251  m_fidMap[dsn] = m_fidMap[dataset] = m_fidMap[gfal_name] = dsn;
252  sc = connectDataIO( PFN, rw, gfal_name, technology, keep_open, connection );
253  if ( sc.isSuccess() ) return sc;
254  if ( m_quarantine ) s_badFiles.insert( dsn );
255  }
256  if ( m_quarantine ) s_badFiles.insert( dsn );
257  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
258  error( "connectDataIO> Failed to resolve FID:" + dsn, false ).ignore();
260  }
261  // keep track of the current return code before we start iterating over
262  // replicas
263  auto appmgr = serviceLocator()->as<IProperty>();
264  int origReturnCode = Gaudi::getAppReturnCode( appmgr );
265  for ( auto i = files.cbegin(); i != files.cend(); ++i ) {
266  std::string pfn = i->first;
267  if ( i != files.cbegin() ) {
268  log << MSG::WARNING << "Attempt to connect dsn:" << dsn << " with next entry in data federation:" << pfn
269  << "." << endmsg;
270  }
271  sc = connectDataIO( PFN, rw, pfn, technology, keep_open, connection );
272  if ( !sc.isSuccess() ) {
273  if ( m_quarantine ) s_badFiles.insert( pfn );
274  m_incSvc->fireIncident( Incident( pfn, IncidentType::FailInputFile ) );
275  } else {
276  m_fidMap[dsn] = m_fidMap[dataset] = m_fidMap[pfn] = dsn;
277  // we found a working replica, let's reset the return code to the old value
278  Gaudi::setAppReturnCode( appmgr, origReturnCode, true ).ignore();
279  return sc;
280  }
281  }
282  log << MSG::ERROR << "Failed to open dsn:" << dsn << " Federated file could not be resolved from "
283  << files.size() << " entries." << endmsg;
285  }
286  return StatusCode::FAILURE;
287  }
288  std::string fid;
289  auto j = m_fidMap.find( dsn );
290  if ( j == m_fidMap.end() ) {
291  IFileCatalog::Files files;
292  switch ( typ ) {
293  case LFN:
294  fid = m_catalog->lookupLFN( dsn );
295  if ( fid.empty() ) {
296  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
297  log << MSG::ERROR << "Failed to resolve LFN:" << dsn << " Cannot access this dataset." << endmsg;
299  }
300  break;
301  case PFN:
302  fid = m_catalog->lookupPFN( dsn );
303  if ( !fid.empty() ) m_catalog->getPFN( fid, files );
304  if ( files.empty() ) {
305  if ( rw == Connection::CREATE || rw == Connection::RECREATE ) {
306  if ( fid.empty() ) fid = m_catalog->createFID();
307  m_catalog->registerPFN( fid, dsn, technology );
308  log << MSG::INFO << "Referring to dataset " << dsn << " by its file ID:" << fid << endmsg;
309  } else {
310  fid = dsn;
311  }
312  }
313  break;
314  }
315  } else {
316  fid = j->second;
317  }
318  if ( typ == PFN ) {
319  // Open PFN
320  auto fi = m_connectionMap.find( fid );
321  if ( fi == m_connectionMap.end() ) {
322  connection->setFID( fid );
323  connection->setPFN( dsn );
324  auto e = new Entry( technology, keep_open, rw, connection );
325  // Here we open the file!
326  if ( !reconnect( e ).isSuccess() ) {
327  delete e;
328  if ( m_quarantine ) s_badFiles.insert( dsn );
329  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
330  error( "connectDataIO> Cannot connect to database: PFN=" + dsn + " FID=" + fid, false ).ignore();
332  }
333  fid = connection->fid();
334  m_fidMap[dataset] = m_fidMap[dsn] = m_fidMap[fid] = fid;
335  if ( !( rw == Connection::CREATE || rw == Connection::RECREATE ) ) {
336  if ( !m_disablePFNWarning && strcasecmp( dsn.c_str(), fid.c_str() ) == 0 ) {
337  log << MSG::ERROR << "Referring to existing dataset " << dsn << " by its physical name." << endmsg;
338  log << "You may not be able to navigate back to the input file"
339  << " -- processing continues" << endmsg;
340  }
341  }
342  m_connectionMap.emplace( fid, e ); // note: only if we disconnect does e get deleted??
343  return StatusCode::SUCCESS;
344  }
345  // Here we open the file!
346  if ( !reconnect( ( *fi ).second ).isSuccess() ) {
347  if ( m_quarantine ) s_badFiles.insert( dsn );
348  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
349  error( "connectDataIO> Cannot connect to database: PFN=" + dsn + " FID=" + fid, false ).ignore();
351  }
352  return StatusCode::SUCCESS;
353  }
354  sc = connectDataIO( FID, rw, fid, technology, keep_open, connection );
355  if ( !sc.isSuccess() && m_quarantine ) {
356  s_badFiles.insert( fid );
357  } else if ( typ == LFN ) {
358  m_fidMap[dataset] = fid;
359  }
360  return sc;
361  } catch ( std::exception& e ) {
362  error( std::string( "connectDataIO> Caught exception:" ) + e.what(), false ).ignore();
363  } catch ( ... ) { error( std::string( "connectDataIO> Caught unknown exception" ), false ).ignore(); }
364  m_incSvc->fireIncident( Incident( dsn, IncidentType::FailInputFile ) );
365  error( "connectDataIO> The dataset " + dsn + " cannot be opened.", false ).ignore();
366  s_badFiles.insert( dsn );
368 }
int getAppReturnCode(const SmartIF< IProperty > &appmgr)
Get the application (current) return code.
Definition: AppReturnCode.h:69
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
SmartIF< ISvcLocator > & serviceLocator() const override
Retrieve pointer to service locator.
Definition: Service.cpp:277
T empty(T... args)
const SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:64
constexpr static const auto SUCCESS
Definition: StatusCode.h:85
T end(T... args)
SmartIF< IFace > as()
Definition: ISvcLocator.h:103
STL class.
Connection * connection(const std::string &dsn) const override
Retrieve known connection.
Gaudi::Property< bool > m_quarantine
Definition: IODataManager.h:46
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:274
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
virtual void fireIncident(const Incident &incident)=0
Fire an Incident.
T what(T... args)
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:50
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:56
Gaudi::Property< bool > m_useGFAL
Definition: IODataManager.h:45
bool isSuccess() const
Definition: StatusCode.h:267
STL class.
StatusCode setAppReturnCode(SmartIF< IProperty > &appmgr, int value, bool force=false)
Set the application return code.
Definition: AppReturnCode.h:49
T insert(T... args)
T find(T... args)
T length(T... args)
const StatusCode & ignore() const
Ignore/check StatusCode.
Definition: StatusCode.h:153
STL class.
T begin(T... args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54
T c_str(T... args)
Base class for all Incidents (computing events).
Definition: Incident.h:17
Gaudi::Property< bool > m_disablePFNWarning
Definition: IODataManager.h:49
T emplace(T... args)
constexpr static const auto FAILURE
Definition: StatusCode.h:86
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
T substr(T... args)
The IProperty is the basic interface for all components which have properties that can be set or get.
Definition: IProperty.h:20
StatusCode reconnect(Entry *e)
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:192
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:58

◆ connection()

IIODataManager::Connection * IODataManager::connection ( const std::string dsn) const
override

Retrieve known connection.

Definition at line 193 of file IODataManager.cpp.

193  {
194  auto j = m_fidMap.find( dataset );
195  if ( j == m_fidMap.end() ) return nullptr;
196  auto i = m_connectionMap.find( j->second );
197  return ( i != m_connectionMap.end() ) ? i->second->connection : nullptr;
198 }
T end(T... args)
T find(T... args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54
IDataConnection * connection
Definition: IODataManager.h:35
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:58

◆ connections()

IODataManager::Connections IODataManager::connections ( const IInterface owner) const
override

Get connection by owner instance (0=ALL)

Definition at line 79 of file IODataManager.cpp.

79  {
80  Connections conns;
81  transform_copy_if(
83  []( ConnectionMap::const_reference i ) { return i.second->connection; },
84  [&]( const IDataConnection* c ) { return !owner || c->owner() == owner; } );
85  return conns;
86 }
T end(T... args)
ABC describing basic data connection.
T begin(T... args)
T back_inserter(T... args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54

◆ connectRead()

StatusCode IODataManager::connectRead ( bool  keep_open,
Connection *  ioDesc 
)
override

Open data stream in read mode.

Connect data file for writing.

Definition at line 89 of file IODataManager.cpp.

89  {
90  if ( !establishConnection( con ) ) {
91  return connectDataIO( UNKNOWN, Connection::READ, con->name(), "UNKNOWN", keep_open, con );
92  }
93  std::string dsn = con ? con->name() : std::string( "Unknown" );
94  return error( "Failed to connect to data:" + dsn, false );
95 }
StatusCode establishConnection(Connection *con)
STL class.
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)

◆ connectWrite()

StatusCode IODataManager::connectWrite ( Connection *  con,
IoType  mode = Connection::CREATE,
CSTR  doctype = "UNKNOWN" 
)
override

Open data stream in write mode.

Connect data file for reading.

Definition at line 98 of file IODataManager.cpp.

98  {
99  if ( !establishConnection( con ) ) { return connectDataIO( UNKNOWN, mode, con->name(), doctype, true, con ); }
100  std::string dsn = con ? con->name() : std::string( "Unknown" );
101  return error( "Failed to connect to data:" + dsn, false );
102 }
StatusCode establishConnection(Connection *con)
STL class.
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)

◆ disconnect()

StatusCode IODataManager::disconnect ( Connection *  ioDesc)
override

Release data stream.

Definition at line 119 of file IODataManager.cpp.

119  {
120  if ( con ) {
121  std::string dataset = con->name();
122  std::string dsn = dataset;
123  StatusCode sc = con->disconnect();
124  if ( ::strncasecmp( dsn.c_str(), "FID:", 4 ) == 0 )
125  dsn = dataset.substr( 4 );
126  else if ( ::strncasecmp( dsn.c_str(), "LFN:", 4 ) == 0 )
127  dsn = dataset.substr( 4 );
128  else if ( ::strncasecmp( dsn.c_str(), "PFN:", 4 ) == 0 )
129  dsn = dataset.substr( 4 );
130 
131  auto j = m_fidMap.find( dataset );
132  if ( j != m_fidMap.end() ) {
133  std::string fid = j->second;
134  std::string gfal_name = "gfal:guid:" + fid;
135  auto i = m_connectionMap.find( fid );
136  m_fidMap.erase( j );
137  if ( ( j = m_fidMap.find( fid ) ) != m_fidMap.end() ) m_fidMap.erase( j );
138  if ( ( j = m_fidMap.find( gfal_name ) ) != m_fidMap.end() ) m_fidMap.erase( j );
139  if ( i != m_connectionMap.end() && i->second ) {
140  IDataConnection* c = i->second->connection;
141  if ( ( j = m_fidMap.find( c->pfn() ) ) != m_fidMap.end() ) m_fidMap.erase( j );
142  if ( c->isConnected() ) {
143  MsgStream log( msgSvc(), name() );
144  c->disconnect();
145  log << MSG::INFO << "Disconnect from dataset " << dsn << " [" << fid << "]" << endmsg;
146  }
147  delete i->second;
148  m_connectionMap.erase( i );
149  }
150  }
151  return sc;
152  }
153  return StatusCode::FAILURE;
154 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
const SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
T end(T... args)
STL class.
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:274
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:50
T erase(T... args)
T find(T... args)
ABC describing basic data connection.
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54
T c_str(T... args)
constexpr static const auto FAILURE
Definition: StatusCode.h:86
T substr(T... args)
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:192
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:58

◆ error()

StatusCode IODataManager::error ( CSTR  msg,
bool  rethrow 
)
protected

Small routine to issue exceptions.

Definition at line 71 of file IODataManager.cpp.

71  {
72  MsgStream log( msgSvc(), name() );
73  log << MSG::ERROR << "Error: " << msg << endmsg;
74  if ( rethrow ) System::breakExecution();
75  return StatusCode::FAILURE;
76 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
const SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
GAUDI_API long breakExecution()
Break the execution of the application and invoke the debugger.
Definition: Debugger.cpp:47
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:274
MsgStream & msg() const
shortcut for the method msgStream(MSG::INFO)
constexpr static const auto FAILURE
Definition: StatusCode.h:86
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:192

◆ establishConnection()

StatusCode IODataManager::establishConnection ( Connection *  con)
protected

Definition at line 200 of file IODataManager.cpp.

200  {
201  if ( !con ) return error( "Severe logic bug: No connection object avalible.", true );
202 
203  if ( con->isConnected() ) {
204  con->resetAge();
205  return StatusCode::SUCCESS;
206  }
207  auto i = m_connectionMap.find( con->name() );
208  if ( i != m_connectionMap.end() ) {
209  Connection* c = i->second->connection;
210  if ( c != con ) {
211  m_incSvc->fireIncident( Incident( con->name(), IncidentType::FailInputFile ) );
212  return error( "Severe logic bug: Twice identical connection object for DSN:" + con->name(), true );
213  }
214  if ( reconnect( i->second ).isSuccess() ) return StatusCode::SUCCESS;
215  }
216  return StatusCode::FAILURE;
217 }
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:64
constexpr static const auto SUCCESS
Definition: StatusCode.h:85
T end(T... args)
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
virtual void fireIncident(const Incident &incident)=0
Fire an Incident.
bool isSuccess() const
Definition: StatusCode.h:267
T find(T... args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54
Base class for all Incidents (computing events).
Definition: Incident.h:17
constexpr static const auto FAILURE
Definition: StatusCode.h:86
StatusCode reconnect(Entry *e)

◆ finalize()

StatusCode IODataManager::finalize ( )
override

IService implementation: finalize the service.

Definition at line 65 of file IODataManager.cpp.

65  {
66  m_catalog = nullptr; // release
67  return Service::finalize();
68 }
StatusCode finalize() override
Definition: Service.cpp:164
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:56

◆ initialize()

StatusCode IODataManager::initialize ( )
override

IService implementation: initialize the service.

IService implementation: Db event selector override.

Definition at line 42 of file IODataManager.cpp.

42  {
43  // Initialize base class
45  MsgStream log( msgSvc(), name() );
46  if ( !status.isSuccess() ) {
47  log << MSG::ERROR << "Error initializing base class Service!" << endmsg;
48  return status;
49  }
50  // Retrieve conversion service handling event iteration
52  if ( !m_catalog ) {
53  log << MSG::ERROR << "Unable to localize interface IFileCatalog from service:" << m_catalogSvcName << endmsg;
54  return StatusCode::FAILURE;
55  }
56  m_incSvc = serviceLocator()->service( "IncidentSvc" );
57  if ( !m_incSvc ) {
58  log << MSG::ERROR << "Error initializing IncidentSvc Service!" << endmsg;
59  return status;
60  }
61  return status;
62 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
StatusCode initialize() override
Definition: Service.cpp:60
SmartIF< ISvcLocator > & serviceLocator() const override
Retrieve pointer to service locator.
Definition: Service.cpp:277
const SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
Gaudi::Property< std::string > m_catalogSvcName
Definition: IODataManager.h:43
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:64
StatusCode service(const Gaudi::Utils::TypeNameString &name, T *&svc, bool createIf=true)
Templated method to access a service by name.
Definition: ISvcLocator.h:76
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:274
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:50
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:56
bool isSuccess() const
Definition: StatusCode.h:267
constexpr static const auto FAILURE
Definition: StatusCode.h:86
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:192

◆ read()

StatusCode IODataManager::read ( Connection *  ioDesc,
void *const  data,
size_t  len 
)
override

Read raw byte buffer from input stream.

Definition at line 105 of file IODataManager.cpp.

105  {
106  return establishConnection( con ).isSuccess() ? con->read( data, len ) : StatusCode::FAILURE;
107 }
StatusCode establishConnection(Connection *con)
bool isSuccess() const
Definition: StatusCode.h:267
constexpr static const auto FAILURE
Definition: StatusCode.h:86

◆ reconnect()

StatusCode IODataManager::reconnect ( Entry e)
protected

Definition at line 156 of file IODataManager.cpp.

156  {
158  if ( e && e->connection ) {
159  switch ( e->ioType ) {
160  case Connection::READ:
161  sc = e->connection->connectRead();
162  break;
163  case Connection::UPDATE:
164  case Connection::CREATE:
165  case Connection::RECREATE:
166  sc = e->connection->connectWrite( e->ioType );
167  break;
168  default:
169  return StatusCode::FAILURE;
170  }
171  if ( sc.isSuccess() && e->ioType == Connection::READ ) {
172  std::vector<Entry*> to_retire;
173  e->connection->resetAge();
174  transform_copy_if( std::begin( m_connectionMap ), std::end( m_connectionMap ), std::back_inserter( to_retire ),
175  select2nd, [&]( Entry* i ) {
176  IDataConnection* c = i->connection;
177  return e->connection != c && c->isConnected() && !i->keepOpen && c->ageFile() > m_ageLimit;
178  } );
179  if ( !to_retire.empty() ) {
180  MsgStream log( msgSvc(), name() );
181  std::for_each( std::begin( to_retire ), std::end( to_retire ), [&]( Entry* j ) {
182  IDataConnection* c = j->connection;
183  c->disconnect();
184  log << MSG::INFO << "Disconnect from dataset " << c->pfn() << " [" << c->fid() << "]" << endmsg;
185  } );
186  }
187  }
188  }
189  return sc;
190 }
Gaudi::Property< int > m_ageLimit
Definition: IODataManager.h:48
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
T empty(T... args)
const SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
T end(T... args)
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:274
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:50
bool isSuccess() const
Definition: StatusCode.h:267
STL class.
ABC describing basic data connection.
T begin(T... args)
T back_inserter(T... args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:54
constexpr static const auto FAILURE
Definition: StatusCode.h:86
T for_each(T... args)
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:192

◆ seek()

long long int IODataManager::seek ( Connection *  ioDesc,
long long int  where,
int  origin 
)
override

Seek on the file described by ioDesc. Arguments as in ::seek()

Definition at line 115 of file IODataManager.cpp.

115  {
116  return establishConnection( con ).isSuccess() ? con->seek( where, origin ) : -1;
117 }
StatusCode establishConnection(Connection *con)
bool isSuccess() const
Definition: StatusCode.h:267

◆ write()

StatusCode IODataManager::write ( Connection *  con,
const void *  data,
int  len 
)
override

Write raw byte buffer to output stream.

Definition at line 110 of file IODataManager.cpp.

110  {
111  return establishConnection( con ).isSuccess() ? con->write( data, len ) : StatusCode::FAILURE;
112 }
StatusCode establishConnection(Connection *con)
bool isSuccess() const
Definition: StatusCode.h:267
constexpr static const auto FAILURE
Definition: StatusCode.h:86

Member Data Documentation

◆ m_ageLimit

Gaudi::Property<int> Gaudi::IODataManager::m_ageLimit {this, "AgeLimit", 2, "age limit"}
protected

Definition at line 48 of file IODataManager.h.

◆ m_catalog

SmartIF<IFileCatalog> Gaudi::IODataManager::m_catalog
protected

Reference to file catalog.

Definition at line 56 of file IODataManager.h.

◆ m_catalogSvcName

Gaudi::Property<std::string> Gaudi::IODataManager::m_catalogSvcName
protected
Initial value:
{this, "CatalogType", "Gaudi::MultiFileCatalog/FileCatalog",
"name of the file catalog service"}

Definition at line 43 of file IODataManager.h.

◆ m_connectionMap

ConnectionMap Gaudi::IODataManager::m_connectionMap
protected

Map with I/O descriptors.

Definition at line 54 of file IODataManager.h.

◆ m_disablePFNWarning

Gaudi::Property<bool> Gaudi::IODataManager::m_disablePFNWarning
protected
Initial value:
{
this, "DisablePFNWarning", false,
"if set to True, we will not report when a file is opened by its physical name"}

Definition at line 49 of file IODataManager.h.

◆ m_fidMap

FidMap Gaudi::IODataManager::m_fidMap
protected

Map of FID to PFN.

Definition at line 58 of file IODataManager.h.

◆ m_incSvc

SmartIF<IIncidentSvc> Gaudi::IODataManager::m_incSvc
protected

Definition at line 64 of file IODataManager.h.

◆ m_quarantine

Gaudi::Property<bool> Gaudi::IODataManager::m_quarantine
protected
Initial value:
{this, "QuarantineFiles", true,
"if unaccessible files should be quarantines in job"}

Definition at line 46 of file IODataManager.h.

◆ m_useGFAL

Gaudi::Property<bool> Gaudi::IODataManager::m_useGFAL {this, "UseGFAL", true, "flag for auto gfal data access"}
protected

Definition at line 45 of file IODataManager.h.


The documentation for this class was generated from the following files: