Gaudi::IODataManager Class Reference

#include <src/component/IODataManager.h>

Inheritance diagram for Gaudi::IODataManager:
Collaboration diagram for Gaudi::IODataManager:

Classes

struct  Entry
 

Public Member Functions

 IODataManager (CSTR nam, ISvcLocator *loc)
 the incident service More...
 
 ~IODataManager () override=default
 Standard destructor. More...
 
StatusCode initialize () override
 IService implementation: initialize the service. More...
 
StatusCode finalize () override
 IService implementation: finalize the service. More...
 
StatusCode connectRead (bool keep_open, Connection *ioDesc) override
 Open data stream in read mode. More...
 
StatusCode connectWrite (Connection *con, IoType mode=Connection::CREATE, CSTR doctype="UNKNOWN") override
 Open data stream in write mode. More...
 
StatusCode disconnect (Connection *ioDesc) override
 Release data stream. More...
 
Connection * connection (const std::string &dsn) const override
 Retrieve known connection. More...
 
Connections connections (const IInterface *owner) const override
 Get connection by owner instance (0=ALL) More...
 
StatusCode read (Connection *ioDesc, void *const data, size_t len) override
 Read raw byte buffer from input stream. More...
 
StatusCode write (Connection *con, const void *data, int len) override
 Write raw byte buffer to output stream. More...
 
long long int seek (Connection *ioDesc, long long int where, int origin) override
 Seek on the file described by ioDesc. Arguments as in ::seek() More...
 
- Public Member Functions inherited from extends< Service, IIODataManager >
void * i_cast (const InterfaceID &tid) const override
 Implementation of IInterface::i_cast. More...
 
StatusCode queryInterface (const InterfaceID &ti, void **pp) override
 Implementation of IInterface::queryInterface. More...
 
std::vector< std::stringgetInterfaceNames () const override
 Implementation of IInterface::getInterfaceNames. More...
 
 ~extends () override=default
 Virtual destructor. More...
 
- Public Member Functions inherited from Service
const std::stringname () const override
 Retrieve name of the service. More...
 
StatusCode configure () override
 
StatusCode initialize () override
 
StatusCode start () override
 
StatusCode stop () override
 
StatusCode finalize () override
 
StatusCode terminate () override
 
Gaudi::StateMachine::State FSMState () const override
 
Gaudi::StateMachine::State targetFSMState () const override
 
StatusCode reinitialize () override
 
StatusCode restart () override
 
StatusCode sysInitialize () override
 Initialize Service. More...
 
StatusCode sysStart () override
 Initialize Service. More...
 
StatusCode sysStop () override
 Initialize Service. More...
 
StatusCode sysFinalize () override
 Finalize Service. More...
 
StatusCode sysReinitialize () override
 Re-initialize the Service. More...
 
StatusCode sysRestart () override
 Re-initialize the Service. More...
 
StatusCode setProperty (const Property &p) override
 
StatusCode setProperty (const std::string &s) override
 
StatusCode setProperty (const std::string &n, const std::string &v) override
 
StatusCode getProperty (Property *p) const override
 
const PropertygetProperty (const std::string &name) const override
 
StatusCode getProperty (const std::string &n, std::string &v) const override
 
const std::vector< Property * > & getProperties () const override
 
bool hasProperty (const std::string &name) const override
 
template<class TYPE >
StatusCode setProperty (const std::string &name, const TYPE &value)
 set the property form the value More...
 
 Service (std::string name, ISvcLocator *svcloc)
 Standard Constructor. More...
 
SmartIF< ISvcLocator > & serviceLocator () const override
 Retrieve pointer to service locator. More...
 
StatusCode setProperties ()
 Method for setting declared properties to the values specified for the job. More...
 
template<class T >
StatusCode service (const std::string &name, const T *&psvc, bool createIf=true) const
 Access a service by name, creating it if it doesn't already exist. More...
 
template<class T >
StatusCode service (const std::string &name, T *&psvc, bool createIf=true) const
 
template<typename IFace = IService>
SmartIF< IFace > service (const std::string &name, bool createIf=true) const
 
template<class T >
StatusCode service (const std::string &svcType, const std::string &svcName, T *&psvc) const
 Access a service by name and type, creating it if it doesn't already exist. More...
 
template<class T >
PropertydeclareProperty (const std::string &name, T &property, const std::string &doc="none") const
 Declare the named property. More...
 
PropertydeclareRemoteProperty (const std::string &name, IProperty *rsvc, const std::string &rname="") const
 Declare remote named properties. More...
 
template<class T >
StatusCode declarePrivateTool (ToolHandle< T > &handle, std::string toolTypeAndName="", bool createIf=true)
 Declare used Private tool. More...
 
template<class T >
StatusCode declarePublicTool (ToolHandle< T > &handle, std::string toolTypeAndName="", bool createIf=true)
 Declare used Public tool. More...
 
SmartIF< IAuditorSvc > & auditorSvc () const
 The standard auditor service.May not be invoked before sysInitialize() has been invoked. More...
 
- Public Member Functions inherited from CommonMessagingBase
virtual ~CommonMessagingBase ()=default
 Virtual destructor. More...
 
SmartIF< IMessageSvc > & msgSvc () const
 The standard message service. More...
 
MsgStreammsgStream () const
 Return an uninitialized MsgStream. More...
 
MsgStreammsgStream (const MSG::Level level) const
 Predefined configurable message stream for the efficient printouts. More...
 
MsgStreamalways () const
 shortcut for the method msgStream(MSG::ALWAYS) More...
 
MsgStreamfatal () const
 shortcut for the method msgStream(MSG::FATAL) More...
 
MsgStreamerr () const
 shortcut for the method msgStream(MSG::ERROR) More...
 
MsgStreamerror () const
 shortcut for the method msgStream(MSG::ERROR) More...
 
MsgStreamwarning () const
 shortcut for the method msgStream(MSG::WARNING) More...
 
MsgStreaminfo () const
 shortcut for the method msgStream(MSG::INFO) More...
 
MsgStreamdebug () const
 shortcut for the method msgStream(MSG::DEBUG) More...
 
MsgStreamverbose () const
 shortcut for the method msgStream(MSG::VERBOSE) More...
 
MsgStreammsg () const
 shortcut for the method msgStream(MSG::INFO) More...
 
MSG::Level msgLevel () const
 get the output level from the embedded MsgStream More...
 
MSG::Level outputLevel () const __attribute__((deprecated))
 Backward compatibility function for getting the output level. More...
 
bool msgLevel (MSG::Level lvl) const
 get the output level from the embedded MsgStream More...
 
- Public Member Functions inherited from extend_interfaces< Interfaces...>
 ~extend_interfaces () override=default
 Virtual destructor. More...
 

Protected Types

typedef const std::stringCSTR
 
typedef std::map< std::string, Entry * > ConnectionMap
 
typedef std::map< std::string, std::stringFidMap
 

Protected Member Functions

StatusCode connectDataIO (int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
 
StatusCode reconnect (Entry *e)
 
StatusCode error (CSTR msg, bool rethrow)
 Small routine to issue exceptions. More...
 
StatusCode establishConnection (Connection *con)
 
- Protected Member Functions inherited from Service
 ~Service () override
 Standard Destructor. More...
 
int outputLevel () const
 get the Service's output level More...
 
- Protected Member Functions inherited from CommonMessaging< implements< IService, IProperty, IStateful > >
void updateMsgStreamOutputLevel (int level)
 Update the output level of the cached MsgStream. More...
 

Protected Attributes

std::string m_catalogSvcName
 Property: Name of the file catalog service. More...
 
int m_ageLimit
 Property: Age limit. More...
 
bool m_useGFAL
 Property: Flag for auto gfal data access. More...
 
bool m_quarantine
 Property: Flag if unaccessible files should be quarantines in job. More...
 
bool m_disablePFNWarning
 Property DisablePFNWarning: if set to True will not report when a file is opened by it's physical name. More...
 
ConnectionMap m_connectionMap
 Map with I/O descriptors. More...
 
SmartIF< IFileCatalogm_catalog
 Reference to file catalog. More...
 
FidMap m_fidMap
 Map of FID to PFN. More...
 
SmartIF< IIncidentSvcm_incSvc
 
- Protected Attributes inherited from Service
IntegerProperty m_outputLevel = MSG::NIL
 Service output level. More...
 
Gaudi::StateMachine::State m_state = Gaudi::StateMachine::OFFLINE
 Service state. More...
 
Gaudi::StateMachine::State m_targetState = Gaudi::StateMachine::OFFLINE
 Service state. More...
 

Additional Inherited Members

- Public Types inherited from extends< Service, IIODataManager >
using base_class = extends
 Typedef to this class. More...
 
using extend_interfaces_base = extend_interfaces< Interfaces...>
 Typedef to the base of this class. More...
 
- Public Types inherited from Service
typedef Gaudi::PluginService::Factory< IService *, const std::string &, ISvcLocator * > Factory
 
- Public Types inherited from CommonMessaging< implements< IService, IProperty, IStateful > >
using base_class = CommonMessaging
 
- Public Types inherited from extend_interfaces< Interfaces...>
using ext_iids = typename Gaudi::interface_list_cat< typename Interfaces::ext_iids...>::type
 take union of the ext_iids of all Interfaces... More...
 

Detailed Description

Definition at line 29 of file IODataManager.h.

Member Typedef Documentation

typedef std::map<std::string,Entry*> Gaudi::IODataManager::ConnectionMap
protected

Definition at line 42 of file IODataManager.h.

typedef const std::string& Gaudi::IODataManager::CSTR
protected

Definition at line 32 of file IODataManager.h.

typedef std::map<std::string, std::string> Gaudi::IODataManager::FidMap
protected

Definition at line 43 of file IODataManager.h.

Constructor & Destructor Documentation

IODataManager::IODataManager ( CSTR  nam,
ISvcLocator loc 
)

the incident service

Initializing constructor

Parameters
[in]namName of the service
[in]locPointer to the service locator object
Returns
Initialized reference to service object

Definition at line 44 of file IODataManager.cpp.

45  : base_class(nam, svcloc)
46 {
47  declareProperty("CatalogType", m_catalogSvcName="Gaudi::MultiFileCatalog/FileCatalog");
48  declareProperty("UseGFAL", m_useGFAL = true);
49  declareProperty("QuarantineFiles", m_quarantine = true);
50  declareProperty("AgeLimit", m_ageLimit = 2);
51  declareProperty("DisablePFNWarning", m_disablePFNWarning = false,
52  "if set to True, we will not report when a file "
53  "is opened by its physical name");
54 }
int m_ageLimit
Property: Age limit.
Definition: IODataManager.h:48
bool m_useGFAL
Property: Flag for auto gfal data access.
Definition: IODataManager.h:50
extends base_class
Typedef to this class.
Definition: extends.h:14
std::string m_catalogSvcName
Property: Name of the file catalog service.
Definition: IODataManager.h:46
bool m_disablePFNWarning
Property DisablePFNWarning: if set to True will not report when a file is opened by it's physical nam...
Definition: IODataManager.h:55
bool m_quarantine
Property: Flag if unaccessible files should be quarantines in job.
Definition: IODataManager.h:52
Property * declareProperty(const std::string &name, T &property, const std::string &doc="none") const
Declare the named property.
Definition: Service.h:215
Gaudi::IODataManager::~IODataManager ( )
overridedefault

Standard destructor.

Member Function Documentation

StatusCode IODataManager::connectDataIO ( int  typ,
IoType  rw,
CSTR  fn,
CSTR  technology,
bool  keep,
Connection *  con 
)
protected

Definition at line 248 of file IODataManager.cpp.

248  {
249  MsgStream log(msgSvc(),name());
250  std::string dsn = dataset;
251  try {
253  if ( ::strncasecmp(dsn.c_str(),"FID:",4)==0 )
254  dsn = dataset.substr(4), typ = FID;
255  else if ( ::strncasecmp(dsn.c_str(),"LFN:",4)==0 )
256  dsn = dataset.substr(4), typ = LFN;
257  else if ( ::strncasecmp(dsn.c_str(),"PFN:",4)==0 )
258  dsn = dataset.substr(4), typ = PFN;
259  else if ( typ == UNKNOWN )
260  return connectDataIO(PFN, rw, dsn, technology, keep_open, connection);
261 
262  if(std::find(s_badFiles.begin(),s_badFiles.end(),dsn) != s_badFiles.end()) {
263  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
265  }
266  if ( typ == FID ) {
267  auto fi = m_connectionMap.find(dsn);
268  if ( fi == m_connectionMap.end() ) {
269  IFileCatalog::Files files;
270  m_catalog->getPFN(dsn,files);
271  if ( files.empty() ) {
272  if ( !m_useGFAL ) {
273  if ( m_quarantine ) s_badFiles.insert(dsn);
274  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
275  error("connectDataIO> failed to resolve FID:"+dsn,false).ignore();
277  }
278  else if ( dsn.length() == 36 && dsn[8]=='-' && dsn[13]=='-' ) {
279  std::string gfal_name = "gfal:guid:" + dsn;
280  m_fidMap[dsn] = m_fidMap[dataset] = m_fidMap[gfal_name] = dsn;
281  sc = connectDataIO(PFN, rw, gfal_name, technology, keep_open, connection);
282  if ( sc.isSuccess() ) return sc;
283  if ( m_quarantine ) s_badFiles.insert(dsn);
284  }
285  if ( m_quarantine ) s_badFiles.insert(dsn);
286  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
287  error("connectDataIO> Failed to resolve FID:"+dsn,false).ignore();
289  }
290  // keep track of the current return code before we start iterating over
291  // replicas
292  auto appmgr = serviceLocator()->as<IProperty>();
293  int origReturnCode = Gaudi::getAppReturnCode(appmgr);
294  for(auto i=files.cbegin(); i!=files.cend(); ++i) {
295  std::string pfn = i->first;
296  if ( i != files.cbegin() ) {
297  log << MSG::WARNING << "Attempt to connect dsn:" << dsn
298  << " with next entry in data federation:" << pfn << "." << endmsg;
299  }
300  sc = connectDataIO(PFN, rw, pfn, technology, keep_open, connection);
301  if ( !sc.isSuccess() ) {
302  if ( m_quarantine ) s_badFiles.insert(pfn);
303  m_incSvc->fireIncident(Incident(pfn,IncidentType::FailInputFile));
304  }
305  else {
306  m_fidMap[dsn] = m_fidMap[dataset] = m_fidMap[pfn] = dsn;
307  // we found a working replica, let's reset the return code to the old value
308  Gaudi::setAppReturnCode(appmgr, origReturnCode, true).ignore();
309  return sc;
310  }
311  }
312  log << MSG::ERROR << "Failed to open dsn:" << dsn
313  << " Federated file could not be resolved from "
314  << files.size() << " entries." << endmsg;
316  }
317  return S_ERROR;
318  }
319  std::string fid;
320  auto j = m_fidMap.find(dsn);
321  if ( j == m_fidMap.end() ) {
322  IFileCatalog::Files files;
323  switch(typ) {
324  case LFN:
325  fid = m_catalog->lookupLFN(dsn);
326  if ( fid.empty() ) {
327  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
328  log << MSG::ERROR << "Failed to resolve LFN:" << dsn
329  << " Cannot access this dataset." << endmsg;
331  }
332  break;
333  case PFN:
334  fid = m_catalog->lookupPFN(dsn);
335  if ( !fid.empty() ) m_catalog->getPFN(fid, files);
336  if ( files.empty() ) {
337  if ( rw == Connection::CREATE || rw == Connection::RECREATE ) {
338  if ( fid.empty() ) fid = m_catalog->createFID();
339  m_catalog->registerPFN(fid,dsn,technology);
340  log << MSG::INFO << "Referring to dataset " << dsn
341  << " by its file ID:" << fid << endmsg;
342  }
343  else {
344  fid = dsn;
345  }
346  }
347  break;
348  }
349  }
350  else {
351  fid = j->second;
352  }
353  if ( typ == PFN ) {
354  // Open PFN
355  auto fi = m_connectionMap.find(fid);
356  if ( fi == m_connectionMap.end() ) {
357  connection->setFID(fid);
358  connection->setPFN(dsn);
359  auto e = new Entry(technology, keep_open, rw, connection);
360  // Here we open the file!
361  if ( !reconnect(e).isSuccess() ) {
362  delete e;
363  if ( m_quarantine ) s_badFiles.insert(dsn);
364  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
365  error("connectDataIO> Cannot connect to database: PFN="+dsn+" FID="+fid,false).ignore();
367  }
368  fid = connection->fid();
369  m_fidMap[dataset] = m_fidMap[dsn] = m_fidMap[fid] = fid;
370  if ( !(rw==Connection::CREATE || rw==Connection::RECREATE) ) {
371  if ( ! m_disablePFNWarning && strcasecmp(dsn.c_str(),fid.c_str()) == 0 ) {
372  log << MSG::ERROR << "Referring to existing dataset " << dsn
373  << " by its physical name." << endmsg;
374  log << "You may not be able to navigate back to the input file"
375  << " -- processing continues" << endmsg;
376  }
377  }
378  m_connectionMap.emplace( fid, e ); // note: only if we disconnect does e get deleted??
379  return S_OK;
380  }
381  // Here we open the file!
382  if ( !reconnect((*fi).second).isSuccess() ) {
383  if ( m_quarantine ) s_badFiles.insert(dsn);
384  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
385  error("connectDataIO> Cannot connect to database: PFN="+dsn+" FID="+fid,false).ignore();
387  }
388  return S_OK;
389  }
390  sc = connectDataIO(FID, rw, fid, technology, keep_open, connection);
391  if ( !sc.isSuccess() && m_quarantine ) {
392  s_badFiles.insert(fid);
393  }
394  else if ( typ == LFN ) {
395  m_fidMap[dataset] = fid;
396  }
397  return sc;
398  }
399  catch (std::exception& e) {
400  error(std::string("connectDataIO> Caught exception:")+e.what(), false).ignore();
401  }
402  catch(...) {
403  error(std::string("connectDataIO> Caught unknown exception"), false).ignore();
404  }
405  m_incSvc->fireIncident(Incident(dsn,IncidentType::FailInputFile));
406  error("connectDataIO> The dataset "+dsn+" cannot be opened.",false).ignore();
407  s_badFiles.insert(dsn);
409 }
int getAppReturnCode(const SmartIF< IProperty > &appmgr)
Get the application (current) return code.
Definition: AppReturnCode.h:70
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
SmartIF< ISvcLocator > & serviceLocator() const override
Retrieve pointer to service locator.
Definition: Service.cpp:324
T empty(T...args)
bool m_useGFAL
Property: Flag for auto gfal data access.
Definition: IODataManager.h:50
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:68
T end(T...args)
SmartIF< IFace > as()
Definition: ISvcLocator.h:106
bool m_disablePFNWarning
Property DisablePFNWarning: if set to True will not report when a file is opened by it's physical nam...
Definition: IODataManager.h:55
STL class.
Connection * connection(const std::string &dsn) const override
Retrieve known connection.
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:319
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
virtual void fireIncident(const Incident &incident)=0
Fire an Incident.
T what(T...args)
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:26
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:60
STL class.
StatusCode setAppReturnCode(SmartIF< IProperty > &appmgr, int value, bool force=false)
Set the application return code.
Definition: AppReturnCode.h:50
bool m_quarantine
Property: Flag if unaccessible files should be quarantines in job.
Definition: IODataManager.h:52
T insert(T...args)
T find(T...args)
T length(T...args)
STL class.
T begin(T...args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
T c_str(T...args)
Base class for all Incidents (computing events).
Definition: Incident.h:17
T emplace(T...args)
SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
T substr(T...args)
void ignore() const
Definition: StatusCode.h:108
The IProperty is the basic interface for all components which have properties that can be set or get...
Definition: IProperty.h:21
list i
Definition: ana.py:128
StatusCode reconnect(Entry *e)
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:244
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:62
IIODataManager::Connection * IODataManager::connection ( const std::string dsn) const
override

Retrieve known connection.

Definition at line 221 of file IODataManager.cpp.

221  {
222  auto j = m_fidMap.find(dataset);
223  if ( j == m_fidMap.end() ) return nullptr;
224  auto i=m_connectionMap.find(j->second);
225  return (i != m_connectionMap.end()) ? i->second->connection : nullptr;
226 }
T end(T...args)
T find(T...args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
list i
Definition: ana.py:128
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:62
IODataManager::Connections IODataManager::connections ( const IInterface owner) const
override

Get connection by owner instance (0=ALL)

Definition at line 96 of file IODataManager.cpp.

96  {
97  Connections conns;
98  transform_copy_if( std::begin(m_connectionMap), std::end(m_connectionMap),
99  std::back_inserter(conns),
100  [](ConnectionMap::const_reference i ) { return i.second->connection; },
101  [&](const IDataConnection* c) { return !owner || c->owner() == owner; } );
102  return conns;
103 }
tuple c
Definition: gaudirun.py:391
T end(T...args)
ABC describing basic data connection.
T begin(T...args)
T back_inserter(T...args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
list i
Definition: ana.py:128
StatusCode IODataManager::connectRead ( bool  keep_open,
Connection *  ioDesc 
)
override

Open data stream in read mode.

Connect data file for writing.

Definition at line 106 of file IODataManager.cpp.

106  {
107  if ( !establishConnection(con) ) {
108  return connectDataIO(UNKNOWN,Connection::READ,con->name(),"UNKNOWN",keep_open,con);
109  }
110  std::string dsn = con ? con->name() : std::string("Unknown");
111  return error("Failed to connect to data:"+dsn,false);
112 }
StatusCode establishConnection(Connection *con)
STL class.
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
StatusCode IODataManager::connectWrite ( Connection *  con,
IoType  mode = Connection::CREATE,
CSTR  doctype = "UNKNOWN" 
)
override

Open data stream in write mode.

Connect data file for reading.

Definition at line 115 of file IODataManager.cpp.

115  {
116  if ( !establishConnection(con) ) {
117  return connectDataIO(UNKNOWN,mode,con->name(),doctype,true,con);
118  }
119  std::string dsn = con ? con->name() : std::string("Unknown");
120  return error("Failed to connect to data:"+dsn,false);
121 }
StatusCode establishConnection(Connection *con)
STL class.
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
StatusCode connectDataIO(int typ, IoType rw, CSTR fn, CSTR technology, bool keep, Connection *con)
StatusCode IODataManager::disconnect ( Connection *  ioDesc)
override

Release data stream.

Definition at line 138 of file IODataManager.cpp.

138  {
139  if ( con ) {
140  std::string dataset = con->name();
141  std::string dsn = dataset;
142  StatusCode sc = con->disconnect();
143  if ( ::strncasecmp(dsn.c_str(),"FID:",4)==0 )
144  dsn = dataset.substr(4);
145  else if ( ::strncasecmp(dsn.c_str(),"LFN:",4)==0 )
146  dsn = dataset.substr(4);
147  else if ( ::strncasecmp(dsn.c_str(),"PFN:",4)==0 )
148  dsn = dataset.substr(4);
149 
150  auto j = m_fidMap.find(dataset);
151  if ( j != m_fidMap.end() ) {
152  std::string fid = j->second;
153  std::string gfal_name = "gfal:guid:" + fid;
154  auto i=m_connectionMap.find(fid);
155  m_fidMap.erase(j);
156  if ( (j=m_fidMap.find(fid)) != m_fidMap.end() )
157  m_fidMap.erase(j);
158  if ( (j=m_fidMap.find(gfal_name)) != m_fidMap.end() )
159  m_fidMap.erase(j);
160  if ( i != m_connectionMap.end() && i->second ) {
161  IDataConnection* c = i->second->connection;
162  if ( (j=m_fidMap.find(c->pfn())) != m_fidMap.end() )
163  m_fidMap.erase(j);
164  if ( c->isConnected() ) {
165  MsgStream log(msgSvc(),name());
166  c->disconnect();
167  log << MSG::INFO << "Disconnect from dataset " << dsn
168  << " [" << fid << "]" << endmsg;
169  }
170  delete i->second;
172  }
173  }
174  return sc;
175  }
176  return S_ERROR;
177 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
virtual StatusCode disconnect()=0
Release data stream.
virtual bool isConnected() const =0
Check if connected to data source.
tuple c
Definition: gaudirun.py:391
T end(T...args)
const std::string & pfn() const
Access physical file name.
STL class.
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:319
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:26
T erase(T...args)
T find(T...args)
ABC describing basic data connection.
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
T c_str(T...args)
SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
T substr(T...args)
list i
Definition: ana.py:128
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:244
FidMap m_fidMap
Map of FID to PFN.
Definition: IODataManager.h:62
StatusCode IODataManager::error ( CSTR  msg,
bool  rethrow 
)
protected

Small routine to issue exceptions.

Definition at line 88 of file IODataManager.cpp.

88  {
89  MsgStream log(msgSvc(),name());
90  log << MSG::ERROR << "Error: " << msg << endmsg;
91  if ( rethrow ) System::breakExecution();
92  return S_ERROR;
93 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
GAUDI_API long breakExecution()
Break the execution of the application and invoke the debugger.
Definition: Debugger.cpp:47
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:319
SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:244
StatusCode IODataManager::establishConnection ( Connection *  con)
protected

Definition at line 228 of file IODataManager.cpp.

228  {
229  if ( !con ) return error("Severe logic bug: No connection object avalible.",true);
230 
231  if ( con->isConnected() ) {
232  con->resetAge();
233  return S_OK;
234  }
235  auto i=m_connectionMap.find(con->name());
236  if ( i != m_connectionMap.end() ) {
237  Connection* c = i->second->connection;
238  if ( c != con ) {
239  m_incSvc->fireIncident(Incident(con->name(),IncidentType::FailInputFile));
240  return error("Severe logic bug: Twice identical connection object for DSN:"+con->name(),true);
241  }
242  if ( reconnect(i->second).isSuccess() ) return S_OK;
243  }
244  return S_ERROR;
245 }
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
tuple c
Definition: gaudirun.py:391
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:68
T end(T...args)
MsgStream & error() const
shortcut for the method msgStream(MSG::ERROR)
virtual void fireIncident(const Incident &incident)=0
Fire an Incident.
T find(T...args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
Base class for all Incidents (computing events).
Definition: Incident.h:17
list i
Definition: ana.py:128
StatusCode reconnect(Entry *e)
StatusCode IODataManager::finalize ( )
override

IService implementation: finalize the service.

Definition at line 82 of file IODataManager.cpp.

82  {
83  m_catalog = nullptr; // release
84  return Service::finalize();
85 }
StatusCode finalize() override
Definition: Service.cpp:193
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:60
StatusCode IODataManager::initialize ( )
override

IService implementation: initialize the service.

IService implementation: Db event selector override.

Definition at line 57 of file IODataManager.cpp.

57  {
58  // Initialize base class
60  MsgStream log(msgSvc(), name());
61  if ( !status.isSuccess() ) {
62  log << MSG::ERROR << "Error initializing base class Service!" << endmsg;
63  return status;
64  }
65  // Retrieve conversion service handling event iteration
67  if( !m_catalog ) {
68  log << MSG::ERROR
69  << "Unable to localize interface IFileCatalog from service:"
71  return StatusCode::FAILURE;
72  }
73  m_incSvc = serviceLocator()->service("IncidentSvc");
74  if( !m_incSvc ) {
75  log << MSG::ERROR << "Error initializing IncidentSvc Service!" << endmsg;
76  return status;
77  }
78  return status;
79 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
StatusCode initialize() override
Definition: Service.cpp:68
SmartIF< ISvcLocator > & serviceLocator() const override
Retrieve pointer to service locator.
Definition: Service.cpp:324
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
SmartIF< IIncidentSvc > m_incSvc
Definition: IODataManager.h:68
std::string m_catalogSvcName
Property: Name of the file catalog service.
Definition: IODataManager.h:46
StatusCode service(const Gaudi::Utils::TypeNameString &name, T *&svc, bool createIf=true)
Templated method to access a service by name.
Definition: ISvcLocator.h:78
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:319
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:26
SmartIF< IFileCatalog > m_catalog
Reference to file catalog.
Definition: IODataManager.h:60
SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:244
StatusCode IODataManager::read ( Connection *  ioDesc,
void *const  data,
size_t  len 
)
override

Read raw byte buffer from input stream.

Definition at line 124 of file IODataManager.cpp.

124  {
125  return establishConnection(con).isSuccess() ? con->read(data,len) : S_ERROR;
126 }
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
StatusCode establishConnection(Connection *con)
StatusCode IODataManager::reconnect ( Entry e)
protected

Definition at line 179 of file IODataManager.cpp.

179  {
181  if ( e && e->connection ) {
182  switch(e->ioType) {
183  case Connection::READ:
184  sc = e->connection->connectRead();
185  break;
186  case Connection::UPDATE:
187  case Connection::CREATE:
188  case Connection::RECREATE:
189  sc = e->connection->connectWrite(e->ioType);
190  break;
191  default:
192  return S_ERROR;
193  }
194  if ( sc.isSuccess() && e->ioType == Connection::READ ) {
195  std::vector<Entry*> to_retire;
196  e->connection->resetAge();
197  transform_copy_if( std::begin(m_connectionMap), std::end(m_connectionMap),
198  std::back_inserter(to_retire),
199  select2nd,
200  [&](Entry* i) {
201  IDataConnection* c = i->connection;
202  return e->connection!=c && c->isConnected() &&
203  !i->keepOpen && c->ageFile() > m_ageLimit;
204  });
205  if ( !to_retire.empty() ) {
206  MsgStream log(msgSvc(),name());
207  std::for_each( std::begin(to_retire), std::end(to_retire),
208  [&](Entry* j) {
209  IDataConnection* c = j->connection;
210  c->disconnect();
211  log << MSG::INFO << "Disconnect from dataset " << c->pfn()
212  << " [" << c->fid() << "]" << endmsg;
213  } );
214  }
215  }
216  }
217  return sc;
218 }
Definition of the MsgStream class used to transmit messages.
Definition: MsgStream.h:24
virtual StatusCode disconnect()=0
Release data stream.
T empty(T...args)
int m_ageLimit
Property: Age limit.
Definition: IODataManager.h:48
virtual bool isConnected() const =0
Check if connected to data source.
const std::string & fid() const
Access file id.
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
tuple c
Definition: gaudirun.py:391
T end(T...args)
const std::string & pfn() const
Access physical file name.
const std::string & name() const override
Retrieve name of the service.
Definition: Service.cpp:319
int ageFile()
Increase age of I/O source.
This class is used for returning status codes from appropriate routines.
Definition: StatusCode.h:26
STL class.
ABC describing basic data connection.
T begin(T...args)
T back_inserter(T...args)
ConnectionMap m_connectionMap
Map with I/O descriptors.
Definition: IODataManager.h:58
SmartIF< IMessageSvc > & msgSvc() const
The standard message service.
T for_each(T...args)
list i
Definition: ana.py:128
MsgStream & endmsg(MsgStream &s)
MsgStream Modifier: endmsg. Calls the output method of the MsgStream.
Definition: MsgStream.h:244
long long int IODataManager::seek ( Connection *  ioDesc,
long long int  where,
int  origin 
)
override

Seek on the file described by ioDesc. Arguments as in ::seek()

Definition at line 134 of file IODataManager.cpp.

134  {
135  return establishConnection(con).isSuccess() ? con->seek(where,origin) : -1;
136 }
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
StatusCode establishConnection(Connection *con)
StatusCode IODataManager::write ( Connection *  con,
const void *  data,
int  len 
)
override

Write raw byte buffer to output stream.

Definition at line 129 of file IODataManager.cpp.

129  {
130  return establishConnection(con).isSuccess() ? con->write(data,len) : S_ERROR;
131 }
bool isSuccess() const
Test for a status code of SUCCESS.
Definition: StatusCode.h:76
StatusCode establishConnection(Connection *con)

Member Data Documentation

int Gaudi::IODataManager::m_ageLimit
protected

Property: Age limit.

Definition at line 48 of file IODataManager.h.

SmartIF<IFileCatalog> Gaudi::IODataManager::m_catalog
protected

Reference to file catalog.

Definition at line 60 of file IODataManager.h.

std::string Gaudi::IODataManager::m_catalogSvcName
protected

Property: Name of the file catalog service.

Definition at line 46 of file IODataManager.h.

ConnectionMap Gaudi::IODataManager::m_connectionMap
protected

Map with I/O descriptors.

Definition at line 58 of file IODataManager.h.

bool Gaudi::IODataManager::m_disablePFNWarning
protected

Property DisablePFNWarning: if set to True will not report when a file is opened by it's physical name.

Definition at line 55 of file IODataManager.h.

FidMap Gaudi::IODataManager::m_fidMap
protected

Map of FID to PFN.

Definition at line 62 of file IODataManager.h.

SmartIF<IIncidentSvc> Gaudi::IODataManager::m_incSvc
protected

Definition at line 68 of file IODataManager.h.

bool Gaudi::IODataManager::m_quarantine
protected

Property: Flag if unaccessible files should be quarantines in job.

Definition at line 52 of file IODataManager.h.

bool Gaudi::IODataManager::m_useGFAL
protected

Property: Flag for auto gfal data access.

Definition at line 50 of file IODataManager.h.


The documentation for this class was generated from the following files: