Changelog

All notable changes to this project will be documented in this file.

Project Coordinators: Marco Clemencic @clemenci, Charles Leggett @leggett, Stefan Roiser @roiser

The format is based on Keep a Changelog.

Unreleased

Changed

Added

Fixed

v32r2 - 2019-09-27

This is mostly a bugfix release, but there are nonetheless a couple of important changes:

• now Gaudi can be used with Python 3 (#56)
• the IAsyncEventProcessorexperimental interface introduced in v32r0 (!878) has been replaced with IQueueingEventProcessor (!966)
• it's now possible to get type information from Python version of DataHandles

Changed

• Replace IAsyncEventProcessor with IQueueingEventProcessor (!966)

Added

• GaudiKernel instructionsetLevel: Add additional AVX512 levels (!954)
• Add Python 3 support (#56, !875, !975, #88, !976)
• Add vecwid256 microarchitecture option to enable -mprefer-vector-width=256 (!965)
• Add type information to Python representation of DataHandles (!951)

Fixed

• Backward compatible interface of IAlgorithm for GaudiPython (#75, !977)
• Misc fixes for GaudiMP (#87, #75, !973)
• Remove unneeded change in refactoring of TES handling (#70, !971)
• Various ToolHandleArray bug fixes (!960)
• Functional ranges v3 0.9.x warning suppression (!968)
• Fix clang -Wpotentially-evaluated-expression warning (!970)
• Switch from Boost test minimal (deprecated) to single header variant (!948)
• Fixes for MacOS (!956)
• Make ContainedObject assignment consistent with copy construction (!967)
• THistSvc::io_reinit: Handle case of empty new filename (!959)
• EventSelector - Always set context ptr to null on release (!961)
• Minor improvements to sanitizers (!955)
• ChronoStatSvc - Clear maps in finalize (!958)
• Minor updates to Gaudi::Functional implementation (!952)
• THistSvc: Fix crash in MP if root file already exists (!949)
• Fix gcc9 warnings (!962)

v32r1 - 2019-07-18

Bugfix release, with some backward compatible changes, mostly meant to pick up LCG 96 and ROOT 6.18/00.

Note: Because of the updated version of Boost in LCG 96, you should use a recent version of CMake (>= 3.14).

Changed

• Add algorithm name to warning in HiveDataBroker (!923)
• Make Configurable.getGaudiType a classmethod (!929)
• Prefer xyz_v<T> over xyz<T>::value, and xyz_t<T> over typename xyz<T>::type (!925)
• Prefer std::{variant,optional} over boost::{variant,optional} (!926)

Added

• Specify JOB_POOL for genreflex custom commands (!933)
• Add an example MergingTransformer algorithm (!934)
• Add loop unroll hint macros to GaudiKernel/Kernel.h (!931)
• Add MsgCounter (!921)
• Introduce a MergingMultiTransformer (!901)

Fixed

• Removed a few unused .cpp files (!945)
• Print counters in Gaudi::Algorithm::finalize (!943, !946, !947)
• Make asynceventprocessor tests more stable (!940)
• Small fixes to toolchain (!938)
• Use the AlgContext guard constructor without EventContext for single thread (!973, #72, #73)
• Hide spurious warning from clang-8 (!941)
• Check that the clang-format command found can be executed (!935)
• Improve scheduler stall debug output (!930)
• Fix CMake modules test on special architectures (!927)
• Fix handling of strings with quotes in properties (!919)
• Remove reference to removed gaudimain.runSerialOld (!918)
• Restore backward compatibility after !878 (!917)
• Revert !869: Create a queue for all schedule-able algorithms (!924)
• Add GaudiKernel linkage to Counters unit test (!920)
• Misc. sanitizer fixes (!916)
• Fix distcc/icecc support (!911)
• Made the LibLZMA dependency in GaudiKernel optional (!909)
• GaudiKernel: check likely macros before defining them (!912)
• Fix IOpaqueAddress lifetime in EvtStoreSvc (!914)

v32r0 - 2019-05-18

This release features many fixes and improvements, but also a few backward incompatible changes:

• removed GaudiGSL package (in preparation for the licensing under Apache 2) (!879)
• change of IEventProcessor interface (!877)
• change (const correctness) of custom containers interfaces (!896)
• removed a few unused components: HLTEventLoopMgr (!876), AlgErrorAuditor (!874)
• drop support for Python custom event loop in gaudirun.py (!878)

Among the various additions you can find:

• Counter-based timers (!787)
• example of an interface for accessing conditions, which require experiment specific implementation (!838)
• a new interface (Gaudi::Interfaces::IAsyncEventProcessor) to allow decoupling of the main thread from the event processing thread(s) (!878)
• a simplified application steering class (Gaudi::Application) (!878)

Changed

• Update clang-format to 8 (!872)
• Factor TES path related code out of GaudiCommon (!904)
• More explicit management of EventContext in IEventProcessor (!877)
• Small clean up in EventLoopMgr implementations (!876)
• Remove GaudiGSL (!879)
• Do not return non-const pointers to elements of const containers (!896)
• Drop backwards compatibility code (!883, !900)
• HiveDataBroker: throw exception in case of same output locations in two different algorithms (!858)
• Delete AlgErrorAuditor (!874)
• PrecedenceSvc: demote 'Verifying rules' message to debug (!873)
• Replace boost::string_ref with std::string_view (!859)

Added

• Add optional extension of EventProcessor for asynchronous processing (!878)
• Add example of ConditionAccessorHolder implementation for integration in Functional (!838)
• Bits for the cross-experiment study on heterogeneous workflows (!828)
• Generalize Gaudi::Functional to support LHCb-style conditions handles (!899)
• Add a synthetic performance stress test for the scheduler (!865)
• New Minimal Event Store: EvtStoreSvc (!882)
• Add Counter-based timers (!787)
• Add support for Gaudi::Algorithm as base class of Functional algorithms (!897)
• Create a queue for all schedule-able algorithms (!869)
• Add support for specific architectures in BINARY_TAG (!853)
• Add getIfExits for DataHandles with AnyDataWrapper (!854)
• Add support for std::chrono::duration in counters (!849)
• Add the possibility to ask for Mode in python data handle property (!845)

Fixed

• Tweaks to Gaudi::Functional (!903)
• HiveDataBroker: Improve error message when unknown requested input (!906)
• Avoid copy of EventContext in Gaudi::Utils::AlgContext (!902)
• Reverse order of tbb task_scheduler_init and global_control (!895)
• Make StatusCodeSvc thread-safe (!885)
• ThreadPoolSvc: prefer std::unique_ptr (!889)
• Ensure thread termination tools only run in initialized threads (!887)
• Updates for MacOS and clang (!888, !894, !893)
• Enable std::iterator_traits<vector_of_const_<T>::iterator> (!855)
• Fix MessageSvc::reinitialize (!871)
• Fixes to EventIDRange (!860)
• ApplicationMgr: Call all state transition for MessageSvc and JobOptionsSvc (!866)
• Fix tunneling of CF decisions to inactive graph regions in multi-parent configurations (!863)
• Use [[fallthrough]] attribute instead of the fallthrough comment (!862)
• Restructure PRG sources and add visitor for Concurrent/Prompt CF inconsistency detection (!861)
• Sort data dependencies in HiveDataBroker debug printout (!868)
• Avoid displaying unused Binomial counters (!830)
• Python 2 & 3 compatibility (!832, !852)

v29r5 - 2019-03-06

This is a bugfix release only to fix a warning with the most recent version of CMake (3.13), see the changes on Gitlab.

v31r0 - 2019-02-11

The main reason for this major release is to the change of the Algorithm base class with a re-entrant execute method (#47, !703). For backward compatibility the base class for algorithms is Gaudi::Algorithm, while the name Algorithm in the top level namespace is used for a backward compatible implementation. This change also required a backward incompatible change in the IAlgorithm interface, so some code may require adaptation.

We also took the occasion to update the version of clang-format we use from 3.9 to 7, and to switch Python formatting from autopep8 to yapf (!788). Because of the differences (fixes and new features) in the versions of clang-format, we also updated the style rules.

As of this version, Gaudi requires C++17 compatible compiler. C++14 compatibility is not required in merge requests, nor guaranteed on the master branch.

On top of that, we have the usual collection of minor fixes and improvements.

Changed

• Use gcc 8 instead of gcc 7, and drop gcc 6.2 in CI jobs (!836)
• Make new counters work with Gaudi::Algorithm (#57, !834, !839)
• Update clang-format version and style rules (!788)
• Make DataObjectHandle::{put,getOrCreate} const (!833)
• Simplify IAlgExecStateSvc interface (!775)
• Simplify Algorithm constructor (!766)
• Improve the scheduler's execution flow simulator and fix the intra-event occupancy logger (!819)
• Reentrant Algorithm base class (#47, !703)

Added

• Introduced CPUCrunchSvc (!840)
• Added a boolean parameter to turn on verbose view algorithm status (!835)
• Added MsgStream support for counters (!826)
• Message: Add option to print full event ID (!824)
• IoComponentMgr: Add interface to retrieve all filenames of component (!823)
• Add example for Consumer<void()> (!797)

Fixed

• Fix lookup/use of nosetests for LCG 95 (!842)
• Check coding conventions in build stage of Gitlab-CI (!848)
• Make gaudi_add_compile_test also work for succeeding compilations (!844)
• Fix build of Gaudi with LCG 95rc1 in ATLAS context (!847)
• Explicitly define StatusCode constants (!841)
• Fix handling of EventContext in Gaudi::Functional::Transformer arguments (!837)
• Extensions to allow EventIDRanges limited by both time-stamps and run/LB pairs (!831)
• Improve ROOT dictionary dependency computation (!825)
• Fix issues in preparation for LCG 95 (!820)
• Improve QMT test wrapper (!812)
• Improvements for tests (!812, !817)
• Fixes/workrounds for thread-safety (!815, !829)
• Code clean up (!808, !811, !814, !809, !827, !843)
• Fixes to thread local initialization (!816, !807)

v30r5 - 2018-11-15

This release is mainly to revert !462 (Reimplemenation of DataHandle), which caused problems in ATLAS code.

Supersedes v30r4

Changed

• refer error in databroker when multiple algorithms declare the same output and remove some error messages (!799)
• Remove redundant code from GaudiKernel/StdArrayAsProperty.h (!790)
• Use generic boost::callable_traits over hand-rolled solution (!785)
• Revert "Reimplementation of `DataHandle` (!462)" (!794)

Added

• Added printing of Inputs/Outputs for HiveDataBroker in debug mode (!801)
• Support Boost >= 1.67 Python library name convention (!743)

Fixed

• Reduce dependency on LHCb env tools in CI jobs (!805)
• Make sure algorithms managed by AlgorithmManager are correctly initialized and started (!804)
• test_LBCORE_716: Use the default linker and not the system linker (!783)
• CountersUnitTest: Fix C++14 compiler warning (!795)

v30r4 - 2018-11-02 [YANKED]

Improvements, modernization and fixes.

Changed

• Reorganization of Gaudi::Parsers code (!759)
• Update LCG to 94 and Gaudi version to v30r4 (!776)
• Update minimum required CMake version to 3.6 (!789)
• Remove m_event_context from Algorithm (!784)
• Reimplementation of DataHandle (!462)
• Improve extension API in EventContext (!770)
• Change return type of interger counters operations, like mean, to float (!786, #50)
• Clean up IDataProviderSvc and IDataManagerSvc interfaces (!674)
• Avoid printing empty counters (!777)
• Use unique_ptr for EventContexts (!732, #37)
• ConcurrencyFlags: move setting from AvalancheScheduler to HiveWhiteBoard (!726)
• Introduce a generic IDataStoreAgent implementation (!661)
• MetaDataSvc: save data for all clients of the JobOptionsSvc (!720)
• Adding actual type of object in data store to error messages (!715)

Added

• Add Gaudi::Property constructors which accept an update handler (!769)
• Optionally capture and print exception backtrace (!653)
• Add build and runtime support for gcc/clang sanitizers (!744)
• Add debug builds to Gitlab-CI and other improvements to CI configuration (!756)
• Introduce IDataBroker interface, and HiveDataBroker implementation (!685, !739)
• Added an accumulate helper function to new counters (!717)

Fixed

• HiveWhiteBoard::freeStore: check whether slot was already free (!779, #49)
• Fix segfault when scheduling EventViews (!781)
• Fixed bug in BinomialCounter when using buffer (!764)
• Clarify scalar operator() signature in MultiScalarTransform (!755)
• Fix potential race condition in TimelineSvc (!750)
• Implicitly convert DataObjectHandleBase to str properties (!773)
• Filter out install and build files from make apply-formatting (!747, )
• Fixed issues reported by gcc/clang sanitizers (!749, !757, !758, !752, !751, !746, !772, !753)
• Make sure a test fails if its reference file is not found (!734)
• Fix Floating Point Exception in counters.h (!760)
• Make the CMake tests more resilient to symlinks (!742)
• Add support for versioned ROOT libraries (!737)
• Make event views bookkeeping thread-safe (!729, #36)
• Avoid using a (possibly invalid) EventContext pointer (!727)
• Mark dependencies as required when the build requires them (!731)
• Use std::filesystem only with C++17 (!738)
• Fixed indentation of new counters to be backward compatible (!717)
• Fixed error handling of qmt parsing errors (!714)
• Stop profiling on finalize if needed (!713)
• THistSvc: fix memory corruption in deReg(string) (!728)
• Avoid possible memory corruption in PluginService (!733)
• Added correct vector_of_const_::operator[] and ::at (!710)

Clean up

• Make VectorMap nothrow_move_constructible (!791)
• Fix some gcc 8 warnings (!782)
• Remove unused mutex member in Algorithm (!767)
• Replaced invalid std::forward with std::move in PluginService (!745)
• Counters: improve doxygen (!754)
• Minor code clean up (!408, !740, !774)

v30r3 - 2018-08-01

Improvements, modernization and fixes.

Deprecated

• Print warning if begin/endRun are used in serial jobs (!666)
• Fully deprecate DeclareFactoryEntries.h (!648)

Changed

• Implement error detection for Hive scheduler/eventloopmgr (!706)
• Modernize memory management in PrecedenceRulesGraph (!684)
• Simplify HLTEventLoopMgr (!686)
• Rewrite of PluginService in modern C++ (!709)
• Sort uids map before printing the content at finalize (in DEBUG) (!702)
• Remove NTUPLEINFO (!688)
• Revert IRegsitry modification from !604 (!704)
• Move to event-wise stall detection (!690)
• Avoid superfluous visitor-to-scheduler handovers of in-view DR algorithms (!689)
• Add set-to-set lookups for faster AlgsExecutionStates matching (!687)
• FileMgr: replace integer with bitset (!667)
• Enable top-level 3T dumps for event views (!675)
• Add support for multiple CF parents to 3T (!672)
• Applied new formatting (see !651, !448)
• Remove DataHandle::commit() (!625)
• Modernize Registry usage in (Ts)DataSvc (!604, #5)
• Improve Gitlab-CI configuration (!651, !448)
• Implemented a new property for IoComponentMgr to hold search patterns for direct I/O input file names (!646)
• Allow retrieval of const services, lost with !526 (!647)
• Add renamePFN and deletePFN methods to FileCatalog (!645)
• Do string conversion in stream operator for AlgsExecutionStates::State (!643)
• Use apply in Gaudi::Functional (!622)

Fixed

• Fixes for clang build (!724)
• Fix dependency of generated PCM files (!705)
• Fix tests in special configurations (!701)
• Fix asynchronous task scheduling (!697)
• Make sure we do not use ninja from LCG (!695)
• Increase timeout for event view tests (!696)
• Fix message service format in GaudiMP (!694)
• CommonAuditor: only print CustomEventType deprecation warning if the property is used (!691)
• Fix for crash when events are deleted while view algorithms are still running (!670)
• Fix environment for Gitlab-CI
• Unset GIT_DIR/GIT_WORK_TREE before cloning xenv (!669)
• Add test and fix of version propagation in KeyedContainer move (!663, #27)
• Fixed the list of files used for formatting check in Gitlab-CI (!664)
• Avoid (false positive) clang warning (!656)
• Handle incompatible values for DataObjectHandleBase props (!650)
• Fixed bug in THistSvc introduced with !594 (!652)

Added

• Add support for EventContext as input to Gaudi::Functional algorithms (!671)
• Add operator() to Property (!679)
• Add producer for KeyedContainer and vector to GaudiExamples (!673)
• Implementation of new counters classes (!629, !676, !677)
• Change log (!649)

Clean up

• Simplify Property forwarding functions (!683)
• Simplify counter implementation (!682)
• Update C++17 / lib fundamentals v2 implementations (!681)
• Modernize GaudiPython (!680)
• Modernize RootCnv (!660)
• Modernize RootHistCnv (!659)
• Protect logging and clean up headers in HiveSlimEventLoopMgr (!692)
• Remove unused variables in Algorithm (!668)
• Remove unreferenced code (!658)

v28r2p1 - 2018-06-15

Bugfix release

Fixed

• Add test and fix of version propagation in KeyedContainer move (!665, #27)

Added

• Gitlab-CI configuration (!665, backport from master)

v29r4 - 2018-04-17

Bugfix release.

Fixed

• Add test and fix of version propagation in KeyedContainer move (!662, #27)
• Handle incompatible values for DataObjectHandleBase props (!654)

Changed

• Improve Gitlab-CI configuration (!655, backport of !651)

Old versions

Details about old versions of the project can be found in the [ReleaseNotes](ReleaseNotes) folder and in the release.notes files in GaudiRelease/doc.