EDM4hep Live Notes

Date: January 12, 2021
Indico: https://indico.cern.ch/event/983626/

This is a document for taking notes during EDM4hep meetings.

Connected: Joseph, Andre, Tao, Weidong, Teng, Jiaheng, Frank, Birgit, Placido, Sang Yun, Thomas, Xingtao, Valentin, Clement, Brieuc, Gerri, Hyeonja

Apologies:

Introduction and General Points

• Happy new Year!

Key4hep presentation at Epiphany

• https://indico.cern.ch/event/934666/contributions/4154229/

• Add to https://key4hep.github.io/key4hep-doc/talks-and-presentations/README.html

  • Also add a link to the FCC presentation https://indico.cern.ch/event/934666/contributions/4154228/

Virtual Chep

• papers(!) due February 28, 2021

  • Submissions should consist of a paper of 6-10 pages, following the instructions for authors. Paper submission is open until 28 February 2021.
  • https://indico.cern.ch/event/948465/page/21562-instructions-to-submitters

• Contribution:

  • Key4hep overview
    • Few paragraphs per contribution?
  • Delphes parts?
  • podio+edm4hep!

• TODO: share the overleaf from the last CHEP proceedings

• TODO: create repository for published documents?

• Why not CERN gitlab/DESY stash?

  • Need cern or desy computing accounts to access non-public repositories
  • (At least at the moment for CERN gitlab, might change)

• Private github repository free now?

  • Unlimited public/private repositories
  • See: https://github.com/pricing

Progress and Discussion

Podio

Benchmarking

[Thomas] Added some scripts and tools to have almost automatic benchmarking and some results using k4SimDelphes to “generate” data for I/O. Could be made fully automatic with a bit more work and if we want it to be fully automatic.

• https://github.com/tmadlener/podio_benchmarking

• Example results: https://github.com/tmadlener/podio_benchmarking/tree/master/results/k4SimDelphes/ee_Z_bbbar

• Teng: Add to github actions, upload plots as artifacts?

• Thomas: takes about 1 hour to run (depending on number of events)

• Teng: Can set schedule time in the yaml file. Very useful for validation

• Andre: The root peak structure understood?

• Thomas: Not yet fully understood/investigated. “working hypothesis” is that root has to resize some internal buffers if an Event/Collection has a certain size, resp. when it hits an Event with a certain size for the first time.

Plans for multi-threading testing of EDM4hep for CEPC

• Using Gaudi-hive functionality to speed up garfield simulation of the driftchamber
• Any documentation sugestions for how this can be done?
• Valentin: sounds very useful, not much experience in FCC
• Andre: What is the status of multi-threading for podio?
• Thomas: Reading is safe by design, writing has to be synchronised (e.g. by Gaudi)
• Frank: Root or also SIO backend?
• Thomas: Difference between standalone readers and what happens in the Gaudi world
• Valentin: Inside Gaudi it should work

Issues

more fixes to unsigned and shadowed variables

• https://github.com/AIDASoft/podio/pull/158

c++ concepts

• BH: add compile time checks for class behaviours: e.g., movable

CollectionIterator does not fully satisfy iterator concept

• https://github.com/AIDASoft/podio/issues/150
• For const collections, would be useful, e.g. using std::find on the collection

Idea: True reference collections

• https://github.com/AIDASoft/podio/issues/146

issue w/ ROOT and (vectors of) non-copyable collections

• happens in ROOT 6.22
• PM: there is a patch available in LCG repository
  • ROOT team is working on a general solution

What are the different branches in the root file?

• Related to use in RDataFrame
• Encode more information in the relation branch names?

Multi-Threading

See minutes of https://indico.cern.ch/event/969468/
–> Open issue(s) for further discussion

• Thomas and Benedikt discussing and started to work, nothing to report yet

“event class” in podio

• Currently being perceived

Writing second file with another tree

• https://github.com/key4hep/K4FWCore/issues/10
• this problem only happens in the GAUDI framework
  • if user tries to write an additional ROOT file
  • to be addressed
• Solved

PRs

• https://github.com/AIDASoft/podio/pulls

Disable collection operator=

• https://github.com/AIDASoft/podio/pull/154
• Review: Benedikt

Meta Data

Usage of “metadata” for user defined data

• need to check if current implementation addresses all use cases
• need test use-cases

Issues following MetaData Developments

• cannot write out event data previously read from file

  • Issue: https://github.com/AIDASoft/podio/issues/103
  • Test: https://github.com/AIDASoft/podio/pull/102
  • TM: have fix, but break framework core handling of collections

• there seems to be a fix in the Gaudi ROOTWriter in FCC-EDM !?

  • historical reason, fixes and developments should be back ported to PODIO
  • …

EventStore

Schema Evolution

• Version for object descriptions, etc.
• Open issue: https://github.com/AIDASoft/podio/issues/86

Features

• Subset collections?

LCIOConverters

• k4LCIOReader:
  • https://github.com/key4hep/k4LCIOReader
  • Moved k4LCIOReader code to key4hep project

EDM4hep

https://github.com/key4hep/EDM4hep/pulls

EDM4hep tools

• Thomas, Clement:
• Where to put tools using EDM4hep? In EDM4hep or separate repository?
• VV: Would be nice to keep EDM4hep stable, so separate repository.
• FG: Concur. Many releases for LCIO just to update tools.

Issues

k4FWCore: cannot read collections back

• https://github.com/key4hep/k4FWCore/issues/27

DD4hep output plugin

–> Moved to DD4hep

Repository is getting too big

• github pages branch is getting large
• Can they be created on the fly?
• Use a different host for the website

Tracker Hit

• Q: Tracker hit input to tracking algorithms?
• A: In LCIO different traker hits: Planar and Cylindrical using inheritance. Still needs to be adressed how to do this in EDM4hep.
• Q: Is inheritance needed?
• Q: What to use for Driftchambers?
• Open issue:

Need review of EDM4hepDelphes output

• Output is not stored if no tracks or towers created (?)

  • Allow also other types to be part of the list
  • Allow construction in a more general way (VV)

• Need to assume there is a complete list of reconstructed particles?

• Relations:

  • FCCSW: Issue with more than one RecoParticle pointing to one MCParticle
  • In LCIO to Delphes, conversion happens later than in edm4hep, maybe delphes does de-duplication?
  • TODO: open issue in edm4hep with reproducer (VV)

–> Moving to separate repository

Reworking output handling

• TM: better address edge cases
• TM + CH: FCC cases, not sure if delphes or output
• https://github.com/key4hep/k4SimDelphes/pull/5
• TM: How and what to test?

PRs

https://github.com/key4hep/EDM4hep/pulls/

Release 1.0

• Need:
  • Plugin
  • Eventheader
  • Meta Data (Event / Run Parameters)

AOB

Generator infrastructure in FCCSW moved to EDM4hep

• VV: show at next key4hep

Framework integration of k4SimDelphes

• On hold until delphes output reworked
• Started with updating the Gaudi documentation
  • New cmake in Gaudi v35

Dual Read-out calorimeter for FCC

• Special data structure used for that simulation
• Tried to use edm4hep, some issues
• Present in a future meeting

Conditions handling in Belle2

• Benedikt, or Martin Ritter

Feedback from FCC tutorial for snowmass

• Common question: What are the different branches in the root file?
• uproot was also used to access the root file
  • Basically re-implemented event store
    • Associations
  • Try to get this into the, e.g., podio repository

Next meeting:

• January 26, 2020