Journals of Accelerator Conferences Website (JACoW)
JACoW
is a publisher in Geneva, Switzerland that publishes the
proceedings of accelerator conferences held around the world
by an international collaboration of editors.
| URL | https://doi.org/10.18429/JACoW-ICALEPCS2025-WECG003 |
|---|---|
| Title | Lessons learned refactoring the EuXFEL’s central data acquisition system |
| Authors |
|
| Abstract | The European XFEL, a world-leading X-ray light source, boasts high brilliance and fast burst repetition rates supported by bespoke MHz-imaging detectors, resulting in data rates close to 20 GB/s. Correlating auxiliary data from various sources with these images requires a centralized Data Acquisition (DAQ) system that ingests data from Karabo, and outputs aggregated data into a common HDF5 format. Facility-side calibration and processing rely on the stability of this data format. We discuss refactoring this major, mission-critical software to scale for future data reduction needs. The system transitioned from a blackboard to a pipelined design modeling unidirectional dataflows. Refactoring was prepared with extensive tests for verification. Unit tests, often following test-driven development, accompanied refactoring and new features. A high continuous integration coverage is enhanced by system-level tests. The refactored system was introduced incrementally over 3 months, following a „test early, fail early“ philosophy. The new system successfully implements critical petabyte-scale data reduction, and enhanced status monitoring capabilities. |
| Paper | download: WECG003.pdf |
| Cite | BibTeX, LaTeX, Text/Word, RIS, EndNote |
| Conference | 20th Int. Conf. Accel. Large Exp. Phys. Control Syst. (ICALEPCS'25) |
| Series | |
| Location | Chicago, IL, USA |
| Date | 20-26 Sep 2025 |
| Publisher | JACoW Publishing, Geneva, Switzerland |
| Editorial Board | Kent Wootton (ANL), Jeffery Biss (FNAL), Gustavo Bruno (ANL), Jeff Kirchman (ANL), Kelly Jaje (ANL), Denise Skiadopoulos (ANL), SuYin Grass Wang (ANL) |
| Online ISBN | 978-3-95450-255-4 |
| Online ISSN | 2226-0358 |
| Received | 26 August 2025 |
| Revised | 28 August 2025 |
| Accepted | 29 October 2025 |
| Issued | 25 November 2025 |
| DOI | 10.18429/JACoW-ICALEPCS2025-WECG003 |
| Pages | 879-884 |
| Copyright | Published by JACoW Publishing under the terms of the Creative Commons Attribution 4.0 license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI. |