Journals of Accelerator Conferences Website (JACoW)
JACoW
is a publisher in Geneva, Switzerland that publishes the
proceedings of accelerator conferences held around the world
by an international collaboration of editors.
| URL | https://doi.org/10.18429/JACoW-ICALEPCS2025-TUBG006 |
|---|---|
| Title | Hunting for hidden bugs: dealing with test flakiness in SKA control software |
| Authors |
|
| Abstract | Test flakiness—when a test intermittently passes or fails without changes to the code—poses a significant challenge in the validation of distributed control systems. This paper presents an investigation into test flakiness in CSP.LMC (Local Monitoring and Control for the Central Signal Processor), a key subsystem of the SKA (Square Kilometre Array) telescope. CSP.LMC is a Python application built on the TANGO framework, that is tested using a multi-level testing approach combining unit, component, and integration tests. To achieve scalable and reproducible deployment, the entire SKA control software runs within a Kubernetes environment. We systematically collect test outcomes and execution benchmarks to monitor system stability over time. A data mining approach is applied to uncover correlations and hidden patterns associated with test instability. Our analysis aims to uncover subtle software issues that are not easily detected through standard test evaluation. Furthermore, we aim to explore how the complexity of both the software architecture and its deployment may introduce sources of non-determinism that can lead to flaky tests. We discuss the impact of flakiness on the reliability of SKA control software and propose practical strategies to benchmark, detect, and mitigate flaky tests in complex distributed environments. |
| Paper | download: TUBG006.pdf |
| Cite | BibTeX, LaTeX, Text/Word, RIS, EndNote |
| Conference | 20th Int. Conf. Accel. Large Exp. Phys. Control Syst. (ICALEPCS'25) |
| Series | |
| Location | Chicago, IL, USA |
| Date | 20-26 Sep 2025 |
| Publisher | JACoW Publishing, Geneva, Switzerland |
| Editorial Board | Kent Wootton (ANL), Jeffery Biss (FNAL), Gustavo Bruno (ANL), Jeff Kirchman (ANL), Kelly Jaje (ANL), Denise Skiadopoulos (ANL), SuYin Grass Wang (ANL) |
| Online ISBN | 978-3-95450-255-4 |
| Online ISSN | 2226-0358 |
| Received | 21 September 2025 |
| Revised | 24 September 2025 |
| Accepted | 31 October 2025 |
| Issued | 25 November 2025 |
| DOI | 10.18429/JACoW-ICALEPCS2025-TUBG006 |
| Pages | 259-265 |
| Copyright | Published by JACoW Publishing under the terms of the Creative Commons Attribution 4.0 license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI. |