Journals of Accelerator Conferences Website (JACoW)
JACoW
is a publisher in Geneva, Switzerland that publishes the
proceedings of accelator conferences held around the world
by an international collaboration of editors.
| URL | https://doi.org/10.18429/JACoW-IPAC2023-THPL038 |
|---|---|
| Title | Ultra fast reinforcement learning demonstrated at CERN AWAKE |
| Authors |
|
| Abstract | Reinforcement learning (RL) is a promising direction in machine learning for the control and optimisation of particle accelerators since it learns directly from experience without needing a model a-priori. However, RL generally suffers from low sample efficiency and thus training from scracth on the machine is often not an option. RL agents are usually trained or pre-tuned on simulators and then transferred to the real environment. In this work we propose a model-based RL approach based on Gaussian processes (GPs) to overcome the sample efficiency limitation. Our RL agent was able to learn to control the trajectory at the CERN AWAKE (Advanced Wakefield Experiment) facility, a problem of 10 degrees of freedom, within a few interactions only. To date, numerical optimises are used to restore or increase and stabilise the performance of accelerators. A major drawback is that they must explore the optimisation space each time they are applied. Our RL approach learns as quickly as numerical optimisers for one optimisation run, but can be used afterwards as single-shot or few-shot controllers. Furthermore, it can also handle safety and time-varying systems and can be used for the online stabilisation of accelerator operation.This approach opens a new avenue for the application of RL in accelerator control and brings it into the realm of everyday applications. |
| Paper | download: THPL038.pdf |
| Cite | BibTeX, LaTeX, Text/Word, RIS, EndNote |
| Conference | 14th International Particle Accelerator Conference |
| Series | |
| Location | Venice, Italy |
| Date | 07-12 May 2023 |
| Publisher | JACoW Publishing, Geneva, Switzerland |
| Editorial Board | Ralph Assmann - Deutsches Elektronen-Synchrotron DESY Peter McIntosh - Science and Technology Facilities Council (STFC/DL/ASTeC) Giovanni Bisoffi - Istituto Nazionale di Fisica Nucleare (INFN/LNL) Alessandro Fabris - Elettra-Sincrotrone Trieste S.C.p.A. Ivan Andrian - Elettra-Sincrotrone Trieste S.C.p.A. Giulia Vinicola - Istituto Nazionale di Fisica Nucleare (INFN/LNF) |
| Online ISBN | 978-3-95450-231-8 |
| Online ISSN | 2673-5490 |
| Received | 03 May 2023 |
| Revised | 23 June 2023 |
| Accepted | 23 June 2023 |
| Issued | 26 September 2023 |
| DOI | 10.18429/JACoW-IPAC2023-THPL038 |
| Pages | 4510-4513 |
| Copyright | Published by JACoW Publishing under the terms of the Creative Commons Attribution 4.0 license. Any further distribution of this work must maintain attribution to the author(s), the published article's title, publisher, and DOI. |