The High-Luminosity LHC (HL-LHC) project at CERN aims at doubling the beam intensity and the brightness. To achieve this unprecedented performance, the LHC injectors were upgraded during the Long Shutdown 2 (2019-2021) to overcome limitations such as space charge and beam instabilities. Despite these upgrades, the reduction of beam loss on the flat bottom in the Super Proton Synchrotron (SPS) to reach the target beam parameters remains a challenge, avoiding unnecessary activation. Losses are due to several factors: uncaptured beam in the SPS due to the bunch rotation in the Proton Synchrotron (PS) prior to the transfer, large transient beam loading during multiple SPS injections, and transverse tails reaching aperture limitations. Investigations were conducted with HL-LHC beam parameters, aiming at disentangling the different sources of losses and defining specific observables. Finally, refining the optimal beam parameters for improved transfer between PS and SPS is the objective of the study, as well as the possible need for new hardware such as an additional RF system for beam stability and capture or a dedicated collimation system.

Reinforcement learning (RL) is a promising direction in machine learning for the control and optimisation of particle accelerators since it learns directly from experience without needing a model a-priori. However, RL generally suffers from low sample efficiency and thus training from scracth on the machine is often not an option. RL agents are usually trained or pre-tuned on simulators and then transferred to the real environment. In this work we propose a model-based RL approach based on Gaussian processes (GPs) to overcome the sample efficiency limitation. Our RL agent was able to learn to control the trajectory at the CERN AWAKE (Advanced Wakefield Experiment) facility, a problem of 10 degrees of freedom, within a few interactions only. To date, numerical optimises are used to restore or increase and stabilise the performance of accelerators. A major drawback is that they must explore the optimisation space each time they are applied. Our RL approach learns as quickly as numerical optimisers for one optimisation run, but can be used afterwards as single-shot or few-shot controllers. Furthermore, it can also handle safety and time-varying systems and can be used for the online stabilisation of accelerator operation.This approach opens a new avenue for the application of RL in accelerator control and brings it into the realm of everyday applications.