A thorough understanding of the Longitudinal Phase Space (LPS) of the electron beam is of great advantage to any modern linear accelerator, and of critical importance for operating a Free Electron Laser (FEL). A diagnostic system equipped with a Transverse Deflecting Structure (TDS) allows full imaging of a beam’s LPS. However, measurements with a TDS are not always easily accessible and are often destructive. In this talk, we present an application of machine learning in the form of a virtual diagnostic which allows for online extraction of the beam’s LPS based on non-destructive measurements. We present how such virtual diagnostics have been developed and tested for three different accelerators: the MAX IV linac and the FELs at FERMI and SwissFEL. We show how a single, general network architecture and training set up can be used to reach reliable predictions of the LPS for all three facilities. For future work, we show how virtual diagnostics could be further developed to suit the specific needs of operations at each facility.