| Title |
Scalable Time Series Documents Store |
| Authors |
- M.J. Slabber, F. Joubertpresenter, M.T. Ockards
SKA South Africa, National Research Foundation of South Africa, Cape Town, South Africa
|
| Abstract |
Data indexed by time is continuously collected from instruments, environment and users. Samples are recorded from sensors or software components at specific times, starting as simple numbers and increasing in complexity as associated values accrue e.g. status and acquisition times. A sample is more than a triple and evolves into a document. Besides variance, volume and veracity also increase and the time series database (TSDB) has to process hundreds of GB/day. Also, users performing analyses have ever increasing demands e.g. in <10s plot all target coordinates over 24h of 64 radio telescope dishes, recorded at 1Hz. Besides the many short-term queries, trend analyses over long periods and in-depth enquiries by specialists around past events e.g. critical hardware failure or scientific discovery, are performed. This paper discusses the solution used for the MeerKAT radio telescope under construction by SKA-SA in South Africa. System architecture and performance characteristics of the developed TSDB are explained. We demonstrate how we broke the mould of using general-purpose database technologies to build a TSDB by rather utilising technologies employed in distributed file storage.
|
| Funding |
National Research Foundation (South Africa) |
| Paper |
download TUBPA06.PDF [0.307 MB / 6 pages] |
| Slides |
download TUBPA06_TALK.PDF [1.781 MB] |
| Export |
download ※ BibTeX
※ LaTeX
※ Text/Word
※ RIS
※ EndNote |
| Conference |
ICALEPCS2017, Barcelona, Spain |
| Series |
International Conference on Accelerator and Large Experimental Control Systems (16th) |
| Proceedings |
Link to full ICALEPCS2017 Proccedings |
| Session |
Data Management and Processing |
| Date |
10-Oct-17 11:30–13:00 |
| Main Classification |
Data Management and Processing |
| Keywords |
ion, interface, database, controls, software-component |
| Publisher |
JACoW, Geneva, Switzerland |
| Editors |
Volker RW Schaa (GSI, Darmstadt, Germany); Isidre Costa (ALBA-CELLS, Cerdanyola del Vallès, Spain); David Fernández (ALBA-CELLS, Cerdanyola del Vallès, Spain); Ãscar Matilla (ALBA-CELLS, Cerdanyola del Vallès, Spain) |
| ISBN |
978-3-95450-193-9 |
| Published |
January 2018 |
| Copyright |
|
