| Title |
Filestore: A File Management Tool for NSLS-II Beamlines |
| Authors |
- A. Arkilic, T.A. Caswell, D. Chabot, L.R. Dalesio, W.K. Lewispresenter
BNL, Upton, Long Island, New York, USA
|
| Abstract |
NSLS-II beamlines can generate 72,000 data sets per day resulting in over 2 M data sets in one year. The large amount of data files generated by our beamlines poses a massive file management challenge. In response to this challenge, we have developed filestore, as means to provide users with an interface to stored data. By leveraging features of Python and MongoDB, filestore can store information regarding the location of a file, access and open the file, retrieve a given piece of data in that file, and provide users with a token, a unique identifier allowing them to retrieve each piece of data. Filestore does not interfere with the file source or the storage method and supports any file format, making data within files available for NSLS-II data analysis environment.
|
| Funding |
Brookhaven National Lab, Departmet of Energy |
| Paper |
download WEPGF044.PDF [0.547 MB / 3 pages] |
| Poster |
download WEPGF044_POSTER.PDF [0.854 MB] |
| Conference |
ICALEPCS2015, Melbourne, Australia |
| Series |
International Conference on Accelerator and Large Experimental Physics Control Systems (15th) |
| Proceedings |
Link to full ICALEPCS2015 Proccedings |
| Session |
Poster session |
| Date |
21-Oct-15 17:15–18:15 |
| Main Classification |
Data Management, Analytics & Visualisation |
| Keywords |
experiment, data-analysis, EPICS, data-acquisition, database |
| Publisher |
JACoW, Geneva, Switzerland |
| Editors |
Lou Corvetti (AS, Clayton, Australia); Kathleen Riches (SLSA, Clayton, Australia); Volker RW Schaa (GSI, Darmstadt, Germany) |
| ISBN |
978-3-95450-148-9 |
| Published |
December 2015 |
| Copyright |
|
