Author: Arkilic, A.
Paper Title Page
Experiment Control and Data Management Systems at the NSLS-II  
  • D. Chabot, D.B. Allan, A. Arkilic, T.A. Caswell, L.R. Dalesio, E. Dill, K.R. Lauer
    BNL, Upton, Long Island, New York, USA
  A Python-based framework for experiment control is under development at Brookhaven National Lab and deployed for use on NSLS-II beamlines. The software supports device control, data collection and distribution, reciprocal space operations, integration with electronic logging, and more. The system is coupled with searchable back-end storage and retrieval components, which together form an end-to-end data management architecture for the NSLS-II. An overview of the data collection system and its accompanying data management infrastructure will be discussed.  
poster icon Poster MOPGF053 [3.834 MB]  
WEA3O02 Recent Advancements and Deployments of EPICS Version 4 1
  • G.R. White, M.V. Shankar
    SLAC, Menlo Park, California, USA
  • A. Arkilic, L.R. Dalesio, M.A. Davidsaver, M.R. Kraimer, N. Malitsky, B.S. Martins
    BNL, Upton, Long Island, New York, USA
  • S.M. Hartman, K.-U. Kasemir
    ORNL, Oak Ridge, Tennessee, USA
  • D.G. Hickin
    DLS, Oxfordshire, United Kingdom
  • A.N. Johnson, S. Veseli
    ANL, Argonne, Ilinois, USA
  • T. Korhonen
    ESS, Lund, Sweden
  • R. Lange
    ITER Organization, St. Paul lez Durance, France
  • M. Sekoranja
    Cosylab, Ljubljana, Slovenia
  • G. Shen
    FRIB, East Lansing, Michigan, USA
  EPICS version 4 is a set of software modules that add to the base of the EPICS toolkit for advanced control systems. Version 4 adds the possibility of process variable values of structured data, an introspection interface for dynamic typing plus some standard types, high-performance streaming, and a new front-end processing database for managing complex data I/O. A synchronous RPC-style facility has also been added so that the EPICS environment supports service-oriented architecture. We introduce EPICS and the new features of version 4. Then we describe selected deployments, particularly for high-throughput experiment data transport, experiment data management, beam dynamics and infrastructure data.  
slides icon Slides WEA3O02 [2.409 MB]  
WED3O02 Databroker: An Interface for NSLS-II Data Management System 1
  • A. Arkilic, D.B. Allan, D. Chabot, L.R. Dalesio, W.K. Lewis
    BNL, Upton, Long Island, New York, USA
  Funding: Brookhaven National Lab, U.S. Department of Energy
A typical experiment involves not only the raw data from a detector, but also requires additional data from the beamline. This information is largely kept separated and manipulated individually, to date. A much more effective approach is to integrate these different data sources, and make these easily accessible to data analysis clients. NSLS-II data flow system contains multiple backends with varying data types. Leveraging the features of these (metadatastore, filestore, channel archiver, and Olog), this library provides users with the ability to access experimental data. This service acts as a single interface for time series, data attribute, frame data access and other experiment related information.
slides icon Slides WED3O02 [2.940 MB]  
WEPGF043 Metadatastore: A Primary Data Store for NSLS-2 Beamlines 1
  • A. Arkilic, D.B. Allan, T.A. Caswell, L.R. Dalesio, W.K. Lewis
    BNL, Upton, Long Island, New York, USA
  Funding: Department of Energy, Brookhaven National Lab
The beamlines at NSLS-II are among the highest instrumented, and controlled of any worldwide. Each beamline can produce unstructured data sets in various formats. This data should be made available for data analysis and processing for beamline scientists and users. Various data flow systems are in place in numerous synchrotrons, however these are very domain specific and cannot handle such unstructured data. We have developed a data flow service, metadatastore, that manages experimental data in NSLS-II beamlines. This service enables data analysis and visualization clients to access this service either directly or via databroker api in a consistent and partition tolerant fashion, providing a reliable and easy to use interface to our state-of-the-art beamlines.
WEPGF044 Filestore: A File Management Tool for NSLS-II Beamlines 1
  • A. Arkilic, T.A. Caswell, D. Chabot, L.R. Dalesio, W.K. Lewis
    BNL, Upton, Long Island, New York, USA
  Funding: Brookhaven National Lab, Departmet of Energy
NSLS-II beamlines can generate 72,000 data sets per day resulting in over 2 M data sets in one year. The large amount of data files generated by our beamlines poses a massive file management challenge. In response to this challenge, we have developed filestore, as means to provide users with an interface to stored data. By leveraging features of Python and MongoDB, filestore can store information regarding the location of a file, access and open the file, retrieve a given piece of data in that file, and provide users with a token, a unique identifier allowing them to retrieve each piece of data. Filestore does not interfere with the file source or the storage method and supports any file format, making data within files available for NSLS-II data analysis environment.
poster icon Poster WEPGF044 [0.849 MB]