Science Archives in the 21st Century
This topic covers the application of submission agreements in an institution with long standing relationships with data providers from the pre and incipient stages of the digital age.
The National Geophysical Data Center (NGDC) is one of three of NOAA.s data centers under the National Environmental Satellite, Data, and Information Service (NESDIS). Compared with the other two data centers who focus purely on climatic and ocean data, NGDC handles heterogeneous data streams that are from multi-disciplines where the data sources range from satellites and ships to ground-based observatories.
NGDC is in a state of transition when it comes to archiving. For over a year now, the data center has moved towards using the OAIS reference model as a framework for defining the relationship between data provider and the archive. For new datasets, there is good support for engaging both sides in a formal submission agreement (SA) process as advocated in the PRODUCER-ARCHIVE INTERFACE METHODOLOGY ABSTRACT STANDARD (CCSDS 651.0-B-1 Blue Book). The archive presents the prospective data providers with a SA, and they accept it as a standard business operating procedure.
However, in the case of instituting the SA on existing data streams with a long time archival arrangement at NGDC, the process is not always so simple. Applying the OAIS concepts to existing data flows presents interesting challenges for well established datasets where minimum requirements for metadata, requests for PDI and codifying all aspects of the arrangements between data provider and archive transmissions were few and far between. This presentation of OAIS infusion at NGDC will highlight some of the aspects of dealing with these well-established datasets. Key points are as follows: a) Brief historical perspective of data ingest; b) The scenario involved with one dataset involving a myriad of data providers; and c) Phased-in approach proposal to incorporate the SA into the data management process.