Digital Archive Directions (DADs) Workshop

• Call for Participation
• Registration Form
• Invited Speakers and Accepted Position Papers
• Position Paper Requirements
• Web Submission Form
• Email Submission Format
• Standardization Project Considerations
• Registered Participants
• Draft Agenda
• Meeting and Hotel Information
• Contact Information

   

• Highlights of the Reference Model
• Reference Model for an Open Archival Information System

1. Identification of Proposed Topic [Required]

1.1 Title

Data Ingest Methodology and Tools

1.2 Contributor(s)

Mike Martin, Steven Hughes, Jet Propulsion Laboratory

1.3 Description of Proposed Project

Over the past 8 years, the Planetary Data System has developed a methodology for the production of archive quality data sets. It is used by all the Planetary Data System Data Engineers as a guide in negotiating agreements with data producers (generally space missions like Galileo or Cassini) and performing all the steps required to define, document, produce and evaluate digital archive submissions. While the current workbook is oriented to the mission interface normally encountered in PDS operations, it could be extended into a general workbook for archive summission procedures. The following chart illustrates the activities involved in the end-to-end process. There are also a number of data validation tools that are used by the data engineers to validate incoming products. The foremost is the volume verifier which assures all components of an archive quality products are included on an archival submission and validates all metadata entries included in the products.

1.4 Justification

Most of the archiving and data processing systems we are familiar with have no formal methodology for negotiating with data producers. This can lead to a wide diversity in the quality and type of documentation and data formats which the archive receives and stores.

1.5 Definitions of Concepts and Special Terms

archive quality data product

A data product which is accompanied with sufficient descriptive material to be interpretable by a user who is not familiar with the circumstances of its production. Carrying all the information needed to be useful over a very long time span.

Volume verifier

A software program that automatically tests archive volumes to see that they conform to the standards of the archive system. Project Data Management Plan - A negotiated agreement between the archive and a producer which identifies and describes all products of potential archival value.

Archive and Transfer Plan

A negotiated agreement between the archive and the producer which identifies specific data products and a schedule for their delivery to the archive.

1.6 Expected Relationship with OAIS Reference Model

This activity would support the administration and ingest functions.

2. Scope of Proposed Standard

2.1 Recommended Scope of Standard

Enhance the PDS Data Preparation Workbook to be a general guide to the practices and procedures of negotiating, receiving, reviewing and accepting archival products. Include relevant material from other similar efforts. Develop a standard set of data validation tools built on the PDS.

2.2 Existing Practice in Area of Proposed Standard

We are not aware of any other rigorous methodology for archive data ingest.

2.3 Expected Stability of Proposed Standard with Respect to Current and Potential Technological Advances

It is expected that the methodology and tools will need revisions as the ingest process becomes more web-based (many procedures still focus on cd-rom volumes), but the basic techniques involved should be permanent.

• Call for Participation
• Registration Form
• Invited Speakers and Accepted Position Papers
• Position Paper Requirements
• Web Submission Form
• Email Submission Format
• Standardization Project Considerations
• Registered Participants
• Draft Agenda
• Meeting and Hotel Information
• Contact Information

   

• Highlights of the Reference Model
• Reference Model for an Open Archival Information System