FEP - Format Developer - PDS


		FEP - Format Developer - PDS Steve Hughes NASA JPL

Comment on this template in the HyperNews Discussion.		1. History and Philosophy 1.1 Identification PDS Standards Version 3.2 The PDS standards architecture is documented in three volumes, a) the PDS Standards Reference Version 3.2, b) the Planetary Science Data Dictionary Version 1R26, c) the Data Preparation Workbook Version 3.0. PDS labels include the keyword PDS_VERSION_ID =PDSn where the currently used value, PDS3, represents the version of the standards. 1.2 Purpose The PDS standards architecture provides a methodology that focuses on the use of meta-data labels to describe science data products. PDS labels describe the data format as well as the context in which the data was collected. A publishing metaphor is used, where the data product is peer reviewed before being placed in the archive and made available through an on-line digital library. The PDS standards recommend the use of simple formats for the data, such as raster format for images and ASCII tables for tabular data. However, PDS labels are regularly used for binary tables other data formats such as FITS and VICAR. The PDS standards allow compressed formats to reduce volume but require that the algorithm (source code) needed for decompression be included in the archive. The PDS discourages the use of data formats that require specialized software for retrieval. 1.3 User Community and Sponsoring Organization The PDS standards architecture has been adopted by the planetary science community as its archive standard. The PDS management council advises the PDS standards development team on evolution of the standards. The PDS is funded by the NASA's office of Space Science. 1.4 Format Evolution The PDS has a standards development team led by a PDS standards administrator and composed of the system engineer and the data engineers. This team addresses standards issues that are identified by the PDS data engineers or the PDS node personnel in support of data archiving functions. Issues are worked by the standards development team and may lead to the submission of a Standards Change Request which is handled according to standard PDS procedures. Major changes to the standards require the approval of the PDS management council. 2. Conceptual Model The PDS standards use an object-based methodology to describe data products. The commonly used objects are IMAGE, CUBE, and TABLE, where TABLE includes the subtype TIME_SERIES and SPECTRUM. Each object has been defined with required and optional attributes (keywords or data elements). Required attributes tend to focus on cataloging information such as time, identification, and target and critical structural attributes such as image lines and line_samples. Optional attributes are available to describe more specific data characteristics and complexity. For example, the image object allows optional attributes to describe multispectral images. All attributes are part of a controlled vocabulary and are defined in the Planetary Science Data Dictionary. A documented process is used to add attributes to the data dictionary. 3. Format Details The PDS standards were developed with hardware and media portability in mind. The emphasis is on designing a data product that is not hardware or media specific. However, the standards are flexible enough that hardware specific data formats have been used and described sufficiently for interpretation by future users. The internal representation of a number in a data product is denoted using a data type identifier. Each data type is described in the standards documentation. A PDS label together with the PDS standards reference provides a software developer with the necessary information for interpreting PDS archive data products. 4. Uses The PDS is primarily an archive format and its primary purpose is to maintain the scientific usefulness of the data for future users. However, the simplicity of the meta-data labels (keyword/value statements) and the simplicity of the recommended formats has made the development of software for display and processing very easy. Software libraries are also available for label parsing and data conversion. The PDS standards are regularly extended to handle additional complexity. 5. Format Developer Software The PDS provides and maintains two PDS label parsers and the Object Access Library (OAL). The OAL provides an API for archive product retrieval, conversion, manipulation, and writing. The OAL is provided for DOS, Windows, MacOS, Solaris and other UNIX platforms. A large number of special applications developed by the PDS nodes are also available. 6. Software Standard Features The PDS standards architecture, including the recommended data formats and the meta-data structures, are considered standard. The processing of meta-data, such as data type interpretation, is also considered standard. However software itself is simply provided as a service. 7. Non-developer Software Many non-PDS developers have developed software to parse PDS labels and access the data. For example, there are many IDL read procedures for PDS labeled data. 8. User Support The PDS maintains the standards architecture documentation. Several software libraries and application are also maintained by the PDS. The PDS also provides data engineering support for the design and development of archive products. The PDS support the entire planetary science community, including most solar system exploration missions with a planetary component. 9. Work In Progress The PDS continues to update its standards architecture, upgrade its data access infrastructure, and is currently focusing on streamlining the data ingestion process. 10. Evolution Plans The PDS would like to provide more user-friendly access to PDS datasets by developing Java-based front-ends that can utilize existing cross-platform code resources (like the object access library). This applies not just to data inspection tools but also to tools for data archive production and validation. It is also planned to convert existing libraries to Java format. The PDS would also like to explore the layer between the object access layer and the application layer, middle-ware, to insulate changes and promote interoperability. 11. Documentation and Related References http://pds.jpl.nasa.gov/ - PDS Home Page http://pds.jpl.nasa.gov/prepare.html - PDS documentation - links to Standards Ref, Data Dictionary, and Data Preparation Workbook http://pdsproto.jpl.nasa.gov/DDColStdVal/newdd/Top.CFM - Data Dictionary - DB Interface 12. Other Comments [Editors Note: This question was added after form was originally completed.]
Comment on this template in the HyperNews Discussion.

Wider Views Formats Evolution Process (FEP) Discussion Forums Page Formats Evolution Process (FEP) Home Page NASA/Science Office of Standards and Technology (NOST) Home Page

URL: http://ssdoo.gsfc.nasa.gov/nost/fep/developer-pds.html A service of NOST at NSSDC. Access statistics for this web are available. Comments and suggestions are always welcome. Author: Steve Hughes / NASA JPL (Steve.Hughes@jpl.nasa.gov) +1.818.354.9338 Curator: John Garrett (John.Garrett@gsfc.nasa.gov) +1.301.286.3575 NASA Official: Code 633.2 / Don Sawyer (Don.Sawyer@gsfc.nasa.gov) +1.301.286.2748 Last Revised: 1999-03-24, Steve Hughes (1999-08-24, John Garrett)