Digital Archive Directions (DADs) Workshop

(A part of the ISO Archiving Workshop Series)


Position Paper

Digital Archive Directions (DADs) Workshop

DATE: June 22-26, 1998

HOST: The National Archives and Records Administration
Archives II
8601 Adelphi Road
College Park, MD 20740-6001



1. Identification of Proposed Topic [Required]

1.1 Title

Needed: An standard for digital video master files.

1.2 Contributor(s)

Grace Agnew, Assistant Director for Systems and Technical Services, Georgia Tech Library

1.3 Description of Proposed Project

The Southeastern University Regional Association is the organizing body for Internet-2 (OC3 and higher) internet initiatives in the Southeast. Internet-2 connectivity involves regional gigaPOP locations connected throughout the Southeast, known at the Southern Crossroads (SoX). Many bandwidth-intensive applications requiring Internet-2 connectivity involve the sharing of video files at 30 fps and higher. A committee has been formed--the SoX Digital Video Initiative--to define regional standards and practices for highly-functional, highly-interoperable digital video for content sharing within the SURA region. As a librarian and member of the committee, my focus is on content--primarly the large video collections held by academic and research libraries in the Southeast. The Sox Digital Video Initiative will be co-hosting a workshop with the Southeastern Library Network (SOLINET) to identify and evaluate video collections for scope, content, condition and use, to develop robust practices and identify encoding standards that will serve the needs of these collections. This workshop will also be an opportunity to collaborate with the collection managers in the Southeastern University Libraries to determine practices and standards that are easily applied and that meet the archival concerns for fidelity to the origial and persistence across time.

1.4 Justification

One clear need that has already emerged is the need for an encoding standard that can serve as a digital master, which can be further encoded and edited for streaming service to patrons at differing bandwidths using differing client software and plug-ins. A digital master standard needs to support: standard playback functionalities, including random access, fast forward/reverse, audio and video synchronization, keyframe or timecode searches, robustness to errors, robust encoding and decoding supporting many hardware and software platforms and editability. In addition, for wide adoption, efficient compression for afforable storage is very critical. Most video encoding at the moment is video encoded for immediate use. Long-term storage and viability have not been a general consideration because digital video as a format is very new and very volatile. New streaming codecs appear and disappear rapidly to compete in the Internet streaming video market and as more is learned about efficient compression and streaming. Digital video has not been created on a large scale, and the heaviest use of digital video has been commercial (music video, advertising, news, etc.) and not archival collections with long-term value, so the issue of long-term storage and display has not arisen. Now that libraries have embraced digital still images and addressed issues of digitizing, indexing and display, they are ready to address their audio and video collections. Unfrotunately, there is no corpus of knowledge yet available addressing digital video in libraries to provide guidance. Libraries are making choices for video encoding based on available hardware and software and recommendations from vendors, such as video server vendors or based on current bandwidth principles, rather than guideing principles for persistence, accuracy and viability of the digital object. Issues to consider in selecting an archival standard are (1) viability and robustness of the standard as defined above; and (2) fidelity to the original. Selecting a standard that is both viable and robust poses the most difficulty. Some suggestions are: (1) Uncompressed D1. This is not a standard, but instead represents uncompressed digital video, frequently direct from a digital source, such as a digital camera, imported to the workstation through firewire. The difficulty with uncompressed D1 is that proprietary firewire technology may be required, D1 is not easily displayed or accessed, the storage requirements are immense, most firewire technologies are currently proprietary with uncertain futures, and the cost of uncompressed D1 is beyond the reach of most libraries. (2) According to Nels Johnson of *Digital Video* magazine, many commercial users saving advertising clips, music video, etc. use uncompressed AVI, since this digital video file format can be created by standard encoding hardware and software and easily displayed through a variety of hardware and software options. The problem is that storage overhead for longer videos is heavy and Microsoft intends to replace AVI with the ASF file format. M-JPEG is another possiblity, since the intraframe compression retains good fidelity to the original and is most probably more recoverable in case of minimal file corruption than interframe compression standards, and is provided by many standard encoding hardware and software platforms. M-JPEG is not an adopted standard, however. The MPEG encoding standards use intraframe and interframe compression, but primarily interframe, using prediction and interpolation algorithms, which could conceivably be problematic when trying to restore a slightly corrupted file. For example, a static image, such as a tree, which is predicted across frames and utilizes pointers to the frame with the original image requires that this original frame is not corrupted over time for accurate playback to continue. MPEG formats have been somewhat proprietary in the past, and currently the newer "editable" MPEG formats remain somewhat proprietary in their editing capabilities. The MPEG formats are robust standards, however, with a clear development path and a regulatory body. The problem, in a nutshell, is that a relatively "safe", robust encoding standard that is robust, versatile and readily accessible is adopted, before significant video digitization occurs in library collections, to insure that fragile videos do not need to be re-encoded and to insure persistence and widespread availability of signoficant video collections via the world wide web. Re-encoding, to serve different purposes, clients, and bandwidths, should occur from the digital archival master.

1.5 Definitions of Concepts and Special Terms

Motion Picture Experts Group. MPEG standards are video and audio encoding standards intended to address robustness, fidelity to the original, flexible, feature-rich playback, and, most-importantly, hardware and software independent, in that any vendor can provide MPEG encoding capabilities.
Motion Joint Photographic Experts Group. Encoding of video as a series of still images with compression within the image. High storage, slow playback, but fairly lossless.
Hardware and software technology to import uncompressed video into a computer for editing, playback, etc. Used with digital video cameras to import digital video in an efficient manner into microcomputers, so that high bandwidth requirements of uncompressed video do not overwhelm the processing capabilities of the computer

1.6 Expected Relationship with OAIS Reference Model

An archival master for digital video would touch many areas of the OAIS, but primarily storage,delivery, access, and migration of information across media, particularly in re-encoded digital video streamed via the web to users.



2. Scope of Proposed Standard [Desired]

2.1 Recommended Scope of Standard

2.2 Existing Practice in Area of Proposed Standard

2.3 Expected Stability of Proposed Standard with Respect to Current and Potential Technological Advances

Wider Views

Overview of the DADs Workshop
Overview of US Effort
Overview of International Effort


A service of NOST at NSSDC. Access statistics for this web are available. Comments and suggestion are always welcome.

Author: Grace Agnew (Grace Agnew@The Org) +1.The Phone
Curator: John Garrett ( +1.301.286.3575
Responsible Official: Code 633.2 / Don Sawyer ( +1.301.286.2748
Last Revised: May , 1998, Grace Agnew (May 26, 1998, John Garrett)