ISO Archiving Standards - Ninth US Workshop- Minutes

The Ninth US Workshop on Data Archive Standards was held at the National Archives and Records Administration's (NARA) Archives II Complex in College Park, MD on September 30 - October 1, 1997. The list of attendees is available at: http://ssdoo.gsfc.nasa.gov/nost/isoas/us09/participants.html.

Action Items
Materials Distributed
Future Meetings
Discussion Items

• Agenda
• Activities Since Last US Meeting
  • Society of American Archivists
  • Research Libraries Group
  • CEOS Archive Task Team
• Current Work Plan
• Summary of Changes to Reference Model
• Detailed Discussion of Reference Model
  • Section 1 - Introduction
  • Section 2 - OAIS Concepts
  • Section 3 - OAIS Responsibilities
  • Section 4 - Detailed Models
  • Section 5 - Migration Perspectives
  • Section 6 - Archive Classifications
  • Section 7 - Illustrative Scenario
  • Annex A - Scenarios of Existing Archives
  • Annex B - Migration of Archives
  • Annex C - Compatability with Other Standards
  • Annex D - Brief Guide to OMT

Action Items

Materials Distributed

Future Meetings

Discussion Items

Agenda

Activities Since Last US Meeting

Society of American Archivists (SAA)

• Bruce Ambacher and Don Sawyer attended this meeting in Chicago and gave papers
• Meeting seen as interesting with over 1000 participants consisting of corporate people, agencies, museums, academia, volunteers.
• Impression that community is, as a whole, just getting their feet wet with electronic forms of data despite considerable experience at some archives such as NARA
• Digital libraries appear to be the major archival thrust for forming views on preservation of electronic forms
• Statement made that there is no difference between archives and digital libraries when it comes to digital information

Research Libraries Group (RLG)

• This group had a booth at above SAA meeting
• Don Sawyer has not yet again contacted Nancy Elkington
• Scott Roley/NARA was present at the SAA and has since invited Don Sawyer to present at the NAGARA meeting in July. The proposed session is "Distributed Archives: Models for Standards, Licensing, and Oversight" and an RLG perspective will also be given in this session.
• Bruce Ambacher handed out a copy of a draft RLG work plan which showed areas in which we have interest and should try to coordinate with
• One such activity relates to a Recommended Practices guide (which we are considering as a New Work Item.)
• Mike Martin has worked with RLG as part of his PDS effort. He has not felt they have accomplished much; probably running into copyright restrictions

CEOS ATT

• Don Sawyer developed draft charts for Claude Huc to present at CEOS Archive Task Team in September
• Claude since reported that they do want to work with us and have a strong action to review the OAIS document

Status of US08 Action Items

OPEN ACTION LIST SUMMARY - U.S. DA Workshops (as of Sept 30 1997)

AI #	Description	Actionee	Date	Status	Comments TBD Review of WB changes Develop work plans for new work Write memo to RLG regarding new work plans
U/9708/02	Check meaning of Migration in "Preservation" Paper	All	TBD	Open
U/9708/03	Rewrite text re Physical Migration per discussion	Mike Martin	TBD	Open	some of this is included in new text
PREVIOUS ACTIONS
U/9707/01	Follow up with Elkington re RLG activities (url http://www.rlg.org/preserve)	Don Sawyer	TBD	Open	Don has yet to get back to Nancy
U/9707/03	Contact Gerry Gibbon regarding Cross Reference material	Don Sawyer	TBD	Open	Gerry has not responded to Don's E-Mail
U/9707/06	Analyze and report on SOMO data Archive study	Mike Martin	TBD	Open
U/9707/07	Upgrade OAIS classification descriptions as per SOMO survey	Don Sawyer Mike Martin	TBD	Open
U/9707/16	Develop Concept paper on differentiating federated archive vs a distributed archive	Paul Grunberger	TBD	Open
U/9707/17	Develop NARA scenario on special collection	Bruce Ambacher	TBD	Open

Current Work Plan

Summary of Changes to Reference Model (Item 3)

• Don Sawyer produced and distributed an upgraded document
• He itemized the areas still OPEN as found in the 'Dear Reader' note of version 1.2

• Don Sawyer then reviewed the revised Table of Contents and indicated areas where the major changes have been made:

• Section 1: Don Sawyer noted there were changes in the Purpose and Scope and hoped this section was settling down

• Section 2: Sections 2.1 and 2.2, which involved moving the environment figure forward and improving readability of information model discussions

• Section 3, now "Responsibilities", has had a number of changes in 3.1, 3.3, 3.4 and 3.5. Section 3.4 is completely rewritten.

• Section 4.1: Mike Martin commented on his update of this section:
  • He noted that most of Paul Grunberger's flows are still there
  • Paul Grunberger has further modified the Matrix
  • Mike Martin has tried to make his nomenclature more consistent with that used by the rest of the document
  • Paul Grunberger feels he still has some orphan flows which need to be searched out

• Section 4.2
  • Don Sawyer noted that the Information Model organization is basically all new
  • There are still some questions as to whether this is the right organization
  • Changes in model feed back to affect terminology in the various sections
  • Lou Reich plans to further smooth the writing

• Section 5 - Migration
  • The special writing session addressed mainly Migration
  • Randy Davis has rewritten this section
• Section 6-Randy Davis has rewritten the scenario to be more generic and complete
• Annex A-There are new scenarios: one from Life Sciences and one from CNES on their archive

Since this material had only been put on the web the previous night, no one had read it. Therefore, a reading session was held with a focus on sections 1.1, 2.1, 2.2, 4.2, and 4.1.

Detailed Discussion of Reference Model - White Book-1.2 (Item 3)

Section 1 - Introduction

• Randy Davis to work on introductory text: He wants to identify why each of the three terms (Open, Archival, Information) appear in the OAIS title
• #4 bullet is proposed for updating as "provides a basis for comparing data models of digital information preserved by archives and for discussing how data models and the underlying information may change over time."
• Alan Wood's mailed comments were: On reading the current Purpose and Scope I think that things have gotten less clear in terms of distinguishing the role of the reference model for digital information. The current version reads:

  "provides a starting point that may be expanded by other efforts to cover long-term preservation of information that is NOT in digital form (e.g., physical media, physical samples);"

  As contrasted to the earlier, preferable statement in the Purpose and Scope of the Oct. 25, 1996 version

  "Its focus is on the preservation of information in digital form, but it also accommodates information that is inherently non-digital (e.g., a physical sample). However, the modeling and preservation of non-digital information is not addressed in detail in this reference model"

• There were other lesser comments which Don Sawyer captured which he will consider for update of section 1

Section 2 - OAIS Concepts

Sections 2.1 and 2.2

• Paul Grunberger felt 2.2.1 and 2.2.1.1 should be at the same level, along with material immediately under 2.1
• Mike Martin sugested that Fig 2-2 is vague and the OAIS term "More meaningful Information" should use the OAIS term "Content Information."
• Don Sawyer said this was to be about information in general, so it could apply to all the information objects, and should be written in that manner
• Paul Grunberger felt the text should include a qualifying sentence to this affect
• Randy Davis would like to see the distinction among: data, knowledge, and information
• There were different opinions as to the meaning of the words and discussion followed
• Don Sawyer feels there is no useful information without a knowledge entity to understand the information
• Paul Grunberger feared that too much treatment would bog down the reader in this area
• Randy Davis liked (data + representation) plus representation.... equals Information object. He would show in some manner the recursion aspect
• Randy Davis would like to be able to express all this as an equation
• Don Sawyer indicated that some Representation Data in itself may need other Representation Data
• Suggestion made to include a more extensive example consisting of ASCII data in a table.
• Information will be introduced with a short description relating to knowledge exchange to bring out the importance of the 'knowledge base' that a person or entity needs to be able to understand any information.
• More Meaningful Information will be replace with just 'Information'.

Sect 2.2.1

• Don Sawyer is trying to convey that there is a finiteness to every data submission, a beginning and an end, which motivates the use of 'information package'. Find a better way to say this.
• Randy Davis did not like Content Information and Context Information used so close together, but we did not see viable alternatives given that Information Preservation paper terms are widely used now.
• Mike Martin expressed concern about Catalog Information being dropped (discussed later)
• In Preservation paper, their Context Information included our Packaging Information. We pulled out the packaging information to keep the Preservation Description Information more inviolate for some types of migrations
• Proposal is to number sections 2.2.1, 2.2.2 and 2.2.3
• Proposal is that 2.2.1.3 become 2.2.3 - Information Package Classes.
• Is fig 2-4 clear? There was extended discussion. The notion of 'specialization' was not clear.
• Betty Brinker: Can the DIP for one archive be a SIP for another archive? - YES
• Suggestion made to not have diagram 2-4 and treat the subject in text
• Don Sawyer to revise text and delete figure 2-4
• Lou Reich feels Preservation Desc. Information is too perfused through the document
• Don Sawyer stated PDI information categories are important at a high level and need to be in section 2. Packaging Information concept is also very important and was missing before. This is needed to help understand what bits can be changed easily and which can not as Information moves thru system.
• Lou Reich: If it is NOT in the AIP one can change it. If it is in the AIP, one cannot change it. He want the Packaging Information outside the Package (IP)
• The SIP needs external information to assist in archiving the information
• Randy Davis showed diagram: Give you two CD-ROMs (as one AIP) and files need to be stored on different media. There may be an interconnection between the two at the representation level. (note added in editing: inside the archive, Claude has modeled this situation by identifying a 'Storage' object.)
• Lou Reich: This is intrinsic to a file system which keeps track of all this activity
• Don Sawyer: All this "Packaging Information (PI)" relates to more than information re packaging; it deals with, among other things, Migration.
• Lou Reich: take this out of the IP and he will agree with all this.
• We may need another entity containing PI and AIP
• Lou Reich: it is wrong to have PI inside the AIP; this is packaging information that is not to be preserved
• Don Sawyer This information is related intimately with the AIP - not necessarily remote from the AIP information
• The question was asked, "THIS IS BRAND NEW STUFF - ARE WE ALLOWED TO INTRODUCE NEW STUFF?"
• Lou Reich feels PI is as important as any other information
• Randy Davis: Agree that packaging info is important in SIP and DIP; but disagreement exists as to its presence in the AIP
• Packaging Info for one function may be viewed as Representation Info in another area
• Agree that IPs are stored in one place while PI may get either stored somewhere else or even discarded and its function implemented differently
• Lou Reich would agree to have PI as a separate, separable part of SIP and DIP and optional as to having it as part of the AIP
• Don Sawyer believes that PI is just as important within the archive, and within the Storage function, as it always exists in some form and needs to be clearly distinguished from CI and PDI by the archive.
• Mike Martin: there is some type of PI that is transient and other that needs to be preserved.
• Don Sawyer Example: VMS files have several types of information attributes and some one has to determine which information is Content Information and PDI, and what is not and therefore does not need to be preserved. The RM must help make this type of distinction clear or implementations will mush these together
• Randy Davis: do away with concept of Information Package
• Is the notion clear re the information to be archived?
• Lou Reich feels that transient information (like the PI) should be outside the IPs
• Randy Davis:Is serious about concept of no IP. He showed and explained:

                                         AIC
                                     /         \
                                 AIU             Rep
                               /     \
                         Content      \
                        /    \         \
                     Data   Rep Info    PDI
  

  (editor's note: It turned out that there was confusion on the relations of IP to SIP, AIP, DIP, AIU, and AIC. That AIP was a superclass for AIU and AIC was not clear. The notion of SIPs, AIPs, and DIPs inheriting from IP was not clear.)

• Lou Reich felt this is very much like figure 4-7
• Lou Reich still feels we should call SIPs and DIPs "volumes' rather that products and treat PI as something NOT part of Information Packages
• Don Sawyer feels PI must be treated as an important Object somewhere
• Paul Grunberger saw the problem as one of semantics. We need to invent names for Content Info and Rep Info as immutable objects and the transient items as something else

Section 3 - Detailed Models

Section 4 - Migration Perspectives

• After reviewing attempt to reorganize this to work better with Int'l actions affecting section 2 reorganization, a decision was made to try to merge and reorganize material in 4.2.1 and 4.2.2 with minimum refernce to Section 2.
• Don Sawyer: What is dividing line between 4.2.2 and 4.2.3?
• Lou Reich: Sect 4.2.2 concentrates on information objects to be stored- CI, preservation info., etc., while 4.2.3 talks more of archive operations. One builds on an information model while the other builds on the activity assuming the model exists
• Some inconsistencies in style noted by Randy Davis (Bold, capitalize, etc)
• Mike Martin did not like basic "currency" phrase
• Lou Reich will combine 4.2.1 and 4.2.2
• Don Sawyer feels this will require a bit more text explaining the Information Object figure - digital/physical split, digital bit sequences, representation recursion.
• Lou Reich feels this has been treated in section 2, more might be viewed as redundant. He also indicated that he plans some changes in the flow of the text, including following the IO with the taxonomy view, which was seen as smoother by the group
• Mike Martin felt more treatment of the Transfer Info Package was advisable
• Don Sawyer asked what was the significance of the word Transfer since all IPs get transferred somewhere
• The following differentiation among AIP, AIU and AIC terms was offered
• -Originally, the AIP was to be the basic storage (atomic) item
• -The Int'ls preferred AIU as the "unit" to be stored, not liking to view any package as an atomic unit. So AIU was created
• -AIC is a collection of AIUs
• -AIP is a sort of short-hand term that can represent either.
• Randy Davis: more description is needed about Information Object
• Randy Davis: need example of physical object and example of digital object
• Lou Reich: feels this will all be resolved in the revised write-up
• Randy Davis suggested that this text should be interspersed with scenarios
• Randy Davis asked for more explanation as to just what constitutes a digital object. (Is a CD-ROM full of files one, or many, digital objects. The answer given was, "depends on how these are addressed."
• Content information is just a special case of an information object
• Randy Davis again mentioned the advantage of giving examples for these figures
• Lou Reich will move figures 4-5 and 4-8 and modify figures 4-6 and 4-7
• page 4-18: Need to make more clear just what is Representation Information
• Mike Martin: In Representation Information section, the text seems repetitive
• Randy Davis will rework Representation Information section
• Need to write description of differentiation between physical-digital and physical-containing digital terms and decide where it should fit
• Need a mapping from SIP to DIP
• Need to Update Migration Text
• Agreed to drop concept of TIP superclass

• Randy Davis asked how rigid are these requirements to be observed, since not all documentation will contain all these materials
• Lou Reich feels now these are objectives and with time will become more observed in practice. We may eventually even have more categories
• It was noted that these are shown as subtypes in one place and in another as container objects.
• Lou Reich stated that perhaps the subtypes should be Preservation Description IOs
• Add "objects" to Figure 4-10

• Discussion on Catalog Information in PDI followed

• Catalog Information may be generated and collected later, eventually becoming part of Context, or Reference, or Content Information, or a separate package to be pointed to. Provenance would need to be updated. Does every change have to go back through Ingest?
• The same thing could be said about derived products, but how do these point backward to source products?
• Lou Reich would like to break down Packaging Info into Delimitation and Identification.
• Randy Davis to write Scenario on Cataloguing, Software and Derived Products

• There was additional discussion re Physical vs digital objects
• Randy Davis: Re software - he used to feel we need to archive software but has changed his mind. It should be ancillary information.
• Don Sawyer: S/W could be part of Provenance if it includes the transformation algorithm relating to a previous product. It may be a part of Context, or referenced by Context, if it is provided as a 'possible aid to access'.
• Don Sawyer believes that a query for CI or PDI, in an object sense, must logically go thru PI information and its associated methods.
• Lou Reich You don't query the AIP directly, you query the container object and let the system decide how to respond

  o There was extended discussion about the meaning, significance, etc of all the structures being introduced in this section. Generally, the inheritance diagrams were not understood. A mistake in Figure 4-16, where AIP should have been AIU, compounded the problem.

• The extra model shown is ONLY within the Archive
• Lou Reich will rework this entire area
• To Lou Reich, PI is not contained, as is PDI and other entities
• Randy Davis: All this aggregation of AIPs into AIUs/AICs is not too clear and wondered if it were necessary
• In Fig 4-16, this should be an AIU
• Randy Davis feels we need to ellucidate an overall architecture of the archive OR get a more complete description of the activity
• Need to build up to Figures 4-17 and 4-18
• Randy Davis would like to see the term AIP disappear and just talk about AIUs and AICs.
• Don Sawyer likes the term AIP because it allows one to talk about common features of the 'package to be preserved' without getting into units and collections of units. This is convenient to give an overview such as is needed and given in section 2.
• Don Sawyer views that the producer and the Archive negotiate what is to be in a SIP, and how these are to be submitted. Multiple SIPs (in some form) provide all this necessary information, but any one SIP may contain only parts of required categories (e.g., CI, Rep. Info, PDI). It was agreed that we should 'finesse' the information modeling issue of how a SIP really inherited from an IP to avoid too much complexity.
• John Garrett would like to break the SIP into two parts, the Content Information and the PI
• Randy Davis: In BNF: Info Package ::= Content Info[PDI] or PDI
• It was agreed to retain IP, but look to drop AIP.
• It was agreed to try PI outside the IP, but with reluctance by some overcome by the need to allow the review to proceed.

• Lou Reich: Send comments re section 4.3 to Lou Reich by Oct 3 (end of week)
• Lou Reich: Send revised section 4.3 to members for comments (These two activities may be done in parallel)
• Lou Reich suggested we split chapter 4 into two sections, between Functional Model and Information Model. We may want the Information Model to come first, but then some functional model terms need to be introduced earlier.

OAIS Version 2.0

• Don Sawyer noted that we need to put the next-version WB on the Web by mid-Oct so the P2 international people can have a chance to review it for discussion during the Oct 27-29 DA meeting at ESRIN.

• Mike Martin to upgrade Functions material, with Paul Grunberger edits considered
• Paul Grunberger to provide electronic edits by 3 Oct.

Section (New) - Federated Archives

• There was discussion as just what Federated Archives meant to help Paul Grunberger start writing
• Paul Grunberger should cover the level of autonomy of each node
• Is the management breakdown different in Federated versus Distributed?
• Lou Reich: Currently accepted perceptions of the practice are most likely not correct
• Is there a single centralized management or a single decentralized management among the nodes?
• If you have a local catalog schema in a federated environment, can one reach the catalog from a remote location or another node
• Paul Grunberger to make statements re nature of federation and how we can model it

Section 5 - Archive Classifications

Section 6 - Archive Classifications

• Don Sawyer said we agreed to look at each class and provide rationalization as to why this particular classification is important (if it is).
• Lou Reich doesn't mind going with the current classification text " as is"
• Mike Martin discussed expanding the utility of each type of archive classification. He feels there will not be any great differentiating significances among the classifications we have now.
• Mike Martin to upgrade classification section

Section 7 - Illustrative Scenarios

• No comments at this time.

Annex A - Scenarios of Existing Archives

• No comments at this time.

Annex B - Migration Issues

• This Annex is to be dropped since we believe the important concepts are now in the migration section
• Lou Reich did not feel transformation text in migration section covered enough cases
• Lou Reich will send comments to Randy Davis by end of week

Annex C - Compatability with Other Standards

• Lou Reich: Delete current text but keep the annex with new text.

Annex D - Brief Guide to OMT

• This is still needed.

The Workshop was adjourned at 1700 hours, Wednesday, October 1, 1997.

Wider Views