ISO Archiving Standards - Fourteenth US Workshop- Minutes

• Agenda
• DADs Workshop Report
• Status of Goldin's Letter
• Status of Action Items from International Meeting
• OAIS Work Remaining
• What Else Needs To Be Done
• Emulation

Action Items

Materials Distributed

Future Meetings

Discussion Items

Agenda

DADs Workshop Report

• Don Sawyer reported that the group is getting close to being finished. Bruce, Jerry and Don Sawyer have some more work to do
• The report still needs an Executive Summary which Don Sawyer plans to writ before the end of the year
• There are some gaps in Bruce’s area which he needs to fill in; some papers in this area are missing
• Lou Reich feels it is currently too long being over 70 pages. He suggests it be divided into two sections: a Summary section and a Detail section containing the details of the working groups.
• Don Sawyer would like to get it out to committee by Christmas
Action Item:

01

Identify "holes" in his area and attempt to develop some material to fill in these areas

Bruce Ambacher

1999-01-21

 

• No one has approached NSF re providing funding for future DA work
• Plan some PR activity to stimulate interest before scheduling another workshop
• Need a 10 Chart overview of the DADS symposium, can be derived from the Executive Summary
• If we got additional money, we would hold an invitational workshop (a-la—DADS-2)
• Symposiums are expensive
• OR we can undertake development of some of the standards as identified under DADS-1
• It probably too late for us to give a presentation at the March IEEE meeting on Mass Data Storage Reference Model
• Need interaction between vendor community and government
• Don Sawyer mentioned the use of electron-beam etching on a stable media like gold has been reported
• Also reported is an optical monitoring of the error detection results to track how the media is degrading
• Propose a long-term preservation issues theme for the next (entire) IEEE conference

Status of Goldin's Letter

• It appears that the letter has not yet been sent to Executive Office, but, instead it has gone to NARA for concurrence.
• Bruce provided a copy of Nara’s "rewrite" of the letter (Item 1). NARA wants to the letter to show a joint thrust by NASA and Nara’s and that the two agencies are planning to establish a joint working group to develop a plan on how to proceed.
• Bruce did not know what has happened and will check
• Don Sawyer reported that Ben had made some improvements in our response but this has not yet gone forward. It had been felt that Haley’s response was too narrowly focussed.
• Our response would go to GSFC management and then, if approved, it to go to Lee Holcomb the Chief Information Officer at NASA Hq.
• Code 600 suggests a proposal to Holcomb on what should be done and also would request accompanying funding
• Mike Martin felt we should develop several proposals to present. This is especially true since six months have passed since the letter was first written. He volunteered to provide a few approaches to present to GSFC manage
• Bob to determine from Lee as to what drove the letter’s being written in the first place, its current status and suggest a meeting of some of us with Lee.
• We want to develop a proposed insert in the draft letter. The words finally agreed to were: Add at the end of the fourth paragraph: "….throughout the Government. This effort can build on current, relevant, agency and interagency efforts."

Status of Action Items from International Workshop

There had been a list of action items with accompanying dates for completion which were reviewed. They included:

• AI #34 Produce Don Sawyer-O-gram
• This is a composite of diagrams. Don Sawyer hopes to have this done by year’s end. Meanwhile Paul Grunberger will attempt to put the revised version is an electronic form. Nestor had made a composite diagram of data and report flows and a copy was FAXed to Mike Martin at JPL
• Some of the minutes’ comments are reflected in this diagram.

 

• AI #35 Distribute material for Section 5
• This has not yet been done; hopefully some comments can be developed during this meeting.

 

• AI #36 Distribute Material for section 4.1
• It was agreed to have material out by Jan. 4 for a Jan 26 to discuss Sections 4 & 5 and DEDSL.
• Bob is to arrange Jan 26 Telecon

 

• AI #37 Distribute material for Section 4.2 for Telecon on Feb 16

• There was no change in this action item

• AI #38 Distribute material for Appendices for telecon on March 16
• There was no change in this action item

 

• AI #39/AI #40 Distribute Whole OAIS/Submit RIDs
• A date of April 01 was set for this distribution. Three weeks were allowed to process RIDs which will be due on April 26. Assuming no major changes, the OAIS Ref Model will be progressed to CCSDS (Red Book) and (ISO DIS) in May/June.

OAIS Work Remaining

Don Sawyer noted that at the IWS, things went well although there had been many comments re the OAIS Reference Model. Most of these were editorial in nature with no big technical problems remaining. Those pertaining to Section 4.1 were reviewed in detail. Some had already been accepted and changes made in the handout (Item 4). (These were not discussed.) Where issues or lack of clarity were present, the items were addressed during this section. The basis for this discussion were Mike Martin's responses to the comments (Item 4). Some of this discussion follows:

• We need more discussion re "Designated Community" in the text and should show it in the diagrams
• We need more detail on Common Services.
• Lou Reich will sent Mike Martin some material from a GIL publication
• Mike Martin is to expand the text to half-page. It should not be too long
• Fig 4-2: Show data flows from QA box to Administration and to Receive Submission; show "QA Results"
• Fig 4-5 Delete word "Agreement" in "Audit Submission Agreement"
• Lou Reich talked about the functions of routine and periodic QA reviews (separate peer review)
• Look at Migration paragraph. Next to last sentence: Mike Martin was asked to make a softer sentence but keep the sense of a warning exists. Refer readers to section 5
• Comments re Figure 4.3 refer to Disaster Recovery. A higher level of error correction, such as Reed-Solomon, can be included
• In Fig 4-4, raise Administer Database to a higher and add line between it and Administrator. Change text re policy like security
• Add box re Physical Security to Fig 4-5 with a connection to Management (for policy).
• Updating designated community: DG wants box entitled Management system configuration. Add bubble Monitor Designated Community with unlabeled connection to Consumer and connection to Manage System Configuration
• Mike Martin will write appropriate text
• In Fig 4-6, add connection between Provide Access Control and Consumer; label with request/response
• Need to show DIP on Consumer side as a primary flow
• Mike Martin is not bothered by a minimum treatment of DIP
• Change Dissemination Request to Order Request
• Propose that Coordinate Access Activities does not respond directly to Consumer
• Change Dissemination request to Order Request (for standing orders) everywhere
• On Fig 4-6 change single line Response to two lines: Result Sets/Report and DIPs
• We generate Result Set and Deliver Result Set
• Everything comes back to Generate Response
• Mike Martin and Paul Grunberger will complete the text and change the diagrams respectively per the meeting discussions

What Else Needs To Be Done

• Some things still undone in Sections 1 and 2
• Coordinate Fig 2-5 as a function of what has been said during this meeting
• Need to complete section 4.1 by Jan 4 (a concept review – not technical changes) for discussion at Jan 26 Teleconference. This is to serve as a check; it will not be included in the OAIS RM itself.
• Need work in the area of Terminology; text to be written in an
• Annex; relationship with other terms
• Section 5
• Annex

Emulation

• Dr Rothenberg had written an article in which he stated that he feels Migration won’t work and the only way to go is Emulation
• Mike Martin and Lou Reich don’t agree with Rothenberg’s conclusions, but it will be necessary to review his study results in more detail
• It is noted that it is extremely difficult to write software to develop an emulator that is usable across platforms. It might be necessary when there are large volumes of data involved
• Mike Martin has distributed his comments
• Lou Reich was in agreement that one should consider the costs of emulation as compared to the value of the data
• We need some cost trade studies and these are missing in the Rothenberg article-Lou Reich will talk to some companies and determine the resources required to do emulation such as Virtual PC and Soft PC.
• Lou Reich will also contact Steve Hughes re PDS experience with XML XML-Don Sawyer had developed some thoughts about Emulation (Item 6) and requested comments
• Mike Martin thinks it is too deep
• Emulation is only a viable approach when the original source code documentation is non-existent and even then, it may not be possible as well as not practical
• Some reorganization of paper suggested
• Bruce to approach CLIR for an advanced copy of Rothenberg paper
• Emphasis is on display since this seems to be the primary emphasis of the storage community
• We need to include treatment of Display in any comments we make re the Rothenberg paper
• Need comments re both the software which tells you what is going on and the software that does it.
• High Level Language does not describe (it masks) all that it is going on within the software
• Extracting data from a file and then using other software to store it again is Transformation
• Emulation will only work in very selected instances
• Our goal is to put something about Emulation into the Ref Model and to have an basic understanding of Emulation.
• One important fact is that one can’t rely of even consecutive versions of a software to work; how much more difficult it is to emulate the operating system
• One problem in data archiving is to validate that data content has not changed when different software/software-versions are used
• Concept of emulation defeats the purpose of archiving
• From Reference Model, if we don’t have information as to what the data look like, we will have major problems a long time down the road
• Re Security, we must preserve the encrypting program or not store encrypted data
• In the Reference Model, we need to talk about the following alternatives:
• Representation/Emulation
• Software Evolution and versions having backward compatibility
• Evaluation of each new version - acceptance of version envelope (how do you know this is true)
• Verification
• When software is no longer evolving, we may be faced with emulation
• Levels of emulation include:
• Bit stream example vs typical vendor interface that requires a significant knowledge of operating system
• Maintaining interface between data and programs
• Standard interfaces: if we have standard hardware platform, emulation would be simpler. Complex platforms would lose the ability to do simple things simply
• Emulation is a point to point solution
• Standard interfaces have to be well document and can still be emulated as the interface changes, but is this approach feasible (Only way to access the data)
• For standard simple emulation
• full emulation
• mothballing hardware and systems which run on it (example would be old tape recorders
• Software to be backwards compatibility
• Simple standard interface
• Plan to send our position to Holsworth for comments
• Standard interface are like calls
• Standard interfaces allow program to use the same instruction/call to get data
• Binary softcode: if you have document, attempt to reconstitute the software, recompile existing software in the new environment
• Software reconstitution is a subcase of Emulation
• JAVA is running on multiple applications. There are lots of system-to-system problems relative to user interface, like single click/double click mouses
• JAVA transportability is awesome
• JAVA is a good example of generating a standard interface
• a standard interface gets wider acceptance and it will receive wider support
• If I use Use JAVA to build my application it will become a good source for my data access
• JAVA means to include source code; it comes with software documentation
• If one analyzes a JAVA case, he finds it is an information rich, highly capable interchange format
• JAVA comes with about 200 pages of source code documentation
• If one had adequate documentation on software, you have adequate information on the data representation
• When trying to recover data from a storage media, get into so many complications with formats, calibration data and things multiplexed into the data stream
• How do we organize the Reference Model to bring in these other aspects? Are they at the same level as Migration? Don Sawyer felt they are NOT; Lou Reich felt they were.
• In the document we need about 5 pages for Migration, one page each on mothballing and Emulation
• We are looking to get enough information together for people to make their choice among:
• Migration
• Transformation
• Replacement
• Emulation
• Reference Model is to look into the future
• How compatible is something with present things: how future- oriented is it
• We should anticipated technology developments
• The Reference Model is an archeology for data systems - Do we need section re Data Archeology?
• Software must be both evolutionary and backwards compatible
• A standard interface to platforms is needed
• Software for access and preservation (understand what the information is)
• Alternatives for Emulation
• Simple environment with standard interfaces (bit stream service)
• More complicated (like JAVA)
• More complex, not standard
• Mothballing systems and software that runs on them
• Need to evaluate these alternatives for data archiving
• We place Major reliance on Transformation
• Mike Martin feels we should add a typical problem with each consideration to expose each option
• Not one that would work with all
• Not one that is probably not possible
• Try to keep examples small
• Lou Reich suggested we take the same example each time but vary assumptions to apply to different types of storage concerns

Wider Views