ISO Archiving Standards - Fifth International Workshop - Minutes

Present:

BNSC	David Giaretta	DG
	Wyn Cudlip*	WC
	Peter Greenbank	PG
CNES	Patrick Mazal	PM
	Denis Minguillon	DM
	Claude Huc	CH
ESA	Nestor Peccia	NP
	Gian Maria Pinna	GMP
	Mario Merri	MM
	Brad Lowe	BL
NASA	Don Sawyer	DS
	Lou Reich	LR
	John Garrett	JGG
	Alan Wood*	AW
CEOS/ATT	Lars Alm*	LA
CEOS/PTT	Richard Goebel*	LA

Table of Contents

• WP700 - Archiving
• Status Report
• Review Status of Action Items (DS)
• Liaisons
• OAIS Promotion Report
• Review of Reference Model White Book, Issue 2
• Additional Archiving Standards
• Archive Workshop Schedules
• WP700 Status Report
• OAIS Promotion Report
• Registered Documents
• Action Item List

WP700 - Archiving

Status Report

White Book 2.1 should be out in Jan/Feb 1998. Need to get wide review in order to be in a position to submit it as an ISO Draft International Standard (DIS) after the May International Workshop.

Two additional Work Packages were defined at the Silver Spring International Workshop:

• WP720 - Submission Standards
• WP730 - Recommended Practices

We must decide on whether these are the correct items.

Review Status of action items (DS)

The status of the archive activity action items is as shown in the Action Item List.

Liaisons

Don Sawyer has presented Reference Model at Society of American Archivists annual meeting.

There will be another presentation early next year - the Research Library Group (RLG) will also attend. It is important to try to co-ordinate with RLG on this work. They have expresed an interest in the Reference Model. It is worth noting that Archivists have a greater volume of information and financial weight than the space agencies and they are taking a lead in solving digital archiving problems.

Commercial groupings (Open Geographic Information Systems) OGIS have related their concepts to OAIS terms in a draft RFP - this is encouraging. Don Sawyer/Lou Reich will send updated versions.

P/9710/40

700

Send updated versions of OAIS RM to OGIS contacts

Don Sawyer

980215

Open

Lou Reich: The Reference Model is mature enough now to present to any other bodies.

Gian Maria Pinna: outside bodies will expect Annex D to be completed - comparison with other Reference Models.

David Giaretta reported that people concerned with archiving of cultural information had expressed interest during the UK workshop. It was suggested that it may be useful to give the OAIS presentation at some conferences on cultural heritage, early next year.

Lars Alm had heard about OAIS Reference Model work from CH at a workshop presentation. During the second meeting held in Stresa (Italy) some ideas on the scope of the ATT were discussed. The OAIS Reference Model version 1.0 was distributed for comments and the idea to adopt it was discussed. The CEOS recommendation for an EO archive model could be an implementation of the OAIS.

This implementation will be based on the concept of Base-Band information - not yet defined - on which the archive will be based. It will be above the raw data format, with all transformations reversible, like Level-1A. Their first thoughts were to try to define a level of processing into which other archives would convert their holdings for interchange.

Furthermore, CEOS ATT is interested in defining an interchangeable standard format between archives - starting with Requirements/Guidelines. For example

• sending tape or other media between archives
• sending info from ground stations to processing station
• user access to data and metadata

Between CEOS archives Base-Band information will be interchanged

Gian Maria Pinna: we should involve industry in this so that commercial ground stations could support this base-band information and interchange format directly.

Nestor Peccia: satellites using CCSDS P1 services - packet telemetry - will go on to use Panel 3 SLE services for interchange from ground stations to processing centre. ESA Earth Observation satellites do not yet use CCSDS Packet Telemetry.

Several strategies were suggested

• CEOS-ATT will define Base-Band content and an interchange format - then promote using P2 methods (EAST etc) to convert individual archives' own formats for interchange. This assumes the archives can be persuaded to do that.
• Use Panel 2 methods to describe and register individual formats to allow them to be interchanged in native format but converted and used by the client (no Base-Band defined).

OAIS Promotion Report

ESA will distribute the Reference Model to others in ESA after February because the second half of the book is not mature enough.

David Giaretta reported on the UK workshop and Symposium.

Review of Reference Model White Book, Issue 2

Lou Reich presented Section 4.2 & 4.3 in some detail, pointing out significant issues along the way.

• There is still a problem with the presentation of OMT diagrams - they are introduced in Section 4 but they are needed earlier in the document.
• The AIU/AIP change needs discussion
• More discussion is needed on the representation model
• The Access function needs more work

Note that in the OMT diagrams the relationship line label text above the line should be read left to right, text below the line should be read right to left.

• Do we need more discussion on SIP's and DIP's?
• Virtual collections is immature
• Data Management diagram - insufficient explanatory text?

Detailed comments

• Make it clear that we are using an "OMT-like" diagram technique - NOT OMT but instead OMT with slight extensions, and document additions to OMT being used.
• Consider breaking Section 4 into 2 or more sections
• Section 1.1, first para - clarification of the term OAIS needed:
  • Sentence 2, "An OAIS is a type of archive..." TO "An OAIS is an archive..." and clarify.
• Section 1.2
  • clarify "shorter-term" archives for first bullet
    • Here we are referring to shorter-term in terms of elapse time - the argument is that when measured by technology change they may be effectively "long-term"
  • Make first sentence say words to the effect that this model MAY be applicable to any archive.
• Section 1.6.2:
  • define shorter-term
    • Expand rational - moving some of the concepts from Applicability - see if "shorter-term" is still needed
  • Administration: change "moment by moment" TO "day to day"
  • remove term Active Archive - only used in section 6.
  • CLIENT - add "see also Consumer"
  • Define "Catalog Information" - or remove it by rephrasing
• Section 2
  • Para 4: add sentence (or new para) about ACCESS.
  • Definition of "OPEN": it should be made clearer earlier in the document that the OPEN in OAIS does not refer to free-access.
    • Add words to Purpose and Scope to explain this.
• Section 2.1:
  • "oversight" may be read as "ignored" - change the word e.g. control
• Section 2.2.1
  • Change Whose has TO who has
  • "converts those characters" TO "are converted " - third & fourth paragraphs
  • change to the recipients knowledge base TO using the recipients knowledge base
• 2.2.2
  • 2nd last para, change last sentence
  • Fig 2.3 : leave as is - no need to add extra boxes
• 2.2.3:
  • add example of SIP without CI
  • draw the distinction between Packaging and Packaging Information i.e not all Packaging Information need be passed in each SIP. The same change applies throughout the document.
• Change DERIVED AIP to DIP sections 3.1 and 3.6
• Fig 2.5 - should queries etc be shown as Data Objects? in 4-19 where they are shown as data flows.
  • Agree to change queries, results sets and orders to data flows i.e. remove bubbles. (GMP suggested this)
• Move 2.3.3 so that Management activity is first (i.e. becomes 2.3.1)
• 2.3.2:
  • Access Aids - explain they may cover information not yet in the archive e.g. operates on Mission Planning information. GMP says 70% of requests are for future aquisitions.
  • Remove MUST from 2/3 down ad-hoc request
• 2.3.3: add "as applicable" when referring to pricing policies

• Section 3
  • Remove "in forms understandable to those communities" from last bullet. Also re-write section 3.6 to concentrate on making info available.
• Section 3.1: last 3 paragraphs should be removed or cut down as they repeat material in section 2.
• 3.2: GMP: does the SIP depend upon the designated community?
  • Change the emphasis to DIP's going to the consumer
• 3.3:
  • change title - FROM "independently usable" TO "independently understandable"
  • combine last 2 paragraphs
• 3.4
  • 3rd para: change last sentence - replace reference to ...hardcopy... TO reference to users knowledge-base
• 3.5: Clarification needed on Copyright issues
• 3.6:
  • refocus on "making info available".
  • Last sentence - change "will" to "may"
  • 2nd para from end - re-write to clarify "approval of the Consumer". Also emphasise that this is about copies of the AIP for Digital data.

• Section 4
  • Need overview section about the various models
• 4.1
  • Add diagrams showing functions - as per NP comments

• Lines from Administration should be dashed instead of gray
• Number all bullet points for easier reference throughout the subsections

• 4.1.1 Common Services
  • Expand
  • Explicitly mention that a lot of real flows are not shown in the diagrams. Only logical flows are shown in the Data Flow Diagrams.

• 4.1.2 Ingest
  • The agreements on submission should be done via Administration, not Ingest
  • Under REVIEW - move higher level review part (not instance by instance) to Administration in order to avoid a linkage between Ingest and Dissemination.
  • Transfer Request - make arrow labels consistent for storage request with transfer request in 4.1.3
• 4.1.3 Archival Storage
  • Address concept of levels of maximum bit-error rates tolerated

(restart 30/1/97)

• Decide on whether AIU/AIC need be in section 4.1 or whether they can be replaced at this point by simply AIP; in order to use AIC etc we must have explained the concept e.g. in the Information Model. See later discussion.
• Hierarchy Management:
  • Usage statistics should be clarified - statistics internally maintained by Hierarchy Management
• Error checking - add example of Error Correction e.g. Reed -Solomon done in addition to any error correction intrinsic to the storage hardware.

• Fig 4-2 : need to make consistent with the text - various small errors
• 4.1.4 Data Management
  • Move bullet "Data Base Administration" to the front - because functions in other bullets require this.
  • Move Handling of Subscription orders to DATA MANAGEMENT
• 4.1.5 Administration
  • Add admin review function to Admin - discussed above
  • Add monitoring function for Designated Community needs and acting on those needs e.g. to know that GIF's are no longer supported by easily available software so a data description is needed - otherwise long-term usage cannot be maintained. This is also related to Migration. These may need to be split into separate bullets.
• 4.1.6 Access
  • Change Prepare Finding Aids TO Provide Finding Aids
  • Improve examples of Finding Aids - current list is to specific/limited
  • Remove dissemination of on-line DIP's from ACCESS - move it all to DISSEMINATION. Currently the diagram shows on-line delivery through ACCESS. ACCESS deals with Metadata, not data.
  • Consider replacing the term ACCESS with some other term e.g. LOCATING or IDENTIFYING&ORDERING.
• 4.1.7 Dissemination
  • GMP: should order confirmation and such customer service be in Dissemination?
  • Add line Order Confirmation from DISSEMINATION to ACCESS and move confirmation to ACCESS.
  • Links between ACCESS and DISSEMINATION should go via DATA MANAGEMENT because the various activities must be recorded within DATA MANAGEMENT.
  • Subscription requests are sent to DATA MANAGEMENT that then generates dissemination requests based on schedule.
  • Delivery function:
    • change "prepare DIP functions" TO "generate DIP functions"
    • change "bills of lading and other shipping records" TO "shipping records"
• 4.2 Information Model
• 4.2.1 Background Concepts
  • change "basic currency" TO ????? - find a word or phrase
  • explain that labels above the association lines are read left to right and the other way for labels below the line
  • change 1st sentence 3rd para. "The CI is that information ..." TO "The CI is the primary information of interest." Then add introduction to DIP, AIP, SIP - focus for AIP is preservation etc.
  • Remove duplication of material from Scenarios: OneSIP-One SIP etc
  • Resolve TBS on figures - references to sections in doc.
  • Fig 4-7: swap boxes DIP and SIP so it reads better from left to right
• 4.2.2 Logical Model for Archival Information
  • 2nd para change "a taxonomy of Information Objects" TO "a taxonomy of those Information Objects" in order to not exclude the possibility of other Information Objects.
• 4.2.2.2 Representation Information
  • Fig 4.9:
    • add a label to the line between Structure Layer and Semantic layer "ADDS MEANING" - below the line.
    • Change aggregation to specialisation
  • Change "Such software methods" TO "Such methods"
  • Drop "As an aside, it is clear...practically achievable".
  • Change sentence "In principle, this recursion continues until..." TO "This recursion should continue until the designated user community's knowledge base is adequate to understand the representation" (or some better sentence).
  • Improve example, add figure and explain Recursion chain.
  • Change "found to be quite temporary" TO "found to be ephemeral"
  • Change ""and the physics meaning" TO "and the physical meaning" or better phrase.
• 4.2.2.3 Preservation Description Information
  • Fig 4-10: not consistent with 4-12, specialisation in one, aggregation in the other. Delete Fig 4-10.
  • State that PDI contains the following types of information : Reference etc.
  • In description of Provenance Info: remove last sentence "This information may be thought of as a special case of Context Information described below"
  • Last paragraph, 1st sentence: define what "it" refers to
• 4.2.2.4 Packaging Info
  • Last sentence: change "must" to some kind of advisory
  • Change to be consistent with 2.2.2 change
• 4.2.3 Logical Model of Information in an OAIS
  • 1st sentence - no verb - correct this
  • 2nd para: remove
• 4.2.3.1 AIP
  • Need to define Package Descriptor before Fig 4-11, or put pointer to definition before Fig.
  • 5th para - change "must more" to "more"
• 4.2.3.2 - actually labeled as 4.2.3.1 repeated
  • change "Archive Information" to "Archival Information"

• 4.2.3.2.1
  • change "unique ordering number" TO " other information including unique ordering identifier ..."
  • change "digital image of the film" TO "digital representation of the film"
  • change "JPEG" to "MPEG"
  • add example show finding aid for "future products"
  • Data-Mining discussion needs reworking. It raises several new issues because it requires access to the large portions of the archival data. Various ideas were discussed:-
    • may need to blur the line between Data and Metadata so that data itself can be queried
    • may need to add line between Data Management and Archival Storage to allow the creation of new metadata
    • in 4.1.6, Prepare Finding Aids - should probably pass request for specialised queries to DATA MANAGEMENT which then passes it to DISSEMINATION, which actually carries out the processing and returns info back via DATA MANAGEMENT.
    • DG suggests changing labels as follows in order to get to the heart of the disagreements:
      • ACCESS change to METADATA ACCESS & PROCESSING
      • DATA MANAGEMENT change to DATA MANAGEMENT & METADATA STORAGE
      • DISSEMINATION change to DATA ACCESS & PROCESSING
    • Lou Reich to re-draft taking these points of view into account
    • Make it clear within the model how new Metadata is created
    • Clarify what the lines in the Functional Model actually mean - ensure that they cannot be misinterpreted by readers as unrestricted data flow.
    • Add figure showing taxonomy of Access Methods
    • Figure 4-15:
      • Add "1+" to lines to indicate where associations are not just 1 to 1.
      • Consider changing "Access METHOD" - methods should not be separate class - should be part of a class.
    • 2nd para sentence beginning "A single AIP can be included" - should be moved or expanded
• 4.2.3.2.3 Collection Descriptions
  • "Collection Description" is not an AIP specialisation and so should not be a subsection
  • Needs to be clarified/re-worked to highlight the fact that "Virtual Collections" (or as was proposed "Temporary Collections") do not necessary carry PDI, due to their "virtual" (or "temporary") existence.
  • The introduction of Temporary Collections enhances the Access function in providing more powerful Finding Aids. Other Finding Aids can be applied to Temporary Collections.
• Fig 4-17
  • AIC to AIP line - should have "1+" label instead of filled circle
  • AIP to Member Description - label with filled circle Member Description end
  • Various other errors in diagram - needs careful checking
• Fig 4-18 and related paragraph
  • Make this a separate subsection of 4.2

• 4.3 High Level Data Flows and Transformations
  • Consider changing diagram 4-19 following one by DG - mapping more easily to Functional Model.
  • Change CAP in fig 4-19 to AIC
  • Last para: justify or delete the second half.
  • Throughout - remove "scientific" where inappropriate
• 4.3.1
  • Update last sentence to include packaging description etc
• 4.3.2
  • last para.: change to something like "Due to the lack of suitable hardware and/or operating system the producer media may not be available indefinitely.
• 4.3.5
  • Make consistent with earlier changes: 1st para request agreement should say that this is negotiated by Administration
  • Clarify or remove "self-describing"
  • Last sentence Change "newly granularity" TO "new granularity"

• 5 Migration Perspectives
  • Rework this section on the basis of when and under what circumstances the Representation, Packaging and Preservation Info change. Include aspects of changes arising from changes in the designated community, changes in technology, media lifetime, archive organisation etc.

Depending upon the outcome of the above action, the following points may be necessary.

• Rearrange/shorten text that introduces each major topic: replication etc.

• 5.1 Replication
  • Add example in Replication
  • Define Replication to include the concept that after replication no changes are required to the h/w or s/w in order to read the information.

• 6 Archive Classifications
  • This section may perhaps be dropped if there is no time for adequate treatment. Alternatively move to an Annex.
  • Needs further elaboration - add rationale of why this is significant
  • Address GMP concern about "Active" and "Final" archive: needs to discriminate on the basis of input flow, e.g. use "retrospective"
  • Remove unnecessary use of "scientific"

• 7 Illustrative Scenarios
  • Provenance and Ref examples must correspond to the CT scenario being used
  • Remove paragraphs "When a SIP is ingested .." and "A SIP is typically..."
  • Remove encrypt & decrypt from FIXITY
  • In the CT scenario there is a need to specify the Designated Community and also how the submission agreement is set up etc.
  • Clarify what the content information is in the CT scenario

• ANNEX B
  • No specific changes noted, except that it should be a section of the model, before illustrative scenario, NOT an Annex.
• ANNEX C
  • May be useful to identify areas of standardisation, but perhaps will not appear in the final document.

Additional Archiving Standards

WP720 - "Archiving Submission Standards" is in the Management Plan.

There may be some relationship with the Base-Band interchange work mentioned by LA. Lou Reich noted that there is a difference between Archive-to-Archive interchange and Archive-to-User interchange. Also the P2 view is to describe what people give you, rather than specifying the one and only format to use. However P2 should be able to contribute to the type of metadata which should be included in any interchange.

Don Sawyer noted that NASA would probably want P2 to draft REQUIREMENTS and SCOPE for this work that would then be put out as an RFP, to try to bring in external effort.

WP730 - "Archiving Recommended Practices" - perhaps the name of the Blue Book planned should be changed - not "Recommended Practices". This is planned to be modeled on ISO9000.

Lou Reich expressed concern that we must attempt to be inclusive, and not omit certain areas simply because the proponents of e.g. STORAGE, have already dropped out of the OAIS development process. He suggests a large workshop to try to draw in others, and especially vendors. Developing standard interfaces to e.g. STORAGE would allow one to switch HSM systems for example, with minimal disruption. DG pointed out that he had presented considerations like this in Munich in 1996.

Wyn Cudlip suggested that CEOS be involved making it a joint workshop. Working title "Areas for Potential Standardisation for Open Archival Information Systems"

Lou Reich proposes that a planning committee draw up proposals for an this - either an International Meeting or parallel US and European meetings (1 or 2 days). Maybe attach to a meeting with another body - e.g. CEOS. There is a joint NASA/IEEE Mass Storage workshop in Spring 1998 that should be considered.

Gian Maria Pinna: should focus on Standards Developers rather than implementors e.g. IEEE Storage, Metadata, etc.

P/9710/43

700

Lead e-mail discussions to draw up work plan and plan for Workshop(s) for Archival Standards. Initially aim for something in April - needs plans completed in December.

Don Sawyer

971215

Open

Archive Workshop Schedules

Editors Note: After the meeting the following schedule for the OAIS RM was suggested by JGG:

White Book (Issue 2.1)	13 January 1998
Comments by US and other interested reviewers	22 January 1998
White Book (Issue 3)	1 April 1998
Agency Comments	15 April 1998
Review Book and Comments at US WS	April 1998
Review Book and Comments at IWS	6-15 May 1998
Draft Red Book (Issue 1)	1 June 1998
Agency Comments	15 June 1998
Red Book (Issue 1)	29 June 1998

P/9710/44	700	Raise the OAIS related work at Management Council. Suggest that agencies not already involved may wish to identify appropriate resources to participate in the OAIS discussions.	David Giaretta	971113	Open
P/9710/45	700	Panel to raise the importance of OAIS and suggest that they identify appropriate projects/individuals from other agencies to participate in OAIS discussions	All	980430	Open

WP700 Status Report

The active work packages in Archiving are WP710, WP720, and WP730.

A. Progress

NASA held two US archiving workshops since the fourth ISO/CCSDS archiving session in Silver Spring. Results of this workshop and US workshops have been put on the WEB.

White Book version 2 of the reference model, which has incorporated most of the Silver Spring agreed updates, was produced following the September US workshop. This version has been registered for the Silver Spring workshop. Also registered was a related paper giving the status of the Silver Spring requested updates.

B. Changes

No changes in direction are identified.

C. Problems

No unusual problems are noted.

D. Forecast

We should be able to get a further significant improvement for WB-2.1 that could be out in January or February 1998. We all need to seek wide, informal, review so that we can look to a solid version in May 1998. The May 1998 version should become an ISO Draft International Standard (DIS) and simultaneously, a CCSDS Red Book. It is possible that we may have to produce this version following the May workshop, which may mean a slip of a month or two. This remains to be seen.

We should also define one or more work packages for additional archive standardization.

E. Milestone Table

WP #	Description	Mgmt. Plan Completion Date	Status	Projected Completion Date
710.2	Draft Archiving Reference Model	96.09.01	Closed	96.10.28
710.3	Draft ISO Committee Reference Model	97.05.31	Closed	97.05.31
710.4	Draft ISO Standard Reference Model	98.05.31	Open

A. Progress

Will assess this during the workshop.

B. Changes

None to date.

C. Problems

None to date

D. Forecast

We need to decide if this is sufficient priority that resources can be found to begin the effort.

E. Milestone Table

WP #	Description	Mgmt. Plan Completion Date	Status	Projected Completion Date

A. Progress

Will assess this during the workshop.

B. Changes

None to date.

C. Problems

None to date

D. Forecast

We need to decide if this is sufficient priority that resources can be found to begin the effort.

E. Milestone Table

WP #	Description	Mgmt. Plan Completion Date	Status	Projected Completion Date

OAIS Promotion Report

Registered Documents

Wider Views