NSSDC/WDC-SI 2001-01

2000 ANNUAL STATISTICS AND HIGHLIGHTS REPORT FOR THE NATIONAL SPACE SCIENCE DATA CENTER

Joseph H. King
National Space Science Data Center
Greenbelt, Maryland 20771
 

Table of Contents

PREFACE

1. INTRODUCTION

2. HIGHLIGHTS

3. DATA MANAGED AT NSSDC, AND 2000 INFLOW AND OUTFLOW

3.1. Data Inflow

3.2. Data Outflow

4. ADDITIONAL NSSDC SERVICES

4.1. NSSDC Information Systems

4.2. NASA/Science Office of Standards and Technology (NOST)

Glossary

Tables 1 - 17

Appendix - Selected 2000 NSSDC-Acknowledged Papers

2000 Annual Report of the National Space Science Data Center

PREFACE

The National Space Science Data Center is pleased to issue this 2000 Annual Report describing (1) the 2000 growth and evolution of NSSDC's data archives, access pathways, and other tools and services, and (2) the 2000 access to those data and services by NSSDC's customer communities. This report has been made WWW-accessible in the hope that readers will avail themselves of the opportunity to link to the services reported herein.

The scope of this report is that of the traditional NSSDC as defined by the NSSDC budget. It should be noted that some of the activities thereby supported are the responsibilities of the Astrophysics and Space Physics Data Facilities, organizational peers of the formal NSSDC within Goddard's Space Science Data Operations Office.

I welcome suggestions for user-benefiting improvements to this Annual Report and to NSSDC services.

Joseph H. King
Head, National Space Science Data Center

1. INTRODUCTION

This report characterizes NSSDC's data holdings, metadata holdings, access pathways, and value-added data products, tools, and services at the end of 2000, with a focus on the 2000 activities leading to that end-of-year state. In addition, this report characterizes the nature and amount of 2000 access to NSSDC's data and services by its many users from various communities..
 
 

2. HIGHLIGHTS

 

The most important result of NSSDC’s 2000 activities is the continuing preservation of growing space science data volumes, ensuring their continuing and future accessibility to the space science, education, and general public communities.  The statistics to follow reveal that NSSDC’s archive has now grown to 17.0 TB of space science data and an additional 3.3 TB of Earth science data.  During 2000, 1.7 TB of data were added to the NSSDC archive which now holds data from over 1400 experiments flown on 420 spacecraft.

Next, NSSDC continues to distribute large amounts of data by network to the space science community and general public, and by offline mailings to the general public. Again, following statistics detail the data volumes disseminated via various pathways to various communities.  We note here that during 2000, NSSDC’s customers downloaded via network just over 2.6 million data files (a 30% increase over 1999!) and received about 2.6 TB of data on mailed media.

NSSDC’s data dissemination is leading to the publication of significant new science.  The Appendix of this Annual Report lists 65 science papers acknowledging NSSDC data or services as contributing to their analyses.  These are papers which have come to the attention of our staff.  Most science journals in which NSSDC data or services may have been used are not reviewed by our staff routinely, so the list represents a lower limit on papers enabled or benefitted by NSSDC.

The year 2000 marked the start of multiple data management advances at NSSDC.  Archive Information Packages (bundles of data files and companion attribute files as prescribed by the ISO/CCSDS Archive Reference model) were defined, created and written to DLT jukebox thereby initiating a gradual transition of NSSDC’s permanent digital archive from offline shelves to nearline jukeboxes.

At the same time the AIPs’ constituent data and attribute files are being written to a new unix-based RAID magnetic disk environment (cf. ftp://nssdcftp.gsfc.nasa.gov/.  Data previously ftp-accessible from other NSSDC systems are also being written to the same location.  The first data being packaged as AIPs are the space physics data and IRAS data on NSSDC’s VMS-based optical disk jukeboxes (NDADS) that are about to be retired, and data newly arriving at NSSDC.  Details on these data management changes are given in the web-accessible NSSDC Newsletter (December, 2000; http://nssdc.gsfc.nasa.gov/nssdc_news/).

NSSDC developed a new family of graphical-display-and-subset interfaces for selected ftp-accessible ASCII and gzipped-ASCII data sets that are awaiting possible conversion to Common Data Format for support by CDAWeb.  This family, called Ftphelper, aids in the data preview and selection process.  See http://nssdc.gsfc.nasa.gov/ftphelper/.

Through its NASA/Science Office of Standards and Technology (NOST), NSSDC provides international leadership in providing increasingly effective digital data archiving. During 2000, the NOST-led Formats Evolution Process Committee prepared and released for community review a paper proposing a possible time series object that could lead to improvements in the science community's ability to understand and process time series data. Also, there was growing worldwide acceptance of the NOST-led, ISO/CCSDS-sponsored Archive Reference model and use of this model in refashioning NSSDC's own data management approach as mentioned above.

There was high-level interest in 2000 in the recovery and digitization of data from the Labelled Release biology experiment flown to Mars on the two Viking Lander spacecraft. NSSDC played a key role, along with the Planetary Data System's Geosciences node and with two members of the original science team, in effecting this. Much keypunching was done from NSSDC-held microfilmed numeric listings and from newly-provided numeric listings on paper. This newly digitally available data set is supporting new "Life on Mars?" analyses.

Two other notable data rescue efforts occurred in 2000. First, a unique IMP 8 magnetotail electron data set available only as numeric listings (and plots) on microfilm was scanned and passed through Optical Character Recognition. The resulting ASCII files were manually cleaned and made ftp-accessible. Also, NSSDC began a systematic conversion of its film archive to computer-accessible TIFF format with the scanning of Magellan 10" x 10" film (Venus imagery) and Mariner-series 70-mm film (Venus and Mars imagery).

NSSDC's Astronomical Data Center (ADC), chartered to acquire, manage, and provide access to astronomical source catalogs, saw monthly accesses to its catalog and visualization servers of 32,000 and 5200, respectively. The ADC created the first online repository of XML data (http://xml.gsfc.nasa.gov/guides/) and remains at the forefront of XML development as applied to astrophysical and astronomical data.

NSSDC provided leadership to the NASA Sun Earth Connection (SEC) Education Forum (SECEF), sponsored by the NASA SEC enterprise of the Office of Space Science. In particular, SECEF played key roles in year-2000 creations of (1) a new web-accessible space science education resource directory and (2) a new "Making Sun-Earth Conceptions" CD-ROM with various sections oriented to various subsets of the K-12 population.

Readers are encouraged to exercise the multiple options on the hierarchical array of WWW pages starting with NSSDC's home page. There are several more functionalities beyond those called out in the preceding paragraphs.

 

3. DATA MANAGED AT NSSDC, AND 2000 INFLOW AND OUTFLOW

There are several ways to characterize the multi-disciplinary NSSDC archive. Byte counts are a common metric for modern archives, and will be reported herein. Numbers and diversity of media volumes managed, and numbers of distinct data sets, are also very important. (In NSSDC's terminology, a data set is typically all the data from a given instrument at a given processing level in a given format.) The diversity of data sets and of media types relate to the intellectual heterogeneity and technical heterogeneity of the archive, respectively, and we shall report on these also.

At the end of 2000, NSSDC had 4,582 distinct data sets and accompanying documentation packages being managed. Table 1 indicates the disciplines from which these data sets come, and a breakout as to whether the data sets are digital or non-digital (film, etc.). The table shows that these data sets come from 1,416 experiments which have flown on 422 mostly-NASA spacecraft. By data set count, space physics is the dominant discipline, accounting for over half NSSDC's data sets. This reflects the fact that in its early years, NASA launched a preponderance of space physics missions and also that space physics spacecraft typically carry more independent experiments than do astrophysics missions.

Note from the table that NSSDC manages almost as many non-digital (mostly film) data sets as digital data sets, although it should be noted that NSSDC has been acquiring almost no non-digital data in recent years and has been gradually converting parts of its film archive to a digital form.

Table 2 is a different characterization of the NSSDC archive, by byte counts and media volume counts. The table shows 20.3 TB of total data, a 3.3 TB subset which is network-accessible, and 75,767 digital media at NSSDC. The byte counts are estimates, involving for some data sets assumptions about the mean numbers of bytes on various media types.

Note that only astrophysics data and space physics data are network-accessible from NSSDC. That planetary data are not network-accessible is the result of the Planetary Data System's making most or all its planetary data network accessible and because of the emphasis of the PDS (through which virtually all planetary mission data has entered the NSSDC archive for the past several years) on CD-ROM production and dissemination. [Exceptions are NSSDC's photo gallery and image catalog which are WWW-accessible from http://nssdc.gsfc.nasa.gov/planetary/; these are largely oriented towards the general public.] NSSDC's Earth science data are gradually being migrated to the EOSDIS data management infrastructure, hence are not good candidates for network-accessibility from NSSDC.

Tables 3a and 3b better characterize NSSDC's network-accessible astrophysics and space physics data, by project and by whether the data are immediately accessible from magnetic disk (online) or from the robotics-based NDADS system (nearline).

This annual report will be the last one to report on NDADS, which will be retired during 2001 in favor of a data access system based on RAID magnetic disk in a unix environment. Most NDADS-resident space physics data, plus NDADS' long wavelength IRAS astrophysics data, are being migrated to the RAID system. Most NDADS-resident astrophysics data will be made network-accessible from other NASA-supported entities.

Table 4 characterizes the digital media managed at NSSDC, not including back up copies. This table is an expansion of Table 2 in which total numbers of unique digital media volumes were given. It should be noted that most volumes are replicable and have one backup volume. However, for "CD-ROM (Titles)" which are not locally replicable, NSSDC typically holds between 20 and 200 copies of each title. For these, NSSDC must replenish stock through a commercial vendor as request activity drives NSSDC stock down. DLT and DVD are expected to become increasingly important at NSSDC.

Table 5 characterizes NSSDC's non-digital archive, by disciplines by form factor. Note that NSSDC has large volumes of non-digital data for each of the discipline areas it supports. It should be noted, however, that very little new data have been arriving at NSSDC in non-digital form in recent years. NSSDC has recently begun an effort to systematically convert this film archive to computer-readable form.

 

3.1 DATA INFLOW

Tables 6 and 7 characterize the inflow of digital data to NSSDC during 2000. In particular, Table 6 shows that NSSDC received approximately 1.7 TB of new data in 2000, via a combination of networks and hard media. Table 6 shows data volumes by project, with the astrophysics and space physics subsets of ISTP/Wind and Ulysses data attributed to their respective disciplines. Dominating the counts are the media-based Level-0 data from the FAST and ISTP missions plus data from the Mars Global Surveyor and EUVE missions. Table 7 characterizes the inflowing media types by discipline. CD-WO media are the dominant input media type overall.

During 2000, NSSDC received approximately 240 GB of data electronically, in addition to the data arriving on the media reported in Table 7. The electronic inflow was dominated by the ISIS ionogram digitization effort (84 GB) and by ISTP and IMAGE data, with lesser amounts from a number of spaceflight projects.

By data set count, which as noted earlier marks the intellectual heterogeneity of NSSDC, 109 distinct and new data sets from 20 flight experiments arrived at NSSDC during 2000.

3.2 DATA OUTFLOW

NSSDC provides user access to its data holdings through multiple electronic interfaces and, in addition, through a user support infrastructure for the mailing of offline digital and non-digital data volumes.  Most electronic interfaces are accessible through NSSDC's WWW home page and include:  (1) special WWW-based interfaces to specific data sets or groups thereof; (2) anonymous/ftp pathways to a range of data files maintained permanently on NSSDC magnetic disk as well as files staged from NSSDC's nearline mass storage environment (NDADS); (3) WWW-based and other interfaces to NDADS (e.g., SPyCAT) for the purpose of selecting data files and having them staged or downloaded. Note that NDADS will be retired during 2001.

The dominant special WWW-based data access interfaces that NSSDC offers to the research community relate to: ISTP key parameter and a growing range of other space physics data (CDAWeb); the OMNI and uniformized-COHO solar wind datasets (through OMNIWeb and COHOWeb, respectively); various atmospheric and ionospheric data (ATMOWeb); IRAS, COBE and SWAS; and the Astronomical Data Center astronomical source catalogs and journal tables. CDAWeb and the below-addressed SSCWeb system are joint services of NSSDC and the Space Physics Data Facility.

The OMNI data set is a 37+ year compilation of multi-spacecraft near-earth solar wind magnetic field and plasma data and energetic particle data, while the COHOWeb database is a uniformized set of files of merged magnetic field, plasma, and position data for each of many deep space spacecraft. Table 8a shows annual statistics for the CDAWeb, OMNIWeb, COHOWeb and ATMOWeb systems. Note the continuing steady growth in usage of these systems. In 2000 they were used by NSSDC's customers to produce over 400 plots, listings, and data files every working day.

Table 8b reports statistics on the usage of NSSDC's executable geophysical models services and its services for magnetospheric (via SSCWeb) and heliospheric orbits. These services have been available for some years now. The models service lets users specify a model, a spatial point of interest, and any other parameters on which the model depends, and have the model parameters computed at the point or along a profile through the point. Table 8b shows that there were about 52,000 such computations done by NSSDC customers in 2000, with geomagnetic, ionospheric and atmospheric models dominating. Ftp access to modelsí software is included in ftp access statistics in Table 9, not in Table 8b.

Table 8b also reveals over 28,000 orbit computations, a 38% increase over 1999. Of these, about 78% use the primarily magnetospheric SSCWeb service and the balance use the Heliocentric Ephemerides page.

A great many NSSDC data sets and other information services are held permanently on magnetic disk for anonymous/ftp access mainly from NSSDC's unix computers. The reader is invited to review all these services from the ftp link on NSSDC home page. Table 9 gives the annual counts of files downloaded, both overall (nearly 2.3 million files in 2000, up by 35% from 1999) and for selected directories with high activity. Note that the Photo Gallery, of high public interest, dominates the statistics. The researcher-downloading of 76,000 data files from the spacecraft_data subdirectory shows the high interest in and great value of this service. These counts do not include data files ftp-downloaded after having been staged from NSSDC's NDADS nearline system; relevant statistics on NDADS access are discussed in the following paragraphs.

Table 10 summarizes the 2000 accesses to NSSDC's nearline mass storage environment (NDADS) through the multiplicity of available interfaces. Note that three measures are offered for each mission supported. A request is the specification of one or multiple "entry ID's" each of which specifies one or more data files. Both request counts and counts of files are reported. Finally as a measure of the breadth of interest in the data from any given mission, we count on a monthly basis, and report annual means of these monthly counts, the numbers of unique electronic addresses (username @ site) accessing each mission's data.

NDADS hosted a totality of 5,568 requests in 2000, for a total of 109 thousand data files. These numbers are lower than in 1999, as most astrophysics data are now accessible from elsewhere and as NSSDC's magnetic disk holdings of space physics data grow. In particular, IUE data has ceased to be the dominant beneficiary of NSSDC's nearline environment as the STScI/MAST service became fully operational during 1999. Broadly based communities accessed IMP 8, ISTP and IRAS data from NDADS.

WWW access statistics are frequently misleading, insofar as they usually individually count the many files (buttons, etc.) that make up a page. Nevertheless, growth in WWW accesses is indicative of continuing and growing use of the WWW-provided services. In 2000, there were 11.2 million hits monthly to NSSDC's two main www servers, nssdc and bolero. This is up by 18% over 1999!

While the dominant mode of dissemination of data to the astrophysics and space physics research communities is via the internet, NSSDC continues to provide a high level of offline data dissemination. Table 11 shows that NSSDC responded to almost 1,200 distinct requests for "significant" products plus (by footnote) 2,700 poster requests. Table 11 also characterizes the user community of NSSDC's offline services. To a very large extent it is the U.S. and international general public, the education enterprise, publishers, etc. and their desire for NASA imagery on CD-ROM and as film products that account for most of NSSDC's offline request activity.

Table 12 gives the counts of requests for offline data sets from various disciplines in 2000, and as integrated over NSSDC's history. (A small fraction of requests which are multi-disciplinary are double counted in this table.) Note particularly the dominance of planetary data over both time scales. This is largely associated with lunar and planetary image data which are widely requested by the general public. The high level of astrophysical offline activity to a large extent reflects requests by the amateur and professional astronomical communities for ADC catalogs on CD-ROM. Most offline space physics request activity was for copies of the IMAGE-based "Solar Storms" video tape.

Table 13 shows the most recent 5-year history of NSSDC's offline data request activity by media type. Several points are noteworthy. The dominant mode of offline digital data dissemination continues to be by CD-ROM. It is of interest to note that every working day of 2000, NSSDC mailed about 18 CD-ROMs to 3 requesters. These numbers are down by 30% from 1999 as more members of the general public are able to access NSSDCís data electronically. Note that magnetic tape dissemination fell to zero in 2000 and is unlikely to be reported on in future years.

Also significant in Table 13 is the fact that requests to NSSDC for film data have not declined over the past 5 years. The number of film products mailed was up significantly in 2000. NSSDC finished one film request every working day.

Finally, very noteworthy is the distribution of over 2,700 copies of the Multi-wavelength Milky Way poster or COBE poster created by the SSDOO/Astrophysics Data Facility as special education/outreach products. These distributions are not included in the counts of Tables 11 and 12.

 

4. ADDITIONAL NSSDC SERVICES

In addition to its archive of scientific data characterized in the preceding part of this Annual Report, NSSDC offers a number of additional services which are described in this Section.

4.1 NSSDC Information Systems

The Automated Internal Management (AIM) database identifies virtually all launched spacecraft, the experiments carried by many of those spacecraft, and data sets primarily as archived at NSSDC. Table 14 identifies the numbers of spacecraft, experiments, and data sets described in the AIM File, along with the numbers of new entries made in 2000. The database serves as the source of information for many of NSSDC's WWW information pages. The NSSDC Master Catalog (NMC) and a number of discipline project pages retrieve information from AIM and build WWW pages "on the fly" so that the latest information is presented to the user. During 2000, there were approximately 137,000 monthly web accesses to the NMC by the public, comparable to the prior year's access rate.

The AIM database also serves as a useful source of comparative information about spacecraft, experiments, etc. such as launch dates, time spans of data, orbital characteristics, time resolution, data availability, etc. The presence of comparative information in a consistent format within a single database makes the database especially valuable for survey reports.

The NSSDC Supplementary Data File (NSDF) is similar to AIM, but differs from it in that NSDF tracks non-spacecraft data, multi-source spacecraft or other data, models and programs, and other NSSDC-held data sets that do not fit the AIM spacecraft/experiment/data set hierarchy. See Table 15 for NSDF statistics.

The Technical Reference File (TRF) tracks individual published papers associated with space flight experiments. The NSSDC ID for the experiment is attached to the reference information so lists of papers relevant to a particular experiment can be reported, and/or provided to persons accessing data from a given experiment from NSSDC. Table 16 shows that 816 papers were newly identified in TRF during 2000 mainly as the result of NSSDC staffers reviewing the Journal of Geophysical Research and the Geophysics Research Letters. The TRF was used to generate the Appendix listing 65 NSSDC-acknowledged papers.

The Interactive Request and Name Directory (IRAND) tracks people who have interacted with NSSDC over the years. It includes full names, one or more addresses, telephone and email information, and what NSSDC distribution lists they are on. The database contains approximately 56,000 entries. This information is also accessed and made available through the PIMS interface on the NSSDC WWW Home Page. Further IRAND statistics are available as Table 17. Note especially that over 7% of the records were updated in 2000, reflecting volatility of such information items as Internet addresses. IRAND also tracks individual staff-involved requests, now more than 80,000 over the years.

The Interactive Data Archive (IDA) is another database of interest. IDA tracks the inventory of NSSDC's digital data volumes (tapes, disks, etc.). IDA had 164,425 records at the end of 2000, with 2,723 records having been added during 2000.

Clearly there is a wealth of information that has been stored over the years in separate database information systems developed for various purposes. The

NSSDC is now evolving these databases into a more up-to-date and consolidated form. During 2000 many of the databases such as AIM, NSDF, and IRAND were merged and migrated to a new dedicated server using the Oracle 8.1.5 operating system. New interfaces were developed to make the information more readily available to NSSDC personnel.  

4.2 NASA/Science Office of Standards and Technology (NOST)

NOST's mission is to facilitate the recognition and use of standards to reduce cost/benefit ratios in the exchange and management of scientific data among NASA entities and the scientific communities they serve. NOST's Web Home Page is at http://ssdoo.gsfc.nasa.gov/nost/.

The NOST strategy is to play a coordinating role in helping the science disciplines identify new standards requirements. NOST participates in partnerships with them, other agencies, and industry on facilitating the adoption of leading-edge technologies with national or international visibility that can be tailored to meet NASA science information management and exchange requirements, and it assists in the process of moving these technologies toward standards with commercial support.
 

4.2.1 Formats Evolution Process (FEP)


Scientific progress continues to be impeded within some science communities, and across the boundaries of traditional discipline domains, by the lack of, or excessive multiplicity of, available standards for data formats and structures. A somewhat more subtle adverse effect of this situation is that the commercial software sector is unable to perceive what formats/structures it should support (e.g., with applications software).  This leads to the situation whereby developers and/or users of the many available "standard" formats must develop/maintain/evolve tools that the commercial sector might otherwise do.

NOST has established a Formats Evolution Process to address these issues.  It involves an open dialogue with the relevant communities and the publication of materials and comments via the Web at http://ssdoo.gsfc.nasa.gov/nost/fepc/.  Descriptions of many formats and usage experience from researchers, archives, and projects are included.  Views from tool developers are also available, and there is a Web based forum by which comments may be posted.  This activity is guided by a Formats Evolution Process Committee which is charged with providing synthesis materials to stimulate community consensus.

A paper entitled "A Possible Time-Series Object" was prepared and released for community review during 2000. This could lead to an improvement in the way in which data formats are used to convey and describe time-series data. This may include agreements on needed attributes, and even standard services or APIs that time-series objects should provide. Also under development by the FEPC is a paper establishing an expanded set of terms and concepts enabling more precise discussion of data format features and limitations, including their associated software.


4.2.2 Consultative Committee for Space Data Systems (CCSDS)

NOST operated NASA's highest level Control Authority office in accordance with the applicable Consultative Committee for Space Data Systems (CCSDS) and ISO standards to formally archive data descriptions for interchange and long term preservation. New descriptions were added for Interball, and for NSSDC data sets being migrated into archival packages on new media.  Currently there are 411 registered data descriptions, up from 350 a year ago.

NOST participated in the development of draft CCSDS/ISO standards applicable to multi-discipline and sub-discipline information interchange.  The primary standards and their usage categories were:

Data Entity Dictionary Specification Language (DEDSL): This draft standard addresses the problem of providing a standard way to document and exchange the various attributes needed to fully define data elements.† It has been harmonized with the conceptual data element standard from ISO known as ISO 11179 and the ANSI X3.L8 standard known as X3.285. The DEDSL is split into two components - one addressing the conceptual model and one addressing interchange forms. The conceptual model and an interchange form using the ISO Parameter Value Language (PVL) have completed formal reviews and will be published this summer.† An interchange form using XML with DTDs is entering formal review, and will be followed by an XML version with XML Schema. These standards support the publication and exchange of data Elements, and groups of data elements, and should lead to more automated access and understanding of data across science disciplines and among organizations.

Reference Model for an Open Archival Information System (OAIS): This standard provides a conceptual model of a digital archive, including a functional view and an information view, and it provides a framework for discussing migration issues and interactions among archives. The model establishes initial criteria for recognition of a true archival function and should lead to improved archival implementations, provide a basis for further standardization, and provide more cost-effective vendor support. It has been adopted as a starting point, in addressing digital preservation issues, by a wide variety of organizations around the world. The reference model draft has undergone formal agency review as well as review as an ISO Draft International Standard. It is under update in response to these comments and is expected to become a full ISO standard in the Fall of 2001. It can be found at http://ftp.ccsds.org/ccsds/documents/pdf/CCSDS-650.0-R-1.pdf.

4.2.3.      Flexible Image Transport System (FITS) Technical Panel and Support Office

The Flexible Image Transport System (FITS) is the standard format for astronomical data interchange, endorsed and supervised by the International Astronomical Union (IAU). NOST participates in the evolution of FITS by commenting on proposed new FITS extensions and conventions and by the development of a formal FITS specification document using the NOST standards accreditation process. The FITS Support Office (FSO) of NSSDC/ADF supports the NOST-convened FITS Technical Panel (including Goddard and extra-Goddard FITS experts) in its work and supports the community by providing a number of services, such as current versions of all documents and WWW information pages.

During 2000, the IAU FITS Working Group adopted the NOST_100-2.0 document as the IAUís new official FITS standard. Other activities of our FITS Support Office included support to the FAME project for FITS usage and discussions with the community to evolve the FITS standards based on our XML development.

4.3.     Astronomical Data Center

In the Astronomical Data Center (ADC) (http://adc.gsfc.nasa.gov), over 3000 astronomical source catalogs and journal tables are maintained online for easy access.  Entire catalogs and tables can be retrieved via FTP. Web-based visualization tools (http://tarantella.gsfc.nasa.gov/adf/visualization/design.html) are available for browsing, plotting, and subsetting the contents of the catalogs and tables before download. Users can query interactively for information on individual plotted data points and search for observations made by NASA missions. To enable interoperability with other data facilities, a programmer's interface (http://tarantella.gsfc.nasa.gov/viewer/AEQdoc.html) has been developed to allow external software to directly access the ADC catalogs without going through the Web browser.

During 2000, the ADC catalog server experienced over 32,000 accesses per month. The server for visualization tools was accessed over 5200 times per month. The outflow of data during 2000 exceeded 120 GB and 400,000 files. This byte count is nearly 40 times greater than in 1999, evidence of the fact that astronomical catalogs are getting much larger.

The ADC is currently involved in a research project to define the metadata of an astronomical repository in eXtensible Markup Language (XML) (http://tarantella.gsfc.nasa.gov/xml/).  An XML toolbox is being developed for importation, enhancement, and distribution of data tables and their metadata documents. The objective is to enable a more focused search of the ADC data holdings and better support of data exchanges with other data centers. In 2000, ADC began releasing test versions of the XML software to the community. See http://xml.gsfc.nasa.gov/ADCSoftwareDownload.html.

4.4.     Common Data Format

The NSSDC Common Data Format (CDF) is a self-describing data abstraction for the storage and manipulation of multidimensional data in a discipline-independent fashion.  CDF is comprised of three parts, the CDF data files that contain both the actual data values and metadata, the CDF software library that is used to create, access, manage, manipulate, etc. CDF files, and a well-defined Applications Programming Interface (known as the CDF Interface) that provides transparent access to underlying software and data.  CDF provides the essential framework for which generic applications (e.g., visualization, statistical) can easily be created. The CDF effort provides a consistency among data formats and structures and allows data to be shared transparently among a variety of projects and applications.

The International Solar Terrestrial Physics (ISTP) Project is NASA's single largest user of CDF.  The ISTP Project is using CDF to store its Key Parameter data.  The NASA IMAGE mission, launched in 2000, will use CDF extensively.  We also note that CDF underlies NSSDC’s OMNIWeb, COHOWeb, CDAWeb and SSCWeb services..

During 2000, NSSDC's CDF office developed Java-based programs to provide an alternative way to browse, edit and export CDF files. These new tools, relying on the CDF-Java APIs developed in 1999, provide a better graphical user interface (GUI) through Java language. Also, a new set of APIs was developed and tested for the Perl language. The advent of the CDF Perl APIs, just like CDF-Java APIs, significantly benefits the CDF user community since a CDF application can now be written in popular Perl scripts and run on any one of the Perl-supported platforms without modifications. This encourages sharing of scientific data analysis code among scientists.

A World Wide Web (WWW) page located at http://nssdc.gsfc.nasa.gov/cdf/ on the Internet provides a description of CDF, access to the software distribution, documentation, papers, a list of Frequently Asked Questions, and facilitates interaction with the CDF support group at the NSSDC.

Approximately 21,220 files were FTP-downloaded from the CDF directory of NSSDC’s anonymous account during 2000, an increase of 33% over 1999.  These were mostly files describing CDF, software tools from the CDF library, etc.  In addition, a great many users browse the CDF pages identified above.