CDF is a scientific data management software package and format based on a multidimensional (array) model. HDF is a Hierarchal Data Format developed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois. The data model of CDF is very similar to HDF5's data model. They both have 2 basic objects: data and attribute. Data is an entity that represents data while attribute is a mechanism used to denote data. HDF5 allows to group similar objects into a group, but CDF doesn't have a grouping mechnism.

Albeit HDF4 and HDF5 are developed by the same organization, the data model of HDF5 is totally different from HDF4 and their formats are incompatible.

9. What documentation is available?

The documentation set consists of the CDF User's Guide (UG), the CDF C 
Reference Manual (CRM), the CDF Fortran Reference Manual (FRM), and the CDF 
Internal Format Description (IFD).  The documents, except the IFD, are each 
approximately 200 pages.  The IFD is about 50 pages.

The CDF documents are available in Adobe Portable Document Format (PDF) files,
and they are located at http://nssdcftp.gsfc.nasa.gov/standards/cdf/doc.

Availability of the CDF 3.0 documentation:

     cdf30ug.pdf  (CDF User's Guide)
     cdf30crm.pdf (CDF C Reference Manual)
     cdf30frm.pdf (CDF Fortran Reference Manual)
     cdf30jrm.pdf (CDF Java Reference Manual)
     cdf30ifd.pdf (CDF Internal Format Description)

If you have trouble printing any of the file(s), send CDFSUPPORT a request 
for hard copies of the documents indicating which documents you need, and 
we will mail you a hard copy.

Some general information on CDF, including this FAQ, and software are also
available from the CDF Home Page on the World Wide Web with the following URL:

	http://cdf.gsfc.nasa.gov


10. What are some publications/references relating to CDF?

A list of publications about or referring to CDF can be found on the
CDF ftp site on nssdcftp.gsfc.nasa.gov in the 
standards/cdf/doc/papers directory
as Bibliography.html. The same directory contains PostScript and/or text
files for a few of the papers.

The following is a list of some references:

Mathews, G. J., and S. S. Towheed, "OMNIWeb: The First Space Physics Data
WWW-Based Data Browsing and Retrieval System," Computer Networks and ISDN
Systems, Proceedings of the Third International WWW Conference, Vol. 27,
No. 6, April 1995, pp. 801-808.

Goucher, G. W., and G. J. Mathews, "A Comprehensive Look at CDF,"
NSSDC/WDC-A-R&S 94-07, NASA/Goddard Space Flight Center, August 1994.

S. Brown, M. Folk, G. Goucher, and R. Rew, "Software for Portable Scientific
Data Management," Computers in Physics, Vol. 7, No. 3, pp. 304-308,
May/June 1993.

Salem, K., "MR-CDF: Managing Multi-Resolution Scientific Data," CESDIS
TR 92-81, NASA/Goddard Space Flight Center, Greenbelt, Maryland, March 1992.

Treinish, L. A. (ed.), "Data Structures and Access Software for Scientific
Visualization," A Report on a Workshop at Siggraph '90, Computer
Graphics, 25, No. 2, April 1991.

Treinish, L. A., and G. W. Goucher, "A Data Abstraction for the Source-
Independent Storage and Manipulation of Data," National Space Science
Data Center Technical Paper, NASA/Goddard Space Flight Center, August 1988.

Treinish, L. A., and M. L. Gough, "A Software Package for the Data-
Independent Storage of Multi-Dimensional Data," EOS Transactions,
American Geophysical Union, 68, pp. 633-635, 1987.

IDL Scientific Data Formats, Version 3.6, Research Systems Incorporated,
Boulder, Colorado, April 1994.



11. How can I contact them?

Comments, suggestions, and questions may be directed to CDF User Support via

• WWW feedback form:

  	http://cdf.gsfc.nasa.gov/cdf/form/feedback.html
  

• Electronic mail:
  

  	Internet   : cdfsupport@listserv.gsfc.nasa.gov
  

• U.S. mail:
  

  	Attention: CDF USER SUPPORT OFFICE
  	National Space Science Data Center
  	Code 632
  	NASA/Goddard Space Flight Center
  	Greenbelt, Maryland 20771-0001
  	U.S.A.
  

• Telephone:
  

  	Voice: (301) 286-9884
  	FAX:   (301) 286-1771
  

Before asking a question, you should check the FAQ and CDF User's Guide.


12. How do I make a bug report?

All bug reports, comments, suggestions, and questions should go to
cdfsupport@listserv.gsfc.nasa.gov.

Attached below is a bug report template. It would be very helpful
for us to locate and fix the bug if all the information requested in
the template is supplied by the reporter. It is very difficult for
us to trace bugs when we don't have enough background information.
You can also fill out the form via a Web browser with the following URL:
http://cdf.gsfc.nasa.gov/cdf/form/helpdesk.html.

       ------------------  Template for bug report  ------------------------
       To: cdfsupport@listserv.gsfc.nasa.gov.
       Subject: [area]: [synopsis]   [replace with actual AREA and SYNOPSIS]

       VERSION:
              [CDF library/toolkit version (e.g., CDF Version 2.5.1).
               You can obtain the version of your CDF distribution
               using the CDFinquire toolkit program and specifying the "/ID"
               argument as in the examples below:
                 From MS-DOS:  cdfinq -id
                 From UNIX:    cdfinquire -id
                 From VMS:     CDFINQUIRE /ID ]

       USER:
              [Name, telephone number, and address of person reporting the bug.
               (email address if possible)]

       AREA:
              [Area of the CDF source tree affected, e.g., lib, tools, tests,
                toplevel. If there are bugs in more than one AREA, please use 
                a separate bug report for each AREA.]

       SYNOPSIS:
              [Brief description of the problem and where it is located]

       MACHINE / OPERATING SYSTEM:
              [e.g. Sparc/SunOS 4.1.3,  HP9000/730-HPUX9.01...]

       COMPILER:
              [e.g. native cc, native ANSI cc, Borland C++ V3.1, MPW, ...]

       DESCRIPTION:
              [Detailed description of problem.]

       REPEAT BUG BY:
              [What you did to get the error; include test program or session
               transcript if at all possible.  If you include a program, make
               sure it depends only on libraries in the CDF distribution, not
               on any vendor or third-party libraries.  Please be specific;
               if we can't reproduce it, we can't fix it. Tell us exactly what
               we should see when the program is run.]

       SAMPLE FIX:
              [If available, please send context diffs (diff -c).]

       [PLEASE make your Subject (SYNOPSIS): line as descriptive as possible.]
       [Remove all the explanatory text in brackets before mailing.]
       [Send to cdfsupport@listserv.gsfc.nasa.gov or to

                Attn: CDFSUPPORT
                NASA/GSFC/SED/ESED/SPDF
                Code 612-4
                Greenbelt, MD 20771, U.S.A. ]

       ------------------  End of Bug Report Template  ----------------------


13. Is there a mailing list for CDF discussions, questions and announcements?

There are two mailing lists:

    1. General Mailing List and
    2. CDF Users Discussion Mailing List.

  *** The general mailing list is for major announcements and events. ***

The general mailing list announces new releases and general announcements.
Anyone who contacts CDFSUPPORT will automatically be added to the
general mailing list, unless requested otherwise.

For administration questions, for additions or deletions from the general
mailing list or for corrections to the list, please send messages to

                cdfsupport@listserv.gsfc.nasa.gov

  *** The CDF users mailing list is for discussions. ***

CDF has a mailing list for electronic discussions between programmers
and users of CDF.  This service has been created to encourage the CDF
user community to share knowledge and experience and to reduce duplication
of effort.

To subscribe to this mailing list, please send an e-mail message to
"Majordomo@nssdc.gsfc.nasa.gov" with the body of the message as the
following one-line command:

	SUBSCRIBE cdf-users

Note that commands in the "Subject:" line are NOT processed.

To send a message to the mailing list, send your message to

	cdf-users@nssdc.gsfc.nasa.gov

For more information about this mailing list, see the mailing list
introduction page via one of the following URLs:

	http://cdf.gsfc.nasa.gov/cdf/html/cdf-users-intro.html

	http://nssdcftp.gsfc.nasa.gov/standards/cdf/doc/mail-archives/cdf-users.intro



14. What programming languages does CDF support?

The CDF library comes with C, Java, and Fortran Application programming 
Interfaces (APIs) that provide the essential framework on which 
graphical and data analysis packages can be created.  In addition, Perl 
APIs are are available as an optional package for download. The CDF library 
allows developers of CDF-based systems to easily create applications that 
permit users to slice data across multidimensional subspaces, access entire 
structures of data, perform subsampling of data, and access one data 
element independently regardless of its relationship to any other data 
element.  


15. Are CDF data sets/files platform-independent?

CDF data sets are portable across any platform supported by CDF.  These 
currently consist of VAX (OpenVMS and POSIX shell), Sun (SunOS & SOLARIS), 
DECstation (ULTRIX), DEC Alpha (OSF/1 & OpenVMS), Silicon Graphics Iris and 
Power Series (IRIX), IBM RS6000 series (AIX), HP 9000 series (HP-UX), 
NeXT (Mach), PC (DOS, Windows 3.x, Windows NT/95/98, Linux, QNX, Solaris, 
Cygwin & MinGW), and 
Macintosh (68K & Power PC running MacOS 7/8/9.x, X or Linux).


16. What's the difference between a single-file CDF and multi-file CDF?

The CDF library gives the user option to choose from one of two file formats:
single-file and multi-file.  Single-file CDF contains the control information, 
metadata, and the data values for each of the variables in one file.  Whereas
multi-file CDF has two parts to it: metadata file and data file (one for 
each variable.

The main advantage of the single-file format is that it minimizes the number
of files one has to manage and makes it easier to transport CDFs across a
network.  The organization of the data within the single file may, however, 
become somewhat convoluted, slightly increasing the data access time.  The 
multi-file format, on the other hand, clearly delimits the data from the 
metadata and is organized in a consistent fashion within the files.  Updating, 
appending, and accessing data are also done with optimum efficiency.

For multi-file format CDFs, certain restrictions are applied.  They are: 

-  Compression: Compression is not allowed for the CDF or any of its variables.

-  Sparseness: Sparse records or arrays for variables are not allowed.

-  Allocation: Pre-allocation of records or blocks of records is not allowed. 
               For each variable, the maximum written record is the last 
               allocated record.


17. What are the differences between the Standard Interface, Internal 
Interface, and the CDF Java APIs? 

The CDF library comes with two types of Application Programming Interfaces 
(APIs) for C and Fortran: Standard Interface and Internal Interface.  
The Standard Interface allows to perform basic operations such as creating 
a CDF file, opening a CDF file, and reading and writing data.  See chapter 
5 of the CDF Fortran Reference Manual or the CDF C Reference Manual for a 
complete list of the APIs that are available for the Internal Interface and 
their detailed descriptions.

The Internal Interface consists of only one routine, CDFlib.  CDFlib 
is used to perform all possible operations on a CDF.  In fact, all of the 
Standard Interface functions are implemented using the Internal Interface.
CDFlib must be used to perform operations not possible with the Standard
Interface functions. (e.g., specifying a single-file format for a CDF, 
accessing zVariables, compressing a CDF file or variables or specifying 
a pad value for an rVariable or zVariable).  Note that CDFlib can also be 
used to perform certain operations more efficiently than with the Standard 
Interface functions.
 
The Standard Interface and Internal Interface do not apply to the CDF Java 
APIs.  The CDF Java APIs can do everything the C and Fortran APIs can do and
more such as copying a variable with or without data).  Detailed description
of the CDF Java APIs can be found at 

http://cdf.gsfc.nasa.gov/cdf/cdfjava_doc/index.html.


18. What's "variable"?

CDF's "variable" is a generic name or an object that represents data
where data can be 0-dimensional (scalar data) or multi-dimensional (up to
10-dimension), and it does not have any scientific context associated it.
For example, a variable can be data representing an independent variable, a
dependent varibale, time and date value, or whatever data might be (e.g.
image, XML file, etc.).  In other words, the variable doesn't contain any
hidden meanings other than the data itself.  One may describe one
variable's relationship with other variable(s) through "attributes". 

There are two types of variables (rVariable and zVariable) and they can
coexist in the same CDF file.  Every rVariable in a CDF must have the 
same number of dimensions and dimension sizes, whereas each zVariable has 
its own dimension and dimension size.  Suppose there are 2 rVariables 
(v1, v2) in a CDF.  Let's say v2 is defined as 2:[20,10] - 2-dimensional 
array with its size of 20 x 10 (20 rows and 10 columns).  Then v1 MUST 
be defined as 2:[20,10] albeit it only needs 1:[8] (since it is a rVariable).  
But if this model is implemented using zVariables, then v1 and v2 can be 
defined as 1:[20,10] and 1:[8] instead of 1:[20,10] and 1:[20,10].  As you 
can see above, since all the rVariables must have the same dimensions and 
dimension sizes, there'll be a lot of disk space wasted if a few variables 
need big arrays and many variables need small arrays.  

So why would you want to use rVariables over zVariables?  There's no reason
to use rVariables at all (since zVariables are much more efficient) if you 
are creating a new CDF file.  But if you are analyzing data files that were 
created with early CDF releases or contain rVariables for some reason, you'll 
need to use rVariables.  One may wonder why there are rVariables and 
zVariables, not just zVariables.  When CDF was first introduced in early 
90's, only rVariables were available.  The inefficiencies with rVariables 
were quickly realized and addressed with the introduction of zVariables in 
later CDF releases. 


19. What compression algorithms does CDF support?

Compression may be specified for a single-file CDF and the CDF library can 
be instructed to compress a CDF as it is written to disk.  This compression 
occurs transparently to the user.  When a compressed CDF is opened, it is 
automatically decompressed by the CDF library. An application does not have 
to even know that a CDF is compressed.  Any type of access is allowed on a 
compressed CDF. When a compressed CDF is closed by an application, it is 
automatically recompressed as it is written back to disk.

The individual variables of a CDF can also be compressed. .  The CDF library 
handles the compression and decompression of the variable values 
transparently.  The application does not have to know that the variable 
is compressed as it accesses the variable's values.

The following compression algorithms are supported by the CDF library:

  - Run-Length Encoding
  - Huffman
  - Adaptive Huffman
  - GZIP

See chapter 2 of the CDF User's Guide for more detailed information for
the compression algorithms mentioned above.


20. Can CDF delete a range of records?

Yes.  The Standard Interface doesn't have a function that allows users to 
delete a range of records, but the Internal Interface and the CDF Java 
APIs provide APIs for deleting a range of records.

The following APIs/method allows to delete a range of records:

   C or Fortran: <DELETE_, rVAR_RECORDS> or <DELETE_, zVAR_RECORDS>
   CDF Java API: deleteRecords(firstRec, lastRec) method in the Variable class 

Chapter 6 of the CDF C Reference Manual and CDF C Reference Manual contain
more detailed description of the <DELETE_, rVAR_RECORDS> and 
<DELETE_, zVAR_RECORDS> APIs/functions.  

Click here and select the "Variable" link in the lower left frame 
for a detailed description of the deleteRecords method. 


21. Does CDF support random file access?

Yes.  One of the strengths of CDF is random file access and ability to 
specify the granularity of data read/write.  Suppose you have a CDF file 
that has 1000 records in it.  You can extract data by specifying:

- the individual record number you want to extract data from

  OR

- the start and end record number from which data is to be retrieved.  You
can also specify the record interval that defines the number of records
to skip between successive reads.  For example, if you specified 100, 200,
and 2 for the start record number, the end record number, and the  record
interval respectively, the CDF system will start reading data from the
record number 100 and read every other record (100, 102, 104, 106, and so
on) until it reaches the record number 200.  If a record is an array
(contains more than one value), you can even specify which element(s) you
want to extract for every record.  

The following methods in the Variable object class provide random file
access for reading:
   getRecord, getScalarData, getSingleData, getHyperData.

The following methods in the Variable object class provide random file
access for writing:
   putRecord, putScalarData, putSingleData, putHyperData.

Click here and select the "Variable" link in the lower left frame
for a detailed description of the methods/APIs described above. 


22. How do I optimize a CDF file or can I read data fast? 

Before reading data (whether it's a sequential or random/direct access),
it's important to minimize the number of internal pointers (a.k.a data
defragmentation) that are used to keep track of where data is located 
within a file before reading data (whether it's a sequential read or 
random/direct access read).  The CDFconvert CDF utility, among many
other functions it performs, allows users to optimize a CDF file and one
can optimize a CDF file by entering the following command at the operating
system prompt:

   CDFconvert <source cdf> <dest cdf>  

       where <source cdf> = the name of the CDF file to be optimized
             <dest cdf>   = the name of the newly optimized file

For example, the following command reads a file called test.cdf and creates
an optimized file called test_new.cdf.

   CDFconvert test.cdf test_new.cdf 
     
NOTE: It's always a good idea to run CDFconvert on newly created CDF 
files since reading data from an optimized file is very fast.  This is 
especially true if the size of the file is big.



23. What is the best way to represent my data using CDF?

There are many ways to represent the same information in any general-purpose
data model, and there is no single "correct" way to store data in a CDF.  The
user has complete control over how the data values are stored in the CDF
(within the confines of the variable array structure) depending on how the
user views the data.  This is the advantage of CDF.  Data values are organized
in whatever way makes sense to the user.

We provide some guidelines in the CDF User's Guide (e.g., "Organizing Your Data 
in a CDF"), but we've found that a little experience helps.  Occasionally,
we have decided it was useful to change the structure of the CDF files after 
experience with how the data are used.


24. Can new versions of CDF applications read CDF files written using older
    versions of the CDF library?

Our goal is to make CDF backward compatible in the sense that CDF files
can always be read by new versions of the CDF library and tools. We have
succeeded in doing so up to CDF 2.7 (the latest CDF library) and will 
continue to follow the principle as much as possible.  

Old CDF libraries are generally NOT able to read CDF files
written by newer versions of the CDF library.  In some cases old CDF libraries
can read CDFs from newer versions, such as CDF V2.3 reading CDF V2.4 files,
but this is NOT recommended.  Our policy is for CDF toolkit applications to
read any CDF data set, but your own applications should use the latest CDF
library.


25. Can my application programs that work with old versions of the CDF
    library always be compiled with new versions of CDF?

As CDF evolves new functions are added, but they are transparent to
applications since all existing functions of the Internal and Standard
interfaces are unchanged.  However, the CDF Version 1.0 "Obsolete" interface,
which was only callable from Fortran, is no longer supported with CDF V2.5.
The Standard Interface is similar to the CDF V1.0 interface with several
additions for new features.  Those functions not available using the Standard
Interface are made available using the Internal Interface.  The Internal 
Interface makes CDF an easily extensible software package, so all applications
using this interface will remain portable with newer versions of the CDF
library.

However, some advanced CDF users have exploited undocumented CDF functions
that were removed from a later version of the library and could not compile
their application.  This practice is NOT recommended because these functions
are for internal use only and subject to change without notice in future
releases.  If some internal CDF function is useful to your application, then
you should notify us, and we will determine if it is appropriate to make it
visible in the next release.  In the meanwhile, the portable approach would
be to copy the corresponding function(s) from the CDF library source, rename
the function name(s) to something else, insert it into your application code,
and update the calling statements with the new function name(s).



26. Is there any commercial or public domain data analysis/visualization 
       software that accepts/supports CDF files?

    Commercial software:
        Interactive Data Language (IDL)
        MathWorks MATLAB Language (MATLAB)
        Application Visualization System (AVS)
        Weisang GmbH & Co. KG Data Analysis and Presentation (FlexPro)

    Public domain software:
        IBM Visualization Data Explorer (DX)

        CDAWlib - A set of IDL routines that underlie the CDAWeb software.
           This is a "library" of routines useful for reading in data 
           that is stored in the CDF format and also for plotting the 
           data variables as time series, images, radar, and 
           spectrograms.  In order to use this s/w you must have IDL 
 	   installed on your local machine.

        CDFx - an IDL software tool for displaying, editing, and listing the 
           contents of CDF data files. It displays image plots, time-series 
           plots, CDF variables; it will list and store CDF data in plain 
           ASCII text; and it will save subsets of data to new CDF files.

           Familiarity with CDF and ISTP CDF conventions will be helpful when 
           using CDFx. ISTP specifications and master CDF file concept 
           discussions are at the SPDF Use of CDF site.

        LinkWinds (contact berkin@krazy.jpl.nasa.gov)

    There are also several NSSDC developed WWW-based data systems that 
    provide access to data stored in CDF and use IDL to generate time 
    series plots of selected variables, which are available via the 
    following URLs:

	CDAWeb - http://cdaweb.gsfc.nasa.gov/cdaweb/
	SSCWeb - http://sscweb.gsfc.nasa.gov/
	COHOWeb - http://nssdc.gsfc.nasa.gov/cohoweb/cw.html
	OMNIWeb - http://nssdc.gsfc.nasa.gov/omniweb/ow.html

    Example visualizations can be found via the URL:
	http://cdf.gsfc.nasa.gov/cdf/html/examples.html



27. Are there any conversion programs available that will convert non-CDF 
       files into CDF files or vice versa?

    
MakeCDF is a CDF application that reads flat data sets, in both
binary and text and generates a ISTP-compliant CDF data set from that data.
It is available at http://spdf.gsfc.nasa.gov/makecdf.html.

    
In a bid to facilitate and promote data exchanges among space 
scientists, the CDF office has developed a web service called 
Data Translation Web Service (DTWS) and a client to 
talk to DTWS via a web browser.   The DTWS is a web service based on the 
Simple Object Access Protocol (SOAP), an XML-based protocol designed for the 
exchange of information in a distributed computing environment, and its client
allows users to translate one or more local and/or remote files into other 
data formats interactively or in a batch mode via a Web browser.  Below is a 
list of the data translation modes that are currently supported by the client:

   
CDF-to-netCDF
   
CDF-to-FITS
   
CDF-to-CDFML (XML representation of CDF)
   
CDF-to-ASCII
   
netCDF-to-CDF
   
netCDF-to-ASCII
   
FITS-to-CDF
   
CDFML-to-CDF
   
HDF4-to-CDF

The DTWS client is available at http://translators.gsfc.nasa.gov.
    
CDFexport is a tool that can generate an ASCII text file or 
subsetted CDF of the selected variables from a CDF, among other things. 
It is part of the CDF distribution package.  (See question 6 above to obtain 
source code for CDF tools.)


28.  Can I convert HDF4, HDF5, FITS, or netCDF to CDF?

Yes.  The CDF office has developed the Data Translation Web Service (DTWS)
through which one can translate one or more local and/or remote files into 
other data formats interactively or in a batch mode. The DTWS service is 
available at http://translators.gsfc.nasa.gov.

As for the translation from HDF4 to HDF5, the NCSA Hierarchical Data 
Format (HDF) office has developed a tool that converts a HDF4 file to HDF5, 
and this tool (h4toh5) is available from the HDF home page 
(http://hdf.ncsa.uiuc.edu/HDF5).


29. How can I contribute my software to the CDF user community?

Contact CDFSUPPORT indicating that you would like to contribute your 
software to the CDF user community.  Please fill out the CDF Directory
Entry form (pub/cdf/doc/forms/direct) and send the completed form to
us.  Your information will then be added to our Directory of CDF User
Applications, and users may contact you for information on your
application.

However, if you want us to distribute your application, we will set up a
directory for you to send the contribution package.  For other users'
convenience your contribution package should include the software itself,
a Makefile if possible, a man-page, test programs, and input data files for
testing.  A README file is also required.  It should describe briefly the
purpose, function, and limitation of the software; on which platforms and
operating systems it runs; how to compile, install, and test it; whom and
where to contact for comments, suggestions, or bug reports.

We encourage CDF users to let us know about their applications, so
other CDF users benefit from it as well.

Some CDF-related software are available from the URL:
http://nssdcftp.gsfc.nasa.gov/standards/cdf/apps.

Comments, corrections, or additions to the CDF FAQ along with any CDF-related comments or questions should be sent to the CDF User Support Office via email to cdfsupport@listserv.gsfc.nasa.gov.

CDF Home Page