FEP - Format Use by a Researcher - Eduardo Santiago - ASCII

1. Format (Format System) Identification

ASCII

2. Original Motivation

ASCII is the ultimate in portability.

3. Data Types

* Processing Level

	o Level 1 and higher

* Object Types

	o Time Series.  Multidimensional
    

4. Support

Some people just dump numbers to ASCII files, without a header indicating what the numbers mean. In those cases, the person who writes the file must provide separate documentation... and this doesn't always happen.

Properly written ASCII files are standalone and self-describing via a header.

5. Software

Simple C, IDL, Perl functions that read or write delimited strings.

6. Environment

UNIX-only (Linux, Solaris) with GNU tools.

7. Usage

I write ASCII for one-shot deals. It is the easiest format for transferring data across institutions, and sometimes even within our group (for instance, to Macintosh and Wintendos users).

Other institutions frequently distribute their data in ASCII. Invariably, if we are to use their data on a regular basis, I write C code to convert their ASCII files to HDF (via cron).

8. Experience

>Relative to its ability to carry and manage research-needed metadata >Relative to its related software

You can do anything with ASCII.

Unfortunately, as often happens in the UNIX world, people often take too much rope and shoot themselves in the foot with it :-).

When writing data to ASCII files, it is imperative that a proper header be included with EVERY file, describing the field names at a minimum, and possibly the delimiter used.

If the file is a one-shot deal, the format of the header doesn't really matter, since a human will always have to examine it.

If the file is a standard product, though, it is expected that custom code will be written to read the files. Therefore, the header must not change from one day to the next. The file layout must not change, either: fields cannot be added or removed without significant work on the part of everyone who uses the data.

9. Desired Functionality

You can't really ask for more out of ASCII. However, it would be nice if _people_ wrote better ASCII files!

To this end, it would help to have standardized headers, and C, Fortran, IDL, (etc) code to write/read them and figure out the data layout.

10. Selection Criteria

See my comments under HDF.

11. Impact on Research

See my comments under HDF.

12. Other Comments

Wider Views