Submission of data to ISTP
There are essentially three steps to submitting data to the ISTP Central Data Handling Facility (CDHF). We describe each of the steps in some detail below or on subsequent pages and give contact names/emails to help smooth the process.
Arrange/Agree the submission with the ISTP Project
Put your data into CDF and make detached SFDU
FTP the files to the CDHF
Arrange/Agree the submission with the ISTP Project
contact: Bill Mish, ”wmish@istp1.gsfc.nasa.gov” or ”u2whm@lepvax.gsfc.nasa.gov”, (301)286-5444.
High-resolution (“event”) digital data is being solicited from every ISTP investigation, and other related space physics missions and observatories.
Data values should be in physical / geophysical units, fully processed with the latest available algorithms and calibrations. The data submitted should be suitable in quality and resolution for primary research studies, representative of the true, useful resolution of your instrument both in time and in other parameters of your measurements as appropriate (i.e. spatial, frequency, energy, angle etc.). Images should be at the best available time and angular resolution, plasma measurements should include full distribution functions as well as moments, particle data should include multiple species over a full set of energy and pitch angle bands, wave distributions should be at the best useful frequency resolution. High-resolution data from ground investigations is requested. Contribution of appropriate and applicable theory products is strongly encouraged.
Experiments producing Key Parameter Data should use the existing Global Despcriptions with the exception that the data type will now be H0>High Resolution Data.
Those with new experiments proposing to submit data should read the remaining information in this section.
Return to top of page
High Resolution Data was described above.
Key Parameter files are in almost all cases submitted as
- daily files
- with one instrument per file
- at approximately one minute resolution.
It will be important to discuss the proposed frequency of transmissions as well as the approximate volume of the files.
The following values must be agreed before submission in order to ensure uniqueness of the data sets. This information is required for the CDHF data base as well as NSSDC’s data base and is also found in the data file (CDF) and the detached header file (SFDU).
- Source_name: need a 2-character code plus a longer, descriptive
name, e.g.,
- Geotail - GE - Geomagnetic Tail Laboratory
- Wind - WI - Wind Interplanetary Plasma Laboratory
- SAMPEX - SX - Solar Anomalous Magnetospheric Particle EXplorer
- MSX - MX - Midcourse Space eXperiment
- DARN - DN - Dual Auroral Radar Network
- Data_type: need a 2-character code plus a longer name. ISTP
allows 10 possible data sets to be submitted per instrument for each
data type. Key Parameters have been the most widely submitted data
type (using the single character K), though higher resolution data
(using the single character H) has also been used. Along with the
single character above, a number 0-9 to identify the particular data
set is also used to define the Data_type, e.g.,
- K0 - Key Parameters
- K1 - Key Parameters
- H0 - High Resolution Data
- H1 - High Resolution Data
- Descriptor: need a 3 or 4-character code plus a longer,
descriptive name, e.g.,
- LEP - Low Energy Particle Experiment
- MFI - Magnetic Field Instrument
- WAV - Radio and Plasma Waves
- GBAY - Goose Bay Station
- Discipline>Subdiscipline: you can choose from the following
- Space Physics>Magnetospheric Science
- Interplanetary Studies
- Ionospheric Science
- Solar Physics>Gamma-Ray Observations
- Infrared Observations
- Radio Observations
- Ultraviolet Observations
- Microwave Observations
- Visible Observations
- X-Ray Observations
- Space Physics>Magnetospheric Science
Files are named according to the short character codes, built from the Source_name, Data_type, Descriptor, the date of the data (4 digit year, 2 digit month, 2 digit day) and the version number of the file (in case of re-processing some files are produced more than one time!). For example, a file name might look like GE_K0_LEP_19950215_V01.cdf or WI_H0_MFI_19960425_V02.cdf.
Return to top of page
Put your data into CDF and make detached SFDU
contact: Mona Kessel, ”mona.kessel@gsfc.nasa.gov”, (301)286-6595.
2 Products are sent to the CDHF:
- data in the Common Data Format (CDF) using the ISTP Guidelines
- detached SFDU header file
All of the necessary information and tools are available from the Space Physics Use of CDF web page with important aspects repeated below for your convenience.
CDF
You will need to obtain the CDF software from NSSDC.
You will need specific, detailed guidance on creat ing ISTP CDF files, especially:
- How to structure data
- What Descriptions to include
An intermediate result of this is a CDF “skeleton table”, a specially formatted ASCII text file that contains the information about the data set. The format of the file allows simultaneous viewing and editing of the information (note: the skeleton table syntax must be followed exactly). Instrument-Specific Skeleton Tables are available, and can then be edited to match your data.
Skeleton Tables must be validated before submission of the data. Send your prototype to KPGSIT@istp1.gsfc.nasa.gov. A report with comments will be returned to you. This is usually an iterative process. You will also receive an ADID number at this time.
The CDF Skeleton is a file similar to the skeleton table in that it only contains information, but its format is binary. The CDF skeleton is created using the CDF toolkit routine called CDFskeleton. (This is also a check that in editing the skeleton table the syntax was preserved. If not, an error message appears on the screen pointing to the first error detected.) You can write your own Fortran or C program to put data into the CDF Skeleton (see the Tutoria l) or use the program makeCDF.
SFDU
Software is available for creating SFDU header files from an existing ISTP CDF file: see makeSFDU.
Return to top of page
FTP the files to the CDHF
contact: Program Assistance Center (PAC), ”pac@istp1.gsfc.nasa.gov”, (301)286-9453.
You will need to apply for a CDHF account in order to transfer files. Click here to open a telnet session and APPLY for your account. Then enter a username of “APPLY” and follow the directions.
The PAC will be your resource on the CDHF end to ensure that the transmissions flow smoothly.
Please provide the PAC with a name/email address on your end to serve as the main contact. All acknowledgements and/or problem reports will go to this main contact.
- FTP of files will be to a general location specified by the PAC (with essentially unlimited quota). You will need to log on to your cdhf account at the beginning of the FTP session to provide the authority for the transfer.
- Send a test CDF and SFDU pair of files and alert the PAC when they are in place.
- The PAC will acknowledge success or problems with the files.
- Once successful, you can start routinely FTPing files as they are produced.
- An email message will be sent to the address you supplied acknowledging receipt of data.
- The PAC will monitor transmissions and let you know of any problems.
Return to top of page
ISTP homeAuthors and Curators
Mona Kessel,
kessel@nssdca.gsfc.nasa.gov, (301)286-6595
Code 632, NASA Goddard Space Flight Center
Greenbelt, MD 20771, USA\
Authorized by R.E. McGuire, Head, Space Physics Data Facility\
(Code 632, NASA/GSFC), Robert.E.McGuire@gsfc.nasa.gov, (301)286-7794
Last Updated: 22 Feb 1996, KCH