NSSDCA ID: PSPG-00883
Availability: Archived at NSSDC, accessible from elsewhere
This description was generated automatically using input from the Planetary Data System.
This volume contains portions of the CRISM Experiment Data Record (EDR) Archive, a collection of image cubes from the Compact Reconnaissance Imaging Spectrometer for Mars on the Mars Reconnaissance Orbiter spacecraft. Images are stored with PDS labels, but are otherwise unprocessed and uncalibrated. This volume also contains ancillary data files and browse images in PNG format, HTML documents that support a web browser interface to the volume, an index file ('imgindx.tab') that tabulates the contents of the volume, and documentation files. For more information on the contents and organization of the volume set refer to the aareadme.txt file located in the root directory of the data volumes. Parameters ========== CRISM observing scenarios are constructed using a set of key variables ('configurations') which include the following: (All are selectable separately for the VNIR and IR detectors). Image source: Image data may be generated using digitized output from the detector, or using one of up to seven test patterns. Row selection: All detector rows having useful signal can be saved, or alternatively an arbitrary, commandable subset of rows can be saved. The number of rows with useful signal is 545, 107 in the VNIR and 438 in the IR. The nominal number of rows for multispectral mode was 73, 18 in the VNIR and 55 in the IR prior to 10 Dec 2006. On that date an extra channel was added to the VNIR for a total of 19. For each detector, there are four options of channel selection to choose from rapidly by command: hyperspectral (545 total channels), multispectral (73 total channels prior to 10 Dec 2006, 74 total channels on and after 10 Dec 2006), and two sets of expanded multispectral (84 and 92 channels prior to 10 Dec 2006, 85 and 93 channels on and after 10 Dec 2006). An analogy is a car radio preset button. New options are set by uploading a data structure to the DPU. Data with any wavelength selection may have a corresponding TRDR, but the wavelength selection affects the wavelength continuity and dimensionality of a TRDR. Pixel binning: Pixels can be saved unbinned or binned 2x, 5x, or 10x in the spatial direction. No pixel binning in the spectral direction is supported. Compression: All CRISM data are read off the detector in 14-bit format and are compressed real-time in hardware. Compression options, in succession, are: (1) Subtraction of an offset, on a line by line basis. Offsets are set by uploading a data structure to the DPU. (2) Multiplication by a gain, on a line by line basis, with the output in 12-bit format. Gains are set by uploading a data structure to the DPU. (Raw 12-bit data are stored onground in data products as 16-bit numbers.) (3) Optionally, conversion from 12 to 8 bits using one of eight look-up tables (LUTs) specified on a line by line basis. These choices are set by uploading a data structure to the DPU. (4) Optionally, lossless Fast + differential pulse-code modulation (DPCM) compression. Pointing: CRISM has two basic gimbal pointing configurations and two basic superimposed scan patterns. Pointing can be (1) fixed (nadir-pointed in the primary science orbit) or (2) dynamic, tracking a target point on the surface of Mars and taking out ground track motion. Two types of superimposed scans are supported: (1) a short, 4-second duration fixed-rate ('EPF-type') scan which superimposes a constant angular velocity scan on either of the basic pointing profiles, or (2) a long, minutes-duration fixed-rate ('target swath-type') scan. Frame rate: Frame rates of 1, 3.75, 5, 15, and 30 Hz are supported. The 1 Hz frame rate is used for hyperspectral measurements of the onboard integrating sphere, because the long exposures possible at 1 Hz are needed for appreciable SNR at the shortest wavelengths. 3.75 Hz is used for hyperspectral measurements of Mars; this is the highest frame rate at which the DPU electronics support onboard compression options over the range of wavelengths imaged onto the detectors with useful SNR. 15 and 30 Hz frame rates are used for nadir-pointed multispectral measurements that return only selected wavelengths. The 5 Hz frame rate is not planned for use in flight, because at that rate the electronics do no support compression of a hyperspectral wavelength selection, and it would produce excessive along-track smear in a nadir-pointed observation. Integration time: Integration times are in increments of 1/480th of the inverse of the frame rate. At 1 Hz, for example, available integration times are 1/480th sec, 2/480th sec...480/480th , and at 15 Hz, 1/7200th sec, 2/7200th sec...480/7200th sec. Calibration lamps: 4095 levels are commandable in each of two lamps at each focal plane, and in two lamps in the integrating sphere. All lamps can be commanded open-loop, meaning that current is commanded directly. For the integrating sphere only, closed loop control is available at 4095 settings. For closed loop control, the setting refers to output from a photodiode viewing the interior of the integrating sphere; current is adjusted dynamically to attain the commanded photodiode output. Note: lamps reach maximum current at open- or closed-loop settings <4095. Shutter position: Open, closed, or viewing the integrating sphere. The shutter is actually commandable directly to position 0 through 32. In software, open=3, sphere=17, closed=32. NOTE: during integration and testing, it was discovered that at positions <3 the hinge end of the shutter is directly illuminated and creates scattered light. Position 3 does not cause this effect, but the other end of the shutter slightly vignettes incoming light. Pointing: CRISM has two basic gimbal pointing configurations and two basic superimposed scan patterns. Pointing can be (1) fixed (nadir-pointed in the primary science orbit) or (2) dynamic, tracking a target point on the surface of Mars and taking out ground track motion. Two types of superimposed scans are supported: (1) a short, 4-second duration fixed-rate ('EPF-type') scan which superimposes a constant angular velocity scan on either of the basic pointing profiles, or (2) a long, minutes-duration fixed-rate ('target swath-type') scan. Pointing configuration affects the contents but not the dimensionality of an EDR. Processing ========== The CRISM data stream downlinked by the spacecraft unpacks into a succession of compressed image frames with binary headers containing housekeeping. In each image, one direction is spatial and one is spectral. There is one image for the VNIR focal plane and one image for the IR focal plane. The image from each focal plane has a header with 220 housekeeping items that contain full status of the instrument hardware, including data configuration, lamp and shutter status, gimbal position, a time stamp, and the target ID and macro within which the frame of data was taken. These parameters are stored as part of an Experiment Data Record (EDR), which consists of raw data. The data in one EDR represent a series of image frames acquired with a consistent instrument configuration (shutter position, frame rate, pixel binning, compression, exposure time, on/off status and setting of different lamps). Each frame has dimensions of detector columns (spatial samples) and detector rows (wavelengths, or bands). The multiple image frames are concatenated, and are formatted into a single multiple-band image (suffix *.IMG) in one file, plus a detached list file in which each record has housekeeping information specific to one frame of the multiple-band image (suffix *.TAB). The text file is based on the 220 housekeeping items. Five of theitems are composite in that each byte encodes particular information on gimbal status or control. These separate items are not broken out, except for the gimbal status at the beginning, middle, and end of each exposure, from which gimbal position is broken out (3 additional items). The housekeeping is pre-pended with 10 additional frame-specific items useful in data validation, processing, and sorting, for a total of 233 items per frame. Further information can be found in the data product SIS in the DOCUMENT directory. The multiple-band image has dimensions of sample, line, and wavelength. The size of the multiple-band image varies according to the observation mode but is deterministic given the macro ID. A typical multiple-band image might have XX pixels in the sample (cross-track) dimension, YY pixels in the line (along-track) dimension, and ZZ pixels in the wavelength dimension, where: XX (samples) = 640/binning, where 640 is the number of columns read off the detector, and binning is 1, 2, 5, or 10; YY (lines) = the number of image frames with a consistent instrument configuration; and ZZ (bands) = the number of detector rows (wavelengths) whose read-out values are retained by the instrument. Data ==== There is only one data type associated with this volume, the raw uncalibrated DNs. Ancillary Data ============== There are various types of ancillary data provided with this dataset: 1. SPICE kernels, used to interpret observational geometry, are available in the GEOMETRY directory. See GEOMINFO.TXT for more details. 2. The EXTRAS directory contains materials that may be helpful but are beyond the scope of the required elements of the archive. This includes a time ordered history of observations and the characteristics of the sites observed, as well as the configuration-managed history of the hardware and software state of the CRISM instrument. See EXTRINFO.TXT for more details. 3. The BROWSE directory contains browse images in PNG format, and HTML documents that support a web browser interface to the volume. See BROWINFO.TXT for more details. Coordinate System ================= The cartographic coordinate system used for the CRISM data products conforms to the IAU planetocentric system with East longitudes being positive. The IAU2000 reference system for Mars cartographic coordinates and rotational elements was used for computing latitude and longitude coordinates. Media/Format ============ The CRISM archive will be made available online via Web and FTP servers. This will be the primary means of distribution. Therefore the archive will be organized as a set of virtual volumes, with each data set stored online as a single volume. As new data products are released they will be added to the volume's data directory, and the volume's index table will be updated accordingly. The size of the volume will not be limited by the capacity of the physical media on which it is stored; hence the term virtual volume. When it is necessary to transfer all or part of a data set to other media such as DVD for distribution or for offline storage, the virtual volume's contents will be written to the other media according to PDS policy, possibly dividing the contents among several physical volumes.
Data Set Overview ================= This volume contains portions of the CRISM Calibration Data Record (CDR) Archive, a collection of calibration files from the Compact Reconnaissance Imaging Spectrometer for Mars on the Mars Reconnaissance Orbiter spacecraft. CDRs are created in two ways: (a) One type of calibration file, for example the wavelength calibration, is either invariant or expected to change only infrequently. These files are stored in directories named using a 2-letter acronym for the contents of the file. (b) The other type of calibration file is a snapshot of some attribute of the instrument that is time-variable, for example the thermal background measured by the IR detector. These files are stored in directories named YYYY_DOY, each one of which contains subdirectories containing distinct types of calibration files. Processing ========== EDRs containing bias measurements or measurements of background, focal plane lamps, or the internal integrating sphere are processed into level-4 CDRs. A level-4 CDR contains derived values needed to convert a scene-viewing EDR into units of radiance. These are in image format, with multiple versions corresponding to different pixel binning states. Other level 4 CDRs are derived from ground measurements. A level-6 CDR contains tabulated information, of two types: (a) a file list of level 4 CDRs generated inflight, or (b) tabulated information, for example, for correcting for detector non-linearity, converting housekeeping to physical units, or converting EDRs from 12 to the original at-sensor 14 bits prior to calibration. Data ==== There are two types of calibration files. CODMAC level 4 CDRs (CDR4s) are in image format, and CODMAC level 6 CDRs (CDR6s) are text files in ASCII format. Both types have a file name that encodes the time at which they are applicable, and the configuration of the data used to derive them (detector, frame rate, binning, wavelength filter, exposure time parameter, etc.). YYYY_DOY/BI directory: ---------------------This directory contains CDR4s representing measurements of the response of each detector to zero signal, i.e., the detector bias. These are measured as dark frames for the VNIR detector. For the IR detector, these are the zero exposure-time intercept of a fit of dark measurements taken at several exposure times. For each detector there are separate files for each frame rate, wavelength filter, and binning state, time-stamped for their mean time of acquisition. YYYY_DOY/BK directory: ---------------------This directory contains CDR4s representing thermal background measured by the IR detector. These are measured as dark frames. There are separate files for each frame rate, exposure time, and binning state, time-stamped for their mean time of acquisition. YYYY_DOY/BP directory: ---------------------This directory contains CDR4s representing bad pixels. There are separate files for each frame rate, exposure time, wavelength filter, and binning state, time-stamped for their mean time of acquisition. YYYY_DOY/SP directory: ---------------------This directory contains CDR4s representing sphere output in DN/ms at the sphere's closed-loop setpoint. There are separate files for each sphere bulb. For the VNIR all data are taken at 1 Hz, but there are versions for each binning state. For the IR there are separate versions for each frame rate, wavelength filter, and binning state. Both the VNIR and IR files are time-stamped for their mean time of acquisition. YYYY_DOY/ST directory: ---------------------This directory contains a table of CRISM low-rate telemetry in raw counts, from the beginning of a UTC calendar day to its end. This is used in preference to the telemetry attached to each image for correction of thermal effects, because of uncorrectable artifacts in the raw values of critical temperature measurements in the image headers. YYYY_DOY/UB directory: ---------------------This directory contains CDR4s representing pixel-by-pixel uncertainties in VNIR bias or IR background images. There are separate files for each frame rate, exposure time, wavelength filter, and binning state, time-stamped for their mean time of acquisition. This product is not a part of the nominal data processing pipeline and instead is used to monitor detector health. YYYY_DOY/ATF directory: ---------------------This directory contains a table of EDRs containing scene data and the corresponding EDRs containing time-dependent calibration measurements needed to calibrate the scene EDRs. It is used to process calibration EDRs to CDRs, and scene EDRs to TRDRs. If there is a discrepancy between the actual EDRs used for calibration and the predicted EDRs (in the BTF directory), the TRDRs resulting from scene EDRs are quality-flagged. YYYY_DOY/BTF directory: ---------------------This directory contains a predicted table of EDRs containing scene data and the corresponding EDRs containing time-dependent calibration measurements. It is constructed from uplinked commands. AS directory: ------------This CDR6 gives the maximum expected 14-bit scene or sphere 14-bit DN level (units from the detector, after correction for the pixel processing table and subtraction of bias) for core observation types to be made by CRISM. There are 5 core observation types defined by combinations of frame rate, exposure time, and scene: 1 Hz observation, exposure parameter 27, observing the internal integrating sphere; 3.75 Hz, exposure parameter 301, observing a nominally non-ice region of Mars; 3.75 Hz, exposure parameter 147, observing a potentially ice-containing region; 15 Hz, exposure parameter 425, observing any region; and 30 Hz, exposure parameter 425, observing any region. The values given here are used with bias images in the YYYY_DOY/BI directories to generate masks of probably bad pixels (in the YYYY_DOY/BI directories). AT directory: ------------This CDR4 gives the atmospheric transmission as measured from a nadir-pointed hyperspectral scan across Olympus Mons, taken 27 November 2006. It is not used during calibration to radiance or I/F; instead it is intended for subsequence correction for atmospheric attenuation. There is a different version for each detector, wavelength filter, and binning state. BS directory: ------------This CDR6 gives the amplitude of the bias step correction for each detector as a function of frame rate and quadrant. The bias step is a step function in detector bias that occurs at a row number (band) correlated with exposure time parameter. BW directory: ------------This CDR6 gives the parameters to calculate the spectral profile for each wavelength bin averaged over IR columns 270-369 or VNIR columns 260-359. This represents the 'sweet spot' of each detector at which spectral smile and keystone are minimum, and is the preferred detector location for extraction of overlapping data for analysis of emission phase functions. CM directory: ------------This CDR4 gives the along-slit angle measured from slit center for each detector. It is similar to the SPICE Instrument Kernel. There is a different version for each detector, wavelength filter, and binning state. CT directory: -----------This CDR6 gives the atmospheric transmission for each wavelength bin averaged over IR columns 270-369 or VNIR columns 260-359. This represents the 'sweet spot' of each detector at which spectral smile and keystone are minimum, and is the preferred detector location for extraction of overlapping data for analysis of emission phase functions. It is a sample of the 'native' atmospheric transmission images in the AT directory. DB directory: ------------This CDR6 gives the correction to detector bias for changes in detector temperature since the last bias measurement, as a function of frame rate and quadrant. DM directory: ------------This CDR4 is a mask of detector dark columns, scattered light columns, and scene columns for each detector. There is a different version for each wavelength filter and binning state. EB directory: ------------This CDR6 gives the correction to detector bias for differences in focal plane electronics temperature since the last bias measurement, as a function of frame rate and quadrant. GH directory: ------------This CDR6 gives the scaling factors needed for each frame rate to calculate and subtract the inter-quadrant electronics ghost in each detector, using bias-corrected signal. HD directory: ------------This CDR6 gives the coefficients for correcting raw housekeeping for perturbations due to changes in lamp or cooler state and frame rate. There is one file for both detectors. HK directory: ------------This CDR6 gives the coefficients to convert housekeeping parameters from raw counts to physical units. HV directory: ------------This CDR6 gives the coefficients to correct housekeeping voltages for perturbations due to changes in current. LC directory: ------------This CDR6 gives the coefficients to correct bias-subtracted DN from each detector at each frame rate for non-linearity in detector response. LI directory: ------------This CDR6 gives 8 to 12 bit lookup tables (inverse of 12 to 8). These are used to restore 8-bit to 12-bit DN values. LK directory: ------------This CDR6 gives 12 to 8 bit lookup tables loaded in the CRISM instrument. LL directory: ------------This CDR4 contains matrices to remove estimated leaked higher order light in the IR detector, by scaling shorter-wavelength signal and subtracting it from longer-wavelength signal. NU directory: ------------This CDR4 is a time-tagged, row-normalized measurement of detector nonuniformity. There is one version for each binning state, wavelength filter, and frame rate. It is only created for the VNIR detector. PP directory: ------------This CDR6 contains the gain and offset to use for each row to convert 12-bit DNs to the native 14-bit DNs produced by each detector, and the 12 to 8 bit lookup tables used for each detector row (band) when lossy compression is employed. PS directory: ------------This CDR4 is used for nearest neighbor spectral resampling. It gives the number of detector rows (bands) by which to shift each column (sample) of an image to minimize the effects of spectral smile. The effect of the pixel shift is to restore the wavelength at a detector element to within one-half detector element of the wavelength in the central columns of the detector. (The wavelength and spectral bandwidth in the central columns are given in the SW and BW directories respectively.) There is a different version for each detector, wavelength filter, and binning. RA directory: ------------This CDR4 contains the ratio of focal plane lamp illumination to an external flat field, normalized on a row-by-row basis. This is a contingency and not part of the normal calibration pipeline, so its creation is TBD. There is a different version for each detector, wavelength filter, and binning. RF directory: ------------This CDR4 contains a nearest-neighbor resampled solar flux image. It is related to the 'native' solar flux images in the SF directory by the pixel- shifting given in the PS directory. There is a different version for each detector, wavelength filter, and binning. RT directory: ------------This CDR4 gives the atmospheric transmission as measured from a nadir-pointed hyperspectral scan across Olympus Mons, taken 27 November 2006. It has been nearest-neighbor resampled in the spectral direction. It is not used during calibration to radiance or I/F; instead it is intended for subsequence correction for atmospheric attenuation. It is related to the 'native' atmospheric transmission images in the AT directory by the pixel- shifting given in the PS directory. There is a different version for each detector, wavelength filter, and binning. There is a different version for each detector, wavelength filter, and binning state. RW directory: ------------This CDR4 contains a nearest-neighbor resampled wavelength image. It is related to the 'native' wavelength images in the WA directory by the pixel- shifting given in the PS directory. There is a different version for each detector, wavelength filter, and binning. SB directory: ------------This CDR4 contains the spectral bandpass width for each detector element. There is a different version for each detector, wavelength filter, and binning. SF directory: ------------This CDR4 contains solar flux at 1 AU for each detector element. There is a different version for each detector, wavelength filter, and binning. It is used together with the solar distance given in an EDR or TRDR label to calculate I/F. SH directory: ------------This 2-layer CDR4 is used to correct flight measurements of the onboard integrating sphere for effects of shutter position irreproducibility. Layer 0 is a corrected sphere image from ground radiometric calibration in units of 14-bit DN. Layer 1 is a shutter mirror non-repeatability correction image, a multiplicative correction that gets scaled by the ratio of layer 0 to the measured flight sphere image. There is a different version for each detector, wavelength filter, and binning. SL directory: ------------This CDR6 gives the 14-bit DN at which each detector is saturated, as a function of quadrant and frame rate. SS directory: ------------This 3-layer CDR4 gives sphere spectral radiance at the closed-loop set point for each sphere bulb, as pixel by pixel coefficients to a 2nd order polynomial function of sphere temperature. There is a different version for each detector, wavelength filter, and binning. SW directory: ------------This CDR6 gives the center wavelength for each wavelength bin averaged over IR columns 270-369 or VNIR columns 260-359. This represents the 'sweet spot' of each detector at which spectral smile and keystone are minimum, and is the preferred detector location for extraction of overlapping data for analysis of emission phase functions. TD directory: ------------This CDR4 gives temperature dependence of detector responsivity, as pixel by pixel coefficients to a 2nd order polynomial function of detector temperature. There is a different version for each detector, wavelength filter, and binning. UR directory: ------------This CDR6 gives uncertainty in absolute and relative sphere signal, for each bulb. This product is not a part of the nominal data processing pipeline and its creation is TBD; it is intended for end users of calibrated data as an indication of limits to data interpretation. VL directory: ------------This CDR6 gives the 14-bit DN at which each detector ON AVERAGE - is saturated. It is used together with the predicted limiting case expected 14-bit scene DN levels (in the AS directory) to produce bad pixel maps. It also gives additional criteria for defining a bad pixel, thresholds for noise and relative pixel response. WA directory: ------------This CDR4 gives the center wavelength for each detector element. There is a different version for each detector, wavelength filter, and binning. WV directory: ------------This CDR6 gives the table of detector rows (bands) loaded into wavelength filters 0-3. There is one version for each detector. File Naming Convention ====================== The file naming convention for level-6 CDRs is as follows. (ProductType)(Level)_(Partition)_(Time)_(Product)_ (SensorID)_version.(Ext) where: Product Type = CDR Level = 6 Partition = n, partition of the spacecraft clock. Time = nnnnnnnnnn, spacecraft start time of applicability of data product; units are spacecraft clock counts, in units of whole seconds. Product = nn, acronym describing data product listed above Sensor ID = S or L (or J=joint) Version = 0, 1,..., 9, a,..., z Ext = TAB The file naming convention for level-4 CDRs is as follows. (ProductType)(Level)(Partition)(Time)_ (Product)(FrameRate)(Binning) (ExposureParameter)(WavelengthFilter) (Side)(SensorID)_ version.(Ext) where: Product Type = CDR Level = 4 FrameRate = n, rate in Hz at which data are taken (0=1 Hz, 1=3.75 Hz, 3=156 Hz, 4=30 Hz, 5 = N/A) Binning = n, number of spatial pixels binned (0=unbinned, 1= 2x binned, 2= 5x binned, 3= 10x binned, 4=N/A) Exposure parameter = nnn, an integer 1-480 indicating commanded exposure time in units of (inverse frame rate)/480; 000 if inapplicable Wavelength filter = n, and integer 0-3 indicating which onboard menu of rows of the detector are represented Side = #, 1 or 2 for focal plane or sphere bulbs; or 0 if N/A Ext = IMG The tables in the YYYY_DOY/ATF and YYYY_DOY/BTF directories have a different file naming convention, as follows. (Product)_(Sensor)_(YYYY)_(DOY)_version where: Product Type = BTF for before-the-fact predicted or ATF for after-the-fact actual Sensor = VN or IR YYYY = year DOY = day of year Version = nn Media/Format ============ The CRISM archive will be made available online via Web and FTP servers. This will be the primary means of distribution. Therefore the archive will be organized as a set of virtual volumes, with each data set stored online as a single volume. As new data products are released they will be added to the volume's data directory, and the volume's index table will be updated accordingly. The size of the volume will not be limited by the capacity of the physical media on which it is stored; hence the term virtual volume. When it is necessary to transfer all or part of a data set to other media such as DVD for distribution or for offline storage, the virtual volume's contents will be written to the other media according to PDS policy, possibly dividing the contents among several physical volumes.
These data are available on-line from the Planetary Data System (PDS) at:
Questions and comments about this data collection can be directed to: Dr. David R. Williams
Name | Role | Original Affiliation | |
---|---|---|---|
Dr. Scott Murchie | Data Provider | Applied Physics Laboratory | scott.murchie@jhuapl.edu |
Dr. Scott Murchie | General Contact | Applied Physics Laboratory | scott.murchie@jhuapl.edu |