Spectops stores its measurement results in plain-text files.
Records: Full measured records (include electrometer signals and diagnostic data)
File names: {year}{month}{day}-{avg}.records
Example: 20130203-block.records
Spectra: Aerosol/ion number density distributions.
File names: {year}{month}{day}-{avg}-{opmode}.spectra
Example: 20130203-block-ions.spectra
Log: Program log file
File names: {year}{month}{day}.log
Comments: User comments file
File names: {year}{month}{day}-{avg}.comments
All file names start with the date in local time zone. Different averaging variants are stored in different files: the {avg} field in the name will be block, 10s, 1s, etc. Also spectra for different operating modes are stored in different files: the {opmode} will be particles, ions, etc.
The record and spectra files have a similar format. Both types of files contain header sections where lines start with the # character and data lines similar to CSV files where the field separator is the TAB character. End of line character is LF.
The header section starts with the line # Spectops spectra in case of spectra files or # Spectops records in case of record files.
All the immediately following lines that start with # constitute the header section. The header section is followed by a single column labels line and then many data lines.
The files start with a number of header lines that all start with the # character. The headers contain descriptions of the data, e.g. name of the spectra, units, names of diagnostic parameters. The headers lines may occur in the middle of the files as well in case the measurements are restarted with a different configuration and the data description changes. (For example when the inversion matrices are replaced).
The headers can be parsed by software. After removing the # characters from the start of the lines the header sections are valid YAML documents (See YAML file format). The Retrospect program uses the header sections process the data.
The exact number of lines in the header section is not specified and may change when the measurement software or instrument configuration files are updated.
The header section is followed by a line with column titles:
begin_time end_time opmode ...
The data records follow the columns line with one record per line.
The times are given in ISO 8601 format with fractional seconds and including the time zone difference:
YYYY |
- |
MM |
- |
DD |
␣ |
hh |
: |
mm |
: |
ss.zzzzzz |
+hh:mm |
| year | month | day | hours | minutes | seconds | ± timezone |
Note that the + or – in front of the timezone specification may be counterintuitive. The hours marked with + have to be subtracted to get the UTC time, marked with – have to be added.
Also note that there is a space character between day and hour, not T as the ISO standard specifies.
The record files are the most important data files of the instrument. They contain the full measured records:
| Field description | Columns |
|---|---|
| Begin time | begin_time |
| End time | end_time |
| Operating mode | opmode |
| Processed average electrometer currents | cur_0 ... cur_n |
| Electrometer current variance estimates | curvar_0 ... curvar_n |
| Raw average electrometer currents | rawcur_0 ... rawcur_n |
| Average electrometer voltages | volt_0 ... volt_n |
| Instrument variables | (various) |
| Diagnostic flags | flags |
!!! Always include record files when archiving data. Information about the health of the instrument and the quality of the measurements is only stored in the records files. Spectra files can be calculated from the record files any time using the Retrospect program.
The instrument records may have a number of diagnostic flags attached to them.
The flags are saved in last field of the data record line as a number of flag codes separated by exclamation marks. For example: !31!11f!92!e0.
The meaning of the flag codes are not defined in the header section, but on lines between the measurement records that start with # flag. For example:
# flag a4: {message: '+ prefilter voltage is too low'}
The reason for this is to limit the size of the header. There is a very large number of different diagnostic flags but most of them will not be raised during normal measurements. So each flag definition is added to the data files only when the flag occurs.
The format of the flag definition line is #flag (code): (contents).
The flag code is an arbitrary string. Usually it is a hexadecimal number.
The contents of the flag is an arbitrary string. Generally it is a JSON (or YAML) encoded object with the fields "message" (explanation of the issue), "type" (indicates seriousness, e.g. note, warning or failure), "elms" (electrometer channels that caused the issue), "vars" (measurement parameters that caused the issue).
When a new header section occurs in the data file then previous flag definitions are no longer valid. A new header section indicates that the measurement program was restarted with a different configuration.
Spectra files contain the actual particle size distributions. A separate file is made for each operating mode.
EAS produces only particles spectra, so there are only the particles spectra files.
NAIS will produce separate spectra files for ions, particles and other modes. Each file will contain both positive and negative analyzer spectra for the corresponding operating mode.
The datafiles contain the number density spectra values in the columns sp_... and their standard deviations (measurement uncertainties) in the columns sperr_....
Log files contain the same program log that is visible in the Log window in Spectops.
Comments files contain the user entered comments. The comments files may be modified (or created if missing) when comments are changed or added in Retrospect.