Configuration

The program reads two YAML configuration files from the working folder (the folder where you run the program):

• config.yaml — Main configuration
• secrets.yaml — Sensitive values (passwords, credentials)

Variable Substitution

Configuration files support ${variable} placeholders:

• ${device_id} is automatically set to the device ID
• Other variables are read from secrets.yaml

This allows sharing configuration files while keeping passwords private in a separate file.

Command Line Options

cic-daq [OPTIONS]

Options:
  -i, --device-id <DEVICE_ID>  Override device ID from config file
  -a, --address <ADDRESS>      Override device address (IP or hostname)
  -l, --log <LOG_DIR>          Enable file logging to specified directory
  -h, --help                   Print help
  -V, --version                Print version

Basic Settings

# Device serial number (required)
# Use "*" for dynamic mode - automatically discover any available CIC-2 device
device_id: CIC-2-0

# Device IP address (optional - uses auto-discovery if omitted)
address: 192.168.1.100

# Require specific software version (optional, e.g., ">=0.5.0" means version 0.5.0 or newer)
require_version: ">=0.5.0"

Fixed vs Dynamic Device ID

The device_id setting supports two modes:

Fixed mode (default): Set device_id to a specific device serial number (e.g., CIC-2-37). The program will only connect to that specific device and wait indefinitely if it's not available.

Dynamic mode: Set device_id: "*" (asterisk) to automatically connect to any available CIC-2 device. This is useful for situations where you want the program to adapt to whatever device is available. Dynamic mode is not recommended when you have more than one CIC in the local network because it cannot be specified which device the program will connect to.

In dynamic mode:

• On startup, the program discovers the first available CIC-2 device
• If the device becomes unavailable, the program attempts to reconnect exclusively to that device for 2 minutes, then restarts discovery and will connect to any available device
• When a different device is found, all state is reset (data processing, storage buffers, etc.)
• The ${device_id} placeholder in configuration is substituted with the discovered device ID

Example dynamic mode configuration:

device_id: "*"

cycle:
  - [ offset, 30 ]
  - [ ions, 30 ]

sinks:
  - !RecordsFile
    source: 1s
    output_folder: data/${device_id}/1s    # Will use discovered device ID

You can also combine dynamic mode with a specific address to probe a known device and determine its ID:

device_id: "*"
address: 192.168.1.100    # Connect to this address, use whatever device is there

Measurement Cycle

The cycle defines a sequence of operating modes. Each entry has a name and duration in seconds:

cycle:
  - [ offset, 30 ]
  - [ ions, 30 ]

The cycle repeats continuously. Two operating modes are available:

• ions — Normal ion measurement mode.
• offset — The inlet filter is switched on, preventing ions from entering the analyzer. This allows measuring the electrometer zero signal and estimating noise levels. Used to correct for electrometer zero drift and evaluate device health. A consistently high offset signal (above 10 fA absolute value) indicates the device may require maintenance.

Data is collected into memory until an offset period completes. At that point, the software uses the previous and current offset periods to estimate zero signals and noise levels, applies corrections to the ion measurements between them, and sends the processed data to storage. Only the most recent offset period is kept in memory for the next cycle.

Measurement cycle synchronization: The cycle is synchronized to wall-clock time, not program start time. For example, with [offset, 30], [ions, 90] (total 120 seconds), offset always runs at :00–:30 and ions at :30–:00 of each 2-minute period. If the program starts at 14:35:47, it will begin mid-cycle in the ions period and complete normally from there. This ensures consistent timing across restarts and makes it easier to compare data from different sessions or devices.

Output Sinks

Multiple output sinks can be configured. Each sink receives processed data records.

RecordsFile Sink

Writes data to plain text files with daily rotation. The file format is compatible with Spectops and Retrospect programs:

sinks:
  - !RecordsFile
    source: block           # Record type to write
    output_folder: data/${device_id}/block

Options:

• source — Record type: block or 1s
• output_folder — Output directory (created if needed)

MQTT Sink

Streams live data to an MQTT broker for processing by other software (e.g., Python scripts) for custom storage or analysis. Also supports streaming high time resolution data useful for Eddy covariance measurements.

The MQTT sink also listens for commands (e.g., disabling blowers, enabling heating) allowing live control of the measurement device.

Optional Home Assistant integration publishes metadata about MQTT topics, making it easy to build a dashboard to check device health and control the devices. This is especially useful when monitoring a large number of devices:

sinks:
  - !Mqtt
    host: mqtt.example.com
    port: 1883
    client_id: cic-daq-${device_id}
    username: ${mqtt_username}
    password: ${mqtt_password}
    transport: tcp                    # tcp, tls, or tls_os
    discovery_topic: homeassistant    # Optional: Home Assistant discovery

Options:

• host — MQTT broker hostname
• port — MQTT broker port (typically 1883 for TCP, 8883 for TLS)
• client_id — MQTT client identifier
• username, password — Authentication credentials
• transport — Connection type:
  • tcp — Unencrypted connection
  • tls — Encrypted connection with custom certificates
  • tls_os — Encrypted connection using system certificates
• discovery_topic — Home Assistant MQTT discovery prefix (optional)

For TLS with custom certificates:

sinks:
  - !Mqtt
    host: mqtt.example.com
    port: 8883
    client_id: cic-daq-${device_id}
    username: ${mqtt_username}
    password: ${mqtt_password}
    transport: tls
    ca: /path/to/ca.crt
    client_cert: /path/to/client.crt
    client_key: /path/to/client.key

ClickHouse Sink

Writes data to a ClickHouse database:

sinks:
  - !Clickhouse
    url: tcp://user:password@clickhouse.example.com:9000/database
    retry_path: retry_buffer.sqlite    # Optional: SQLite buffer for retries
    tables:
      - source: 1s
        table: measurements_1s
      - source: block
        table: measurements_block
        constant_columns: # Optional: additional columns
          - [ station_id, "station1" ]

Options:

• url — ClickHouse connection URL
• retry_path — SQLite file for temporarily storing data when database is unreachable (optional)
• tables — List of source-to-table mappings:
  • source — Record type
  • table — Target table name
  • constant_columns — Additional constant values to insert

Startup Parameters

Optional section to set device parameters on connection. Use null to reset a parameter to device default.

startup_parameters:
  heater_level: 1
  red_flow_rate_target: 30.0
  blue_flow_rate_target: 30.0

Common parameters:

• heater_level — Heater level 0–3
• red_flow_rate_target, blue_flow_rate_target — Flow rate targets (l/min)

Additional parameters are available but generally do not need to be modified for normal operation.

Record Types

The program generates several record types at different time resolutions:

Use _preliminary versions for debugging. Final versions (block, 1s) include offset-corrected currents using zero signal estimates from the surrounding offset periods.

Complete Example

device_id: CIC-2-11
address: 192.168.1.50

cycle:
  - [ offset, 60 ]
  - [ ions, 120 ]

sinks:
  # Local file storage
  - !RecordsFile
    source: block
    output_folder: data/${device_id}/block
  - !RecordsFile
    source: 1s
    output_folder: data/${device_id}/1s

  # MQTT for real-time monitoring
  - !Mqtt
    host: mqtt.local
    port: 8883
    client_id: cic-daq-${device_id}
    username: ${mqtt_username}
    password: ${mqtt_password}
    transport: tls_os
    discovery_topic: homeassistant

  # Database storage
  - !Clickhouse
    url: tcp://${ch_user}:${ch_password}@db.example.com:9000/measurements
    retry_path: /var/lib/cic-daq/retry.sqlite
    tables:
      - source: 1s
        table: cic_1s
      - source: block
        table: cic_block

startup_parameters:
  blowers_enabled: true
  heater_level: 1

Corresponding secrets.yaml:

mqtt_username: cic_client
mqtt_password: secret_password
ch_user: cic_writer
ch_password: database_secret