Introduction

This document defines the latest CSV specification for the Emteq DAB CSV specification.

Note

Please observe the #Format/Version when parsing CSV data.

Note

The specification is actively being updated and changes can be identified in the chapter Format Revisions.

Rules

Formatting

  1. # prefix on comment-lines (See Meta-Data Comments)
  2. , separated data for all areas of document
  3. #Format/Version,CSV1.0.0 as the first line of the file
  4. Unix line ending LF (MS-DOS CR+LF prior to CSV1.0.7)
  5. Comment lines may appear at any time
  6. CSV Header-row shall always be present on the first non-comment line
  7. CSV Data-row shall contain the same number of delimited fields
  8. Last line shall always include line ending characters

Field-Naming

  1. Data is named using ‘Path like' syntax.
  2. First character of module is upper-case i.e. UpperCamelCase/
  3. First character of member field is lower-case i.e. .lowerCamelCase
  4. Path syntax uses / class-module scopes
    e.g. Module/Class or Module/Class/Subclassetc.
  5. Path syntax uses . for referencing member field
    e.g. Module/Class.firstMember
  6. Path syntax uses [] postfix for Array of data
    e.g. Module/Class.valueArray[],0,1,2,3,4,5,6,7
  7. Path syntax uses {} postfix for Key-Value data
    e.g. Module/Class.valueArray{},valueA,0,valueB,1,valueC,2

Examples

This chapter contains examples for each formatting rule.

  • First Line

#Format/Version,CSV1.0.0

  • Meta-Data

#Device/Version.serialId,DAB6N0tqTM5OVMgICA4KiEQ/w

  • Meta-Data Key-Value

#Imu/Version{},firmware,3.017,bootloader,21,accelerometer,251,magnetometer,50,gyroscope,15

  • Meta-Data Array-Value

#Emg/Properties.ids[],RightFrontalis,RightZygomaticus,...,LeftFrontalis

  • Header-row

Frame#,Time,Faceplate/FaceState,Faceplate/FitState

  • Data-row

123,0.01052,1,1,4,4,0,0,35078,1247797,31721

Data Definitions

Meta Data

This chapter contains detail about all CSV Meta-Data

Note

Comment-lines are used to store Meta-Data thatdescribes how to use the contained CSV Data

#Format

#Format/Version

Version of the CSV specification which the data file is based upon
e.g. #Format/Version,CSV1.0.3

#Format/Deprecate

Deprecation flag for a CSV class means that the data is not intended to be supported in future specification or has been replaced.

Please observe the #Format/Version when parsing CSV data. Developers should prepare to migrate away from use of deprecated classes as they will be removed in a future/next revision of the specification.
e.g. #Format/Deprecate,Imu/* indicates the all of Imu/ types are planned for removal.

See the change-log and respective class specific data documentation for details

#File

#File/Generator

Application that generated this CSV file as a Name and Version pair
e.g. #File/Generator,dab2csv,v0.4.0

#File/Normalised

Generator setting for whether all measurement data is to be normalised'.
e.g. #File/Normalised,YES

When a file is ‘Normalised' the data will have been pre-scaled to the output unit specified in {Measurement}/Properties

Disk-space requirement is significantly increased when converting data to Normalise-CSV format

#File/Source

Source DAB filename from which the CSV was converted from
e.g. #File/Source,recording_2020_01_02_1230.dab

When using dab2csv this field can only be populated if the filename is provided on the command line. i.e. This field is omitted when stdin pipe is used.

#Protocol

#Protocol/Version

Binary communications protocol revision that the Emteq DAB device supports. This must be compatible with the tool used to generate the data-file.
e.g. #Protocol/Version,MASK.1.3

Current dab2csv is only backwards-compatible and must be updated to convert more recent protocol revision data.

#Protocol/Log.message

Developer/diagnostic log message with generic content. Some common messages:

AdsMiss & AdsLate

The EMG device failed to read a number of measurements. This can be caused by:

  1. DAB device busy initialising - can occur near beginning of high-rate data capture and may be resolved in future firmware revisions.
  2. Host computer is not reading the USB fast enough. This is an issue with the Host-Computer and/or Software. This sort of issues may be fixed with updated Host PC software or closing other apps or removing unnecessary USB devices etc.

#Stream

#Stream/Generator

Creator of measurement data as Name and Value pair
e.g. #Stream/Generator,Dab,v0.6.2

#Stream/Source

#SelfTest

#SelfTest/Begin

Beginning of Emteq DAB power-on-self-test (POST)

#SelfTest/End

Indicates the result of power-on-self-test (POST)

#Firmware

#Firmware/Build.buildTag

The revision of application firmware that is running on the Emteq DAB device that generated this data file.
e.g. #Firmware/Build.buildTag,v0.4.6-0

#Firmware/Bootloader.buildTag

The revision of bootloader firmware that is running on the Emteq DAB device.
e.g. #Firmware/Bootloader.buildTag,v1.8.0

#Device

#Device/Version.serialId

The unique hardware device indicator that identifies the Emteq DAB device embedded inside your Emteq headset.
e.g. #Device/Version.serialId,DABqWeRtYuIoPaSd123/w

This should be included in support communications as provides a means to identify the device that any issues are related too.

#Device/Version.hardware

The hardware revision of the Emteq DAB board this data file was created from.

This information is required when performing firmware updates to ensure that the application is compatible with the device revision.

#Faceplate

#Faceplate/Version.serial

The unique identifier for the flexible-mask faceplate that is attached to the Emteq DAB device.
e.g. #Faceplate/Version.serial,oQwErTyUiO

#Faceplate/Version.hardware

The hardware revision of the flexible-mask faceplate that is attached to the Emteq DAB device.

This may not be populated for early-production devices

#Emg

#Emg/Version.model

Hardware device model which is used for EMG acquisition. The model may have specific measurement characteristics which may be published if/as these are determined by Emteq.

#Emg/Properties.sensorCount

The number of EMG channels that are collected in this data file.
e.g. #Emg/Properties.sensorCount,7

#Emg/Properties.contactMode

Method of skin-contact that is in-use. The current modes are:

  • DC - Provides ‘Binary' ON/OFF contact information for each electrode on the mask independently. This data is reported in Emg/ContactStates
  • AC - Provides accurate ‘Analog' measurement of skin-contact impedance for each electrode pair on the mask. This data is reported in Emg/Contact with a threshold applied to give Emg/ContactStates

#Emg/Properties.measureHertz

Underlying hardware data-acquisition rate of EMG data.

#Emg/Properties.rawToVoltageDivisor

CSV data is unprocessed by default and a scalar can be applied to produce normalised measurement in the specified unit.

e.g. #Emg/Properties.rawToVoltageDivisor,25165824.0,volt

Note

#File/Normalised of YES indicates the scalar conversion is intrinsically applied to the data

This scalar may be used to convert Emg/Raw, Emg/Filtered, and Emg/Amplitude values as volts
volts = raw / rawToVoltageDivisor

#Emg/Properties.contactToImpedanceDivisor

CSV data is unprocessed by default and a scalar can be applied to produce normalised measurement in the specified unit.

e.g. #Emg/Properties.contactToImpedanceDivisor,0.5,ohm

#File/Normalised of YES indicates the scalar conversion is intrinsically applied to the data

This scalar may be used to convert Emg/Contact to Impedance as Ohms
ohms = contact / contactToImpedanceDivisor

#Emg/Properties.ids[]

Primary-muscle location of each Emg electrode.

Note

Order and assignment of Leads may change based on hardware version or manual electrode positioning (specific hardware versions).

It is highly recommended that:

  • Muscle-Ids shall be verified for every data file to ensure the data contains the desired biophysical measurements
  • Muscle-Ids shall be referenced when using EMG data that is numeric indexed e.g. Emg/Raw[0] is data from the first primary-muscle identifier.

#Emg/Properties.positions[]

Face-location of each Emg electrode.

This data is not yet defined/populated on any current Firmware

#Emg/CalibrationState

Emteq internal information about the Factory-Calibration status of the device

#Emg/Calibration[]

Emteq internal information about the Factory-Calibration status of the device

#Emg/Config/{Raw | Contact | Filtered | Amplitude}.hertz

The output-rate of the respective Emg data.

This applies to all data after this point in the data file but may be overridden by subsequent changes.

As the data-rate may be configured by the application on the Host computer either:
A) Avoid changing the setting during data-capture
B) ensure that the data-rate change is processed when it occurs mid-recording

#Imu

#Imu/ is deprecated as of CSV1.0.5, removed in CSV1.1.x

#Imu/Properties.accelerationDivisor

See #Accelerometer/Properties.rawDivisor

#Imu/Properties.magnetometerDivisor

See #Magnetometer/Properties.rawDivisor

#Imu/Properties.gyroscopeDivisor

See #Gyroscope/Properties.rawDivisor

#Imu/Properties.accelerometerHertzMax

See #Accelerometer/Properties.hertzMax

#Imu/Properties.magnetometerHertzMax

See #Magnetometer/Properties.hertzMax

#Imu/Properties.gyroscopeHertzMax

See #Gyroscope/Properties.hertzMax

#Imu/Config/Accelerometer.hertz

See #Accelerometer/Config.hertz

#Imu/Config/Magnetometer.hertz

See #Magnetometer/Config.hertz

#Imu/Config/Gyroscope.hertz

See #Gyroscope/Config.hertz

#Accelerometer

#Accelerometer/Version.model

Hardware device model.
e.g. #Magnetometer/Version.model,Bno055

The model may have specific measurement characteristics which may be published if/as these are determined by Emteq.

#Accelerometer/Properties.hertzMax

Maximum configurable data-acquisition rate.

v0.6.x firmware does not provide full configurability of IMU data rate.

#Accelerometer/Properties.rawDivisor

CSV data is unprocessed by default and a scalar can be applied to produce normalised measurement in the specified unit.
The scalar divisor used to convert raw Accelerometer values to m/s^2
e.g. mss = accRaw / accelerationDivisor

#Accelerometer/Calibration.source

Origin of sensor-data calibration information.

Source Value Condition
None 0 No calibration source
Factory 1 Calibration is applied from factory
Staged 2 Calibration is determined by initiating a staged procedure being performed e.g. Accelerometer places stable on all 6-sides.
Online 3 Calibration is determined online at runtime (background process)
User 4 Calibration is determined by user-provided calibration sent to device

#Accelerometer/Calibration.state

Condition/Maturity state of sensor-data calibration.

Source Value Condition
None 0 Indicates data as not calibrated, ‘potential' significant Bias or Offset @note is Source==None then user or factory calibration must be set (emteq internal)
Minimal 1 Indicates data calibration has begun, reduced Bias or Offset
Basic 2 Indicates data calibration is nearing maturity, significant confidence in removal of Offset or Bias
Mature 3 Indicates data as fully calibrated, highest confidence in removal of Offset or Bias

#Accelerometer/Calibration

Emteq internal information about the Calibration data for the device.

#Accelerometer/Config.hertz

The output-rate of the measurement data.

This applies to all data after this point in the data file but may be overridden by subsequent changes.

#Gyroscope

#Gyroscope/Version.model

Hardware device model.
e.g. #Gyroscope/Version.model,Bno055

The model may have specific measurement characteristics which may be published if/as these are determined by Emteq.

#Gyroscope/Properties.hertzMax

Maximum configurable data-acquisition rate.

v0.6.x firmware does not provide full configurability of IMU data rate.

#Gyroscope/Properties.rawDivisor

CSV data is unprocessed by default and a scalar can be applied to produce normalised measurement in the specified unit.
The scalar divisor used to convert raw Gyroscope values to degrees/s
e.g. degPerSecond= gyroRaw / gyroscopeDivisor

#Gyroscope/Calibration.source

Origin of sensor-data calibration information.

See #Accelerometer/Calibration.source for details of the source enumeration

#Gyroscope/Calibration.state

Condition/Maturity state of sensor-data calibration.

See #Accelerometer/Calibration.state for details of the 0-3 scale

#Gyroscope/Calibration

Emteq internal information about the Calibration data for the device.

#Gyroscope/Config.hertz

The output-rate of the measurement data.

This applies to all data after this point in the data file but may be overridden by subsequent changes.

#Magnetometer

#Magnetometer/Version.model

Hardware device model.
e.g. #Magnetometer/Version.model,Bno055

The model may have specific measurement characteristics which may be published if/as these are determined by Emteq.

#Magnetometer/Properties.hertzMax

Maximum configurable data-acquisition rate.

v0.6.x firmware does not provide full configurability of IMU data rate.

#Magnetometer/Properties.rawDivisor

CSV data is unprocessed by default and a scalar can be applied to produce normalised measurement in the specified unit.
The scalar divisor used to convert raw Magnetometer values to microTesla
microTesla = magRaw / magnetometerDivisor

#Magnetometer/Calibration.source

Origin of sensor-data calibration information.

See #Accelerometer/Calibration.source for details of the source enumeration

#Magnetometer/Calibration.state

Condition/Maturity state of sensor-data calibration.

See #Accelerometer/Calibration.state for details of the 0-3 scale

#Magnetometer/Calibration

Emteq internal information about the Calibration data for the device.

#Magnetometer/Config.hertz

The output-rate of the measurement data.

This applies to all data after this point in the data file but may be overridden by subsequent changes.

#Time

#Time/Seconds.referenceOffset

Data-time offset that may be added to Time values to give a J2000 time value as Seconds since 00:00:00 01-01-2000 (J.2000 Epoch).

J2000 Timestamp can be converted to Unix-Timestamp by adding 946684800
e.g. unixTime = referenceOffset + 946684800

When #Time/Epoch is J2000 this offset will be 0.0

Conversion to Unix Timestamp may overflow if using 32-bit data types so overflow must be taken into consideration.

#Time/Seconds.unixOffset

Data-time offset that may be added to Time values to give a Unix time value as Seconds since 00:00:00 01-01-1970 (Unix Epoch). e.g. unixTime = time + unixOffset

When #Time/Epoch is Unix this offset will be 0.0

#Time/Epoch

The ‘Epoch' that is used for the Time field. The Epoch specifies the arbitrary fixed date from which all values are relative to.

  • Session = Time at the start of the file is used as the Epoch. Time field will count up from 0.0
  • Unix [default] = Unix Epoch will be used. Time will be seconds since 00:00:00 01-01-1970
  • J2000 = J2000 Epoch will be used. Time will be seconds since 00:00:00 01-01-2000
Note

Prior to CSV1.1.1 the epoch was Session time. Since Dab2Csv v0.7.4 this can be confiugred while defaulting to Unix when not specified.

Irrespective of which Epoch is selected, #Time/Seconds.unixOffset can be used to adjust time to absolute Unix values.

Data-time at which the data recording began specified as Seconds since 00:00:00 01-01-1970 (Unix Epoch).

#Powerline

#Powerline/Config

{Documentation incomplete} [Introduced in v0.5.0]

Field Data

Frame#

Row index for human readable data references.

The Frame# will start at 1 and increment +1 for each data record row

Note

Frame# may not map directly to Time, in general a fixed rate can be assumed but gaps in data can/do occur under certain criteria (See ‘AdsMiss & AdsLate’)

Time

Relative Time in Seconds at which data was measured in the device hardware.

This may be converted to J2000 or Unix time values, the offset values to apply are available from #Time/Seconds.referenceOffset or #Time/Seconds.unixOffset respectively.

The configured Epoch can is also specified in #Time/Epoch.

Since CSV1.1.1 the Epoch defaults to `Unix`, previously the `Session` Epoch was used for all CSV data files.

Faceplate

Faceplate/FaceState

Discrete OFF>ON face information indicating when the device is detected as being worn by a user.

State Value Condition
Error -1 Fit detection failed e.g. No-Face-Plate
Off 0 No face contact detected
On 1 Face contact detected

Faceplate/FitState

Abstract continuous measurement of Mask ‘Fit' with higher values representing the ideal state of system performance/quality.

Average should a minimum threshold to start an session. Lower-levels than this shows degradation in fit while higher values indicate good preferential fit.

State Value Condition
Failure -1 Fit detection failure e.g. No-Face-Plate or Sensor failure
None 0 Not on-face
Measuring 1 Fit detection failed e.g. No-Face-Plate
Failing 2 .. 3 Minimal to detect face i.e. 3-4 pairs any contact
Failing 4 .. 5 Minimal to detect face i.e. 5-6 pairs any contact
Below-Average 6 .. 7 Basic function i.e. 3 pairs settled contact, 5-7 pairs full contact
Average 8 General function i.e. 5 pairs settled contact, 7 pairs full contact
Very-Good 9 Good on all sensors i.e. 7 pairs settled contact
Excellent 10 Optimal on all sensors i.e. 7 pairs settled contact + Excellent impedance
Perfect 11 Ideal sensor Impedance. Not possible on user-face.

Faceplate/FitState.any#

Supplementary data counting number of electrodes which have any Contact on either electrode of the pair.

Note

Removed as of #CSV1.1.0. See Emg/ContactStates

Faceplate/FitState.both#

Supplementary data counting the number of electrode-pairs which are both in contact.

Note

Removed as of #CSV1.1.0. See Emg/ContactStates

Faceplate/FitState.settled#

Supplementary data counting the number of electrode pairs with settled contact.

Note

Removed as of #CSV1.1.0. See Emg/ContactStates

Emg

Note

Order and assignment of Leads may change based on hardware version or manual electrode positioning (specific hardware versions).

Please see #Emg/Properties.ids[] prior to reading EMG data.

Emg/ContactStates[0..6]

Discrete (OFF>ON>STABLE>SETTLED) contact information for each pair of EMG electrodes.

The fidelity of this signal depends on #Emg/Properties.contactMode as follows:

  • contactMode is DC this provides per-electrode contact state
  • contactMode is AC this provides per-pair electrode contact state (Derived from Emg/Contact)
Note

There is no ContactState for the reference electrodes on the Emteq MASK in currently released firmware.

[Faceplate/FitState](#faceplatefitstate) provides an simplified abstraction to determine EMG contact overall.

Binary Representation

The ContactState is a 8-bit value split into two nibbles (half-a-byte i.e 4-bits), one for the ‘Positive', and one for the Negative electrode contact respectively.

Emg/ContactStates [8-bits]
NegativeContact [4-bits] PositiveContact [4-bits]

Each electrode contact can be one of the following states:

State Binary-Nibble Condition
Off 0000 Lifted - No skin contact
On 0001 Contact - Initial, possibly intermittent skin contact
Stable 1000 Not likely to change; firmly established
Fault 1110 Fault-state - Indicates a faulty contact
Settled 1111 Stable with Saturated filters indicating higher confidence in measurements Emg/Filtered and Emg/Amplitude

Emg/Contact[0..6]

Impedance measurement of electrode-to-skin contact when #Emg/Properties.contactMode is AC mode.

The value may be converted to Ohms and to do so refer to the chapter on #Emg/Properties.contactToImpedanceDivisor

Emg/Raw[0..6]

Raw Analog signal from the EMG measurement device representing the device data in a bit-precise unmodified form i.e. No filtering stages have been performed

  • All other EMG measurements are derived.
  • Measurement rate is 2KHz for AC and 1KHz for DC contact-modes

The value may be converted to Volts and to do so refer to the chapter on #Emg/Properties.rawToVoltageDivisor

Emg/RawLift[0..6]

Supplementary data for Emteq Internal. This represents the AC contact mode signal (see #Emg/Properties.contactMode) and may be removed in future revision.

The value may be converted to Volts and to do so refer to the chapter on #Emg/Properties.rawToVoltageDivisor

Emg/Filtered[0..6]

Filtered EMG data to contain only in-band EMG measurements in the frequency range 100-450Hz

  • 50Hz signal removed
  • AC-Lift 500Hz signal removed
  • Low frequency drift and movement-artifacts removed (*)
  • High frequency noise removed

The value may be converted to Volts and to do so refer to the chapter on #Emg/Properties.rawToVoltageDivisor

Movement artifacts may occur in-band so are not suppressed as of <=v0.5 firmware and <=v9.3.8 hardware.

Emg/Amplitude[0..6]

Amplitude of the muscle EMG in-band signal acquired by a moving-window RMS over Emg/Filtered

The value may be converted to Volts and to do so refer to the chapter on #Emg/Properties.rawToVoltageDivisor

HeartRate

HeartRate/Average

Average beats-per-minute (BPM) of the cardiac cycle as measured from the Photoplethysmographic (PPG) sensor on the user's forehead.

This is updated at 1Hz in all firmware versions <= 0.5

Ppg

Ppg/Raw.ppg

Raw Photoplethysmographic (PPG) sensor reading which detects variation in blood volume within the skin of the user's forehead.

See HeartRate/Average where BPM is of direct interest

Ppg/Raw.proximity

Raw proximity sensor reading (see notes below)

As of <= v0.5 firmware the Proximity function output is not implemented. The Raw sensor will measure ambient incident light.

Tip

Faceplate/FaceState, Faceplate/FitState, and for granular information the Emg/ContactStates are more reliable measures of mask being worn.

Powerline

Powerline/Frequency

{Documentation incomplete} [Introduced in v0.5.0]

Imu

#Imu/ is deprecated as of CSV1.0.5, removed in CSV1.1.x

Imu/Accelerometer.{x | y | z}

See #Accelerometer/Raw.{x | y | z}

Imu/Magnetometer.{x | y | z}

See #Magnetometer/Raw.{x | y | z}

Imu/Gyroscope.{x | y | z}

See #Gyroscope/Raw.{x | y | z}

Accelerometer/Raw.{x | y | z}

IMU sensor reading of linear-acceleration for the X, Y, and Z axes.

The orientation of X, Y, and Z axis are not defined by the Emteq system and must be calibrated for a user or session.

The value may be converted to m/s^2 and to do so refer to the chapter on #Accelerometer/Properties.rawDivisor

Magnetometer/Raw.{x | y | z}

IMU sensor reading of magnetic-field strength on the X, Y, and Z axes which can be used to derive absolute orientation or compensate for Gyroscopic drift.

The orientation of X, Y, and Z axis are not defined by the Emteq system and must be calibrated for a user or session.

The value may be converted to micro-Tesla and to do so refer to the chapter on #Magnemeter/Properties.rawDivisor

Gyroscope/Raw.{x | y | z}

IMU sensor reading of angular-velocity on the X, Y, and Z axes.

The orientation of X, Y, and Z axis are not defined by the Emteq system and must be calibrated for a user or session.

The value may be converted to degrees/s and to do so refer to the chapter on #Gyroscope/Properties.rawDivisor

Online Documentation

The latest version of the emteqPRO CSV specification document can be accessed from the flowing link:

https://support.emteqlabs.com/data/CSV.html

Format Revisions

Refer to #Format/Version field for identifying the revision of the CSV specification that a file uses.

CSV1.0.1

CSV1.0.2

CSV1.0.3

CSV1.0.4

CSV1.0.5

CSV1.0.6

CSV1.0.7

  • Correct [] position in #Emg/Calibration.liftCurrent[]
  • Add FLX prefix on faceplate/Version.serial
  • Line-endings as LF (/n) instead of CR+LF to reduce file size and standardize Linux compatibility

CSV1.0.8

Internal, see #CSV1.1.0

CSV1.1.0

  • Added #Time/Seconds.unixOffset (Previewed as CSV1.0.8)
  • Removed deprecation warning for obsolete Imu/* (Present since CSV1.0.5)
  • Imu/* comment fields removed (Deprecated since CSV1.0.5)
  • Imu/* field headings renamed to their respective Accelerometer/Raw, Magnetometer/Raw, and Gyroscope/Raw (Rename raised since CSV1.0.5)
  • Removed Faceplate/FitState.any#, Faceplate/FitState.both#, and Faceplate/FitState.settled# where Emg/ContactStates shall be used instead
  • Fixed CSV1.1.0 Emg/ContactState should be Emg/ContactStates (with ‘s') to be in-line global schema

CSV1.1.1