Skip to main content

IMU

The IMU data is normalized to a flat CSV file per video. IMU has been normalized into a single format, with associated timestamps to the canonical videos and video components.

Unprocessed IMU data is available. Some IMU data is available on video components and can be parsed with gpmf-parser.

note

IMU data is not attached to (all) the video component containers, due to many video components having metadata stripped from them

Sample​

component_idx,component_timestamp_ms,canonical_timestamp_ms,gyro_x,gyro_y,gyro_z,accl_x,accl_y,accl_z
0,0.0,0.0,0.025559105431309903,0.25026624068157616,0.11182108626198084,-2.278177458033573,-1.0311750599520384,-1.0311750599520384
0,4.975124378109452,4.975124378109452,0.01597444089456869,0.27689030883919064,0.10862619808306707,-2.2709832134292567,-0.8609112709832134,-0.8609112709832134
0,9.950248756218905,9.950248756218905,-0.014909478168264113,0.29073482428115016,0.0926517571884984,-2.3669064748201443,-0.4268585131894485,-0.4268585131894485

canonical_timestamp_ms may be NULL in some instances, due to the canonical videos trimming to the video stream region.

Technical Details​

Since each canonical video has their video stream region trimmed, to normalize the IMU data:

  1. IMU timestamps relative to the video stream are found (via a simple offset)
  2. Timestamps are then offset with respect to where each video component starts in the canonical videos
note

An assumption is made about the first IMU timestamp, specifically it being aligned to the original container's t=0 point

Artifacts / Known Issues​

Please read to understand the limitations

  1. Some videos with IMU do not have all components covered, i.e. there are more video components than associated IMU components.
  2. Missing acceleration values for some files
  3. IMU timestamps can be significantly large or small
    • Due to some unknown artifact when extracting the IMU data
  4. IMU timestamp can be non-monotonic
    • Either due to the above, or:
    • This is simply due to the packets of the data from the original extracted IMU data
      • In this case, the resulting canonical timestamps will be appropriately computed.
      • You should sort the data by the canonical timestamp
  5. No callibration of the IMU was taken place, there is likely measurement bias present for IMU data.

You can query the above issues from the top level metadata file.

Download​

python3 -m ego4d.cli.cli --output_directory=<dir> --dataset=imu