Problems about sensor synchronization

[wheator]

Hello,
I use SICK lidar and Vision camera synchronized to conduct multi-sensor processing. But i encountered the problem that these two sensors have different frequency, the lidar is about 35Hz, the camera is 20Hz, the lidar is faster than camera. When using the component "Snapshot", I use the lidar output to trigger the camera then the problem comes. So I wanna ask how can I fix this synchronization problem? Thank you very much!

[support@intempora.com]

Hello,

In your application, the Snapshot component would not really "trigger" the camera but forward the most recent image acquired by the camera each time a laser scan reaches the trigger input.
This means you should over-sample the video from 20 Hz to 35 Hz on the Snapshot output.

If you want to write a component for fusing the two data streams (from the camera and from the laser scanner), you have several options depending on when you want your code to be executed:

• Triggered execution:
  
  • Declare two inputs on your component, one with type MAPS::FifoReader (the trigger one), and one with type MAPS::SamplingReader (the re-sampled one)
  • In the Core() function, call 2 successive StartReading on Input(0) then on Input(1) : the first call will be blocking until a new sample reaches the first input, then the second call will provide the last sample available on the second output at that time (the second call to StartReading is non-blocking).
  Depending on the order chosen for your 2 inputs, you will either run your component at the frame rate of the laser (35 Hz) or at the frame rate of the camera (20 Hz)
  
• Reactive execution:
  
  • Declare your 2 inputs with type MAPS::FifoReader
  • In the Core() function make a single call to StartReading on the 2 inputs at the same time:
    

    Code: Select all

    MAPSInput* my_inputs[2];
my_inputs[0] = &Input("laser_in");
my_inputs[1] = &Input("image_in");
int input_that_answered;

MAPSIOElt* ioelt_in = StartReading(2,my_inputs,&input_that_answered);
if (ioelt_in == NULL)
   return;

switch (input_that_answered) {
   case 0: // Got a sample from the laser
      ...
      break;
   case 1: //Got a sample from the camera
      ...
      break;
}   


• Synchronized execution:
  
  • Declare your 2 inputs with type MAPS::FifoReader
  • Use function SynchroStartReading with the last optional argument "synch_tolerance".
    This function will return 2 samples when it gets 1 on each input with associated timestamps that differ from less than the specified synch_tolerance.
    So what tolerance should you set ?
    Images arrive at 20 Hz, which means every 50 ms.
    Laser scans arrive at 35 Hz, which means every 30 ms approx.
    So, each time you get an image, you should have a laser scan within + or - 15 ms around the image timestamp.
    In order to take some margin, a synch tolerance of 20 ms should be ok.
    This way, you will resynch the two input streams based on samples timestamps (which means date of acquisition) whatever the latencies introduced by intermediary components.

For more details, check the documentation file "RTMaps User Manual" chapters 3.3.8 and 3.3.9, "RTMaps Developer Manual" chapter 3.1.4, and source codes available in the "rtmaps_samples" SDK project in folder "Chapter 2 - Multiple inputs components".

Warning : the first example provided in the code samples Chapter 2 is the common error to avoid (calling 2 successive StartReading methods on 2 inputs declared as FifoReader...).

Hope this helps,
Best regards,