When a camera is affixed on a dynamic mobile robot, image stabilization is the first step towards more complex analysis on the video feed. This paper presents a novel electronic image stabilization (EIS) algorithm for highly dynamic mobile robotic platforms. The algorithm combines optical flow motion parameter estimation with angular rate data provided by a strapdown inertial measurement unit (IMU). A discrete Kalman filter in feedforward configuration is used for optimal fusion of the two data sources. Performance evaluations are conducted using a simulated video truth model (capturing the effects of image translation, rotation, blurring, and moving objects), and live test data. Live data was collected from a camera and IMU affixed to the DAGSI Whegs mobile robotic platform as it navigated through a hallway. Template matching, feature detection, optical flow, and inertial measurement techniques are compared and analyzed to determine the most suitable algorithm for this specific type of image stabilization. Pyramidal Lucas-Kanade optical flow using Shi-Tomasi good features in combination with inertial measurement is the EIS algorithm found to be superior. In the presence of moving objects, fusion of inertial measurement reduces optical flow root-mean-squared (RMS) error in motion parameter estimates by 40%. Abstract © 2010 IEEE.
IEEE/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010
M. J. Smith, A. Boxerbaum, G. L. Peterson and R. D. Quinn, "Electronic image stabilization using optical flow with inertial fusion," 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 2010, pp. 1146-1153, doi: 10.1109/IROS.2010.5651113.