BumbleBee’s vision processing system consists of modular communication, movement and vision filtering packages that can be combined and tuned to complete each mission task. Each visionprocessing unit runs as a separate ROS node and
is responsible for a single task, providing both movement through ROS SMACH state machines, and vision output, while cooperating with otherunits running in parallel through mission planner. BumbleBee’s front and bottom facing Microsoft Lifecam Cinema cameras provide sufficient visual feedback for the vision processing system. Improved vision algorithms are applied for better identification of the required objects.
The vision nodes receive image input from the cameras in the Bayer encoded bgr8 format through the ROS protocol. The ROS
images are converted to OpenCV images via the ROS cvBridge. To deal with changing water and lighting conditions, various image enhancement techniques such as image sharpening, white balancing, gray world and adaptive thresholding values are applied to obtain better image contrast. A combination of vision filters are used to detect, classify and track objects. These include HSV colour thresholding, contour detection and Hough transforms provided by the OpenCV computer vision library. The vision processing code is written in Python and an annotated processed image is published as a ROS image. An annotated image is presented in Figure 15. Centroid calculation is performed using Hu moment analysis to align the camera’s center with the centroid identified. The centroid is tracked at each frame while the vehicle maneuvers into position, before performing further object identification and manipulation to complete the task at hand.
Interactive vision tuning systems are developed using PyQt to experiment with vision processing parameters at real time. These systems receive a live update from the cameras and the vision processing units and provide analysis of image statistics such as colour histograms. The ROSdynamic reconfigure tool is also used to quickly adjust parameters. These configuration parameters are stored until the next system reboot and hence can be used for subsequent runs.