AgileX LIMO Computer Vision
This project was a part of ME 416 at BU. During this project, my team and I were tasked with making an AgileX LIMO robot follow a tape pathway using computer vision.
Our Approach:
After we first connected MATLAB to one of the LIMO systems, we tested one of our carrot-identification programs from Homework 2. We determined that the LIMO could successfully track a roll of orange tape in front of the integrated camera. We then modified the carrot-identification program so that the LIMO will turn its wheels toward the averaged position of the orange pixels the camera sees. The angle of the wheels is proportionally scaled to the average horizontal position of all detected orange pixels.
We then programmed the robot to identify blue tape rather than orange tape in a well-lit environment, and we programmed the robot to move forward and turn toward any blue pixels in the camera frame. This program worked exceptionally well when following a blue line on the ground, but we then realized that the robot was susceptible to following other blue objects in the room, like a blue jacket hanging on a nearby chair. We knew that we should limit our blue-object-detection range to the bottom portion of the camera, and after some trial and error, we decided to analyze the bottom 120 pixels of the camera frame. For context, the camera displays a total of 480 rows of pixels, so the program only observes the bottom quarter of the camera frame.
Obstacles:
One of the biggest challenges was determining how the camera on the LIMO reacted to the lab's lighting. In optimal room lighting, the LIMO is able to detect the blue tape using the blue mask within the color range set in the code. However, it struggled to recognize the blue tape when the lighting changed. Additionally, when the light level in the surrounding area drops, the LIMO has green-tinted headlights that can make recognizing the blue tape even more difficult.
To fix this problem, we implemented a way to calibrate the blue mask as the LIMO runs and experiences different lighting conditions. When the LIMO takes in the blue mask data, it maps it to a normal distribution centered on the pre-set range of blue values manually put into this code. This allows the LIMO to dynamically adjust its recognition of blue to the current lighting conditions.
Another challenge we faced was the LIMO’s orientation after performing a 90-degree turn. Initially, the LIMO was unable to turn at 90 degrees, and it would overshoot any sharp turns it encountered. The first solution implemented was to add a ‘shuffle’, similar to a car parallel parking. This worked somewhat well, but it had a high degree of error and often left the LIMO unable to continue on its path when it lost sight of the blue mask when shuffling back and forth.
To fix this problem, we implemented counter-steering into the program. Whenever the LIMO detects a 90-degree turn, it will take a few steps back, steer in the opposite direction a few times, and then perform the turn, placing it relatively centered along the tape. In addition to this, the LIMO can distinguish if it needs an extra step back depending on how soon or how late it recognizes the turn, making the program more reliable.
Our route and why:
The route we created for our LIMO consisted of a few T-intersections, curved paths, straight paths, and a dead end. We put different colored tape along the sides of the route to see if the LIMO would detect the wrong path. When given a branching maze which was composed of intersections with changing colors, the LIMO follows only the blue color, satisfying milestone 1 of the maze-solving sidequest. We also put some obstacles along the route to see how that might affect the LIMO. We also laid a route that approached the glass door to test how well our program responded to the glare of light entering the room. All of these factors created a suitable environment to test how robust our LIMO line-following program is.
A video of the robot following the tape is below:
Comments