Face Tracking Mirror Project

Over the summer, I wondered if I could make a practical project that someone in my family could use. I also wanted to develop my programming skills, especially Python, as I had mainly used MATLAB during my undergraduate experience.

After talking to my family about small problems I could solve, I landed on designing a mirror that tracked my mother's face so she could apply makeup with both hands without constantly readjusting the mirror. The solution to this problem was almost instantly clear: I would make an ordinary mirror and endow it with the ability to spin on its own, based on feedback from a camera running a face-tracking algorithm.

The first step was designing a rough version of the mirror in Onshape, focusing on making it the correct size and footprint so the system would fit practically on a small table or vanity. In addition, I worked to make the mirror rotate smoothly and easily when the system is powered off, so a user wouldn't need to use face tracking to use their own mirror. Below is an image of this rough CAD.

From here, I decided on electronic components for the mirror based on my rough CAD design. My actuators for spinning the mirror were some small servo motors. I also purchased some bearings, a small buck converter, a Raspberry Pi 4 for onboard camera vision, and an RPi Camera Module 3 NoIR. I chose the camera specifically because it can see in a wider range of lighting conditions.

Next, I redesigned the CAD model to include mounts for these electronics, as well as more specific sizings for bearings and counterboring holes for fasteners. This design is shown below.

With the initial design complete, I began 3D-printing the components and developing a basic face-tracking script with OpenCV. I began by using my computer's webcam to track my face, drawing a bounding box around it, then tracking the centroid's position and printing it to my computer screen.

Once I felt the face tracking system had been adequately polished, I set up my Raspberry Pi 4 and its camera, ran a stream of the feed and face tracking to ensure the system would work on the hardware.

With all the necessary hardware and software components in place, I assembled the first model of my mirror. Once the mirror was built, I wrote a second face-tracking script that included a simple PID controller to drive it.

After some tuning and testing, I identified a few flaws with the mirror, including significant jitter when the pan axis attempted to spin to different setpoints. Eventually, I realised that the PID controller was trying to move to a point below the servo motors' resolution, resulting in shaky motion near the target position, even at low proportional gain values.

Another problem with this system was that when the mirror's tilt axis was too far down, the camera couldn't see the user. This problem could not be resolved by either improving the controller or adding offsets to target angle positions, as the camera would just lose a user completely in very common positions.

The final major problem encountered was that the controller did not handle a user changing their distance from the mirror well. A user close to the mirror, within a foot, would have a much better experience, as the controller had been tuned to work well at that distance. However, when a user moved back by 1-2 feet, the mirror would move too slowly to consistently catch the user's face.

To solve the jitter problem, I redesigned the torque transmission system for the servo motors to use smaller gears. The tilt-axis problem was also simple to solve: I moved the camera to the bottom of the mirror rather than the top. This allowed the mirror to see a far larger number of potential user positions.

I also resolved the drop in tracking performance for users at a distance by tweaking my PID controller. I first needed to find the distance between a user and the mirror. I did this by taking readings of the bounding box at various distances from the mirror and creating a function that fits the bounding box's dimensions to a user's distance from the mirror. I then added a term to my PID controller that scaled the control signal to account for this distance.

Below is the final CAD for this system.

I also made a very basic Bluetooth remote using an ESP32 Arduino Nano to control the system. The remote can move the mirror in steps of varying granularity in both the pan and tilt axes, and switch between manual movement and face tracking. The remote can also be used to make the mirror sweep through its full range to find a user and engage face tracking.

Some images of the final system are below!

Finally, I did some more testing and small problem-solving until I thought the system was adequate to show to my mother, who was very impressed and offered some feedback I may incorporate in a future version of this device. Chief among those points, I was informed that a mirror was not likely to be bright blue!

Ultimately, I was beyond elated by her reaction, and I would love to revisit this idea in the future.