I’ve been seeing lots of cool smart mirror projects and prototypes online over the past few years. I was surprised to find out that building the mirror itself was actually not too difficult and I started getting really excited about the software side of things. Many of the prototypes I saw online had really clunky experiences so I set out to build my own smart mirror and rethink the user experience.
I bought a small 12” x 12” sample of mirrored glass, a basic white frame, and mounted an old Android phone behind it. I began testing out different visualizations for the date, time, and weather with static designs I created in Figma. I played around with how notifications might appear and listing long bits of text like a twitter feed. These static prototypes helped me realize that less is really more when it comes to a smart mirror. I paired down the most basic information I wanted to display to just the date, time, and weather.
Building the prototype
Building a more personalized experience
I started thinking about how I might make my smart mirror more personalized. Maybe it could show my calendar, or to-do list, or news feed? This was a challenge, because I didn’t want my personalized content to be displayed on the mirror all the time since the mirror was mounted in my kitchen. I needed a way to trigger a personalized experience only when the mirror was actively used. I thought about touch or voice interactions but both had drawbacks. Touching the mirror would just create smudges, and talking to the mirror felt unnatural to me. I wanted it to just know who I was as soon as I walked up to it – and cater the experience to me. I needed facial recognition.
Experimenting with computer vision and facial recognition
I started experimenting with the Kairos API to handle facial recognition. I used the API to train a model to identify me from an image I submitted. After lots of trial and error, I got it to reliably recognize me in many settings (day, night, close up, far away, wearing glasses, etc.). But I was still just submitting test images through the API. The next challenge was integrating this into my mirror’s software.
Getting this timing down took a ton of trial and error. I also had to experiment with different loading states during the 1 second of processing so a user noticed that the mirror recognized them in the first place. I designed the mirror to give a response even when it detected a face that it didn’t know – basically a more generic experience and I experimented with interactions to log the user out, settling on a simple 10 second timeout after the user walked away.
“It’s like magic”
Once I remounted the smart mirror and started playing around with it, the whole experience felt like magic. At this point, the mirror could recognize me and simply say “Hello, Josh!” – it wasn’t even displaying anything that exciting but it was so cool how fluid the experience felt to just walk up to it and have it respond to you. I trained it to recognize a few of my friends and invited them over to see what they thought. It was so fun!
I am working on a larger version of my smart mirror prototype next and plan to expand on the facial recognition functionality – giving more customized experiences to each user who walks up to the mirror. I’m also exploring more details to display in the mirror like a Spotify Now Playing widget.