DoodlAR

2019 MIT Hacking Arts Wayfair Challenge Winner

DoodlAR is an award-winning project that combines augmented reality (AR) and machine learning (ML) technologies to bring imagination and creativity to the product search experience. The project was developed by Team RedBull, consisting of Dongphil Yoo, Youjin Chung, Seyoung Kim, Ji Young Chun, and Jason Yeon. DoodlAR was awarded the Wayfair Challenge award at MIT Hacking Arts 2019.

AR + ML

The project uses a state-of-the-art AR pipeline that can seamlessly integrate with the Wayfair website marketplace. The AR technology enables image target recognition with 3D models, allowing users to visualize their doodles in real-time and see how they would look in their space. The ML component of DoodlAR is a real-time vector-based classifier inference system that can run on mobile devices. The team has used a CNN image classifier with Google Quick Draw to train the model to recognize and analyze the user's doodles. This ML technology is robust, accurate, and fast, making it perfect for real-time usage scenarios.

User Experience Design

The DoodlAR experience is centered around the user, with a primary focus on their needs and preferences. The team has designed a user interface that is clean, intuitive, and easy to navigate. The interface allows users to effortlessly doodle their ideas and see their designs come to life in real-time with the AR technology. The team's user experience design has been carefully crafted to ensure that the entire process is enjoyable and engaging for the user. The goal is to make the product search experience fun and interactive, allowing users to unleash their creativity and imagination.

Visualizing Designs

The AR technology that DoodlAR uses enables users to visualize their designs and see how their furniture will look in their space. This feature adds an additional layer of engagement and excitement to the user experience. With DoodlAR, users can experiment with different furniture arrangements and layouts, allowing them to make informed decisions about their purchases.

Future Plans

The team's future plans for DoodlAR are ambitious. They plan to expand the technology by introducing VR mode and social media integration to enhance the user experience. DoodlAR could also be used to curate search results based on user doodles for any product search on the web. The team intends to license the technology and explore other potential applications, such as educational and entertainment platforms.

In conclusion, DoodlAR is an award-winning and innovative solution that combines AR and ML technologies to bring imagination and creativity to the product search experience. The project's advanced technology and user-focused design provide an intuitive and enjoyable experience for users looking to plan and visualize their floor designs with their furniture. With the AR pipeline seamlessly integrating with the Wayfair website marketplace and the real-time vector-based machine learning classifier inference running on mobile devices, DoodlAR is a robust and innovative solution for modern ecommerce.

Role

Customize the Quick Draw CNN Classifier model for furniture recognition
Build the AR pipeline on Unity3D using AR APIs.

Team RedBull

Dongphil Yoo
Youjin Chung
Seyoung Kim
Ji Young Chun
Jason Yeon

Reference

Quick, Draw! Dataset by Google
DoodleNet by Yining Shi