A Gaming App Experience for Car Connoisseurs
1
Background
Since 1949 MotorTrend has been the car enthusiast’s go-to source for the latest cars, news, reviews, show coverage and more. MotorTrend’s mission today is to entertain a motoring world across every platform, and deliver content to a monthly audience of 26 million across web, TV, print, social, the only auto-dedicated subscription VOD service, and live events. One key to the company’s popularity is its skill of conveying excitement over the latest car models and trends to automotive superfans.
In late 2018, MotorTrend came to Code and Theory with the idea of creating a mobile app gaming experience based around that moment of excitement when you first see a new car in the flesh. The basic idea was developed into a capture and collect gaming concept with a goal of building out a Beta version to test with enthusiasts at the Detroit Auto Show (NAIAS) in early January.
2
Challenge
The challenge was to figure out how we could create a memorable branded game experience at a live car show event. Our target was the North American International Auto Show (Detroit, MI), which spanned 2.5 weeks and attracted car enthusiasts of vastly varying demographics. MotorTrend’s desire to innovate the way its users engage with its brand gave us the opportunity to create a modern way to experience the traditional car show via emerging technologies, gaming mechanics, and experience design.
3
The Technology
From a technology perspective, this project presented some interesting challenges. For the app to work, we needed to be able to accurately predict the make and model of a car based on a native camera image under unpredictable live event conditions. We also needed to run Machine Learning (ML) predictions on a user’s device (instead of a central server) in order to (a) provide real-time feedback to the user, (b) reduce dependency on uncertain network conditions, and (c) remain cost-effective to scale since cloud ML services charge per request.
The final development push presented a unique client-agency collaboration opportunity at the live event. In the week leading up to the show, our on-site developers, designers and QA engineers worked alongside the clients, who were also assisting with beta testing and collecting training data.
Throughout the week, the C&T and MotorTrend teams collectively balanced new features, ongoing testing & QA activities, and the app store submission and approval process while meeting clear deadlines. Despite last minute changes to the game rules and prizing logic to meet Google Play and App Store requirements, we pushed through as a team and made it happen. We were in the same space and camaraderie was high—especially when the event kicked off.
Technology Used:
- iOS: Hybrid Swift/Objective-C Codebase
- Android: Kotlin with AndroidX ViewModel
- Image Recognition: TensorFlow and TensorFlow Lite for on-device car image recognition/classification
- Custom ML Model: Bash/Python/Objective-C codebase
- Custom training data consisting of over 250,000 labeled images of cars sourced from MotorTrend, public data sets, and images captured by the Code and Theory team.
- Functional Reactive Programing (FRP) Frameworks: RxSwift, ReactiveReSwiftRouter, RxJava
- Automated build and distribution system: Fastlane, Testflight, HockeyApp
4
Solution
The Car Spotting app was designed around gaming principles and a new interaction mechanic based on Machine Learning technology. From a design perspective, the user experience was carefully crafted to support the nuances of image recognition mechanics, while also keeping the user engaged throughout the overall experience. The interface was designed to be simple and intuitive so that users at the show could be up and running with ease. From a technical perspective, the application used a combination of a trained on-board Machine Learning model and a cloud service application to accurately recognize the target cars. This combination of design and technology became the base mechanism that not only helped drive the experience but also became part of its identity.
As the “Official App of the North American International Auto Show,” there was great buzz supported by marketing swag and brand ambassadors, which resulted in large-scale participation and learnings that helped to shape plans for future enhancements and roll outs. Over the course of the event, users took 105,000 photos of cars, with 86.5% of users attempting at least one capture, 52% of users successfully unlocking at least one target car, and 20% of users unlocking all 17. User engagement time averaged out at a staggering 24 minutes per user and the application also made the Apple App Store’s top 150 list within 3 days.
