Lessons Learned from Building a Crowdsourced Food Database
I started FoodNoms around a year and a half ago. My mission was to build an alternative to MyFitnessPal, Lifesum, Cronometer, LoseIt!, etc. that was fast, well-designed, privacy-oriented, and integrated into the Apple ecosystem.
From day one, I knew that the biggest challenge by far was going to be the food database.
Even though FoodNoms launched with a limited, US-only database, it managed to attract enough attention and customers to get some traction. I was legitimately surprised this happened. I know the food database is often the main, and sometimes sole, criteria people evaluate when trying food tracking apps.
I was delighted to discover that I wasn't the only one that felt that there were things that are actually more important than the food database: the design, the features, and the privacy policy.
Still, I knew that in order for FoodNoms to really grow, it would need to provide better solutions to those that do consider the food database a "maker" or "breaker". Six months after I launched FoodNoms 1.0, the FoodNoms Community Database launched, which made it possible for users to submit additions and changes to the regular FoodNoms database.
The Community Database ended up growing faster than I anticipated, average around a thousand submissions a week! Clearly the database was addressing a real problem.
However, I realized pretty quickly that a lot of my initial assumptions weren't going to hold up. After seven months of learnings, I shipped the first major update to the Community Database last week.
I'm excited to share more details about what has changed, what learnings led to those changes, and some exciting metrics!
Lesson #1: FoodNoms Users Will Exceed My Expectations
Both the number of foods being submitted every week, and the quality of those foods, far exceeded my expectations. After a few months, I realized that the system I had set up initially was not going to be sustainable, and I needed to find a way to introduce more automation to help out the moderation team.
In order to help my moderation team, I built a warning flag system to detect common errors, like "missing brand name", "calories out of expected range", "iron amount exceeds p99", etc. After observing several months of human moderators be the front line for the Community Database, I felt more confident that these warnings were quite effective and could instead be the "front line of defense" against low-quality submissions.
Today, human moderators are responsible for handling reported issues and going through a backlog of flagged foods. This is a lot more sustainable for the team, and allows user-submitted foods to make it into the Community Database much faster.
Lesson #2: Barcodes Must Be Normalized
I made a design decision with the Community Database that all contributed foods must have a barcode. There must be some unique identifier for each food, so that we can dedupe submissions. I for sure don't want FoodNoms search results becoming polluted with a bunch of garbage results, with no way of being able to verify metadata or nutrition amounts. I know that my customers don't want that either.1
Now, I have something to admit. I had a pretty huge bug in the FoodNoms server code before this release where I was not properly normalizing barcode numbers.
Why does barcode normalization matter? Here's an example:
Let's say a user scans a barcode and the app sends a request to the server with barcode "0076580004912". In the database, there isn't a food matching that barcode, so the server returns a 404. But there is a food with barcode "076580004912". See how it's the same exact barcode, minus the extra "0" at the beginning? That's because the food in the database is using UPC-A format, while the request was using GTIN-13 format. GTIN-13 is actually a superset of UPC-A, so in this case, there was an unnecessary barcode lookup miss.
There are a few other cases where barcodes need to be normalized. Sometimes barcode strings contain dashes, or are missing the final checksum digit, or are longer strings that can be trimmed down to GTIN-13.
In addition to properly normalizing all barcodes, the database now enforces that only one food item will exist for any given barcode.
Lesson #3: Corrections to USDA Branded Food Data Must Be Possible
Since shipping the first version of the Community Database, I have since learned that there are a sizeable number of errors in the USDA branded food database.
The previous version of the Community Database allowed reporting issues to foods originating from the USDA, but due to some internal workings of how the database was architected, I wasn't able to accept those fixes into the Community Database.
I made this a priority to fix with this latest update. Now, any branded food in the database can be corrected.2
Lesson #4: Reporting Issues Deserves a Clear, Explicit UX Flow
"How do I edit a food in the Community Database?" has been a pretty popular support question these past few months.
In the latest update to the FoodNoms app, I made some changes to make this more clear and explicit:
The "Thank You!" modal also teaches you that when you make a correction, that correction is saved to your library so next time you scan that food item, it will use your correction even before it is moderated/accepted.
Lesson #5: Getting Users Opted-In Is a Challenge
Only ~10% of new users logged into the Community Database on their first day. This is a big problem, because users that are logged into the Community Database are significantly more likely to get results from scanning a barcode.
Previous versions of FoodNoms were pretty passive with prompting users to log in. There is a dismissable CTA in the "Log Food" screen and an area in the Settings tab. Clearly, these two spots were not doing enough to encourage users to opt in.
I decided to experiment with a new onboarding step to educate the user and prompt them to sign in as they're just getting started:
The Impact Is Huge
I love it when I can measure the impact of changes I make – especially when the numbers look so good!
First number to look at: the barcode scan success rate of users in the US that are logged in (i.e. best case scenario).
From mid-thirties to mid-seventy percent success rate! THAT'S HUGE! That means we're actually getting more hits than misses now:
It's hard to attribute which change had the most impact here, but it's clear that collectively, these were all really positive changes.
How about for non-US customers? While the absolute value is not as peachy, it has also seen a huge relative increase – literally overnight:
The total number of user submissions is also increasing:
The new onboarding step is also making a significant impact on the number of users opted into the Community Database:
This is all fantastic news for FoodNoms. This will have a compounding impact, as I know that if a user successfully scans a barcode within their first day, they are more likely to be retained than a user that logs a food from any other source (i.e. label scanning or searching). More barcode hits = more users retained = more users submitting data = more barcode hits!
Conclusion
A lot of work has gone into this update. Many weekends and late nights. I'm so pleased all that hard work seems to be paying off. It's super exciting to see such positive early results here; it really gives me a new optimism for the future of FoodNoms.
Still a lot of work left to do here, but I think for now I'm going to let some things bake and shift my attention back to new features and marketing efforts. Follow me @ryanashcraft or @food_noms on Twitter to stay tuned. 😎
- This comes at a big tradeoff – users who want to help grow the FoodNoms restaurant database are not able to do so. I think this is a tradeoff worth making. I realize that the database is severely lacking with restaurant foods, and I'm working on ways to shore up some of those major gaps soon – initially focusing on popular chain US fast food restaurants.
- I'm not aware of a way to share these changes back to the USDA. I'm also not sure if they'd want to accept these sort of "untrusted" changes, anyway.