Moving Needles

In my last blog post, I discussed the "big bets" that I made with FoodNoms 2020.5: an overhauled onboarding, pricing changes, new differentiating features, and free trials.

It has been several weeks since the update went live, so now is a good time to reflect and assess the successes and failures of that release.

Recap

As I explained my last blog post, FoodNoms isn't growing.

In order for an app to grow, it needs two things: (1) a healthy, continuous supply of new users and (2) features that are easy to grasp and provide real long-term value. From reviewing analytics data for FoodNoms, it was easy to spot symptoms that indicate big problems in both of these areas:

• Download counts are low.
• Retention is quite high for users that activate (i.e. they perform a significant action that shows real value like logging a food item), but a large majority of users don't activate. With FoodNoms 2020.5, I focused my attention on addressing these problems – with a strong focus on the activation issue. This was a risky release. The pricing changes involved giving key features away for free and the new onboarding experience took a long time to design and develop (i.e. I could've spent that time working on something else).

For each one of the changes in 2020.5, I had specific goals that could be quantifiable evaluated:

1. Overhauled onboarding: significantly increase percentage of users that activated
2. Free macro and nutrient goals: significantly increase the average number of goals per user, which in turn may improve retention
3. Historical charts and stats: have a high percentage of users repeatedly use this feature, indicating that I provided value
4. Free trials: significantly increase the number of users that got to experience FoodNoms Plus
5. New App Store screenshots and keywords: significantly increase App Store page views and conversion rate

Post-Launch Blues

When 2020.5 shipped and I wrote the "Making Bigger Bets" blog post, I was honestly feeling quite pessimistic about the future of FoodNoms. These are huge problems I'm facing, and I know the odds are stacked against me.

A few days after the release, I reviewed early metrics and didn't see any indication that I was going to meet any of my goals. Even though I know how hard it is to move a metric, I was gutted. My disappoint turned to pessimism and a feeling of helplessness.

My pessimism was made worse by the growing fears of COVID-19 and a sense of burnout. I realized that I needed to focus on my mental health and take a step back. I took time off from FoodNoms and didn't look at the data for weeks.

2020.5 Postmortem

On Saturday evening, I checked the data again to see how things were going. And to my delight, I discovered that I MOVED THE NEEDLE!

I've worked on several teams that were proudly "metrics-driven". In order to be truly metrics driven, you must predefine a quantifiable goal, ship product changes, assess the impact, and repeat. When working on metrics-driven teams, you expect failure. Most goals will not be met. You can't let this get in your head. Just because you fail to meet a goal, doesn't mean you failed.

That said, when you do meet a goal, it feels amazing. You made a real impact! Since this doesn't happen often, the sense of accomplishment can truly be appreciated.

So when I realized that I had actually met some of my goals, I was ecstatic. This was the first time I had taken a problem, implemented some changes, and saw that I made a real difference!

Let me share some of the data with you.

Goal #1: Improve Activation Rate with an Overhauled Onboarding Experience

Prior to 2020.5, a staggering 63% of users downloaded the app and never logged a single food item. Not even once.

This is what led me to focus intently on activation. Reminder: I currently define an "activated user" as a user that has logged a single food item. Therefore, prior to 2020.5, only 37% of users activated.

My hypothesis was that a new onboarding that helped explain the core concepts, personalize the experience, and bring some personality into the mix would help boost activation. (See my previous blog post for more details and before/after screenshots.)

Since 2020.5 shipped, the activation rate has increased to over 50%! That's a 35% increase! This was the goal that I was most worried about. There's still plenty of room for improvement here, but I can confidently say that the new onboarding is a significant improvement over the previous design.

There's a side benefit with these onboarding changes: I now have more granular information about where I'm losing people. The onboarding funnel looks pretty healthy, but there's a big drop off from people opening the "Log Food" screen and actually logging a food item. It's clear now that the biggest opportunity to improve this metric is to redesign this UI and focus on improving the food database.

Conclusion: I'm considering this a big success! 🎉

Goal #2: Improve Retention with Free Goals

Before 2020.5, I discovered that the number of goals you have correlates with how well you retain.

I thought that perhaps this is causal – if you can encourage users to have more goals, they will retain better. Part of the reason that decided to give away more goal types was an attempt to test this hypothesis.

Since 2020.5 was released, the average goal count per user has climbed up from ~1.5 to ~2 goals per user.

I'm glad that more users have more goals. But was there an impact on long-term retention? It's too early to call.

Conclusion: Good early signs of slow change, but inconclusive whether this has real impact on retention.

Goal #3: High Repeated Usage for New Goal Features

With version 2020.5, I introduced new features for goals: the ability to view historical charts, calendar-based weekly and monthly charts, and a new "Top Food" and "Top Entries" list. All of these features are available within a new screen I call the "goal details view".

My goals for this feature tested how useful these features are to my users: how often do my users re-use these features, and how many users use it at least once?

To evaluate repeated use, I measured the percentage of users that came back to reuse the feature for n number of days or more. 36% of users that viewed the new goal details screen once viewed it a again on another day. 28% viewed it on a third day.

If you compare it to other events in the app, you'll find that it isn't a gamechanger, but it's definitely not a dud. It has higher repeated usage than viewing the settings screen (which is featured prominently in the UI with its own dedicated tab in the tab bar):

However, what percentage of users actually discover the feature? Only around 4-5%:

This is in the same ballpark as another feature: scanning a nutrition label, which is currently buried in the app behind several taps from the main screen. Perhaps there is a discoverability issue here with the UI or perhaps most users just aren't that interested in goals.

Conclusion: This feature isn't a gamechanger, but it also isn't a dud. I'm considering this a success.

Goal #4: Increase Percentage of Users That Experience FoodNoms Plus with Free Trials

If you look at the percentage of users using the app that have FoodNoms Plus, the trend overall has been positive even prior to 2020.5. This isn't necessarily a good thing. Plus users retain better, and without a health supply of new users, this will naturally trend up:

However, if you only look at users that were new within each weekly interval, you'll see that since 2020.5 was released, the average percentage of users with FoodNoms Plus has climbed from ~6% to ~11%. Yay!

The new "Try FoodNoms Plus for Free" pane in the onboarding experience is clearly working. The percentage of users that get FoodNoms plus within the first hour of usage has shot up to >15%:

Conclusion: Trials are working!

Goal #5: Increase Page Views and Conversion with new App Store Screenshots

I didn't talk about this in the last post, but I did another revision of my App Store screenshots and keywords. I changed the background from light gray to orange, made the text bigger, and changed the copy to have more personality and focus on different strengths of the app:

It does seem that these changes didn't help my impressions (number of times the app showed up in the App Store at all), page views, and download conversion rate. If anything, they may have hurt.

Conclusion: I definitely need to iterate more here.

Conclusion

I don't think I hit it out the park with any of my goals, but I'm thrilled that I am moving in the right direction!

This release was painful, but extremely valuable. I'm walking away with lessons learned (and re-learned):

1. Taking big risks pays off (at least sometimes).
2. Setting quantifiable goals keeps you focused and makes it easy to evaluate whether or not you're truly making a difference.
3. Wait for the data to bake before reaching conclusions!

---

Thanks for nerding out with me on this stuff. Stay tuned for more updates on the development of FoodNoms. 😎