Building In Public 2: Startup = A Bunch of Simultaneous Experiments 👩‍🔬

Growing up my favourite subject was science. I was (and remain) a very curious kid. I tend to ask a lot of questions and love to understand why. I particularly liked the concept of the scientific method — hypothesis -> design -> test -> repeat. It is brilliant. A logical and methodical way to acquire and build new knowledge.

When I got into the startup world I (of course) read the Lean Startup by Eric Reis. This reintroduced the scientific method to me in the form of validated learning. I kept the learnings from the Lean Startup in the back of my mind as we hacked along at the start of Steppen. However, somewhere along the way we forgot about it. We began spending a long time (a week which is long in early stage) implementing features we thought made sense with little prior testing and even less follow up after implementation.

We ended up implementing a bunch of features (namely contact access, location access, invites and tag searching…) because some social apps had them, so we assumed, ‘Hey they have these features, we should have them too.” These features are among Steppen’s theoretically best and most complete features but are also the least used. After taking some time to reflect on product progress, we realised a lot of the major features we had worked hard to implement had been ‘failures’ as no one was seeing them or wanted to use them. From a personal perspective it was a humbling realisation. As the product person I was the one who had been prioritising what to built and I had been wrong. It sucked.

But after a goal setting session with our mentor Simon the team regained focus. We had forgotten the basic startup principle. Everything is a test. Do the hackiest thing you can do to get the information you need to then decide if it is worth you investing more time doing this. It was time to run a bunch of product experiments!

I’ll walk you through how we now approach the product (and the business as a whole) using (our now most successful feature implementation) recommended workouts.

Approach to Recommended Workouts

More Generally the Product Framework

While I used recommended workouts to highlight the framework we now use it is important to realise that framework is applied for all product experimentation. Below I explain the importance and what each dot point leads us to consider.

Implementation of Recommended Workouts

Below is is the design for how recommended workouts look. After setting up their account, the user selects a goal, based on that goal we recommend them workouts. We add 3 new workouts suggestions every 2 days.

User selects a goal, based on that goal we recommend them workouts. We add new workouts every 2 days.

Results of Experiment: Recommended Workouts

Implementing this feature took a single day meaning we could start getting data on this experiment nearly instantly. So what were the results of this experiment….Below I have included two graphs.

Comparing these results to what we had recorded success to mean- “ 30–40% of users clicking on suggested workouts to check them out.” We were spot on! Future aspects to test, uncover and optimise for:

L: Funnel of users moving from selecting a recommended workout (40%) to starting that workout (20%) . R: % of daily active users selecting a recommended workout, this number fluctuates between 15% and 30%.

Experimenting With the Entire Business

While the example I have detailed is looking at applying an experimental framework within the product, we now apply this mindset to all aspects of our business — marketing, retention & content acquisition. It is a discipline we are still learning and getting into. For sure there are still times we steam ahead without first thinking why, but as a team we do a good job of holding one another accountable to this experiment based approach.

Already I have found this framework much better in grounding us in getting things out fast and iterating to perfect. It has motivated the entire team to be hungry for information and moved the emphasis on learning not tasks (or features). Let me know your thoughts and happy to disucss further!

Until next month friends (and in the meantime there will be many more tests…),

— Cara 🚀

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Cara

Co Founder @Steppen. Working to democratise workouts. Lover of avo and halloumi.