Victoria opens with a story almost everyone in a product team has lived through. A feature is requested by multiple customers. The team builds it carefully. It ships. And then nothing happens. Weeks pass. Usage is flat. The dashboard doesn’t move.
This happened at one of her clients- a SaaS tool for managing software subscriptions. Companies had repeatedly asked for accounting software integrations, so the team built them. And then nobody used them.
What fixed it wasn’t a rebuild. It was five customer interviews. What they found: users couldn’t see the feature. The button label meant nothing to them. They were looking for the logo of their accounting software- something visually familiar that told them “this is what you asked for.” It wasn’t there, so they assumed the integration hadn’t been built yet.
One design change- making the logos prominent- drove adoption and triggered requests for additional integrations from customers who hadn’t previously known to ask.
The lesson Victoria draws from this is pointed: ideally the usability testing would have happened before implementation, not after. The problem wasn’t the feature. It was the absence of research.
Before going into methods, Victoria is deliberate about what user research can and can’t do. It cannot tell you what business impact to pursue. Whether you’re trying to grow revenue, reduce cost, expand into a new market, or improve retention- that’s a strategic decision, and outsourcing it to users doesn’t work. Customers can’t design your business model for you.
What user research can do is help you find opportunities and validate them. Once you know the direction you want to go, continuous discovery- Teresa Torres’s term for small, regular research activities done weekly or more often- becomes the engine for figuring out which opportunities are real and which ones just feel real.
Victoria recommends Torres’s book Continuous Discovery Habits for anyone who wants to go deeper. But her practical message is simpler: even if you can’t commit to weekly research, doing some research before building a new feature is far better than doing none.
The most flexible tool in the kit. Structured or unstructured, one-on-one or in a group, with existing users or prospective ones- user interviews are where you go when you don’t yet know enough to ask specific questions. Victoria recommends The Mom Test as the essential guide to doing them well.
Case study: The sleep app.
A long-established e-commerce company selling mattresses and pillows wanted to build a sleep app. The internal favourite idea was a personal sleep coach- affordable, differentiated, exciting. Before building anything, the team ran ten interviews with prospective users. None of the ten had ever heard of a sleep coach. None could say they’d want one. The idea that had generated the most internal enthusiasm collapsed under thirty minutes of conversation.
What the interviews did surface: people wanted to know which habits actually affect sleep quality, and they wanted help tracking whether those habits were working for them. That insight became the app’s entire proposition. It launched as an MVP, then as a full product- built around what users actually wanted, not what the team had assumed they would.
Case study: The job board.
A recruitment platform noticed that recruiters weren’t posting available jobs to the public job board, even though doing so would drive more applications. The hypothesis was that the process was too time-consuming. Running a customer advisory board- a group of B2B customers in ongoing conversation with the product team- confirmed it. The posting flow was long, the fields were exhausting, and starting from scratch every time was genuinely painful. The fix: reducing mandatory fields and pre-filling data from previous postings. Job posting rates climbed from 60% to 75%, which improved the platform’s value for job seekers too.
Surveys scale where interviews can’t. They’re less rich but more statistically grounded, and they’re the right tool when you already have hypotheses and want to test them across a large user base.
Case study: The real estate tool.
A property management platform for individual landlords was exploring new revenue streams. One idea: a multi-user subscription, so owners could share access with a spouse or a property manager. Initial conversations with individual users suggested it sounded interesting. But before committing to build what the engineering team assessed as a very large and expensive piece of work, the team ran a broad survey.
The results were unambiguous. Willingness to pay was very low. Interest across the general user base- not just the power users who’d expressed enthusiasm in earlier conversations- was limited. The team didn’t build it. The survey cost a fraction of the implementation that was never needed.
Surveys are good for validating hypotheses with quantitative data, for prioritising between known options, and for measuring user satisfaction over time. They are not the right starting point when you don’t yet know what to ask.
Where interviews explore motivation and surveys measure scale, usability testing is about behaviour. What do people actually do when they use the product- not what they say they do, not what they intend to do?
Case study: The taxi driver app.
A mobility service was experiencing a specific and damaging problem: drivers were accidentally charging passengers twice. Customer complaints were increasing, support costs were rising, and churn was following. The company’s working assumption was that this wasn’t fraud- it was a usability failure somewhere in the driver app. To find it, they tested four interface designs across drivers in different countries, measuring completion speed and recording where people clicked.
The designs showed no meaningful difference. But the country-by-country breakdown did. In certain markets, the error rate was significantly higher. The investigation led to the translation: in one language, the button label for ending a journey was ambiguous enough that drivers were completing the action twice. The translation was fixed. Double-charging dropped significantly.
The most statistically robust method- and the one with the most constraints. AB testing requires real production traffic, enough volume to reach significance, and a clear metric to measure against. It can’t tell you why something happens, only whether it does. And it can be a significant resource investment, which means it deserves to be applied selectively.
Case study: The outfit carousel.
A fashion and beauty e-commerce site wanted to test a carousel on product pages showing how items could be worn together as an outfit. The hypothesis: users would discover more products they liked, browse longer, and ultimately buy more. The AB test confirmed half of it. Time on site did increase. But gross merchandise value- the guardrail metric the team had committed to protecting- went down. Users were browsing more and buying less. The change was reverted.
AB testing can’t tell you why the outcome happened, only that it did. If understanding the cause mattered, the next step would be user interviews. In this case, the data was enough.
One more note of caution: Victoria shares the example of a translation tool that AB tested a feature the team already knew they were going to ship regardless- because a future release only worked with one version. The test was essentially theatre. Building it, running it, analysing the data, paying for the tooling- all waste. Any method can be overused. The question worth asking before you start is always: what decision will this research actually help us make?
The four methods map onto different situations:
User interviews are for discovery- when you don’t know enough yet to ask specific questions.
Surveys are for validation and prioritisation- when you have hypotheses and want to test them at scale.
Usability testing is for understanding behaviour with specific interfaces- before release, or to diagnose underperformance.
AB testing is for incremental metric improvement- when you have sufficient traffic and a clear measure of success.
None of these methods tells you what business impact to aim for. All of them help you find the best path to the impact you’ve already decided to pursue.
The throughline across every case study in this talk is the same: teams that talked to users before committing to a solution made better decisions, faster. Teams that skipped that step either built the wrong thing, built the right thing wrong, or spent time and money discovering something a five-person interview series would have surfaced in a week.
