What is it about live intercept participant recruiting that is so powerful?
Intercepts allow us to align many cross-functional teams around understanding of the actual TurboTax experience. We do so by observing and interviewing customers with real intent.
Intercept-based participant recruiting is a vehicle to get our teams as close to our customers as possible.
Joanna VodopivecPrincipal Design Researcher, IntuitTrue Intent
The reason Intuit invests in intercept participant recruiting is mostly because of the need to understand the customer intent. We can ask someone recruited from a panel to try and do their taxes, but you can imagine how fake that would be. We've tried it in the past but it's simply not the same. Intercepts allow us to get deeper into why they came to our site and how they got started (what was the entry point).
The live intercept interviews are as contextual as you can get. Every tax season, we run a series of highly targeted intercepts at different points in the marketing and pre-logged experience on the TurboTax website. Not only does this live aspect generate a huge amount of internal excitement, but it gives us end-to-end insight into the customer journey from first impression to completing taxes.
Not only that, but we also see food being delivered and hear the sounds of kids playing video games in the background, spouses walking in to answer specific tax questions, people shuffling through the stack of papers they pull out of a filing cabinet – anything that happens in real life. Those elements of real moments and real lives of our customers mean everything for the quality of our research.
What would you say to someone who doesn’t do live intercepts for participant recruiting?
You’re missing the highest quality participants, especially from more broad geographic locations, compared to any panel. We get everyone from tiny towns in the middle of nowhere to big cities, and that reflects the general population of the United States which is who we see in our product. Everyone has to file taxes at some point. We believe in getting people as early in the funnel as possible, so if you want to understand the end-to-end customer journey, recruiting pre-login on your marketing site is key.
Engineering won’t do it?Customer obsession is key at Intuit, and Ethnio is a pivotal tool for intercept-based participant recruiting, allowing us to scale our customer insights.
Code snippets are everywhere now, so you should have no problem getting a code snippet on a marketing site. It does take some advocating and explaining the value of what an intercept will gain us. Our dev partners are often very busy so patience and gratitude go a long way.
Uncertainty
People are afraid of getting anyone and everyone via live intercepts, but we use attributes and variables to only show the screener invite to a very narrow subset of people. What sampling methodology is to proper quant study design, targeting is to intercept recruiting.
When to choose intercepts
This depends on your research goals and what you’re trying to uncover. If intent is critical for your research goals, it’s clear that live intercepts are the best option. For example, we can’t talk to people on our site about taxes in August. It’s not the right time, traffic is extremely low and most people don’t want to deal with that then.
Even if intent isn’t the factor in your research study design, it’s a great way to get non-tech savvy and older demographics that typically are not part of any panel. This process has actually become much easier in the last couple years, thanks to the post-Covid increase in remote work, especially with Zoom being so popular and user friendly.
No Professional Testers. No liars.
We typically ask our intercept participants if they have ever participated in any kind of previous research. Vast majority of them have not. This is clearly as honest feedback as you can get. No need for any scenarios to imagine. In our case, if someone isn’t planning on doing their taxes, why would they come to our site?
The opposite of this experience is when you ask this same exact question to a traditional panel audience from agencies or companies that sell their panel by demographics – are they being honest? Are they providing answers that we want to hear? There is always a bit more risk in talking to professional test-takers.
This brings us to the next point.
Play by play feed
I’ve been involved in conducting live intercepts over the past three years. It’s incredible to see how excited all our PMs (and other extended stakeholders) even at the planning stage.
Once we get going, they leave meetings to join the customer sessions because there’s something magic about joining an intercept interview in progress: watching someone actually use a product or service you helped design and build.
Another way to engage with the intercepts is to follow the session comments and transcript that the researcher (and often other stakeholders) contribute to in the research Slack thread. One of our VP's said he loves following it live or reading it at the end of his busy day to tap into what is truly happening in our products ✨.
These Slack messages are almost like live blogging an event or a sporting game, with high engagement. Simply because you don’t know what’s going to happen, the interest and impact is much more substantial compared to inviting people to the research lab for scheduled sessions. We will usually tag specific employees responsible for a feature, a design or service delivery so they can learn first-hand what we are observing.
At the end of the intercept interview, we often invite the participant to continue capturing their feedback via a diary study, allowing us to capture the rest of their journey. This is a critical method our team uses during every tax season. We find these participants rarely drop off (as you know, typically about half of diary study participants will leave). Diary studies are a critical method we use during tax season.
Ethnio intercepts are especially enlightening when looking at customer types that are impossible to recruit using any other method. This can be especially enlightening when looking at customer types that are impossible to recruit using any other method. For example, low income or high earner self-employed customers – both are incredibly difficult segments to truly understand, and both are important to Intuit. When we intercept those types of participants, it can change the entire understanding of key product areas.
Impact on Product at Executive Level
After the first round of intercepts one of my former colleagues ran at Intuit, a VP of Product saw the power of these live observations compared to previous lab sessions. It was clear to him that it’s a very different experience when you are peaking into prospective or existing customers’ lives, and seeing them incredibly engaged. He’s been a champion of this research approach, and helped make sure findings are implemented in our products, as a direct result of that engagement.
Quick Fixes
Bugs sometimes are observed and then an engineer (some are truly our heroes) gets tagged and will create a Jira ticket which means it immediately gets fixed. This is super rewarding to see and makes an impact for potentially millions of customers. Even small design changes are often worked on after a product designer observed them with us on Zoom.
Powerful Sessions
Certain stories from live intercepts are referenced for years. There was a single participant that led to 19 initiatives and a tiger team that was developed to go after the problem. A team even worked with infrastructure partners to provide better service.
Turbotax Live Full Service has a live customer service aspect, and so we often get to observe our internal employees at work, including tax professionals. The feedback is then used to inform backstage processes and tools, and to rethink how our supporting systems should work. For example, it can help us understand potential unhappy paths.
There was a single participant that led to 19 initiatives and a tiger team - that's the power of recruiting via live intercepts.
Unexpected Customer Discovery
You find people you didn’t even know you should recruit for research. I saw this at Groupon as well where we assumed a manager or business owner would be the one coming to our site to investigate how to run a Groupon promotion, but turns out it was sometimes their teenage kid, a spouse, a receptionist or an office manager. We had no idea, and uncovered whole new audience segments through this kind of live research. These types of insights help change some of organizational assumptions.
Problems with Live Intercepts: patience is key
The first initial point of contact with potential participants is getting them on the phone and explaining:
who we are
what we’re doing
Zoom details
NDA (non-disclosure agreement)
Incentive, incentive, incentive!
Sometimes there can be resistance at this point, because we’re asking a lot. Especially in terms of a topic like taxes, where we need the participant to screen share sensitive information. We ask our users permission to observe and record their interactions with our TurboTax product and most people are happy to help as their feedback helps improve our products and services. [See alzheimer’s case study for other sensitive info approach]
In other words, you might have to call 15 people to get 10 completes. As long as you plan for this extra time and are mentally ready, everything runs smoothly. Your patience will be paid off.
In other words, the biggest mental hurdle to get past is this type of recruiting requires a lot more flexibility in approach. That applies to the logistics above and availability of the researcher(s). One of the most critical decisions to make during planning is where and when to place the intercept invite. We refer to the Ethnio funnel often to show our stakeholders how we need a lot of traffic to truly target the right cohort:
Another approach we experimented with is to use branching logic and ask people to both indicate if they are available for a live session NOW and if not, pick some scheduling times in the near future, ideally within 24 hours. We then follow up with them promptly via email and ask to pause on any progress in our product until the interview. We’ve had some success with combining this with our typical live intercepting.
Pro Tips
We also have used intercepts to recruit “high intent” participants to get feedback on early clickable prototypes and have worked closely with our PMs and dev partners to help create A/B testing links enabling us to get feedback about specific tests we were running in the product.
Quant + Qual
Another overlooked aspect of live intercepts is mixing quant and live qual. Absolutely love being able to mix qual & quant in the same Ethnio intercept. I can ask closed-ended questions when I am not available to run live interviews, then switch to qual screening and scheduling the same day with the push of a button. It allows me to maximize the data I am collecting in the same exact place in the funnel.
Conclusion
Ultimately our goal, as a research team, is to have the most impact for our customers. At Intuit, customer obsession has always been one of our core values. I see intercept-based participant recruiting as a vehicle to get our teams as close to our customers as possible without having to bring our entire team in the field across different geographic regions, time zones, etc.
Essentially, live recruiting allows us to scale the impact of insights throughout the organization.