Pg. 85, Userpalooza (electronic edition)
I’ve been doing field research long enough to recognize how incredibly useful Nick Bowmast's "Userpalooza" would’ve been to me when I was just starting out. I wanted to share some of what I took away from it, with the hope that it'll spark some ideas for you and direct you towards Nick's book as an excellent resource - whether you’re just beginning your research journey or have been at it for years.
It feels like the apprenticeship model has, despite evolving in other fields, stubbornly stuck around in UX/user/design research. While today there are more resources than ever for aspiring researchers, I’ve always learned the most around how to do better research is when I get to go in-field alongside folks who are really damn good at it. With a colloquial, no-BS style, Nick makes Userpalooza feels like the next best thing to running an actual study next to a seasoned pro like himself. Here’re a few of my favorite morsels:
Pg. 24, Userpalooza (electronic edition)
Whenever I see a framework, I try to judge its utility based upon, “How might this have helped me in past projects or client conversations?” The reason I'm drawn to Nick’s “Explore > Evolve > Evaluate" approach above is that it takes what can to a non-expert feel like a daunting and indecipherable grab bag of research methods (A/B testing? Contextual inquiry? Shadowing? Intercepts? etc.) and reframes the conversation around the desired outcome.
I'm always on the lookout for valuable tools like this that can gently steer the conversation away from the pressure to execute precisely what was promised in a Statement of Work (usually written before reality had its say in the matter of “what is the most important thing to focus on?”). Next time I feel like I'm bumping up against a conversation with a someone who's focused upon a particular method, expressing a sentiment like “We need to do (a/b testing, "ethnography", etc.)___)!”, I look forward to sharing this framework with the client to reorient the conversation towards a perspective of “If we’re hoping to learn about _______, then I'd recommend a method like _____.”
Another useful research-scoping metaphor I'll likely be borrowing addresses the common (and completely understandable) skepticism of clients concerning a relatively small sample size of a typical qualitative study as it compares to a typical market research survey’s “n” of hundreds or thousands. As Nick nicely frames it, if a larger sample size approach sought to allow one to understand all of the grains of sand on a beach, a smaller-n approach could be compared to:
[taking] a pinch of each from: high-, middle- and low-tide zones; near the rocks; by the boat ramp: where the sunbathers gather; and that area by the old wharf where the seaweed washed up after a storm. With only a sprinkle of sand in total, there's a fair chance that you'd actually covered it.
pg. 32, Userpalooza (electronic edition)
In the endless search for better ways to navigate around the "more = more" thinking that tends to dominate the sample size discussion, I like the idea of an edifying jaunt down to the beach to refocus the conversation.
Coming fairly early on in the typical project, sample selection is one of the most important moments of synchronization with clients or stakeholders. Talking to the wrong people up front can place the entire engagement in jeopardy, as it is easy for people down the line who, in response to what may be a challenging or hand-to-accept finding, can question either the size or make-up of the sample. While the "sand at the beach" metaphor is a nice way around the question of sample size, there's another nice framework that helps start the crucial conversation with clients around not just the size of your sample, but also the composition of the sample in terms of their relationship to your offering, with the direction ultimately depending upon the scope of the project.
Pg. 36, Userpalooza (print edition)
If pursuing a more exploratory, generative answers, like "what is causing declining transit ridership?" or "what role will convenience stores play ten years from now?", you'd be better served by steering more towards "Inspiration at the edges." You wouldn't want to speak with only subway superfans, or the delivery drivers who visit convenience stores more than three times a day. It'd be just as important to speak with folks who refuse (or are currently unable) to ride public transportation, or who take it as a point of pride that they avoid convenience stores.