Interviewing stakeholders to elicit product requirements
study purpose
This study was guided by the following three research questions: 1) How can design interview quality be reliably assessed? 2) What interviewing practices differentiate novice and more informed design interviewers? & 3) What relationship exists between the execution of high quality interviews and the relevance of information elicited for product requirements development?
methods
Eight students in their fourth year of engineering volunteered as participants. Students participated in an eight hour design task where-in they were tasked with developing product requirements and specifications for a ‘toy to help in the cognitive development of young children.’ Participants were given access to the internet, library books, guidelines, standards, reference toys, opportunity to observe children playing with various toys, & opportunity to interview various stakeholders. All interviews were recorded and all requirements and specifications were collected.
findings
The figures below show the findings from this study. I first show the results of the statistical model developed based on the methodology of assessing interview transcripts based on best practices. Here we see how effective each participant was in using interviewing best practices while controlling for interviewee, interview number, & question number.
A statistical model was developed to see if specific participants used interview best practices at a statistically higher rate than their peers. We can see that Participants 5 & 6 did while Participant 8 used significantly fewer.
Below, I show how the top and bottom performing students compare with respect to their overall use of best practices during interviews.
When comparing the top and bottom performing designers, we can see that only specific best practices differentiated these groups such as ‘encouraging deep thinking’ & ‘being flexible & opportunistic.’
Finally, we see whether a higher quality interview actually leads to information which is more relevant to product requirements development.
We also see a moderate correlation between the quality of the interviews conducted and whether the interviews led to relevant information (e.g. higher quality interviews led to more relevant information elicited).
Finally, we can visualize how diverse the designers use of interviews to elicit product requirements were. While some participants used diverse stakeholders interviews to cover many requirements, others used interviews in a much more superficial manner.
Participants also varied greatly in how broadly they asked questions of stakeholders that were directly relevant to their requirements. Here we see that Participant 4 asked many stakeholders about many requirements while Participant 2 asked a much smaller number of stakeholders about a limited set of requirements.
Mohedas, I., Daly, S. R., Sienko, K. H., Huynh, L., Cravens, G., (in review) “Novice to informed designer’s stakeholder interviewing practices during requirements elicitation.” Design Studies.