Santos et al. find 13-24 interviews most common in software engineering studies
New analysis of 10 years of papers reveals wide variation in interview sample sizes.
A new preprint by Ronnie de Souza Santos, Italo Santos, Mauricio Rodrigues, and Cleyton Magalhaes examines how interview sample adequacy and saturation are operationalized in empirical software engineering. The team reviewed papers published between 2016 and 2025 across major SE venues, focusing on sample sizes, saturation discussions, and justifications for sample adequacy.
The analysis reveals substantial variation: studies with fewer than 12 interviewees were common, often tied to specialized industrial contexts or constrained access, while the most recurrent sample size range was 13 to 24 participants. Notably, saturation and sample adequacy justifications were heterogeneous, with many papers relying on implicit or contextual reasoning rather than explicit methodological discussion. The authors call for more transparent reporting to strengthen qualitative rigor in SE research.
- Analyzed SE interview studies from 2016-2025 across top venues.
- Most common sample size range: 13-24 interviewees.
- Many papers lack explicit saturation justifications, relying on contextual reasoning.
Why It Matters
Provides empirical benchmarks for interview study design and pushes for greater methodological transparency in software engineering research.