Sample size for qualitative research

Introduction

This current article considers the seldom-written-about-but-much-questioned issue of sample size in qualitative research. This paper is inspired and informed by the author’s experiences in commercial marketing research, academic management research and as an editor for qualitative academic papers. Further, as an author of such papers and in the role of editor, the views of many reviewers have been read over the past 31 years in research, and these have also inspired this current paper. Reviewers clearly need guidance in this area, and researchers could also benefit from this discussion as they struggle to design qualitative research in terms of sample size.

Furthermore, qualitative research has recently come under criticism for its lack of rigour in terms of there being little or no justifications given for the sample sizes that are actually used in research (Marshall et al., 2013). Marshall, Cardon, Poddar and Fontenot considered 81 qualitative studies and concluded that scant attention was paid to estimating or justifying sample sizes.

The question of what sample size is needed for qualitative research is frequently asked by individual researchers (Dworkin, 2012) but not frequently discussed in the literature (Onwuegbuzie and Leech, 2005). Few studies approach this issue, and as much qualitative research does not involve the making of statistical generalisations, many qualitative researchers report that sample size is not an issue in qualitative research (Onwuegbuzie and Leech, 2005). However, for reviewers it clearly is an issue as described below.

Furthermore, the related issue of what sample size is needed for qualitative research findings to have some validity is also one which many paper reviewers are concerned about enough for them to mention in their reviews. Reviewers typically, nonetheless, do not definitively answer their own questions regarding what size a sample should be. Comments from reviewers are usually to do with the sample size (whatever size it is), being too small, and they commonly state that this should be noted in the limitations sections of an academic research paper. This current paper reviews some of the sparse literature on this subject, investigates a case study from the physical sciences and one from management and comes to some tentative conclusions.

The concept of data saturation, which is the point at which no new information or themes are observed in the data from the completion of additional interviews or cases, (Guest et al., 2006) is a useful one in terms of discussing sample size in qualitative research. This approach implies that a single case study or interview is never enough, because data saturation can only be known after at least two cases, and usually more, are examined. This idea of sampling until data saturation is reached can be used as a justification for the use of a particular sample size in any qualitative research which is guided by this idea.

However, in practical terms, although the idea of saturation is very helpful at the conceptual level, it provides little guidance for estimating actual sample sizes, prior to data collection (Guest et al., 2006). For example, it is difficult to give cost and timing estimates for research where the sample size has not been pre-determined. This impracticality may be a reason why the data saturation approach does not appear to be used in practice, even in academic research.

For example, in a meta-analysis of 560 academic qualitative studies, the distribution of sample sizes used was found to be non-random, with a statistically significant proportion of studies, presenting sample sizes that were multiples of ten (Mason, 2010). This strongly suggests that a premeditated approach to sample size determination was used, and this is not wholly congruent with some of the principles of qualitative research (Mason, 2010). Clearly, there is confusion and a gap between theoretical expectations and practice.

This is corroborated by the investigation of 81 qualitative studies mentioned earlier (Marshall et al., 2013). This investigation found that those qualitative researchers who used data/theoretical saturation as an indicator that their sample size was sufficiently large did not explain this in sufficient detail, or in a way that was persuasive or entailed the presentation of any evidence to support the claim for data saturation (Marshall et al., 2013).

The idea underlying data saturation as a guide to sample size is the idea that once saturation is reached, the results must be capable of some degree of generalisation. Generalisation is traditionally seen as a central aim of science, as a process of theory formulation for further applications (Mayring, 2007). However, as Mayring notes, the concept of generalisation has been criticised, for example because of the context specificity of all scientific findings.

Discussion

Despite the apparent limitations of samples which involve a single case or single research participant as discussed above, it has nevertheless been noted that individual (single sample) case studies can provide reliable indications for the directions in which future research can go. Individual cases can also provide a new, deep and nuanced understanding of previously unexplored phenomena. Furthermore, qualitative researchers have noted that often a researcher can (unknowingly) have all the data they need from their first piece of data collection (Sandelowski, 1995). It is also argued that case studies have been undervalued in terms of their ability to generate theoretical generalisations (Tsang, 2014). This is demonstrated below from the discussion of two examples, one from the physical sciences and one from management research.

First, in medicine, it has been noted that findings from single case studies can have findings which can be generalised from and implications which are global in importance. The discovery of penicillin is a case in point. Alexander Fleming noticed an accidental case where mould was growing as a contaminant on the jelly in one of his culture plates (like Petri dishes). The mould appeared to have an inhibitory effect on the surrounding growth of bacteria. He called the mould Penicillin notatum (American Chemical Society, 1999). Publishing his findings in 1929 in the British Journal of Experimental Pathology, he wrote that the broth from the mould had marked inhibitory, bactericidal and bacteriolytic properties to many of the more common pathogenic bacteria (Fleming, 1929). His work was taken up by Howard Flory and Ernst Chain at Oxford University who developed penicillin as a medicine, with the eventual help of US drug companies.

Penicillin was so apparently successful and generally applicable that it did not initially undergo full randomised trails prior to use in humans. Nonetheless, the development of penicillin is noted as being one of the greatest breakthroughs in modern medicine (American Chemical Society, 1999).

In management research, the longitudinal examination of an individual CEO who was highly psychopathic is also a plausible example of such a single-case approach being ground breaking and informative (Boddy, 2015). This is particularly so because corporate psychopaths appear to have a common modus operandi and to be relatively stable personalities over time (Boddy et al., 2015) (just as penicillin has stable properties). The study of one corporate psychopath CEO, it was compellingly argued, can therefore inform how other psychopathic CEO’s will likely behave.

More theoretically, the research philosophy or paradigm adopted and discussions of an appropriate sample size are related (Onwuegbuzie and Leech, 2005; Boddy, 2005b). Some researchers associate size considerations with an approach to science based on positivism, which is an approach to scientific inquiry which many qualitative researchers reject (Lincoln and Guba, 2000). However, it should be noted that some researchers do use a qualitative element of research to set the parameters for a further, positivist quantification. This usually means that they apply a positivist approach to qualitative research (Boddy, 2005a, 2005b) and, under this approach, a criticism of sample size because of smallness may well be justified. This is because the qualitative sample size has to be representative of the population under consideration as a breadth of inquiry is anticipated.

This is the approach recommended (pp. 25-28) by qualitative market researchers who suggest that researchers draw up a grid (such as sex by brand usage) to make sure that each segment of the population is covered by the research (Gordon and Langmaid, 1990). Academic researchers also suggest this grid or matrix type approach to qualitative sample size determination (Stake, 2000).

Commentators suggest that qualitative sample sizes of ten may be adequate for sampling among a homogenous population (Sandelowski, 1995). Others state that qualitative sample sizes of 20-30 are typically (pp. 56) conducted by researchers to establish data saturation using a grounded theory approach to qualitative inquiry (Creswell, 1998). However, no evidence is presented as the basis for this latter sample size claim. Marshall and colleagues refer to a sample size of 20 as being small for a grounded theory-type approach to qualitative research and to 40 being a large sample size for the same type of study. This gives a range of what sample size they would consider appropriate, and later in the same paper, they recommend a range of 20-30 interviews for grounded research and 15-30 interviews for case studies.

Bearing in mind their North American background, such a recommended range would certainly be smaller in number at both ends of the spectrum, e.g. from UK qualitative researchers. US qualitative researchers tend to adopt larger sample sizes than other qualitative researchers (Marshall et al., 2013).

In terms of the upper limits to sample size, Sandelowski is one of the few commentators on sample size in qualitative research to note that a sample can be too large. A sample which is very large does not permit the deep, case-oriented analysis that is the raison-d’etre of qualitative inquiry (Sandelowski, 1995), at least in constructivist or in-depth approaches to scientific research. In terms of how large is too large, few have ventured an opinion. Sandelowski suggests that 50 interviews is a large sample for a qualitative study. Boddy (2005b, 2005a) mentions once being asked, as a commercial marketing researcher to conduct 1,000 in-depth interviews by a US positivist researcher. Upon learning that, given resources available, this would take over a year and cost about US$1m. The US researcher re-evaluated what was meant by “in-depth”. However, such a sample size would undoubtedly be “too large”, because the sheer volume of data would inhibit meaningful, timely, qualitative analysis. This current author’s view is that any qualitative sample size over 30 (per market/geography) becomes too unwieldy to administer and analyse.

Therefore, in a single market/country or relatively homogeneous population, any qualitative sample size at or over 12 focus groups or more than 30 in-depth interviews could be considered large and would require justification. Corresponding with this viewpoint, in one of the few studies investigating actual theoretical saturation, the authors found data saturation starting to become evident at six in-depth interviews and definitely evident at 12 in-depth interviews among a sample of women in two countries (Guest et al., 2006). This suggests that multiples of 12 in-depth interviews may be more appropriate than the multiples of 10 that were commonly found in a meta-analysis (Mason, 2010) of qualitative research in practice.

In a review of sample sizes in qualitative studies in the information systems discipline, the authors note that (North) American studies tend to have larger sample sizes than those from other countries (Marshall et al., 2013). They state that they cannot account for this difference. However, in a discussion of the different, USA versus UK, approaches to qualitative research using focus group discussions (UK)/focus group interviews (USA), the author notes that US researchers tend to implicitly follow a positivist epistemology (Boddy, 2005a, 2005b). This logically results in their favouring larger sample sizes. On the contrary, for UK researchers, the concern is more about gathering in-depth information rather than quasi-measurement and so smaller sample sizes are intuitively more appealing.

In making a justification for an adopted sample size, qualitative researchers should make reference to the scope of the study and nature of the topic (Morse, 2000), the contact time to be spent on each individual research participant (respondent) (Marshall et al., 2013) and the homogeneity of the population under consideration (Trotter, 2012). In practical terms, attempts should be made to make sure that the sample is as representative of the population as possible (Bock and Sergeant, 2002), albeit that it may be a very tightly defined or unusual population.

Conclusions

Qualitative research often concerns developing a depth of understanding rather than a breadth, particularly when undertaken under a non-positivist paradigm, such as that involving depth psychology or a constructivist approach to research. As such, we must conclude that in these cases a single case study involving a single research participant can be of importance and can generate great insight. This logically means that the smallest acceptable sample size in these types of qualitative research is a sample of one. In many cases, therefore, the observation that many reviewers would be tempted to make, that such a sample is too small or cannot be generalised from, is not a valid criticism, particularly if the researcher has justified the sample size. One case can produce an in-depth understanding that furthers knowledge as in the case of a psychopathic CEO. Furthermore, as the example of the discovery of penicillin demonstrates, a single case can also have findings, which do validly apply across many areas.

Exceptions to this guide to sample size may be where the qualitative research is being undertaken under a positivist approach to research, for example with a view to developing a quantitative measurement instrument such as a questionnaire. In this example, it would be useful to have a more representative understanding of likely incidence rates so that questions can be prioritised in terms of inclusion in any questionnaire or other instrument. This would necessitate sampling a greater number of respondents and, in general, at least one representative of each segment of the population under consideration in the wider research should be sampled in the qualitative research.

Thus, the issue of what constitutes an appropriate sample size in qualitative research is only really answerable within the context and scientific paradigm of the research being conducted. In constructivist or in-depth qualitative research, a single example can be highly instructive.

In positivist qualitative research, a representative sample is arguably needed, involving representatives of each of the sub-segments of the total population to be researched. Researchers and reviewers may take these arguments into consideration when respectively deciding what sample sizes to use and in deciding whether to criticise the sample size used in any qualitative research that is being evaluated.