« How to Write A Dissertation Proposal – A Statistician’s Viewpoint | Home | Should I use One-tailed or Two-tailed Hypothesis Tests? »
What is the smallest sample size I can use for my study?
By Steve Creech | February 26, 2011
This is actually a tougher question than you might think. The text book way to determine a sample size is to conduct a literature review to determine what effect size you are looking for. For example, suppose you wanted to know if there is a correlation between job satisfaction and the perceived leadership style of the supervisor among non-supervisory employees.
Suppose there are 10 published articles on this very subject, all used the exact same instruments you are planning to use and all used rigorous methodology. Suppose the strength of correlation between job satisfaction and leadership style ranged from .20 to .35 and the average among the 10 papers was .27. Then, your best guess as to how strong the correlation will be in your study is .27. Then, you would conduct a statistical power analysis to determine how large a sample size you need in order to detect a correlation of .27.
In most cases the above scenario won’t work. Most doctoral students don’t replicate previous studies. Therefore, most doctoral students do not have any articles to refer to in order to estimate the expected effect size. The next best thing is to find several articles that reported results of studies that were very similar to yours. For example, they studied the correlation between job satisfaction and leadership style, they used the same leadership style questionnaire, but a different job satisfaction instrument.
Again, in most cases, you will not find articles that come close enough to what you are studying to be of any use. Therefore, in this case, you have no idea what the effect size will be. On the one hand, you want to use a very large sample size, just in case the effect size is small. On the other hand, for all you know, the effect size will be large, so you don’t want to waste your time and money on a large sample when a small sample would do. The logical thing to do is select a sample size large enough to detect a medium effect size.
In the real world (as opposed to the text book approach), doctoral students are usually restricted by time, cost and other constraints that limit their sample size. The sample size is determined in part based upon what is doable given the constraints, and then the power analysis can demonstrate what effect size can be detected with that sample size, and in effect, that justifies the sample size.
For example, when selecting a sample, you have to think about how you can get access to the target population. For example, if you want to study job satisfaction and perceived leadership style among non-supervisory employees in the fast food industry, you might go door-to-door to a number of fast food restaurants, ask to speak to the manager, and ask permission to conduct a survey of his/her employees. This would be a lot of work and would not be a practical approach. So, you think of a better way, perhaps you could contact corporate headquarters for one particular fast food chain and get permission that way. Then, the corporate administrator could assist with disseminating your survey among the employees. These practical considerations will often determine the target population you are “able” to study. This may not be the ideal target population, but it may be a necessary limitation of the study to make it doable.
Now that you have selected a “doable” target population, you need to estimate how many members of the target population are eligible for your study (e.g. non-supervisory employees). You might be able to find that out from the corporate administrator that is assisting you. Let’s say it is 1,000 employees. Now, since your survey is voluntary, your actual sample size is going to be a function of how many people agree to participate, sign informed consent, and complete the survey.
You can do a literature review for typical survey response rates. They vary according to a number of factors but over the years the most common response rate I have seen cited is 20%. So, for your sample size justification section, you basically say: I have access to a target population of approximately 1,000 non-supervisory employees. All 1,000 members of the target population will be invited to participate in the study. Based upon papers by …., typical survey response rates are approximately 20%. Thus, the anticipated sample size is approximately 200.
Now, perform your statistical power analysis using a sample size of 200, 80% power, and an alpha level of .05 and solve for the effect size. Now you will be able to say… a convenience sample of approximately 200 will be used, based upon a power analysis, a sample size of 200 will produce 80% power at the .05 level of significance to detect an effect size of ….
Topics: Dissertation Advice |
April 12th, 2011 at 5:49 am
this makes so much sense!!!! :)
May 17th, 2012 at 7:28 pm
good article