Effect Size and Sample Size Calculations
Statistical Consulting  - Company Message
My Blog

Effect Size and Sample Size Calculations


What are Effect Size, Power, and Sample Size Calculation and why do We Care?

By Brian Hunter, M.A.

     You may have heard about these three terms and find them confusing when approaching a dissertation or project that requires you to calculate them and interpret them into the project.

What are the three term definitions from the title?

     Effect size (ES) is a name given to a group of statistics that measure the magnitude or strength of a treatment or phenomena effect.  ES measures are the common metric of meta-analysis studies that summarize the findings from a specific area of research.  This tells us how easy it is or difficult it may be to find an effect when doing a research project.

     The power of any test of statistical significance is defined as the probability that it will correctly reject a false null hypothesis. The question becomes how much power do you want in doing your test?
Sample Size calculation is done to ensure that enough participants or observations are gathered to ensure that the hypothesis testing has enough power to detect and true effect if it is actually present.  Sample size calculation, therefore, depends on effect size and power.

How do we calculate Sample Size?

     First, we need to know the default alpha level, the power level expected, the effect size of the phenomena under study and the statistical procedure that will be used to test our hypothesis before calculating Sample Size.  Whew, that is a great deal of things to know.  Where do we begin?  We start by conducting what is known as a Power Analysis.

What is a Power Analysis?

1.  The primary purpose of power analysis is to estimate sample size. First, the researcher must specify the power level they want to achieve.  The default power level is usually .80 to .95 depending on your field of interest.

2.  The calculations for power depend on the effect size of the phenomena under study in the population.  You can use published experiments similar to the one you will be conducting or a meta-analysis done on your topic of interest as a guide to finding or calculating for yourself the effect size.

3.  Use the default alpha level for your field.  In behavioral sciences we use an alpha level of 0.05.

4.  Choose what statistical test you will use to test your hypothesis.

5.  Then, you choose you power level which can be from .80 to .95 which means you are 80% to 95% sure you have enough power to reject a false null hypothesis and prevent a Type II error.  

Now that you have done this, what is next?

How do we conduct a Sample Size Calculation?

      Once you know your power level, effect size, alpha level and statistical test for the hypothesis, you may use a public domain program known as G*power (Faul, Erdfelder, Lang, & Buchner, 2007).  Faul et al. (2007) developed this program at the University of Düsseldorf and have made it available to the public for free.  So, G*Power is able to compute power analyzes for many different hypothesis tests such as t tests, F tests, χ2 tests, z tests and some exact tests. G*Power can also be used to compute effect sizes and to display graphically the results of power analyzes. The program may be download for free with the given permission from the developers at http://www.gpower.hhu.de/en.html
            
Let’s look at an example of how to do this…..

     So, we decide that we are willing to have a power of .80 and we find from a published meta-analysis that the effect size of the phenomena we are studying is d=.30.  The effect size in the case is relatively small. We want to do a One-Way ANOVA with three groups (Treatment 1, Treatment 2, and Placebo) on the dependent variable of depression level to test our hypothesis regarding differential effects among two treatments and a placebo.

     Now, open up G*power and choose F-testsand then choose ANOVA, fixed effects, one way, omnibus, set power to .80, effect size to .30 and the number of groups to 3.  G*power does the calculation and produces two graphics you see below,  We found that we  need a total sample size of 111 to have enough power (.80) to detect an effect size of .30.  Please see Table 1 and Figure 1.

Table 1

F tests - ANOVA: Fixed effects, omnibus, one-way
______________________________________________________________________________
 
Analysis:    A priori: Compute required sample size
Input:         Effect size f                               =  0.30
                   α err prob                                 =  0.05
                   Power (1-β err prob)                  =  0.80
                   Number of groups                  =  3
Output:       Noncentrality parameter λ      =  9.9900000
                   Critical F                                =  3.0803869
                   Numerator df                         =  2
                   Denominator df                      =  108
                   Total sample size                    =  111
                   Actual power                          =  0.8034951
_____________________________________________________________________________
 
 
Was that so difficult?  

     Once you understand what is involved and where to find those values and procedures, this whole idea of sample size estimation turns out to not be so intimidating.  See the developer of G*power instructions for use and download below. Good Luck in your research endeavors.
 
Download the Short Tutorial of G*Power (PDF) written for G*Power 2 but still useful as an introduction.  For more help, see the papers about G*Power in the References section below.
 
     If you use G*Power for your research, then we would appreciate your including one or both of the following references (depending on what is appropriate) to the program in the papers in which you publish your results:

References

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. Download PDF

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyzes using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods41, 1149-1160. Download PDF

Feedback

To report possible bugs, difficulties in program handling, and suggestions for future versions of G*Power please  send us an e-mail.

Download G*Power 3.1.9.2 for Windows XP, Vista, 7, and 8 (32 and 64 bit) (about 20 MB). Please make sure to choose “unpack with folders” in your unzip tool.
 



9 Comments to Effect Size and Sample Size Calculations:

Comments RSS
NinjaEssays on Wednesday, November 22, 2017 12:33 AM
This topic is relayed to the statistics subject and the topic that you share in this article is so wonderful to read. This always wants to search this type of the information. You did a good job in this article which is so unique.
Reply to comment
 
essay paper on Sunday, June 10, 2018 10:26 PM
I can see that this topic requires an advanced knowledge in order to understand. There’s a lot of technical terms that we’re not familiar with. This topic can really help me with my research project. Each formulation in this article is clearly explained. Maybe some people can’t appreciate this subject and they think that it’s only a waste of time to study these lectures. But the knowledge that we can get on this topic can be applied in many important fields of work.


do my assignment for me on Wednesday, December 13, 2017 5:13 AM
Waste of time is not a good activity. Always try to consume your time with positive thing that gives you a positive result, like wise if you watch T.V in two hour it will never give you any advantage rather than if you will spend time in studies that will gives you fruit full results in exams. Graduate students should spend time in literature review and scientific research for better results .
Reply to comment
 
Sample on Thursday, May 03, 2018 5:49 PM
Sample


Simon Powers on Thursday, May 17, 2018 4:17 AM
Regular job.
Reply to comment


rushessay.com on Wednesday, June 06, 2018 3:52 PM
It was really cool to read this review. I will send you some bug reports if I will find any bugs.
Reply to comment


top wine openers on Monday, June 25, 2018 4:42 AM
But it costs little high than the mechanical wine openers.
Reply to comment


Gina Aragon on Wednesday, August 15, 2018 2:10 PM
For a limited time we have lowered the rate on targeted website traffic. We have visitors from virtually every country on Earth. Each visitor is targeted by both country and keywords that you submit when you start your free trial period. If you need more visitors or product sales try our service free for seven days and we will send you 500 free visitors during the trial. There are no contracts and if you cancel during the trial period you will not be charged anything! Start your trial today: http://stpicks.com/2rusd Unsubscribe here: http://stpicks.com/2ruse
Reply to comment


Robin Valliere on Sunday, September 02, 2018 11:08 AM
For a limited time we have lowered the rate on targeted website traffic. We have visitors from virtually every country on Earth. Each visitor is targeted by both country and keywords that you submit when you start your free trial period. If you need more visitors or product sales try our service free for seven days and we will send you 500 free visitors during the trial. There are no contracts and if you cancel during the trial period you will not be charged anything! Start your trial today: http://insl.co/11 Unsubscribe here: http://stpicks.com/2ruse
Reply to comment

Add a Comment

Your Name:
Email Address: (Required)
Website:
Comment:
Make your text bigger, bold, italic and more with HTML tags. We'll show you how.
Post Comment
RSS

Recent Posts

Hypothesis Testing
Type I and Type II Error
Surviving the Dissertation Process
Effect Size and Sample Size Calculations

Categories

Statistics
powered by