Question:
I know sample size can influence the chance of
statistical significance but can sample size influence effect size. In
other words... If a study showed no statistical significance with a small
effect size could we expect that the effect size will increase if sample
size increased?
Replies:
Roughly speaking the standard deviation (scatter due to random error) is
proportional to (N)^-1/2 the inverse square root of the sample size N. So
the larger the sample size the smaller will be the standard deviation. As
you can see if you "plug in" a couple of values of N you see that doubling
the sample size from 10 to 20 reduces the standard deviation by a factor
of 0.316 to 0.224 but doubling the sample size from 100 to 200 reduces the
standard deviation from 0.100 to 0.071. A smaller relative amount. Having
said that there are many areas of science where "data can be pulled out of
the noise" by measuring very large sample sizes -- one example is high
energy physics, astronomy, and epidemiology where often thousands or even
millions of measurements may need to be made to separate out the "signal"
from the "noise". Of course, in those areas very sophisticated statistical
tools and methods must be used to ensure the results are valid and not due
to systematic or instrumental errors.
NEWTON is an electronic community for Science, Math, and Computer Science K-12 Educators, sponsored and operated by Argonne National Laboratory's Educational Programs, Andrew Skipor, Ph.D., Head of Educational Programs.
