Statistical tests of equivalence

Wellek (2003) illustrates the application of a series of familiar statistical tests corresponding to null and alternative statistical hypotheses of general form

H0: $$\theta \leq $$-t or $$\theta \geq$$ t and HA : -t $$ \leq \theta \leq$$ t

where $$\theta$$ is a function of parameters of interest (e.g. a difference between two group means) and t is the effect size of minimal interest (e.g. minimum difference in a pair of group means which is of clinical interest).

Equivalence tests are also known as reverse tests because they switch around the 'usual' hypotheses of form

H0: $$\theta$$ = t and HA: $$\theta \ne$$ t

and so the emphasis is on verifying rather than rejecting hypotheses such as equality of group means or zero correlations. Failing to reject a null hypothesis is not the same as showing it to be valid.

Lew MJ (2006) illustrates equivalence tests for two-sample (un)paired t-tests. The critical p-values (which are presented in the tables at the back of this paper for selected common group sizes, type II error and effect sizes) may be computed for any sample sizes using the R codes below for any group sizes.

If the p-value from the student's t-test on the raw data is greater than bout2 there is no difference between the observed group means in detecting effect size, d, type II error, beta, for equal group sizes, n.

[TYPE INTO R THE DESIRED INPUTS D, N, AND BETA USING VALUES IN FORM BELOW].

d <- 0.5
n <- 11
beta <- 0.05

[THEN COPY AND PASTE THE BELOW INTO R]

cv <- sqrt(qf(p=beta,df1=1,df2=(2*n)-2,,ncp=((n*n*d*d)/(2*n))))
bout <- 2*pt(q=cv,df=(2*n)-2)-1
bout2 <- 1- bout
print(bout2)

As above but allowing different group sizes, n1 and n2.

[TYPE INTO R THE DESIRED INPUTS D, N1, N2 AND BETA USING VALUES IN FORM BELOW].

d <- 1.5
n1 <- 10
n2 <- 10
beta <- 0.02

[THEN COPY AND PASTE THE BELOW INTO R]

cv <- sqrt(qf(p=beta,df1=1,df2=n1+n2-2,,ncp=((n1*n2*d*d)/(n1+n2))))
bout <- 2*pt(q=cv,df=n1+n2-2)-1
bout2 <- 1- bout
print(bout2)

SAS and FORTRAN programs with help guides are available [http://zima04.zi-mannheim.de/wktsheq/ for free download] which run equivalence analyses for other statistical tests using methodology described in Wellek (2003).

[:FAQ/mcnequiv:R code for equivalence test using McNemar's test.]

References

Lew MJ (2006) Principles: When there should be no difference - how to fail to reject the null hypothesis. Trends in Pharmacological Sciences 27(5) 274-278. [http://www.sciencedirect.com/science Available to CBSU users on ScienceDirect.]

Wellek S (2003) Testing of statistical hypotheses of equivalence. Chapman and Hall/CRC Press.