Lilliefors/Van Soest’ s test of normality 1 Hervé Abdi & PaulMolin

Lilliefors/Van Soest’ s test of normality 1 Hervé Abdi & PaulMolin



Overview
The normality assumption is at the core of a majority of standard
statistical procedures, and it is important to be able to test this
assumption. In addition, showing that a sample does not come
from a normally distributed population is sometimes of impor-
tance per se. Among the many procedures used to test this as-
sumption, one of the most well-known is a modi?cation of the
Kolomogorov-Smirnov test of goodness of ?t, generally referred to
as the Lilliefors test for normality (or Lilliefors test, for short). This
test was developed independently by Lilliefors (1967) and by Van
Soest (1967). The null hypothesis for this test is that the error is
normally distributed (i.e., there is no difference between the ob-
served distribution of the error and a normal distribution). The
alternative hypothesis is that the error is not normally distributed.
Like most statistical tests, this test of normality de?nes a cri-
terion and gives its sampling distribution. When the probability
associated with the criterion is smaller than a given a-level, the
1
In: Neil Salkind (Ed.) (2007). Encyclopedia of Measurement and Statistics.
Thousand Oaks (CA): Sage.
Address correspondence to: Hervé Abdi
Programin Cognition and Neurosciences,MS: Gr.4.1,
The University of Texas at Dallas,
Richardson, TX 75083–0688, USA
E-mail: herve@utdallas.edu http://www.utd.edu/~herve
1

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
alternative hypothesis is accepted (i.e., we conclude that the sam-
ple does not come from a normal distribution). An interesting pe-
culiarity of the Lilliefors’ test is the technique used to derive the
sampling distribution of the criterion. In general, mathematical
statisticians derive the sampling distribution of the criterion us-
ing analytical techniques. However in this case, this approach fails
and consequently, Lilliefors decided to calculate an approximation
of the sampling distribution by using the Monte-Carlo technique.
Essentially, the procedure consists of extracting a large number of
samples fromaNormal Population and computing the value of the
criterion for each of these samples. The empirical distribution of
the values of the criterion gives an approximation of the sampling
distribution of the criterion under the null hypothesis.
Speci?cally, both Lilliefors and Van Soest used, for each sample
size chosen, 1000 random samples derived from a standardized
normal distribution to approximate the sampling distribution of a
Kolmogorov-Smirnov criterion of goodness of ?t. The critical val-
ues given by Lilliefors and Van Soest are quite similar, the relative
2
error being of the order of 10- .
According to Lilliefors (1967) this test of normality ismore pow-
erful than others procedures for a wide range of nonnormal con-
ditions. Dagnelie (1968) indicated, in addition, that the critical
values reported by Lilliefors can be approximated by an analyti-
cal formula. Such a formula facilitates writing computer routines
because it eliminates the risk of creating errors when keying in the
values of the table. Recently, Molin and Abdi (1998), re?ned the
approximation given by Dagnelie and computed new tables using
a larger number of runs (i.e., K = 100,000) in their simulations.
2 Notation
The sample for the test ismade of N scores, each of themdenoted
X i. The samplemean is denoted M X is computed as
N
1
M X = X i, (1)
N
i
2

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
the sample variance is denoted
N 2
(X i -M X)
2 i
S , (2)
X = N -1
and the standard deviation of the sample denoted S X is equal to
the square root of the sample variance.
The ?rst step of the test is to transform each of the X i scores
into Z-scores as follows:
X i M X
Z i = - . (3)
S X
For each Z i-score we compute the proportion of score smaller
or equal to its value: This is called the frequency associated with
this score and it is denoted S (Z i). For each Z i-score we also com-
pute the probability associated with this score if is comes from a
“standard” normal distribution with a mean of 0 and a standard
deviation of 1. We denote this probability byN (Z i), and it is equal
to Z 1 1
i
2
N (Z i) = exp - Z i . (4)
-8 2p 2
The criterion for the Lilliefors’ test is denoted . It is calculated L
fromthe Z-scores, and it is equal to
L =max{|S (Z i)-N (Z i)|, |S (Z i)-N (Z i 1) } . (5)
i - |
So is the absolute value of the biggest split between the proba- L
bility associated to Z i when Z i is normally distributed, and the fre-
quencies actually observed. The term|S (Z i)-N (Z i-1)| is needed
to take into account that, because the empirical distribution is dis-
crete, the maximum absolute difference can occur at either end-
points of the empirical distribution.
The critical values are given by Table 2. critical is the critical L
value. TheNull hypothesis is rejectedwhen the criterion is greater L
than or equal to the critical value critical. L
3

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
3 Numerical example
As an illustration, we will look at an analysis of variance exam-
ple for which we want to test the so-called “normality assump-
tion” that states that the within group deviations (i.e., the “resid-
uals”) are normally distributed. The data are from Abdi (1987, p.
93ff.) and correspond to memory scores obtained by 20 subjects
who were assigned to one of 4 experimental groups (hence 5 sub-
jects per group). The score of the sth subject in the ath group is
denoted Y a,s, and the mean of each group is denoted M a.. The
within-group mean square MS S(A) is equal to 2.35, it correspond
to the best estimation of the population error variance.
G. 1 G. 2 G. 3 G. 4
3 5 2 5
3 9 4 4
2 8 5 3
4 4 4 5
3 9 1 4
Y a. 15 35 16 21
M a. 3 7 3.2 4.2
TheNormality assumption states that the error is normally dis-
tributed. In the analysis of variance framework, the error corre-
sponds to the residuals which are equal to the deviations of the
scores to themean of their group. So in order to test the normality
assumption for the analysis of variance, the ?rst step is to com-
pute the residuals from the scores. We denote X i the residual cor-
responding to the i th observation (with i going from 1 to 20). The
residuals are given in the following table:
Y as 3 3 2 4 3 5 9 8 4 9
X i 0 0 -1 1 0 -2 2 1 -3 2
Y as 2 4 5 4 1 5 4 3 5 4
X i -1.2 .8 1.8 .8 -2.2 .8 -.2 -1.2 .8 -.2
4

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
Next we transform the X i values into Z i values using the fol-
lowing formula:
X i
Z i = (6)
MS S(A)
because MS S(A) is the best estimate of the population variance,
and the mean of X i is zero. Then, for each Z i value, the frequency
associated with S (Z i) and the probability associated with Z i un-
der the Normality condition N (Z i) are computed [we use a table
of the Normal Distribution to obtain N (Z i)]. The results are pre-
sented in Table 1.
The value of the criterion is (see Table 1)
L =max{|S (Z i)-N (Z i)|, |S (Z i)-N (Z i 1) } .250 . (7)
i - | =
Taking an a level of a = .05, with N = 20, we ?nd (from Table 2)
that the critical value is equal critical = .192. Because is larger L L
than critical, the null hypothesis is rejected and we conclude that L
the residuals in our experiment are not distributed normally.
4 Numerical approximation
The available tables for the Lilliefors’ test of normality typically re-
port the critical values for a small set of alpha values. For example,
the present table reports the critical values for
a = [.20, .15, .10, .05, .01].
These values correspond to the alpha values used for most tests
involving only one null hypothesis, as this was the standard pro-
cedure in the late sixties. The current statistical practice, however,
favors multiple tests (maybe as a consequence of the availability
of statistical packages). Because usingmultiple tests increases the
overall Type I error (i.e., the Familywise Type I error or a PF), it has
become customary to recommend testing each hypothesis with a
corrected a level (i.e., the Type I error per comparison, or a PC)
such as the Bonferonni or Sidák corrections. For example, using ?
a Bonferonni approach with a familywise value of a PF = .05, and
5

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
Table 1: How to compute the criterion for the Lilliefors’ test of
normality. N i stands for the absolute frequency of a given value
of X i, F i stands for the absolute frequency associated with a given
value of X i (i.e., the number of scores smaller or equal to X i),
Z i is the Z-score corresponding to X i, S (Z i) is the proportion of
scores smaller than Z i, N (Z i) is the probability associated with
Z i for the standard normal distribution, D 0 =| S (Z i) -N (Z i) |,
D -1=| S (Z i)-N (Z i-1) |, and max is the maximum of {D 0,D -1}.
The value of the criterion is = .250. L
X i N i F i Z i S (Z i) N (Z i) D 0 D -1 max
-3.0 1 1 -1.96 .05 .025 .025 .050 .050
-2.2 1 2 -1.44 .10 .075 .025 .075 .075
-2.0 1 3 -1.30 .15 .097 .053 .074 .074
-1.2 2 5 -.78 .25 .218 .032 .154 .154
-1.0 1 6 -.65 .30 .258 .052 .083 .083
-.2 2 8 -.13 .40 .449 .049 .143 .143
.0 3 11 .00 .55 .500 .050 .102 .102
.8 4 15 .52 .75 .699 .051 .250 .250
1.0 2 17 .65 .85 .742 .108 .151 .151
1.8 1 18 1.17 .90 .879 .021 .157 .157
2.0 2 20 1.30 1.00 .903 .097 .120 .120
testing J = 3 hypotheses requires that each hypothesis is tested at
the level of
1 1
a a
PC= J PF = 3×.05 = .0167 . (8)
With a Sidák approach, each hypothesis will be tested at the level ?
of
1 1
J 3
a a
PC= 1-(1- PF) = 1-(1-.05) = .0170 . (9)
As this example illustrates, both procedures are likely to require us-
ing different a levels than the ones given by the tables. In fact, it is
rather unlikely that a table could be precise enough to provide the
wide range of alpha values needed formultiple testing purposes. A
6

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
more practical solution is to generate the critical values for any al-
pha value, or, alternatively, to obtain the probability associated to
any value of the Kolmogorov-Smirnov criterion. Such an approach
can be implemented by approximating the sampling distribution
“on the ?y" for each speci?c problemand deriving the critical val-
ues for unusual values of a.
Another approach to ?nding critical values for unusual values
of a, is to ?nd a numerical approximation for the sampling distri-
butions. Molin and Abdi (1998) proposed such an approximation
and showed that it was accurate for at least the ?rst two signi?-
cant digits. Their procedure, somewhat complex, is better imple-
mented with a computer and comprises two steps.
The ?rst step is to compute a quantity called A obtained from
the following formula:
2 L 2
-(b 1+N)+ (b 1+N) -4b 2 b 0- -
A = , (10)
2b 2
with
b 2= 0.08861783849346
b 1= 1.30748185078790
b 0= 0.37872256037043 . (11)
The second step implements a polynomial approximation and
estimates the probability associated to a given value as: L
Pr( ) ˜ -.37782822932809+1.67819837908004A L
2 3
-3.02959249450445A +2.80015798142101A
4 5
-1.39874347510845A +0.40466213484419A
6 7
-0.06353440854207A +0.00287462087623A
8 9
+0.00069650013110A -0.00011872227037A
10
+0.00000575586834A . (12)
For example, suppose that we have obtained a value of = L
.1030 froma sample of size N = 50. (Table 2 shows that Pr( ) = .20.) L
7

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
To estimate Pr( ) we need ?rst to compute A, and then use this L
value in Equation 12. FromEquation 10, we compute the estimate
of A as:
2 L 2
-(b 1+N)+ (b 1+N) -4b 2 b 0- -
A =
2b 2
2 2
-(b 1+50)+ (b 1+50) -4b 2 b 0-.1030-
=
2b 2
= 1.82402308769590 . (13)
Plugging in this value of A in Equation 12 gives
Pr( ) = .19840103775379 ˜ .20 . (14) L
As illustrated by this example, the approximated value of Pr( ) is L
correct for the ?rst two decimal values.
References
[1] Abdi, H. (1987). Introduction au traitement statistique des
données expérimentales. Grenoble: Presses Universitaires de
Grenoble.
[2] Dagnelie, P . (1968). A propos de l’emploi du test de
Kolmogorov-Smirnov comme test de normalité, Biométrie et
Praximétrie 9, 3–13.
[3] Lilliefors, H. W. (1967). On the Kolmogorov-Smirnov test for
normality with mean and variance unknown, Journal of the
American Statistical Association, 62, 399–402.
[4] Molin, P ., Abdi H. (1998). New Tables and nu-
merical approximation for the Kolmogorov-
Smirnov/Lillierfors/Van Soest test of normality. Tech-
nical report, University of Bourgogne. Available from
www.utd.edu/~herve/MA_Lilliefors98.pdf.
[5] Van Soest, J. (1967). Some experimental results concerning
tests of normality. Statistica. Neerlandica, 21, 91–97, 1967.
8

Table 2: Table of the critical values for the Kolmogorov-Smir-
nov/Lillefors test of normality obtained with K = 100,000 samples
for each sample size. The intersection of a given row and column
shows the critical value critical for the sample size labelling the row L
and the alpha level labelling the column. For N > 50 the critical
.83 N
value can be found by using f N=+ -.01.
N
N a = .20 a = .15 a = .10 a = .05 a = .01
4 .3027 .3216 .3456 .3754 .4129
5 .2893 .3027 .3188 .3427 .3959
6 .2694 .2816 .2982 .3245 .3728
7 .2521 .2641 .2802 .3041 .3504
8 .2387 .2502 .2649 .2875 .3331
9 .2273 .2382 .2522 .2744 .3162
10 .2171 .2273 .2410 .2616 .3037
11 .2080 .2179 .2306 .2506 .2905
12 .2004 .2101 .2228 .2426 .2812
13 .1932 .2025 .2147 .2337 .2714
14 .1869 .1959 .2077 .2257 .2627
15 .1811 .1899 .2016 .2196 .2545
16 .1758 .1843 .1956 .2128 .2477
17 .1711 .1794 .1902 .2071 .2408
18 .1666 .1747 .1852 .2018 .2345
19 .1624 .1700 .1803 .1965 .2285
20 .1589 .1666 .1764 .1920 .2226
21 .1553 .1629 .1726 .1881 .2190
22 .1517 .1592 .1690 .1840 .2141
23 .1484 .1555 .1650 .1798 .2090
24 .1458 .1527 .1619 .1766 .2053
25 .1429 .1498 .1589 .1726 .2010
26 .1406 .1472 .1562 .1699 .1985
27 .1381 .1448 .1533 .1665 .1941
28 .1358 .1423 .1509 .1641 .1911
Table continues on the following page . . .

H. Abdi & P .Molin: Lilliefors / Van Soest Normality Test
Table 3: . . . Continued. Table of the critical values for the Kolmogo-
rov-Smirnov/Lillefors test of normality obtained with K = 100,000
samples for each sample size. The intersection of a given row and
column shows the critical value critical for the sample size labelling L
the row and the alpha level labelling the column. For N > 50 the
.83 N
critical value can be found by using f N=+ -.01.
N
N a = .20 a = .15 a = .10 a = .05 a = .01
29 .1334 .1398 .1483 .1614 .1886
30 .1315 .1378 .1460 .1590 .1848
31 .1291 .1353 .1432 .1559 .1820
32 .1274 .1336 .1415 .1542 .1798
33 .1254 .1314 .1392 .1518 .1770
34 .1236 .1295 .1373 .1497 .1747
35 .1220 .1278 .1356 .1478 .1720
36 .1203 .1260 .1336 .1454 .1695
37 .1188 .1245 .1320 .1436 .1677
38 .1174 .1230 .1303 .1421 .1653
39 .1159 .1214 .1288 .1402 .1634
40 .1147 .1204 .1275 .1386 .1616
41 .1131 .1186 .1258 .1373 .1599
42 .1119 .1172 .1244 .1353 .1573
43 .1106 .1159 .1228 .1339 .1556
44 .1095 .1148 .1216 .1322 .1542
45 .1083 .1134 .1204 .1309 .1525
46 .1071 .1123 .1189 .1293 .1512
47 .1062 .1113 .1180 .1282 .1499
48 .1047 .1098 .1165 .1269 .1476
49 .1040 .1089 .1153 .1256 .1463
50 .1030 .1079 .1142 .1246 .1457
0.741 0.775 0.819 0.895 1.035
> 50
f N f N f N f N f N
10