[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Kolmogorov-Smirnov test
From: |
Mike Miller |
Subject: |
Re: Kolmogorov-Smirnov test |
Date: |
Thu, 17 Nov 2005 13:25:55 -0600 (CST) |
On Thu, 17 Nov 2005, Brendan Drew wrote:
Continuing on this theme, the following code doesn't do what one would
expect:
k = 10000;
x = zeros(1,1000);
for i = 1:1000
x(i) = kolmogorov_smirnov_test_2(randn(1,k), randn(1,k));
end;
hist(x);
I was hoping that Hamish was just exceedingly unlucky; however, the
histogram produced by this code is alarmingly uniform for several values
of k. Unless I've misunderstood the K-S test completely (which is
entirely possible), I would expect the distribution to be heavily
weighted towards the high confidence values. I'll run some more tests
later in the evening, but I suspect that something isn't quite right --
either with randn or with the K-S test implementation.
In statistical testing, a valid p-value has a uniform distribution when
the null hypothesis is true. In the K-S two-sample test, the null
hypothesis is that the two distributions are the same. We reject the null
when p is small. The probability of p < .05, for example, equals .05 when
the null is true, and this is because p is uniform when the null is true.
Mike
--
Michael B. Miller, Ph.D.
Assistant Professor
Division of Epidemiology and Community Health
and Institute of Human Genetics
University of Minnesota
http://taxa.epi.umn.edu/~mbmiller/
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------