help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Kolmogorov-Smirnov test


From: Mike Miller
Subject: Re: Kolmogorov-Smirnov test
Date: Thu, 17 Nov 2005 13:25:55 -0600 (CST)

On Thu, 17 Nov 2005, Brendan Drew wrote:

Continuing on this theme, the following code doesn't do what one would expect:


k = 10000;
x = zeros(1,1000);
for i = 1:1000
x(i) = kolmogorov_smirnov_test_2(randn(1,k), randn(1,k));
end;
hist(x);

I was hoping that Hamish was just exceedingly unlucky; however, the histogram produced by this code is alarmingly uniform for several values of k. Unless I've misunderstood the K-S test completely (which is entirely possible), I would expect the distribution to be heavily weighted towards the high confidence values. I'll run some more tests later in the evening, but I suspect that something isn't quite right -- either with randn or with the K-S test implementation.


In statistical testing, a valid p-value has a uniform distribution when the null hypothesis is true. In the K-S two-sample test, the null hypothesis is that the two distributions are the same. We reject the null when p is small. The probability of p < .05, for example, equals .05 when the null is true, and this is because p is uniform when the null is true.

Mike

--
Michael B. Miller, Ph.D.
Assistant Professor
Division of Epidemiology and Community Health
and Institute of Human Genetics
University of Minnesota
http://taxa.epi.umn.edu/~mbmiller/



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------



reply via email to

[Prev in Thread] Current Thread [Next in Thread]