Chan considered exact non-asymptotical tests based on two types of asymptotical test statistics Z #/ and Z #$ for the classical and non-classical null hypotheses. A lot of material on exact non-asymptotical tests for two independent binomially distributed variables has been published for the classical null hypothesis H : P !P *0. Little is known, however, for shifted null hypotheses such as H : P !P * . We agree with the author that there is some need to develop exact tests, because the true levels of the usual asymptotical tests are quite unsatisfactory, in particular for small to medium sample sizes. We have, however, some problems with his paper.
For "xed * and the resulting statistic Z*"Z #/ ( *), the author de"nes the p-value through max Pr(Z*)Z* "P3D) where the domain of P is the interval D"[0, 1! *]. According to the author's formula (1), this interval is only the boundary of the null space. For a valid p-value the search for the maximum must therefore be extended over the whole null space. This could be done, for example, by investigating p( )"max Pr(Z*)Z* "P3[0, 1 ! ]) as a function of , 1* * *. The procedure proposed by Chan would be a valid procedure if p( ) is a decreasing function of . This was a claim of Santner and Snell using the di!erence of proportions as the statistic Z, and this was subsequently also claimed by StatXact. To the best of our knowledge, a proof has not yet been published. We are preparing a paper which gives a proof for the fact that, for a broad class of orderings of the sample space and for continuous boundary curves, the search for the maximum may indeed be restricted to the boundary. For our proof we have to assume that the statistic Z satis"es the convexity condition (C), "rst introduced by Barnard. (C) requires that Z(i!1, j ))Z(i, j ) and Z(i, j#1))Z(i, j ) for all (i, j ). It guarantees that the p-value for a particular outcome (i, j ) is not smaller compared to points &more distant from the null hypothesis' (i, j#1) or (i!1, j ). Almost all test statistics or ordering criteria used so far satisfy (C): the usual normal approximation with pooled or unpooled variance, Fisher's exact test; the di!erence of proportions, and Barnard's test. The Berger and Boos p-value does not always satisfy (C). A counter example for the Berger and Boos method is provided for n "6 and n "22, s "3 and s "18 with the speci"cations "0)01, Z-statistic with pooled variance and a &classical' one-sided null hypothesis. We calculated the p-value at (3, 19)"0)10762'0)10136, which is the p-value at the point 18). It is unknown whether the test statistics Z #/ and Z #$ indeed satisfy (C), and a proof seems to be di$cult due to the complex determination of the denominator of the test statistic. Therefore it remains open whether a search through the boundary is su$cient to "nd the overall maximum in the null space.
We were wondering why the author did not mention the work of Barnard who introduced the maximum principle which is also applicable and of high value for non-classical null hypotheses as discussed here. Brie#y, Barnard constructs an ordering criterion on the sample space in a sequential manner starting with the most extreme point (0, n ) in the upper left corner, and then adds from the set of adjacent points the one that increases the probability under the null hypothesis by the smallest amount, therenby taking into