The recent paper by Sandvik et al. proposes a method for determining the size of internal pilot studies in the case of two groups and continuous outcome variables. Their method could be slightly improved by taking into consideration follow-up time and accrual rate.
The basic idea of their method is to choose the size of the internal pilot as large as possible, but keeping the risk of exceeding the true optimal size (the one belonging to the true variance) under a prespecified level.
Let S denote the initial estimate of the standard deviation, obtained from n patients and let C be the upper limit of the probability of exceeding the true sample size (C fixed by the investigators). Then Prob( (S(n!1)/ !L\ )"C, that is, the size corresponding to the 100 (1!C) per cent one-sided lower confidence limit of will be the size of the internal pilot.
The authors comment that in case of values for which (n!1) is close to !L\ (their ratio is greater than 0)75), the use of maximum pilot size has to be avoided. For the probability level of 0)05 this happens for n*55.
Birkett and Day showed through a convincing number of simulations that, whether the true study variance is small or large in relation to S, an internal pilot of 10 patients per treatment arm would suffice and 'this would give reliable estimates of the true study variance without adversely affecting the power and -level'. They also raise the problem of time delay caused by the internal pilot when patient recruitment has to be stopped until the new study size is estimated (continuing recruitment during the internal pilot would increase the risk of exceeding the true optimal study size).
Therefore, an adjustment of the pilot size taking into account recruitment rate and follow-up time would not affect too much the precision of the variance estimate and it would shorten the study duration with one follow-up time. If the recruitment rate is of m patients/day and the follow-up time is t, mt patients could be recruited during the waiting period of time caused by the internal pilot. Decreasing the sample size corresponding to S(+(n!1)/ !L\ , with mt would keep the number of patients enrolled until the end of the internal pilot below the true optimal size with a probability C. This can be applied only if the adjusted size of the pilot exceeds 20.
Invoking the example constructed by the authors, in a randomized two-arm clinical trial the initial estimate of the within-group standard deviation is 8)7, obtained from 28 patients. For a clinically relevant difference of 5 units, a significance level of 5 per cent and a power of 90 per cent, 64 patients are required in each arm. The lower one-sided 5 per cent confidence limit of the standard deviation is 7)14 and its corresponding sample size under the same conditions is 43 patients/arm. If the follow-up time is 8 weeks and the accrual rate is 5 patients/week, the overall pilot size of 86 can be decreased by 40.
To compare the two above pilot sizes, the simulations made by Birkett and Day were repeated (applying exactly the same steps but written in SAS 6)11); 11,875 simulations were made to determine with a standard error of 0)002 and 3600 were made to obtain with a standard error of 0)005.