In a recent issue of Statistics in Medicine, Thompson, Smith and Sharp (TSS) discussed an issue that we have both separately considered in this journal before. They consider approaches through meta-analysis to examining the relationship of the efficacy of treatment to the 'underlying risk', where this risk (in some sense) is regarded as being the prognosis for a patient given the control treatment. This issue has received some attention in Statistics in Medicine in recent years. For example, Skene and Wakefield in a paper discussing Bayesian approaches to analysing multi-centre trials examined the relationship between the treatment effect and the control-group response. Brand and Kragt looked at frequentist approaches to meta-analysis for a very similar problem. In both of these papers, however, it was inappropriately implicitly assumed that the observed response rate in the control group could safely be used as an underlying index of risk. A similar analysis was also carried out by Smith et al. in a paper in the British Medical Journal, although they did draw attention to the problematic nature of such analyses themselves.
The basic problem is one of measurement error in the control group response rate. This induces a correlation between the observed 'treatment effect' (the difference between treatment and control group response) and the control group response, even in the absence of any variation in the true effect of treatment from trial to trial. An analogous problem, of which medical statisticians have long been aware, exists when difference from baseline in an open trial is correlated with the baseline measurement. This is a special instance of regression to the mean.
A number of authors have now proposed solutions to this measurement error problem, 7-9 including TSS. We have, however, some reservations regarding the method proposed and discuss these below. First, however we make a number of general points concerning relevant matters.
GENERAL BACKGROUND CONSIDERATIONS 1. The first point concerns the care that must be taken in modelling hierarchical structures. For some purposes whether certain factors of coincidental interest are treated as fixed or random is irrelevant, but for others it is not.