In this issue, Shmueli has commented on our short letter previously published in this journal (Stevens et al., 2006) in which we replicated the analysis of Torrance et al. (1996) in estimating a relationship between visual analogue scale (VAS) and standard gamble (SG) based on the power curve relationship. We also extended this analysis to empirically test alternative functional forms.
We thank Shmueli for his comments and interest in our work. Shmueli has made three separate comments. The first is that 'The estimated cubic function overfits the four data points and is questionable with respect to the implied attitude toward relative risk.'
We took a purely empirical approach to the analysis in this work, as stated in our paper. We also noted that there is no theoretical framework for the cubic function. However, our emphasis was that if you take a purely empirical approach, we do not find the power curve to perform well, especially compared to other functional forms. This has also been shown in a larger data set by Dolan and Sutton (1997) to which we make reference in the paper.
We recognize that the results of our work and our conclusions have strong implications which is why we recommend further research to establish the most appropriate functional form or the avoidance of mapping between VAS and SG completely by using SG directly.
The main purpose of our paper was to highlight the weakness of estimating a mapping function on such a small number of data points. Something that both the HUI2 and HUI3 valuation studies do (Feeny et al., 2002). We also wished to show that other functional forms can be estimated that pass through the points [(0,0),(1,1)] and that they may perform better than the power curve.
The second comment states that 'The evaluation of the functional forms in terms of the individual predictions' Mean Absolute Error is misleading.'
In the paper, the predictions are not calculated as you describe (using individual level data). They are calculated using mean VAS values for each health state. When describing how we calculate mean absolute error in the text, when we refer to each observation, we are referring to the health states and not to the individuals.
We have rerun the models and predictions to check our results and have discovered that there were errors in the published linear and power predictions. We provide the correct results here, together with the associated MAE and RMSE. Neither change by more than 0.08.
We think that your results may differ from ours based on rounding errors as the larger number of decimal places is not shown in the paper. There was also a slight error in the number of health states the cubic function predicted to within 0.05. This should be 3 not 4 and we have corrected this in the table too.
These corrected results do not change the conclusions or the arguments of this paper. We thank you for identifying the errors and giving us the opportunity to correct them.