Article summary V follow up

Summary

The Stampfer commentary pointed out some of the weaknesses inherent in the application of the “intention-to treat” principle to analyze the observational data by Hernán et al. The advantage of a randomized trial would be lost if only considering adherent individuals. ITT principle could overcome this shortcoming, but poor adherence would yield inaccurate estimates. In observational data, with repeated assessments, the actual exposure of the individuals can be identified, reducing misclassification. However, application of the ITT principle to observational data essentially combined the most important limitations of each study design and the worst features of both designs emerged. And the author also stated that the method by Hernán et al made an analysis that is already complex and difficult to follow even more complicated by requiring additional assumptions and models. The writer also criticized that this novel method added no new insights and should not be used for routine use because of the complexity and “Black box”.

Hernán and Robins addressed each of these criticisms. First, they raised the 3 new insights in their analysis. Second, they explained why Stampfer’s claim that there is empirical evidence of bias in adherence-adjusted HR estimates was wrong. The authors also justified their claims that the conventional method is the black box and potentially biased. The NHS observational estimates are biased for healthy initiator bias, healthy continuer bias or Misclassification of the hormone exposure of initiators. They eliminated the misclassification bias by beginning follow-up at the estimated date of therapy initiation. The healthy initiator bias and healthy continuer bias could be empirically estimated with (and essentially only with) an ITT analysis. In a nutshell, the author clarified their analysis combines the strengths of observational studies (large sample size, long follow up, multiple longitudinal measurements) and a major strength of randomized trials.

Reaction

There are three commentaries on the paper by Herna´n et al. The discussants disagree sharply in their assessments of the value of this analytic strategy. The commentary by Stampfer was negative and the problems he raised were very important. Also, applying ITT to observational data was also my confusion. Herna´n et al addressed each of these criticisms clearly and informatively. The interesting part in this response by Herna´n et al was how they quantified the degree of healthy initiator bias and the healthy continuer bias. And why their adherence-adjusted HR estimator is unbiased for the causal effect of continuous therapy. However, the adherence-adjusted estimates are less precise than conventional NHS estimates.

Questions

1. Could we use this method for routine use and how?
2. Herna´n et al formulated the question of interest explicitly “what would be the relative risk of CHD comparing hormone therapy initiators and non initiators had they adhered to their initial treatment status during the entire follow-up?”, what’s the question for the randomized clinical trials?
3. Why their methodology (start of follow-up at the time of therapy initiation rather than time of questionnaire return could combine the strengths of both studies?

[1] Stampfer MJ. ITT for observational data: worst of both worlds?. Epidemiology. 2008 Nov 1;19(6):783-4. [2] Hernán MA, Robins JM. Authors’ response, part I: observational studies analyzed like randomized experiments: best of both worlds. Epidemiology. 2008 Nov 1;19(6):789-92.