Surrogate end points in clinical research: hazardous to your health. – PubMed – NCBI – Obstet Gynecol. 2005 May
Surrogate end points in clinical research pose real danger.
A surrogate end point is an outcome measure, commonly a laboratory test, that substitutes for a clinical event of true importance.
Resistance to activated protein C, for example, has been used as a surrogate for venous thrombosis in women using oral contraceptives.
Other examples of inappropriate surrogate end points in contraception include the
- postcoital test instead of pregnancy to evaluate new spermicides,
- breakage and slippage instead of pregnancy to evaluate condoms, and
- bone mineral density instead of fracture to assess the safety of depo-medroxyprogesterone acetate.
None of these markers captures the effect of the treatment on the true outcome.
A valid surrogate end point must both correlate with and accurately predict the outcome of interest.
Although many surrogate markers correlate with an outcome, few have been shown to capture the effect of a treatment (for example, oral contraceptives) on the outcome (venous thrombosis).
As a result, thousands of useless and misleading reports on surrogate end points litter the medical literature.
New drugs have been shown to benefit a surrogate marker, but, paradoxically, triple the risk of death.
Thousands of patients have died needlessly because of reliance on invalid surrogate markers.
Researchers should avoid surrogate end points unless they have been validated; that requires at least one well done trial using both the surrogate and true outcome.
The clinical maxim that “a difference to be a difference must make a difference” applies to research as well. Clinical research should focus on outcomes that matter.
Below is a more recent article with the same view:
Our goals in medicine are
(i) to improve the quality of patients’ lives,
(ii) help them to live longer, and
(iii) to do so at a reasonable cost.
These are our true endpoints: health status, survival, and cost.
It is thus entirely consistent with this point of view that these are the fundamental concepts that can be united in a formal cost–utility analysis.
These endpoints remain the best measures of efficacy in clinical trials comparing a new therapy to placebo or to an active control. All other measures may then be seen as surrogate endpoints or surrogates.
Thus, even serious events such as myocardial infarction and stroke may be considered surrogates, as their effect is to adversely affect the critical endpoints of health status, survival, and cost. However, in common usage a surrogate is a relatively easy to measure endpoint, available over a relatively short timeframe that is used in place of the true endpoints.
Generally, surrogates are not events, but rather measurements (physiological, laboratory, or test results, e.g. biomarkers) that predict events.
Thus, surrogates are most commonly measures that we can record, often with much shorter timescales than is necessary for events.
Surrogates are usually continuous variables, often but not necessarily with reasonable approximations of a normal distribution.
Continuous variables, especially if approximating a normal distribution, will allow for much smaller sample sizes than dichotomous variables as well as shorter periods of follow-up and lower costs
Thus, compared with clinical outcome trials, studies with surrogate endpoints can be conducted rapidly and with much less resource use and expense than endpoint studies.
Surrogates can be used in observational studies as well as in randomized trials.
For instance, low-density lipoprotein cholesterol could be used as a surrogate for cardiovascular events in a non-interventional observational study.
However, the most common and perhaps most critical issue is the use of surrogate endpoints in randomized trials comparing different therapies.
Any surrogate should be consistently measurable and sensitive to the intervention.
We give an approximate, perhaps somewhat arbitrary rank to each variable’s validity as a true surrogate. A surrogate is most useful when it
(i) consistently predicts events in the future and
(ii) if the response of the surrogate to an intervention predicts the response to the intervention in an endpoints trial.
The variable perhaps most often thought of as a consistently useful surrogate in interventional trials is the simple measurement of blood pressure. Lowering blood pressure using different therapies has consistently resulted in reduced events, in particular stroke.
However, even this relationship is not straight forward. For instance, while blood pressure is related to event rates to pressures <120 mmHg systolic, there is insufficient evidence that lowering blood pressure with pharmaceutical in patients with hypertension to <140 mmHg systolic will reduce event rates
Furthermore, in some trials similar blood pressure reduction led to different effects on hard endpoints such as mortality and stroke.
Why surrogates fail
A potential surrogate is often considered as an intermediate endpoint in a clinical trial because it is found to predict outcome in observational studies.
Thus, blood pressure and LDL cholesterol are well known (and in fact accepted by regulatory agencies such as the Federal Drug Administration or the European Medical Agency) to predict outcome, and have been used as surrogate endpoints in clinical trials. Indeed, clinical outcome studies did show that interventions which favourably affected these surrogates did in general reduce the incidence of cardiovascular events.
However, there are also many examples where a therapy was shown to favourably affect a surrogate, but was not found to reduce cardiovascular events.
Thus, serum HDL cholesterol level has an inverse relationship with cardiovascular events. Furthermore, both niacin and cholesterol ester transfer protein blockers have been shown to increase serum HDL cholesterol.
Nonetheless, recent trials with these agents have not shown efficacy in reducing cardiovascular events; indeed, one of them even increased mortality in spite of marked increases in HDL cholesterol.
As a third example, hormone replacement therapy in post-menopausal women will favourably affect the serum lipid profile. However, randomized trials of hormone replacement therapy have not been shown to reduce cardiovascular events.
It is also possible that the problem with a therapy that works on a surrogate but failed in an outcomes trial was due to inadequate dosing, too short a time period, an inappropriate patient population or too small a population.
Why do some potential surrogates seem to work well, while others fail? We can gain insight by considering the nature of causality. A true surrogate should be in the causal path of a true endpoint.
To establish causation requires a deep understanding of the pathophysiology of the disease process and hence is a stronger criterion than just noting an association.
If there is association but not causation, then the relationship between a surrogate and outcome events may be confounded.
A confounder is a variable that predicts outcomes and has higher prevalence in the group with the potential surrogate of interest.
A true surrogate will always be affected before the clinical endpoint as it often precedes late disease states that lead to myocardial infarction, stroke, or death.
However, it is wrong to think that a temporal relationship is all that is necessary to establish causality. This has been recognized since ancient times as noted by the famous humorous statement – ‘post hoc ergo propter hoc’, which is Latin for ‘after this, therefore because of this’. Temporality is just one component of what is necessary to consider causal relationship.
Hill considered the steps necessary to establish a causal relationship between any risk factor, i.e. surrogate, and future events.
The Bradford Hill criteria are:
(i) temporal relationship, the cause must always come before the effect,
(ii) strength of association,
(iii) dose–response relationship,
(iv) consistency of the relationship,
(v) biological plausibility,
(vi) consideration of alternatives,
(vii) experimental verification,
(viii) specificity, that is a specific cause for a specific effect, and
(ix) coherence, that is compatible with existing knowledge.
The relationship of surrogates to clinical endpoints
The statistical relationship of surrogates to clinical endpoints is multi-faceted and needs careful explanation.
Surrogates, such as biomarkers, are usually continuous variables while events are generally binary or categorical. The relationship of a surrogate to the clinical endpoint is more complex than that of a risk factor to an endpoint in that a therapy’s value is based on its effect on the clinical endpoint.
How can surrogates be used?
Given the complex relationship between interventions, surrogates, and endpoints, how can surrogates best be used?
It is unusual for a surrogate to be so reliable that it can replace clinical endpoints for regulatory approval and medical decision-making purposes.
Studies with surrogate endpoints will generally be much less expensive and much more rapid to conduct than studies with clinical endpoints.
However, it is very difficult to be confident about the relationship between the surrogate and the clinical endpoint.
Studies with surrogates will also generally have a smaller number of patients and a shorter time span. This limits such studies for evaluation of safety, where safety endpoints may have no pathophysiologic relationship with the surrogate.
Thus clinical endpoint trials will remain essential, although even endpoint trials may not have adequate power for safety.
Given problems that have been noted in the past with regulatory bodies approving therapies based on surrogates, it is likely that the demand for endpoint studies for regulatory approval will be the norm, and use of surrogates for licensing new therapies will be the exception.
Trials with surrogate endpoints that do not show efficacy may obviate the need for endpoint trials, saving time, expense, and avoiding unnecessary patient risk. Surrogates will remain of interest in developing new therapies and providing pathophysiologic insight and guiding the development of clinical endpoint trials.
the potential surrogate should be considered in specific disease states, for specific therapies and for specific clinical outcomes.
The surrogate should also respond to therapy consistently
The use of surrogates is complex, and there is no single criterion or standard that can readily be applied.
An understanding of causality and consideration of the relevant practical criteria is important, but the adoption of a surrogate must always be considered on a case by case basis.
The place of surrogates in phase II trials may be reasonable as a guide to pivotal phase three trials. However, the uncertainty of surrogates must limit their use in phase III trials, where the unreliability of surrogates alone for registration is recognized, so as to avoid potential risk to public health.