Population Statistics do NOT Apply to Individuals

Statistics and the Rise of Medical Fortunetellers – Michel Accad, MD – Tex Heart Inst J. 2009; free full-text PMC2801944

This article makes the increasingly forgotten, yet increasingly important, point that population statistics do NOT apply to individuals.

There was a time when the foretelling of future events was an undertaking of prophets, palm-readers, and weathermen. In recent years, however, the medical profession seems to have embraced this activity with a great deal of enthusiasm.

But hyperbole is also what comes to mind when one examines the claims that authors of such reports make.  

For example, the article “Cardiac troponin I predict s short-term mortality in vascular surgery patients”2 does not provide a method to identify the specific individuals who are headed for the funeral home. Nor does the report “Transesophageal echocardiography predicts mortality in critically ill patients with hypotension”3 offer a means to determine the fate of a given person in shock.

Of course, the skill of prediction has always been one of medicine’s defining features.

A physician is expected to collect clinical data, formulate a clinical impression, and predict a likely course of events for the patient’s illness.

Before the advent of modern therapeutics, establishing the prognosis was one of the most important functions a physician was called upon to perform.

But in the golden age of bedside medicine,4 master clinicians and academicians—rooted in a humble appreciation of the complexity of human biology and behavior—recognized the limitations of their prescience. In those days, the focus of interest was the outcome for the individual patient.

In the current age of population medicine, however, the “case” rarely merits our attention.5

The subject about whom a prediction is made is not a particular person but a group deemed suitably homogeneous to be turned into a cohort to whom tests are applied and for whom subsequent clinical events are tallied.

The keenly wished-for result is a rate of outcomes sufficiently larger or smaller than the rate for the comparative group, in order to sanction the observation as “predictive” through the use of the much-misunderstood P value.6

Invariably, the prediction comes to naught when applied to an individual within the group, but this point is all too frequently overlooked.

The diligent minimization of biasing factors, the detached execution of the experiment, and the statistical anointment by the P value in carefully designed studies provide such a sense of objectivity that the temptation to relate the findings to a specific patient becomes irresistible, regardless of the limitations of such an extrapolation.

When clinicians make predictions, they place their reputations at stake, especially if they share their forecasts with their patients or colleagues.

The prediction is understood to carry the full weight of its meaning and is rarely uttered explicitly or casually.

For academicians and researchers, a predictive declaration has a different audience. Such statements may be aimed at journal editors, funding agencies, news media, or venture capitalists, but to practitioners they sound increasingly hollow.

C-Reactive protein (CRP) may be the darling of the medical fortunetellers, but among clinicians, it is sometimes known as “Can’t Really Predict.”

The application of statistics to medicine has greatly improved our ability to discern phenomena not otherwise visible to the naked eye.

Yet Alvan Feinstein himself, considered by many to be a founder of modern clinical research, was keenly aware of what he called “the distraction of quantitative models”: an over-reliance on mathematical tools and methods imported from nonclinical domains and “applied to goals for which they are not aimed.”9

Will misleading claims of insight and inflated pronouncements of prediction continue to permeate the medical literature?

I ask, but I dare not answer.