If You See A Study Labeled "Landmark," Throw It In The Trash

If You See A Study Labeled “Landmark,” Throw It In The Trash

November 12, 2015 / Rocky D. Bilhartz / 6 Comments

I’m going to tell you the real goal for a clinical trial in medicine these days.

It’s to be landmark.

It’s what the trial sponsors seek and their investigators envy.

Landmark is the ultimate goal.

In fact, trial sponsors will frequently call their own study landmark even before it gets published in a scientific journal. Kind of like getting your Super Bowl ring fitted during the preseason.

The sponsors of a recent blood pressure study, known as the SPRINT trial, did just that. And, just this week, their self-awarded landmark study was finally published for all to review.

I previously told you whether or not you should take more notice of your blood pressure at home, so I feel a brief discussion about this trial might also be helpful.

THE SPRINT TRIAL

The SPRINT trial enrolled over 9,000 people in the United States and Puerto Rico. For every patient enrolled, there was general consensus that the patient needed to be taking medication for high blood pressure.

I’ll say that again.

Everyone in the trial, without question, was felt by medical personnel to require blood pressure medication. Therefore, this wasn’t a trial of if you needed blood pressure medication. It was essentially a trial of how much.

Patients were divided into two groups based upon systolic blood pressure (SBP) goals. One group was given an intense target (SBP less than 120mmHg). The other group was given a standard, or less-intense target (SBP less than 140mmHg). Of note, no patients with diabetes or prior stroke were enrolled.

The trial was stopped early, at a median follow up of just over 3 years. Like I told you, the trial was landmark. Statistically, the investigators said the results were clear, and that the trial need not to be continued any longer.

The intense blood pressure group won.

As you might have expected, the intense group required more medications to get their results. On average, each patient needed one more medicine (a mean of 2.8 versus 1.8 blood pressure medications, respectively, between the two groups).

By adding (on average) one more medication per patient, the mean blood pressure of the intense group was 13 points lower than the less-intense group (121.5 mmHg verses 134.6 mmHg, respectively).

And, as I told you, the intense group, with the lower average blood pressure readings, won. Their primary outcomes were better. How better? I’ll let you decide.

1) The intense group had less deaths.

But, there is something you must know.

For 98.5% of the patients enrolled in the study, it did not matter to what group you were assigned. Intense or less intense therapy was inconsequential to you dying. For 98.5% of the patients in either group, this outcome was the same. Regardless.

2) The intense group had less heart failure.

But, again, here is what you should know.

For greater than 99% of the patients enrolled, the group assigned didn’t matter when it came to developing heart failure. What was going to happen, just happened. Anyway.

3) The intense group had more patients suffer from serious adverse events like too low of blood pressure (hypotension), passing-out spells (syncope), and sudden kidney injury (acute renal failure).

But, yes, you already know.

For 99%, 99%, and 98% of the patients, respectively, these events occurred with the same frequency no matter which of the group’s average number of pills you were receiving.

I could go on and on. In fact, there were many more intriguing statistical findings. For example, the lower your blood pressure was at the beginning of the trial (e.g. if you were in the subgroup of people with a “seemingly decent” SBP ≤ 132mmHg at the start), the more numerical benefit you received regarding the primary outcome by making your blood pressure even lower on intense therapy.

But, lower, lower, lower, wasn’t even obtainable most of the time. That’s right. The intended target for the intense group (SBP < 120mmHg) wasn’t even achieved in over half the patients. In these people, the target was really nothing more than a theoretical construct.

As for how any of this might apply to an individual patient outside of this trial is really anyone’s guess. All we’ve really established are the averages of this and that, and apparently, these may be irrelevant for nearly every end-point examined in about 99% of the people involved.

But, again, this is what we call landmark.

And, by saying that word, we almost succeed at dodging the real question:

What are you, the patient, to do about your blood pressure?

Well, I’ll tell you what I intend to do with mine.

I’ll entrust it to a doctor. A doctor who knows that science and technology alone will never define my best course of action. A doctor who, as Dr. Michel Accad has previously stated, understands Mutton’s Law of optimal patient care. Basically, someone who…

…knows what to do and when to do it.

share+

Clinical Trials, In the News

sanram

Dr. Bilhartz,

1. I’m trying to understand what you’re saying. Could you explain the point in which you mentioned that for 98-99% of the patients enrolled in the study, it didn’t matter which group they were on when it comes to likelihood of death or cardiac events? What do you mean by this, and how does this affect the validity of their conclusions?
2. Why would it have been important for them to include diabetics or those who had suffered a stroke? What other changes would you make on the design of this study to make it more informative?
3. What effect do you expect the conclusions of this “landmark” trial to have on the management of blood pressure?

Respectfully,

Santiago
- Rocky D. Bilhartz
  
  Medical therapies deemed “beneficial” from clinical trials usually provide less absolute benefits for most patients than many people realize.
  
  Let’s treat 100 patients with “Medicine X” for 5 years. Only 1 patient suffers a heart attack.
  
  Let’s treat 100 similar patients with “Nothing” for 5 years. 2 patients suffer a heart attack.
  
  We conclude that Medicine X is better than Nothing.
  
  Medicine X reduced heart attacks by 50% compared to Nothing (2 heart attacks vs. 1 heart attack.) Medicine X results in a 50% relative risk reduction in heart attacks! You should take it.
  
  Well, maybe.
  
  But, just know that 98% of the patients treated with Nothing did just fine. These people did not need Medicine X. They evidently were not going to have a heart attack anyway.
  
  Moreover, for Medicine X to successfully prevent 1 heart attack in 5 years (2 heart attacks vs. 1 again), Medicine X had to be given to 100 people. Basically, a lot of people needed to be taking Medicine X in order for 1 lucky person to be saved. Everyone else was just paying for Medicine X and risking side effects from it.
  
  1) My point is not to discredit the use of needed therapies, but merely to point out that the absolute benefits of most therapies are less than people think. People should know this.
  
  2) The methods used in a study are often as important as the data collected. Essentially, you can only “see” what your methods allow you to see. My point about excluding diabetics or prior stroke patients isn’t meant to be critical of the authors’ methods. You must create some boundaries for all trials. I’m merely pointing out that many people with rather common conditions (like diabetes) were excluded from this trial’s evaluation. Extrapolating results to those individuals may be fraught with error.
  
  3) The SPRINT trial is another set of data in the complex world of understanding blood pressure. Ultimately, for some patients in SPRINT, obtaining a “lower” blood pressure wasn’t even possible (many patients never reached goal). For a vast majority of patients in the trial, “lower” blood pressure didn’t prevent bad outcomes and it was associated with increased frequency of side effects. Other trials, evaluating slightly different patient populations, did not necessarily come to the same conclusion as SPRINT (that “lower” is better). I must take all of these things into context when I’m looking at a real individual patient in my office trying to decide what is best to do.
  
  The real answer is to admit that we still aren’t certain how to manage blood pressure. We aren’t certain how to measure it (home or office, standing or sitting) and even how much to treat it. We have some general ideas, but perhaps, we need to rethink our understanding. Many patients are being treated with less clear benefits. The goal is to better target who needs therapy and who doesn’t.
  
  You may enjoy this short article on “redefining hypertension.” It outlines many of my own thoughts: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2929868/
  
  Kind regards,
  RB
  - sanram
    
    Thank you for going through the numbers and the scenarios you were thinking about when you were sharing the facts. I can follow your logic now.
    
    1. I read somewhere that a considerable proportion of the results of clinical trials/experiments cannot be reproduced. Is this accurate, and is this related to this issue? If so, does this have to do with the study design or that the results are taken at face value and not analyzed properly?
    
    2. How prevalent is the problem of overestimation and over-reliance on the benefits that are claimed in clinical trials? Is this related to the push for more “guidelines” in patient care and to treat everyone as part of a population rather than an individual?
    - Rocky D. Bilhartz
      
      1. This phenomenon has been well-described, appearing most recently in the field of psychology (Article here: https://www.washingtonpost.com/news/speaking-of-science/wp/2015/08/27/trouble-in-science-massive-effort-to-reproduce-100-experimental-results-succeeds-only-36-times/).
      
      Many have speculated reasons why. Some of those are mentioned in the article above. And, of course, you nailed some of it yourself.
      
      Study design and methods used are obviously critical.
      
      The other major issue with medical trials is that, in many ways, the day a study gets published it is essentially “out-of-date.”
      
      For example, the trial has already ended. Data was collected in the past. The study was performed previously on a group of people with seemingly infinite individual variables that can never again truly be replicated. Most of these variables are even inadequately controlled for during the trial itself (e.g. what they ate for lunch, what bus they got on in the city, where they parked their car, etc…). Scientists often assume that all these dynamic variables do not matter. That may be true. But, if so, why isn’t it easier to replicate results?
      
      This doesn’t mean we shouldn’t study things. We should. But, we should be aware of the limitations our methods provide. We should be open to the fact that the only certainty appears to be that things will “change.” And, we should remember that it’s not just the “things.” How we interpret the same “things” changes too.
      
      2) You could list a number of things that contribute to this one… including economic and political/social influences. Economic: I have (profit from) a drug you need (must buy). Political: I will make you healthier (if you elect me). I (the society of whomever) will create “My” guidelines for all to “worship” and follow. Social: I really want to help you. (I don’t think that I can; oh, wait, there is a 1% chance; I have hope.) Take this medicine.
      
      The role you mention regarding population medicine is real too. And, I don’t think that all intentions that built the “quality” and “guideline” movement arose from evil foundations. We are just seeing many of the unintended consequences that result from turning focus away from the individual and more toward a nebulous entity.
      
      We make the mistake of believing that grouping everyone into this nebulous entity will make things better because it somehow will make things simpler. Just do “this” with everyone. Same blood pressure, same vitamin, same vaccine, same medicine, same routine.
      
      The truth is that working with this nebulous entity is really much more complex of a thing to do. Using local knowledge of time and place will be paramount to being able to do that well. Unfortunately, central authority control is not well-equipped for the dynamic movements that are needed.
      
      Kind regards,
      RB
      - sanram
        
        1. In discussing the reproducibility of studies, shouldn’t there be mention of the distinction between the type of disease/treatment that is being studied? Without knowing specific examples off the top of my head, it seems that there are certain conditions that are relatively straightforward (perhaps you could enlighten me with an example from your field) and we have already found an appropriate treatment for. Because of this, there’s little disagreement on how they should be handled. Then there seems to be an entirely different category of conditions that are more complex (we don’t know what causes them/we should revise if we should think of them as diseases or we know what causes them, but because they may be tied to personal behavior, forcing people to change their ways is still difficult), and despite the best efforts/investments, have been difficult to treat. In trying to control costs by conducting studies that form the foundation of eventual guidelines (evidence-based medicine, I think they label it), it seems like this issue is ignored. There seems to be an assumption that everything that physicians treat is not only easy but also cheap to treat. Any thoughts?
        
        2. Depending on the spin of the article that writes headlines such as, “only a small portion of scientific studies are reproducible”, I think you’re left with the impression that a. you shouldn’t trust science as much as you do or b. you are being intentionally deceived by someone. What would you tell someone who formulates this conclusion?
        
        3. What about meta-analyses? Are they helpful in controlling for some of these problems? It seems like these are proposed to be the solution to the variability in results from individual studies.
      - Rocky D. Bilhartz
        
        Regarding 1), there is some relevance to the relationship between the complexity of a problem and the best manner at which to solve it. Malcolm Gladwell, in his best-selling book, “Blink,” discusses the idea that often highly complex problems with multitudes of variables are best solved by NOT using analytical (formula based) models. You will find this short post relevant as well: http://alertandoriented.com/how-experts-really-decide/
        
        However, just note that you will have some debate as to what is an “easy” problem or not to solve. As I’ve discussed in a number of my posts, even the simplest of vital signs (blood pressure) has complex decision making attached to it.
        
        2) I agree with (a) — you should not trust empirical science as much we do in medicine. As for (b), all people have the potential to deceive. Yet, I also want the “expert” to help me sort through the uncertainty. Thus the ongoing quest for the expert who has my best interests at heart. 🙂
        
        3) Meta-analyses are really used for the purpose of establishing “statistical significance” for a number of smaller studies that individually did not have such power (in numbers) by themselves. They are really for statisticians not clinicians. I incorporate their conclusions in my own knowledge base, but with a great deal of uncertainty (as I should).