This is excerpted from The Memory Clinic. Although published back in 2013, I find myself circulating this to folks who email about articles they see in the paper.
Do-It-Yourself—Believing the Newspaper Headlines
If I could wave my magic wand, I would imbue each caregiver with the ability to sort critically through the information on dementia “breakthroughs” available on the Internet. Although they are reading with some caution, caregivers forward all sorts of cure claims they find on the Internet to me for an opinion. Some of this spurious material even makes its way as links onto otherwise trustworthy websites.
Here are some general rules of thumb you can use to evaluate whether the information is valid. If the article is about diagnostic testing, you’d like to see:
- Validation of the test between people who have the illness confirmed by autopsy and an age-matched sample without the illness
- Validation among participants who resemble your demographics (age, sex, race, ethnicity) and risk for dementia (presence or absence of family history)
- Availability of the test to the general public—It may be exciting and promising, but where is it being conducted and what does it cost? New technology typically comes at a price and is not necessarily covered by your health insurance
If the article is about a new “breakthrough” treatment, it should explain:
- Were the participants included in the study affected by the target illness? Was the diagnosis made by the same guidelines that your doctor is using for you? There is a difference to you in results from a study conducted on patients at risk for a disease versus patients already diagnosed with the disease. Both types of studies may be valid, but the applicability of the results to you may differ. My patient Mimi recounts on every visit the findings of a Reader’s Digest article in which a patient with Alzheimer’s disease had a curative surgery. She insists she wants “someone to go in there [knocking on her head] and fiddle around, to bring back aaaaaall my memories.” But even if the surgical result had been replicated in a decent number of Alzheimer’s patients (it hasn’t), Mimi doesn’t understand that she does not have Alzheimer’s disease. The article doesn’t pertain to her.
- Is there a good rationale behind the use of the test drug? Researchers try to use logical means against the target illness—if we can’t justify that the intervention acts on something with a known association to the disease mechanism (e.g., the abnormal protein, or a biochemical process known to be deranged in the dementia), you’d need to be sure the study was soundly designed to take much meaning from the study. Sometimes the rationale is based on observation; the curcumin trials arose from the observation that those who have an East Indian diet with a large amount of curcumin were at lower risk than others internationally for Alzheimer’s disease. This led to a diet intervention trial and simultaneous bench studies checking the effects of curcumin on phosphorylated (sticky) tau.
- Was the study designed to test the drug against a placebo or another drug considered the current standard? The FDA and other public health drug agencies typically will not approve the drug unless it has shown clear superiority over a placebo or at least equivalence against another drug already available.
- Were the outcomes of all the enrolled participants detailed? Sometimes the study is positive only for a small segment of the population and the other participants dropped out before the study ended, which skews the final statistical analysis. It’s important to know who dropped out in case you more closely match that group than you do the study-completers.
- If there were a lot of study drop-outs, did the investigators calculate the results by bringing the last observation forward? This is the authors’ way of acknowledging that there were some hold-ups in the study for participants.
- Sometimes participants leave a study early because they are experiencing adverse effects. What were the side effects of the drug reported in the study? Are they life-threatening? Did they occur with the same frequency in the placebo group? Even if there is a high risk of a few side effects, the benefit of the drug may outweigh the risk in individual decision-making. The relative economics of pharmaceutical drug decision-making is a new science unto itself. The fact that most dementias occur in elderly persons who are at high risk of mortality due to age alone can shift the decision-making algorithm for some families.
- Is this article aimed at caregivers seeking affordable treatment, or at venture capitalists looking to invest in a drug that will sell well? The time from the end of Phase I to the end of Phase III when the drug can be widely accessible with a prescription can run 10–15 years minimum. When the newspaper headline proclaims, “Breakthrough!”, check to see if they’re talking about a pre-clinical, Phase I, or Phase III result. A breakthrough announcement in the newspaper may predate a prescription drug on your pharmacist’s shelf by 20 years.
- There’s a difference between “statistically significant” and “clinically significant.” If you can afford to enroll thousands of patients in a clinical trial, that number can influence the statistics to highlight a very small difference that would not be just a chance finding. That is a statistically significant difference. If my experimental intervention were to wave hello to each patient every morning in passing, and 1,800 of the 2,000 patients showed a change in blood pressure from 170/85 to 165/85, the investigators could report a statistically significant reduction in blood pressure (it did go down!) from a “therapeutic wave.” This amount of change in blood pressure doesn’t make much of a difference in life outcomes like stroke, heart attack, or sudden death. This finding therefore wouldn’t necessarily be clinically The most responsible of investigators would choose a clinically significant amount of change (e.g., a larger drop from 170 to 140 in systolic blood pressure) before performing the study, then would report that they found only statistically significant change at a level below clinical significance when they published the results. A responsible scientific journal’s peer reviewers would insist that the results be stated in this balanced way.
- Is the newspaper or a health website the only place where the results have been published? If so, peer researchers may not have agreed with the design, results, or phrasing of the report from the
- “What is the drug?” is important, but emerging work is also proving that “When is the drug useful?” is just as important. Consider the complexity of estrogen supplementation, for instance. We are still at the stage of discovering which abnormal protein collections are key to target with treatment first. Although Alzheimer’s disease patients have abnormal tau and amyloid, it is not clear which target should be the priority, and the answer may differ depending upon the age of the patient or level of cognitive impairment already in place.
- Whether it’s a drug or a lifestyle intervention, the same questions above should be applied to the There are some suggestions that will seem like “common sense,” such as, more exercise is better for you, but this may not be applicable to a patient with terrible arthritic pain on walking, or it may have greater impact at a different time of the lifespan than the one you’re now in.
Newspaper articles often bear word limits and cannot express all these details. They will tell whether the study was an open-label drug study (implying no placebo arm) or placebo-controlled. To answer the critique questions, you may have to go to the original articles themselves, which are often available online through a hospital or university library. To get into print, studies must detail all of these parameters. Peer reviewers demand it. When in doubt, ask your doctor to help you interpret the applicability of the new findings to you specifically. It’s what we are trained to do.