It's quite intriguing how the more I learn about statistics, the more my trust in medical science seems to waver.
With that in mind, would you mind taking a look at ivmmeta.com? Ivermectin has undoubtedly been contentious during COVID. However, my primary concern is not whether ivermectin works or not, but rather the epistemological issue it raises.
I now find myself in a position where I must choose between a worldview in which the establishment can effectively insulate itself from undesired information, and a distrust in scientific research. If the establishment is correct, then most of the 95+ studies on ivmmeta.com must be fraudulent. Conversely, if the establishment is wrong, then large-scale studies like RECOVERY and ACTIV-6 must be fraudulent. Neither option is very appealing.
I tend to lean towards ivmmeta.com for reasons similar to those you discuss in this blog. (The quality of the establishment's arguments did not help.) I would greatly appreciate it if you could examine the site and share your thoughts on whether it is fraudulent or if their arguments hold merit.
Some of the studies on that site are fraudulent, but most are probably just studies that suffer from small numbers and/or failure to control for any number of relevant variables.
Just as a side note, the idea that it's a good heuristic to always disagree with the establishment's opinion is pretty silly, considering the establishment also endorses electricity, internet, toothpaste, bridges, and everything else the contrarians use. In the case of medicine it is particularly silly, because while obviously the medical establishment can get something wrong, the claim made by anti-vax people that doctors and medical scientists want to kill people or not help them should lead one to abandon modern hospitals and modern medicine altogether, and yet, you don't see much anti-vax activity targeting organ transplants, pacemakers, paracetamol, cataract surgeries, cancer treatments, antibiotics, insulin medication, kidney stone removal, baby incubators, anesthesia, and so on.
Not "can't", but "in most cases will not". In probability terms of the heterogeneity distribution, the standard deviation is very narrow, i.e., most people's effects will be very close to the population average.
I don't understand this - are you claiming that the standard deviation of treatment response (so the standard deviation of the difference between, for example, numerical disease score before and after treatment) is usually very narrow? If so, that seems to be contradicted by the entire field of psychiatry - consider, for example, response to antipsychotics, where many patients will see no response at all and many will see complete remission of symptoms. Or cancer immunotherapy, as the other commenter pointed out. I don't know if your central thesis about interaction effects is necessarily wrong, but I really don't understand how you conclude from that that treatments rarely "have dramatically different effects for otherwise similar people".
Varying outcomes from treatment does not imply interactions are present. Can you link me any convincing study showing an interaction between psychiatric treatment and patient characteristics?
Considering my response to this has made me really doubt your thesis.
Some people will be started on antihypertensive drugs. Some of them will have a significant reduction in blood pressure - more than 10mmHg systolic, say. Some will have a modest reduction in blood pressure - only 3mmHg systolic. An unlucky few might even have an increase in blood pressure. I expect the change people see in their blood pressure would follow a normal distribution.
It's really hard to predict who will respond to which antihypertensive drugs. There are blood tests that have some ability to predict. For example, for people who have hyperaldosteronism (which will generally show up on a blood test), any blood pressure drug other than an antimineralocorticoid like spironolactone is basically a waste of time. So in the case of hyperaldosteronism, there is an interaction between drug response and patient characteristics.
But wait - isn't a patient's response to an antihypertensive drug itself a patient characteristic? You might object that it's not predictable. However, that's not the case. It can be predicted: just do a short (individual) trial of the antihypertensive drug, and measure the reduction in blood pressure. In the case of a patient with hyperaldosteronism who is tried on something other than an antimineralocorticoid, they'll see no change in their blood pressure. But the reason doesn't really matter, at least from a clinical perspective - what matters is just the patient's response, whether it be due to something identifiable like hyperaldosteronism or some complex mess of factors we don't understand.
Though I'm not aware of anyone studying this rigorously, I would think with near certainty that this trial would very strongly predict how well the patient would respond to this drug in the future. In other words, how well a patient has responded to a drug in the past is a patient characteristic that strongly predicts their future response to that drug. This is an interaction between patient characteristics and clinical outcomes.
I think you're right that, currently, claimed interactions between patient characteristics and clinical outcomes rarely turn out to be useful (with important exceptions). However, it seems clear to me that this is only due to poor understanding of the factors that impact clinical outcomes; these interactions do exist, but are very difficult to measure or quantify in most cases without doing a trial of the drug. But if you do a trial, then in many cases you can measure a proxy for the overall effect of the patient characteristics on response to the drug.
Is personalized medicine a waste of time, though? I'm hesitant to say that it is across the board, but I haven't seen many cases where there's really convincing evidence for it. I don't think we really understand medicine well enough for personalized medicine to be useful outside of relatively rare circumstances.
(To answer the question you posed about patient characteristics in psychiatry, the obvious example is blood levels of psych drugs, which vary significantly from patient to patient; some patients will have very low blood levels relative to other patients on the same dose of the same drug with the same route of administration. This paper, for example, discusses response to the antipsychotic clozapine: https://pubmed.ncbi.nlm.nih.gov/34374073/ )
Recall that in my post I said cancer therapy is an exception to the point because of unique mutations in each cancer necessitating personalized treatment.
It's quite intriguing how the more I learn about statistics, the more my trust in medical science seems to waver.
With that in mind, would you mind taking a look at ivmmeta.com? Ivermectin has undoubtedly been contentious during COVID. However, my primary concern is not whether ivermectin works or not, but rather the epistemological issue it raises.
I now find myself in a position where I must choose between a worldview in which the establishment can effectively insulate itself from undesired information, and a distrust in scientific research. If the establishment is correct, then most of the 95+ studies on ivmmeta.com must be fraudulent. Conversely, if the establishment is wrong, then large-scale studies like RECOVERY and ACTIV-6 must be fraudulent. Neither option is very appealing.
I tend to lean towards ivmmeta.com for reasons similar to those you discuss in this blog. (The quality of the establishment's arguments did not help.) I would greatly appreciate it if you could examine the site and share your thoughts on whether it is fraudulent or if their arguments hold merit.
Some of the studies on that site are fraudulent, but most are probably just studies that suffer from small numbers and/or failure to control for any number of relevant variables.
Just as a side note, the idea that it's a good heuristic to always disagree with the establishment's opinion is pretty silly, considering the establishment also endorses electricity, internet, toothpaste, bridges, and everything else the contrarians use. In the case of medicine it is particularly silly, because while obviously the medical establishment can get something wrong, the claim made by anti-vax people that doctors and medical scientists want to kill people or not help them should lead one to abandon modern hospitals and modern medicine altogether, and yet, you don't see much anti-vax activity targeting organ transplants, pacemakers, paracetamol, cataract surgeries, cancer treatments, antibiotics, insulin medication, kidney stone removal, baby incubators, anesthesia, and so on.
Not "can't", but "in most cases will not". In probability terms of the heterogeneity distribution, the standard deviation is very narrow, i.e., most people's effects will be very close to the population average.
I don't understand this - are you claiming that the standard deviation of treatment response (so the standard deviation of the difference between, for example, numerical disease score before and after treatment) is usually very narrow? If so, that seems to be contradicted by the entire field of psychiatry - consider, for example, response to antipsychotics, where many patients will see no response at all and many will see complete remission of symptoms. Or cancer immunotherapy, as the other commenter pointed out. I don't know if your central thesis about interaction effects is necessarily wrong, but I really don't understand how you conclude from that that treatments rarely "have dramatically different effects for otherwise similar people".
Varying outcomes from treatment does not imply interactions are present. Can you link me any convincing study showing an interaction between psychiatric treatment and patient characteristics?
Considering my response to this has made me really doubt your thesis.
Some people will be started on antihypertensive drugs. Some of them will have a significant reduction in blood pressure - more than 10mmHg systolic, say. Some will have a modest reduction in blood pressure - only 3mmHg systolic. An unlucky few might even have an increase in blood pressure. I expect the change people see in their blood pressure would follow a normal distribution.
It's really hard to predict who will respond to which antihypertensive drugs. There are blood tests that have some ability to predict. For example, for people who have hyperaldosteronism (which will generally show up on a blood test), any blood pressure drug other than an antimineralocorticoid like spironolactone is basically a waste of time. So in the case of hyperaldosteronism, there is an interaction between drug response and patient characteristics.
But wait - isn't a patient's response to an antihypertensive drug itself a patient characteristic? You might object that it's not predictable. However, that's not the case. It can be predicted: just do a short (individual) trial of the antihypertensive drug, and measure the reduction in blood pressure. In the case of a patient with hyperaldosteronism who is tried on something other than an antimineralocorticoid, they'll see no change in their blood pressure. But the reason doesn't really matter, at least from a clinical perspective - what matters is just the patient's response, whether it be due to something identifiable like hyperaldosteronism or some complex mess of factors we don't understand.
Though I'm not aware of anyone studying this rigorously, I would think with near certainty that this trial would very strongly predict how well the patient would respond to this drug in the future. In other words, how well a patient has responded to a drug in the past is a patient characteristic that strongly predicts their future response to that drug. This is an interaction between patient characteristics and clinical outcomes.
I think you're right that, currently, claimed interactions between patient characteristics and clinical outcomes rarely turn out to be useful (with important exceptions). However, it seems clear to me that this is only due to poor understanding of the factors that impact clinical outcomes; these interactions do exist, but are very difficult to measure or quantify in most cases without doing a trial of the drug. But if you do a trial, then in many cases you can measure a proxy for the overall effect of the patient characteristics on response to the drug.
Is personalized medicine a waste of time, though? I'm hesitant to say that it is across the board, but I haven't seen many cases where there's really convincing evidence for it. I don't think we really understand medicine well enough for personalized medicine to be useful outside of relatively rare circumstances.
(To answer the question you posed about patient characteristics in psychiatry, the obvious example is blood levels of psych drugs, which vary significantly from patient to patient; some patients will have very low blood levels relative to other patients on the same dose of the same drug with the same route of administration. This paper, for example, discusses response to the antipsychotic clozapine: https://pubmed.ncbi.nlm.nih.gov/34374073/ )
Immunotheraphy for cancer has a ~15% response rate. You can't just assume things like that.
Recall that in my post I said cancer therapy is an exception to the point because of unique mutations in each cancer necessitating personalized treatment.