I think you’re being kind in your remarks about the DANMASK commentary. People who should and did know better (EBMers) drew some bad faith conclusions. Surely Bayesian methods have to have a role here, when we need to understand accumulating data and estimate efficacy ‘on the fly’?
As a layperson this conclusion - the use of air purifiers with HEPA-14 filters did not reduce ARIs compared with the control - seems inaccurate then.
I get the statistical concern but since the study actually estimated that infections were 43% lower in the HEPA group, shouldn’t the conclusion be - while not determinate, it appears that the use of air purifiers with HEPA-14 filters reduce ARIs with further study required to be statistically proven?
I would put it more strongly: the conclusion that "the use of air purifiers with HEPA-14 filters did not reduce ARIs compared with the control" is simply wrong! The correct conclusion is that application of a 95% confidence interval to the results of the study does not allow the null hypothesis to be rejected.
Isn't all of this a sort of war on phisycs, also? As an engineer, I know that HEPA filters work by design, so the answer about efficacy in ARIs reduction shouldn't be more of carefully engineering how and when I filter to be enough to show up in results? Same as in "mask don't work" when many studies just measure a policy effectiveness than proper mask filtration efficacy. In case, sorry for the ignorance and I could have messed up a few English terms also, thanks!
Agree need to properly consider different lines of evidence. The challenge is that overall effectiveness is ultimately what matters for epidemic response, e.g. we found most close contacts on cruises are in environments like restaurants, where there will be less mask wearing (https://www.nature.com/articles/s41467-022-29522-y). So would need additional measures on top to make up the difference (e.g. pre-event testing) – and ideally want to quantify how much extra needed for control.
Adam, how can one set a meaningful and yet achievable effect size then? I was also disappointed with the comments coming out from Danmask as well as the Bangladesh study on mask.
Eric Topal posted a discussion with the dramatic increased air quality with the inexpensive upgrade to a "better air filter" being installed in various buildings.
I will leave it to the more skillful readers to search for the aforementioned 'chat'. Statistical evidence aside, with a better air quality being the end result, QOL measures improve which is positive.
It's discouraging that insufficient statistical power was never even mentioned as a factor in this JAMA article. We've been fighting this statistical battle for a long time. If memory serves me correctly it was a 4% difference between the treatment and control groups in that original large aspirin- reduces-heart-attack study that made them discontinue the study for ethical reasons, so the control group could get aspirin. But then that aspirin study had large sample sizes.
This is an excellent breakdown of the issue, well done
I think you’re being kind in your remarks about the DANMASK commentary. People who should and did know better (EBMers) drew some bad faith conclusions. Surely Bayesian methods have to have a role here, when we need to understand accumulating data and estimate efficacy ‘on the fly’?
As a layperson this conclusion - the use of air purifiers with HEPA-14 filters did not reduce ARIs compared with the control - seems inaccurate then.
I get the statistical concern but since the study actually estimated that infections were 43% lower in the HEPA group, shouldn’t the conclusion be - while not determinate, it appears that the use of air purifiers with HEPA-14 filters reduce ARIs with further study required to be statistically proven?
Yep, unfortunately some papers can still be quite wide of mark on how to sensibly interpret estimates with some uncertainty.
Excellent - thanks. I couldn't believe it when I saw they powered for a 50% reduction!
I would put it more strongly: the conclusion that "the use of air purifiers with HEPA-14 filters did not reduce ARIs compared with the control" is simply wrong! The correct conclusion is that application of a 95% confidence interval to the results of the study does not allow the null hypothesis to be rejected.
Isn't all of this a sort of war on phisycs, also? As an engineer, I know that HEPA filters work by design, so the answer about efficacy in ARIs reduction shouldn't be more of carefully engineering how and when I filter to be enough to show up in results? Same as in "mask don't work" when many studies just measure a policy effectiveness than proper mask filtration efficacy. In case, sorry for the ignorance and I could have messed up a few English terms also, thanks!
Agree need to properly consider different lines of evidence. The challenge is that overall effectiveness is ultimately what matters for epidemic response, e.g. we found most close contacts on cruises are in environments like restaurants, where there will be less mask wearing (https://www.nature.com/articles/s41467-022-29522-y). So would need additional measures on top to make up the difference (e.g. pre-event testing) – and ideally want to quantify how much extra needed for control.
Adam, how can one set a meaningful and yet achievable effect size then? I was also disappointed with the comments coming out from Danmask as well as the Bangladesh study on mask.
Thank you as always Adam!
Eric Topal posted a discussion with the dramatic increased air quality with the inexpensive upgrade to a "better air filter" being installed in various buildings.
I will leave it to the more skillful readers to search for the aforementioned 'chat'. Statistical evidence aside, with a better air quality being the end result, QOL measures improve which is positive.
It's discouraging that insufficient statistical power was never even mentioned as a factor in this JAMA article. We've been fighting this statistical battle for a long time. If memory serves me correctly it was a 4% difference between the treatment and control groups in that original large aspirin- reduces-heart-attack study that made them discontinue the study for ethical reasons, so the control group could get aspirin. But then that aspirin study had large sample sizes.