“Close but no cigar” unit tests and bias in MCMC
Andrew Gelman
by Bob Carpenter
7h ago
I’m coding up a new adaptive sampler in Python, which is super exciting (the basic methodology is due to Nawaf Bou-Rabee and Tore Kleppe). Luckily for me, another great colleague, Edward Roualdes, has been keeping me on the straight and narrow by suggesting stronger tests and pointing out actual bugs in the repository (we’ll open access it when we put the arXiv paper up—hopefully by the end of the month). There are a huge number of potential fencepost (off by one), log-vs-exponential, positive-vs-negative, numerator-vs-denominator, and related errors to make in this kind of thing. For example ..read more
Visit website
Do research articles have to be so one-sided?
Andrew Gelman
by Andrew
7h ago
It’s standard practice in research articles as well as editorials in scholarly journals to present just one side of an issue. That’s how it’s done! A typical research article looks like this: “We found X. Yes, we really found X. Here are some alternative explanations for our findings that don’t work. So, yeah, it’s really X, it can’t reasonably be anything else. Also, here’s why all the thickheaded previous researchers didn’t already find X. They were wrong, though, we’re right. It’s X. Indeed, it had to be X all along. X is the only possibility that makes sense. But it’s a discovery, it’s ab ..read more
Visit website
N=43, “a statistically significant 226% improvement,” . . . what could possibly go wrong??
Andrew Gelman
by Andrew
1d ago
Enjoy. They looked at least 12 cognitive outcomes, one of which had p = 0.02, but other differences “were just shy of statistical significance.” Also: The degree of change in the brain measure was not significantly correlated with the degree of change in the behavioral measure (p > 0.05) but this may be due to the reduced power in this analysis which necessarily only included the smaller subset of individuals who completed neuropsychological assessments during in-person visits. This is one of the researcher degrees of freedom we see all the time: an analysis with p > 0.05 can be labele ..read more
Visit website
Simulation to understand two kinds of measurement error in regression
Andrew Gelman
by Andrew
4d ago
This is all super-simple; still, it might be useful. In class today a student asked for some intuition as to why, when you’re regressing y on x, measurement error on x biases the coefficient estimate by measurement error on y does not. I gave the following quick explanation: – You’re already starting with the model, y_i = a + bx_i + e_i. If you add measurement error to y, call it y*_i = y_i + eta_i, and then you regress y* on x, you can write y* = a + bx_i + e_i + eta_i, and as long as eta is independent of e, you can just combine them into a single error term. – When you have measurement err ..read more
Visit website
Intelligence is whatever machines cannot (yet) do
Andrew Gelman
by Bob Carpenter
4d ago
I had dinner a few nights ago with Andrew’s former postdoc Aleks Jakulin, who left the green fields of academia for entrepreneurship ages ago. Aleks was telling me he was impressed by the new LLMs, but then asserted that they’re clearly not intelligent. This reminded me of the old saw in AI that “AI is whatever a machine can’t do.” In the end, the definition of “intelligent” is a matter of semantics. Semantics is defined by conventional usage, not by fiat (the exception seems to be an astronomical organization trying to change the definition of “planet” to make it more astronomically precise ..read more
Visit website
Evidence, desire, support
Andrew Gelman
by Andrew
4d ago
I keep worrying, as with a loose tooth, about news media elites who are going for the UFOs-as-space-aliens theory. This one falls halfway between election denial (too upsetting for me to want to think about too often) and belief in ghosts (too weird to take seriously). I was also thinking about the movie JFK, which I saw when it came out in 1991. As a reader of the newspapers, I knew that the narrative pushed in the movie was iffy, to say the least; still, I watched the movie intently—I wanted to believe. In the same way that in the 1970s I wanted to believe those claims that dolphins are smar ..read more
Visit website
Delayed retraction sampling
Andrew Gelman
by Andrew
5d ago
Colby Vorland writes: In case it is of interest, a paper we reported 3 years, 4 months ago was just retracted: Retracted: Effect of Moderate-Intensity Aerobic Exercise on Hepatic Fat Content and Visceral Lipids in Hepatic Patients with Diabesity: A Single-Blinded Randomised Controlled Trial https://www.hindawi.com/journals/ecam/2023/9829387/ Over this time, I was sent draft retraction notices on two occasions by Hindawi’s research integrity team that were then reneged for reasons that were not clear. The research integrity team stopped responding to me, but after I involved COPE, they eventua ..read more
Visit website
How large is that treatment effect, really? (my talk at NYU economics department Thurs 18 Apr 2024, 12:30pm)
Andrew Gelman
by Andrew
6d ago
19 W 4th Street, Room 517: How large is that treatment effect, really? Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University “Unbiased estimates” aren’t really unbiased, for a bunch of reasons, including aggregation, selection, extrapolation, and variation over time. Econometrics typically focus on causal identification, with this goal of estimating “the” effect. But we typically care about individual effects (not “Does the treatment work?” but “Where and when does it work?” and “Where and when does it hurt?”). Estimating individual effects is releva ..read more
Visit website
“He had acquired his belief not by honestly earning it in patient investigation, but by stifling his doubts. And although in the end he may have felt so sure about it that he could not think otherwise, yet inasmuch as he had knowingly and willingly worked himself into that frame of mind, he must be held responsible for it.”
Andrew Gelman
by Andrew
1w ago
Ron Bloom points us to this wonderful article, “The Ethics of Belief,” by the mathematician William Clifford, also known for Clifford algebras. The article is related to some things I’ve written about evidence vs. truth (see here and here) but much more beautifully put. Here’s how it begins: A shipowner was about to send to sea an emigrant-ship. He knew that she was old, and not overwell built at the first; that she had seen many seas and climes, and often had needed repairs. Doubts had been suggested to him that possibly she was not seaworthy. These doubts preyed upon his mind, and made him ..read more
Visit website
Here’s something you should do when beginning a project, and in the middle of a project, and in the end of the project: Clearly specify your goals, and also specify what’s not in your goal set.
Andrew Gelman
by Andrew
1w ago
Here’s something from from Witold’s slides on baggr, an R package (built on Stan) that does hierarchical modeling for meta-analysis: Overall goals: 1. Implement all basic meta-analysis models and tools 2. Focus on accessibility, model criticism and comparison 3. Help people avoid basic mistakes 4. Keep the framework flexible and extend to more models (Probably) not our goal: 5. Build a package for people who already build their models in Stan I really like this practice of specifying goals. This is so basic that it seems like we should always be doing it—but so often we don’t! Also I lik ..read more
Visit website

Follow Andrew Gelman on FeedSpot

Continue with Google
Continue with Apple
OR