Thomas Hühn writes: I’m thinking about doing a Bayesian analysis of a very small subset of PISA or TIMSS data. Those large-scale education surveys do not report students achievement scores as single numbers, but they report five or ten numbers, so called plausible values. Those plausible values have been sampled from a constructed probability distribution. The user guides and methodology papers strongly warn against taking those five plausible values as five observations, and also against taking the mean of those five plausible values and doing statistical analysis on that. Instead you’re sup ..read more

A Beatles fan pointed me to this news item from a few years ago, “A Songwriting Mystery Solved: Math Proves John Lennon Wrote ‘In My Life.'” This surprised me, because in his memoir, Many Years from Now, Paul McCartney very clearly stated that he, Paul, wrote it. Also, the news report is from NPR. Who you gonna trust, NPR or Paul McCartney? The question pretty much answers itself. But I was curious, so I read on: Over the years, Lennon and McCartney have revealed who really wrote what, but some songs are still up for debate. The two even debate between themselves — their memories seem to diff ..read more

Fri 3 May 2024, 11am at Chambers Hall, Ruan Conference Room – lower level: Audience Choice: Bayesian Workflow / Causal Generalization / Modeling of Sampling Weights The audience is invited to choose among three possible talks: Bayesian Workflow: The workflow of applied Bayesian statistics includes not just inference but also building, checking, and understanding fitted models. We discuss various live issues including prior distributions, data models, and computation, in the context of ideas such as the Fail Fast Principle and the Folk Theorem of Statistical Computing. We also consider some ex ..read more

I’ve been thinking a lot about how to evaluate MCMC samplers. A common way to do this is to run one or more iterations of your contender against a baseline of something simple, something well understood, or more rarely, the current champion (which seems to remain NUTS, though we’re open to suggestions for alternatives). Reporting comparisons of averages (and uncertainty) Then, what do you report? What I usually see is a report of averages over runs, such as average effective sample size per gradient eval. Sometimes I’ll see medians, but I like averages better here as it’s a fairer indication o ..read more

Oscar Oelrich, Shutong Ding, Måns Magnusson, Aki Vehtari, and Mattias Villani write: Bayesian model comparison is often based on the posterior distribution over the set of compared models. This distribution is often observed to concentrate on a single model even when other measures of model fit or forecasting ability indicate no strong preference. Furthermore, a moderate change in the data sample can easily shift the posterior model probabilities to concentrate on another model. We document overconfidence in two high-profile applications in economics and neuroscience. To shed more light on t ..read more

Erling Rognli writes: I just wanted to bring your attention to a positive stats story, in case you’d want to feature it on the blog. A major journal in my field (the Journal of Child Psychology and Psychiatry) has over time taken a strong stance for using Bayesian methods, publishing an editorial in 2016 advocating switching to Bayesian methods: Editorial: Bayesian benefits for child psychology and psychiatry researchers – Oldehinkel – 2016 – Journal of Child Psychology and Psychiatry. And recently following up with inviting myself and some colleagues to write a brief introduction to Bayesian ..read more

There is a new paper in arXiv: “Supporting Bayesian modelling workflows with iterative filtering for multiverse analysis” by Anna Elisabeth Riha, Nikolas Siccha, Antti Oulasvirta, and Aki Vehtari. Anna writes An essential component of Bayesian workflows is the iteration within and across models with the goal of validating and improving the models. Workflows make the required and optional steps in model development explicit, but also require the modeller to entertain different candidate models and keep track of the dynamic set of considered models. By acknowledging the existence of multiple ca ..read more

I’ll be speaking twice at Carnegie Mellon soon. CMU statistics seminar, Fri 5 Apr 2024, 2:15pm, in Doherty Hall A302: Bayesian Workflow: Some Progress and Open Questions The workflow of applied Bayesian statistics includes not just inference but also model building, model checking, confidence-building using fake data, troubleshooting problems with computation, model understanding, and model comparison. We would like to toward codify these steps in the realistic scenario in which researchers are fitting many models for a given problem. We discuss various issues including prior distributions, d ..read more

This one from 2017 is good; I want to share it with all of you again: Our recent discussion with mathematician Russ Lyons on confidence intervals reminded me of a famous logic paradox, in which equality is not as simple as it seems. The classic example goes as follows: Abraham Lincoln is the 16th president of the United States, but this does not mean that one can substitute the two expressions “Abraham Lincoln” and “the 16th president of the United States” at will. For example, consider the statement, “If things had gone a bit differently in 1860, Stephen Douglas could have become the 16th pr ..read more

Tom Vladeck writes: I thought you may be interested in some internal research my company did using a conjoint experiment, with analysis using Stan! The upshot is that we found that vaccine hesitant people would require a large payment to take the vaccine, and that there was a substantial difference between the prices required for J&J and Moderna & Pfizer (evidence that the pause was very damaging). You can see the model code here. My reply: Cool! I recommend you remove the blank lines from your Stan code as that will make your program easier to read. Vladeck responded: I prefer a lo ..read more