It seems like everyone is releasing some cool model right now, so why not join the fun? Here at Count Bayesie we are introducing a new model unlike anything you’ve seen before: Linear Diffusion! A diffusion model that uses only linear models as its components (and currently works best with a very simple “language” and MNIST digits as images). You can get the model and code on Github right now. If you’d like to learn more about Linear Diffusion, read on! Here is a basic overview of Linear Diffusion’s architecture: Visualization of the modifications to the standard diffusion model used by Linea ..read more

A good chunk of my career has involved running, analyzing and writing about A/B tests (here’s a quick Bayesian overview of A/B testing if you aren’t familiar). Often A/B tests are considered to be the opposite, statistical, end of the data science spectrum from AI and machine learning. However, stepping back a bit, an A/B test just tells you the probability of an outcome (whether variant A is better than variant B) which is not that different than a deep neural network used for classification telling you the probability of a label. With the rapid advances in Natural Language Processing (NLP) i ..read more

After watching the recent 3Blue1Brown video on convolutions I realized that there is a surprising lack of articles on convolutions as they apply to probability. If you search online for "convolution" you will discover about a billion posts on "convolutional neural networks", which is a bit unfortunate since CNNs don't technically use convolutions (they use correlations, though there's a reasonable argument it doesn't matter) and convolution plays a very important and practical role in probability. Not to mention that viewing convolution from the perspective of probability might be one of the e ..read more

As you are probably aware Elon Musk has long been wavering in his decision whether or not to purchase Twitter (ticker: TWTR) for a price that would amount to $54.20 a share. For most of the summer it looked like he was going to fight the purchase, and then in early October, claimed he would go ahead with it. If you look at the stock price of TWTR for the last few months you can see that this has had a stabilizing effect on the price of TWTR during a particular tumultuous market. If Musk buys TWTR the future price will be certain and not stochastic, this impacts the current price Notice that a ..read more

The other day I received a really interesting inference question from @stevengfan8 on Twitter. Here is the text of the original question: Suppose you are trying to predict the temperature on a future day. You have an initial prediction of N(75, 5). You ask 10 scientists what their predictions are. The question you pose to each scientist is of the format “Do you think the temperature will be above or below 78 (this can be any number that is not 75)?” You also have historical predictions in this same framework from past days from the same group of scientists and can assume that not every scient ..read more

In this post we're going to take a deep dive into understanding Modern Portfolio Theory, a strategy for optimizing assets in an investment portfolio to balance the rate of risk and reward. We'll be solving this optimization problem using JAX and differentiable programming. In order to understand Modern Portfolio Theory (MPT), we'll start by exploring how we can model a single stock's price movements as cumulative samples from a normal distribution, then we'll extend this model to include multiple, partially correlated stock prices as cumulative samples from a multivariate normal. Then we can s ..read more

One of the, somewhat odd, morning rituals I've developed is what I would call Bayesian media analysis. More plainly: skeptically reading the news. I wake up in the morning and read the news, not to find out "what's happening" as much as what I'm being told is happening. I enjoy reading articles to see if there is something that stands out as inconsistent with the main story, and seeing how I might be able to glean important information even in a noisy media environment. As I touched on in the last post about warming weather in NJ, one of the reasons that I advocate learning probability and sta ..read more

A few years back I used to mentor data scientists in a bootcamp. One of the requirements was that the students complete a capstone project to demonstrate what they've learned. Many students struggled coming up with ideas for this, often trying to pick ambitious projects that they didn't now how to start. My advice was always the same: Start simple. Take anything you are interested in, the list out the basic facts you believe about this, then start using data to provide evidence for these facts. Despite living in the "information age", my experience is that we very rarely actually have any da ..read more

Statistics is, at its heart, the study of how little you actually know about anything. Even for the beginner the first thing you learn is how much you don't know about the data you have. But it gets worse over time. Without a doubt, the more I study statistics the less I know I know about it. For example, 10 years ago I knew everything about logistic regression and today I know almost nothing about it all! A wonderful object lesson in this is the logit-normal distribution. It seems innocuous at first glance: a logit-normal distributed random variable is one whose logit (log-odds) is normally d ..read more

People (well the tiny subset of them deeply and passionately interested in statistics) tend to argue a lot about the difference between "Bayesian" and "Frequentist" approaches to solving problems. Unfortunately these discussions remain largely at the level of trolling, memes and jokes (at least on Twitter), or, during moments of more enlightened discourse, entirely in the realm of philosophical speculation. As much as I love a good speculative discourse, statistics is fundamentally an applied epistemology. Everything about this topic that's interesting comes out in praxis. It is much more inte ..read more