For a sample $\mathbf{x} = (x_1, x_2, \ldots, x_n)$, the tau measure of location is defined as follows (described in [[wilcox-introduction-to-robust-estimation-and-hypothesis-testing]], Edition 5, Section 3.8.1): $$ \hat{\mu}_{\tau}(\mathbf{x}) = \left( \sum_{i=1}^n w_i x_i \right) / \left( \sum_{i=1}^n w_i \right), $$ where $$ w_i(\mathbf{x}) = W_c \left( \frac{x_i - \operatorname{median}(\mathbf{x})}{\operatorname{MAD}(\mathbf{x})} \right), \quad W_c(x) = \left( 1 - (x/c)^2 \right)^2 \cdot I(|x| \leq c), $$ where $I$ is the indicator function, $\operatorname{MAD}$ is the median absolute devi ..read more

We continue exploring various use cases of the [[lowland-multimodality-detection]]. In this post, we will consider a brief example of using weighted samples ..read more

We continue exploring various corner cases for the Lowland multimodality detection. In this post, we consider an example that illustrates the usefulness of THDQE ..read more

When researchers focus on model design, they often worry whether the model is correct or not. I believe that we should accept the fact that all the models are wrong. The world is too complex to be captured by a single model: we are never able to acknowledge all the variables. Therefore, the answer to the question “Is the model correct?” is always “No”. It should not bother us: from the pragmatic perspective, it is irrelevant whether the model is correct or not. If we embrace the model misspecification, we can switch our attention to the question “What is the impact of deviations from the model ..read more

I have just published a preprint of a paper ‘Quantile-Respectful Density Estimation Based on the Harrell-Davis Quantile Estimator’. It is based on a series of my research notes. The paper preprint is available on arXiv: arXiv:2404.03835 [stat.ME]. The paper source code is available on GitHub: AndreyAkinshin/paper-qrdehd. You can cite it as follows: Andrey Akinshin (2024) “Quantile-Respectful Density Estimation Based on the Harrell-Davis Quantile Estimator” arXiv:2404.03835 Abstract: Traditional density and quantile estimators are often inconsistent with each other. Their simultaneous usage ..read more

In A better jittering approach for discretization acknowledgment in density estimation, I discussed the jittering approach that improves Quantile-Respectful Density Estimation for discrete distributions and continuous-discrete mixtures. In this post, I will show a brief example of how such an approach improves the accuracy of the Lowland multimodality detection ..read more

I continue the topic of Quantile-Respectful Density Estimation in the context of Multimodality Detection. In this post, we briefly discuss the handling of the QRDE boundary spikes in order to correctly detect the near-border modes ..read more

In How to build a smooth density estimation for a discrete sample using jittering, I proposed a jittering approach. It turned out that it does not always work well. It is not always capable of preserving the original distribution shape and avoiding gaps. In this post, I would like to propose a better strategy ..read more

Cohen’s d is one of the most popular measures of the effect size. Unfortunately, it was designed for the normal distribution, which may make it a misleading measure in the non-normal case. And the real distributions are never normal. When we discuss deviations from normality, we should treat the illusion of normality not as an atomic mental construction, but rather as a set of independent assumptions, each of which may be violated independently. In this post, I take a look at what kind of issues we may have when the symmetry assumption is heavily violated ..read more

Statistics is one of the most confusing, controversial, and depressing disciplines I know. So many different approaches, so many different opinions, so many arguments, so many person-years of wasted time, and so many flawed peer-reviewed papers. What we want from statistics is an easy-to-use tool that would nudge us toward asking the right questions and then straightforwardly guide us on how to design proper and relevant statistical procedures. What we have is a bunch of vaguely described sets of strange equations, a few arbitrarily chosen magical numbers as thresholds, and no clear understand ..read more