One image can be worth thousands of words. Image by authorIntroduction A confusion matrix is a convenient way to present the types of mistakes a machine learning mode makes. It is an N by N grid with numbers, where the value in the [n, m] cell represents the number of examples annotated with the n-th class recognized as the m-th class. In this tutorial, I will focus on creating a confusion matrix and a heatmap. The color palette will be used to display the sizes of different groups, making it easy to notice similarities or significant differences in group sizes. This kind of visualiz ..read more

Here’s a tutorial on what PSO is and how to use it Photo by James Wainscoat on Unsplash There is a joke that cracks me up: “Did you know that, before the clock was invented, people had to actively roam around and ask people the time?” There is obviously no need to explain the joke, but if we were to overthink it a little bit (like good mathematicians do) we can say that the joke is about the fact that the information of a particle of a group can be used to inform all the other particles. This concept is actually way deeper than the joke I just said and can be exploited furt ..read more

|AI|CONTINUAL LEARNING|DEEP LEARNING LIMITS| Understanding the limits of deep learning and the quest for true continual adaptation image by the author using AI “The wise adapt themselves to circumstances, as water moulds itself to the pitcher.” — Chinese Proverb “Adapt or perish, now as ever, is nature’s inexorable imperative.” — H. G. Wells Artificial intelligence in recent years has made great progress. All of these systems use artificial neurons in some form. These algorithms are inspired by their biological counterparts. For example, the neuron aggregates information fr ..read more

A python tool to tune and visualize the threshold choices for binary and multi-class classification problems Adjusting the thresholds used in classification problems (that is, adjusting the cut-offs in the probabilities used to decide between predicting one class or another) is a step that’s sometimes forgotten, but is quite easy to do and can significantly improve the quality of a model. It’s a step that should be performed with most classification problems (with some exceptions depending on what we wish to optimize for, described below). In this article, we look closer at what’s actually hap ..read more

Understanding ratios is key to unlocking deeper insights into your business’s health and driving smarter decision-making Image by Jill Wellington from Pixabay Imagine you’re a farmer with two apple orchards. One day, you decide to count the apples in each orchard. Orchard A has 1,000 apples, while Orchard B has only 500. At first glance, it seems like Orchard A is the clear winner. But what if I told you that Orchard A has 10,000 trees, while Orchard B has only 500? Suddenly, Orchard B doesn’t look so bad, does it? This is the power of proportions. In business, as in farming, raw num ..read more

Image generated by Dall-eData science is at its best out in the real world. I intend to share insights from various productionized projects I have been involved in. During my years working as a Data Scientist, I have met a lot of students interested in becoming one themselves, or newly graduated just starting out. Starting a career in data science, like any field, involves a steep learning curve. One, very good, question that I keep getting is: I have learned a lot about the theoretical aspects of data science, but what does a real world example look like? I want to share s ..read more

Image by authorA Step-by-Step Tutorial on Implementing Retrieval-Augmented Generation (RAG), Semantic Search, and Recommendations The accompanying code for this tutorial is here. My last blog post was about how to implement knowledge graphs (KGs) and Large Language Models (LLMs) together at the enterprise level. In that post, I went through the two ways KGs and LLMs are interacting right now: LLMs as tools to build KGs; and KGs as inputs into LLM or GenAI applications. The diagram below shows the two sides of integrations and the different ways people are using them together. Image b ..read more

Agentic Design | Artificial Intelligence | Application Development LangGraph — Intuitively and Exhaustively Explained Building powerful LLM agents within constraints. “Cutting a Path” by Daniel Warfield using Midjourney. All images by the author unless otherwise specified. Article originally made available on Intuitively and Exhaustively Explained. In this article we’ll explore “LangGraph”, a cutting-edge tool for making LLM agents that are actually useful. First we’ll review what an “LLM agent” is, a few popular agentic design approaches, and some of their practical shortcomings. We’ll t ..read more

Monotonic constraints are key to making machine learning models actionable, yet they are still quite unused [Image by Author] Causality is quickly becoming an essential component in every data scientist’s toolkit. And for a good reason. Indeed, causal models are highly valuable in business because they provide more reliable estimates for “what-if” scenarios, particularly when used to make decisions that impact business outcomes. In this article, I’ll show how a simple change — literally adding one line of code — can transform a traditional ML model (like Random Forest, LightGBM ..read more

Exploring popular reinforcement learning environments, in a beginner-friendly way This is a guided series on introductory RL concepts using the environments from the OpenAI Gymnasium Python package. This first article will cover the high-level concepts necessary to understand and implement Q-learning to solve the “Frozen Lake” environment. Happy learning ❤ ! A smiley lake (Image taken by author, made using OpenAI Gymnasium’s Frozen Lake environment) Let’s explore reinforcement learning by comparing it to familiar examples from everyday life. Card Game — Imagine playing a card game: W ..read more