Photo by Zdeněk Macháček on Unsplash Data modeling is a process of creating a conceptual representation of the data and its relationships within an organization or system. Dimensional modeling is an advanced technique that attempts to present data in a way that is intuitive and understandable for any user. It also allows for high-performance access, flexibility, and scalability to accommodate changes in business needs. In this article, I will provide an in-depth overview of data modeling, with a specific focus on Kimball’s methodology. Additionally, I will introduce other techniques ..read more

Learn from Software Engineers and Discover the Joy of ‘Worse is Better’ Thinking source: unsplash.com Recently, I have had the fortune of speaking to a number of data engineers and data architects about the problems they face with data in their businesses. The main pain points I heard time and time again were: Not knowing why something broke Getting burnt with high cloud compute costs Taking too long to build data solutions/complete data projects Needing expertise on many tools and technologies These problems aren’t new. I’ve experienced them, you’ve probably experienced ..read more

Machine Learning unknowns that researchers struggle to understand — from Batch Norm to what SGD hides What We Still Don’t Understand About Machine Learning. (by author) It is surprising how some of the basic subjects in machine learning are still unknown by researchers and despite being fundamental and common to use, seem to be mysterious. It’s a fun thing about machine learning that we build things that work and then figure out why they work at all! Here, I aim to investigate the unknown territory in some machine learning concepts in order to show while these ideas can seem bas ..read more

Python Concurrency — A Brain-Friendly Guide for Data Professionals Moving data around can be slow. Here’s how you can squeeze every bit of performance optimization out of Python. Photo by Matthew Brodeur on Unsplash Python is often criticized for being among the slowest programming languages. While that claim does hold some weight, it’s vital to point out that Python is often the first programming language newcomers learn. Hence, most of the code is highly unoptimized. But Python does have a couple of tricks up its sleeve. Taking advantage of concurrent function execution is stupidly ..read more

How to use Python and OSMnx to create beautiful visuals of global cities’ road networks. Road networks are beautiful bird-eye view representations of cities. However, their importance reaches far beyond eye candies used as wallpapers. In fact, for the trained urbanist eyes, the visual structure of a city’s road network already hints at lots of information about the masterplan (if there was any) and the possible paths of development a city took, as well as the potential pitfalls and problems currently, or in the near future could affect the city. These can range from public and private transpor ..read more

A visual explanation of the math behind decision trees and gradient boosting Image generated using DALL.E A decision tree is a non-parametric supervised learning algorithm that can be used for both classification and regression. It uses a tree-like structure to represent decisions and their potential outcomes. Decision trees are simple to understand and interpret and can be easily visualized. However, when a decision tree model becomes too complex, it does not generalize well from the training data and results in overfitting. Gradient boosting is an ensemble learning model in which w ..read more

Navigating the Latest GenAI Announcements — July 2024 A guide to new models GPT-4o mini, Llama 3.1, Mistral NeMo 12B and other GenAI trends Image Created by Author with GPT-4o to represent different modelsIntroduction Since the launch of ChatGPT in November 2022, it feels like almost every week there’s a new model, novel prompting approach, innovative agent framework, or other exciting GenAI breakthrough. July 2024 is no different: this month alone we’ve seen the release of Mistral Codestral Mamba, Mistral NeMo 12B, GPT-4o mini, and Llama 3.1 amongst others. These models bring signif ..read more

What Does the Transformer Architecture Tell Us? Image by narciso1 from Pixabay The stellar performance of large language models (LLMs) such as ChatGPT has shocked the world. The breakthrough was made by the invention of the Transformer architecture, which is surprisingly simple and scalable. It is still built of deep learning neural networks. The main addition is the so-called “attention” mechanism that contextualizes each word token. Moreover, its unprecedented parallelisms endow LLMs with massive scalability and, therefore, impressive accuracy after training over billions of parame ..read more

Whether you are a Data Engineer, Machine Learning Engineer or Web developer, you ought to get used to this tool How the antic sun shines upon PydAntic users. Image by Vladimir Timofeev under license to Ilija Lazarevic. There are quite a few use cases where Pydantic fits almost seamlessly. Data processing, among others, benefits from using Pydantic as well. However, it can be used in web development for parsing and structuring data in expected formats. Today’s idea is to define a couple of pain points and show how Pydantic can be used. Let’s start with the most familiar use case, and ..read more

How to stop worrying and love the data Generated by the author using Midjourney Version 6 Definition: eval (short for evaluation). A critical phase in a model’s development lifecycle. The process that helps a team understand if an AI model is actually doing what they want it to. The evaluation process applies to all types of models from basic classifiers to LLMs like ChatGPT. The term eval is also used to refer to the dataset or list of test cases used in the evaluation. Depending on the model, an eval may involve quantitative, qualitative, human-led assessments, or all of the above ..read more