A tricky problem when deploying LLM-based chatbots is working out how to stop them from talking about topics that you don’t want them to talk about. Even with the cleverest prompts, with enough effort and ingenuity, users will figure a way around the guard rails. However, I recently came across a library called Semantic Router, which amongst other things, seems to provide a solution to this problem. In this blog post, we’re going to explore Semantic Router and see if we can create a chatbot that only talks about a pre-defined set of topics ..read more

I’ve been playing around with the outlines library and needed to install llama.cpp as a result. I ran into trouble when trying to offload model layers to the GPU and in this post, I’ll explain how to install llama.cpp so that you don’t have the same issues. This was how I installed the library initially: CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python And then let’s try to load a GGUF model with some layers offloaded to the GPU ..read more

In the 0.10 version of DuckDB, a breaking change was made that stops implicit casting to VARCHAR during function binding. In this blog post, we’re going to look at some ways to work around this change when fixing our DuckDB code from 0.9 or earlier. I have a CSV file that looks like this: from 'people.csv' select *; Output ┌─────────┬─────────────┐ │ name │ dateOfBirth │ │ varchar │ int64 │ ├─────────┼─────────────┤ │ John │ 19950105 │ └─────────┴─────────────┘ The dateOfBirth column isn’t an int64, but that’s how DuckDB has inferred it ..read more

One of my favourite tools in the LLM space is Ollama and if you want to learn how to use it, there’s no better place than Matt Williams' YouTube channel. His videos get a lot of comments and they tend to contain a treasure trove of the things that people are thinking about and the questions that they have. Matt recently did a video about embeddings in Ollama and I thought it’d be fun to try to get a high-level overview of what’s happening in the comments section ..read more

I’ve been playing around with the YouTube API to analyse comments on YouTube videos and needed to use pagination to get all the comments. In this blog post, we’ll learn how to do that. But before we do anything, you’ll need to go to console.developers.google.com, create a project and enable YouTube Data API v3. Figure 1. YouTube Data API Once you’ve done that, create an API key. Figure 2. Creating an API key Create an environment variable that contains your API key ..read more

For quite some time I’ve been wondering how to get access to an environment variable in a ClickHouse Local and finally today I have a solution, which we’ll explore in this blog post. My reason for wanting to do this is so that I can pass through a ClickHouse Cloud password to use in a remoteSecure function call. I wanted to do this as part of a blog post I wrote showing how to do Hybrid Query Execution with ClickHouse ..read more

I was recently working with a CSV file that contained a bunch of words and I wanted to render them on the console so that you could see all of them at once without any scrolling. i.e. I wanted the rendering of the CSV file to wrap across columns. I learned that we can do exactly this using the paste command, so let’s see how to do it. Imagine we have the CSV file shown below ..read more

I was recently reading Simon Willison’s blog post about embedding algorithms in which he described how he’d used them to create a 'related posts' section on his blog post. So, of course, I wanted to see whether I could do the same for my blog as well. Note I’ve created a video showing how to do this on my YouTube channel, Learn Data with Mark, so if you prefer to consume content through that medium, I’ve embedded it below ..read more

LLaVA (or Large Language and Vision Assistant), an open-source large multi-modal model, just released version 1.6. It claims to have improvements over version 1.5, which was released a few months ago: Increasing the input image resolution to 4x more pixels. This allows it to grasp more visual details. It supports three aspect ratios, up to 672x672, 336x1344, 1344x336 resolution. Better visual reasoning and OCR capability with an improved visual instruction tuning data mixture ..read more

This week Ollama released a Python/PyPi library to go with their awesome tool for running LLMs on your own machine. You still need to download and run Ollama, but after that you can do almost everything from the library. In this blog post, we’re going to take it for a spin. I’ve created a video showing how to do this on my YouTube channel, Learn Data with Mark, so if you prefer to consume content through that medium, I’ve embedded it below ..read more