Australia/Lord_Howe is the weirdest timezone Lord Howe Island - part of Australia, population 382 - is unique in that the island's standard time zone is UTC+10:30 but is UTC+11 when daylight saving time applies. It's the only time zone where DST represents a 30 minute offset. Via lobste.rs Tags: timezones ..read more

Creating a LLM-as-a-Judge that drives business results Hamel Husain's sequel to Your AI product needs evals. This is packed with hard-won actionable advice. Hamel warns against using scores on a 1-5 scale, instead promoting an alternative he calls "Critique Shadowing". Find a domain expert (one is better than many, because you want to keep their scores consistent) and have them answer the yes/no question "Did the AI achieve the desired outcome?" - providing a critique explaining their reasoning for each of their answers. This gives you a reliable score to optimize against, and the critiques me ..read more

I try to publish weeknotes at least once every two weeks. It's been four since the last entry, so I guess this one counts as monthnotes instead. In my defense, the reason I've fallen behind on weeknotes is that I've been publishing a lot of long-form blog entries this month. Plentiful LLM vendor news A lot of LLM stuff happened. OpenAI had their DevDay, which I used as an opportunity to try out live blogging for the first time. I figured out video scraping with Google Gemini and generally got excited about how incredibly inexpensive the Gemini models are. Anthropic launched Computer Use and Ja ..read more

Bringing developer choice to Copilot with Anthropic’s Claude 3.5 Sonnet, Google’s Gemini 1.5 Pro, and OpenAI’s o1-preview The big announcement from GitHub Universe: Copilot is growing support for alternative models. GitHub Copilot predated the release of ChatGPT by more than year, and was the first widely used LLM-powered tool. This announcement includes a brief history lesson: The first public version of Copilot was launched using Codex, an early version of OpenAI GPT-3, specifically fine-tuned for coding tasks. Copilot Chat was launched in 2023 with GPT-3.5 and later GPT-4. Since then, we h ..read more

Generating Descriptive Weather Reports with LLMs Drew Breunig produces the first example I've seen in the wild of the new LLM attachments Python API. Drew's Downtown San Francisco Weather Vibes project combines output from a JSON weather API with the latest image from a webcam pointed at downtown San Francisco to produce a weather report "with a style somewhere between Jack Kerouac and J. Peterman". Here's the Python code that constructs and executes the prompt. The code runs in GitHub Actions. Tags: vision-llms, drew-breunig, llm, generative-ai, ai, llms, github-actions, prompt-engineering ..read more

I released LLM 0.17 last night, the latest version of my combined CLI tool and Python library for interacting with hundreds of different Large Language Models such as GPT-4o, Llama, Claude and Gemini. The signature feature of 0.17 is that LLM can now be used to prompt multi-modal models - which means you can now use it to send images, audio and video files to LLMs that can handle them. Processing an image with gpt-4o-mini Using a plugin to run audio and video against Gemini There's a Python API too What can we do with this? Processing an image with gpt-4o-mini Here's an example. First, insta ..read more

Matt Webb's Colophon I love a good colophon (here's mine, I should really expand it). Matt Webb has been publishing his thoughts online for 24 years, so his colophon is a delightful accumulation of ideas and principles. So following the principles of web longevity, what matters is the data, i.e. the posts, and simplicity. I want to minimise maintenance, not panic if a post gets popular, and be able to add new features without thinking too hard. [...] I don’t deliberately choose boring technology but I think a lot about longevity on the web (that’s me writing about it in 2017) and boring techn ..read more

If you want to make a good RAG tool that uses your documentation, you should start by making a search engine over those documents that would be good enough for a human to use themselves. — Panda Smith Tags: search, ai, rag, llms ..read more

Hugging Face Hub: Configure progress bars This has been driving me a little bit spare. Every time I try and build anything against a library that uses huggingface_hub somewhere under the hood to access models (most recently trying out MLX-VLM) I inevitably get output like this every single time I execute the model: Fetching 11 files: 100%|██████████████████| 11/11 [00:00<00:00, 15871.12it/s] I finally tracked down a solution, after many breakpoint() interceptions. You can fix it like this: from huggingface_hub.utils import disable_progress_bars disable_progress_bars() Or by setting the HF ..read more

python-imgcat I was investigating options for displaying images in a terminal window (for multi-modal logging output of LLM) and I found this neat Python library for displaying images using iTerm 2. It includes a CLI tool, which means you can run it without installation using uvx like this: uvx imgcat filename.png Via rich/discussions Tags: llm, cli, python, uv ..read more