In which I explain why I’m spending all this time exploring how LLMs might be usefully ensnared for dignified ends. I’m a digital archaeologist. One of the implications of that phrase is that I am interested in artefacts whose existence is primarily digital: the application of electricity to pathways etched in silicon in certain social, political, and/or economic contexts gives rise to emergent effects that impact the world. There are other implications, but let’s stick with that one. They are tools, but they also shape those who use them. They are material culture. Therefore, digital artefact ..read more

How it all goes …in which I discuss the logic and functioning of two jupyter notebooks that use Simon Willison’s LLM to act as a kind of discovery agent for a history api and an archaeology api. Both notebooks are available for copying and improving and use! I have been playing with GPT-Researcher, figuring out ways to make it use different data sources. I eventually got it to work with Open Alex, Open Context, and Chronicling America, and to write particular kinds of research reports. It was tricksy, and getting it to even start took a wee bit (tacit/hidden knowledge at play once again). I ta ..read more

Coronation Hall I come from west quebec. Almost as far west in quebec as you can get – there’s maybe another sixty minutes down the road (which is nothing) before it all starts to peter out. It’s not natural apple country. That came later. When I was a kid, it was a place where most of the english families were tied to farming, and the french families were tied to logging. When I was a really little kid, my dad had a dairy farm. It was his second; the first burned down and all the animals were lost. That was before I was born, but it was something I always grew up knowing. There were lots of d ..read more

Yesterday and today I have put in a lot of energy modifying the intriguing ‘GPT Researcher’ project so that it will use the Open Alex academic search engine. And it all works, and it’s very neat! I devise some new report types too – see yesterday’s post and today’s update at the bottom. This afternoon, it occurred to me that maybe I’d like my students next year to try extending GPT Researcher for themselves. Since they’re historians, what might this kind of tool be useful for? Having a pattern at hand now for building a retriever, creating a custom report, and updating the front end to run the ..read more

GPT-Researcher, from Tavily, is a ‘GPT based autonomous agent that does online comprehensive research on any given topic’ That’s cool. I learned about it from James Smithies, who posted to Mastodon about it; after which he and I had a bit of a discussion about things one might want to do with it. Another thing that interests me is the potential to increase what might be termed ‘disciplinary intelligence’ in AI tools. Is there any value in developing specialist historical tools (tuned to support history as a discipline’s standards, values, ethics etc), or will the general paid research tools w ..read more

(Crossposted from the XLab) In the past, we have applied many different distant reading techniques to social media posts related to the human remains trade. As part of a new analysis, we are exploring a computer-facilitated close reading of social media posts. Here we detail a workflow that assumes one already has a collection of images that capture posts and their associated texts (perhaps as the outcome of some sort of screen-shotting process using something like e.g. this from Simon Willison). OCR The first step is to perform the OCR on the screenshots. Different languages might be present ..read more

(Crossposted from the XLab) The dataset for the Panama Papers is available here. #archaeology #histodons #dh Wouldn’t it be great to examine it for people connected to the antiquities trade? In this post I try to do that, using our knowledge graph generated from the Trafficking Culture Encyclopedia. Behind that link you will find: Offshore Leaks (2013) Panama Papers (2016) Bahamas Leaks (2016) Paradise Papers (2017) Pandora Papers (2021) The ICJI people provide guidance for pulling those materials into neo4j to explore. But what I want to do is go through the data, all these csv tables, and ..read more

If you’ve read any of the previous posts, you’ll know I’m trying to teach gpt4 how to think like an archaeologist.There’s still stuff I’m trying to suss out – I intend to plumb it into OpenContext which you do by writing yaml that explains how the Open Context API works, I think – and the differences between using the OpenAI playground versus the ChatGPT interface to configure things is still not clear to me. But I thought, what the hell, maybe it’s time for other people to play with this thing in its current limited form to see what shakes out. What it’s good for. What it’s dangerous at. Give ..read more

This is a follow-up to what I posted the other day on the XLab site  I’m finding that interactions configured via the ChatGPT assistant builder versus the playground assistant builder are very different. The Playground version follows the instructions much more carefully. In my instructions, I ask it to stop and ask for confirmation from the user, but the ChatGPT version just jumps right ahead. Here’s an experiment. I will ask the Playground version AND the ChatGPT version the same prompt, with the same image: ‘Identify the contexts in this image, and then using their relative positioning ..read more

OpenAI released a whole bunch of stuff yesterday, and updated their python api which broke a lot of my workflows. However, Simon Willison has already updated his excellent LLM package to deal with all this, and he also built into it access to the GPT4 Turbo model preview. Over the last year or so I’ve been engaged in a project to extract triples from unstructured text about the antiquities trade, and then create a knowledge graph embedding model from them. With the release of GPT4 Turbo and Simon’s LLM, all of my code can be boiled down to a one-liner: for f in articles/*.csv; do cat "$f" | l ..read more