MIRI updates Aaron Scher and Joe Collman have joined the Technical Governance Team at MIRI as researchers. Aaron previously did independent research related to sycophancy in language models and mechanistic interpretability, while Joe previously did independent research related to AI safety via debate and contributed to field-building work at MATS and BlueDot Impact.  In an... Read more » The post September 2024 Newsletter appeared first on Machine Intelligence Research Institute ..read more

MIRI updates Rob Bensinger suggests that AI risk discourse could be improved by adopting a new set of labels for different perspectives on existential risk from AI. One drawback of “AI doomer” (a label sometimes used in online discussions) is that it does not have a consistent meaning. AI researcher John Wentworth guesses that a central difference between his and Eliezer Yudkowsky’s views might be that Eliezer expects AI to not use abstractions which are similar to those used by humans. Eliezer clarifies that he expects similar abstractions for predictive parts of the natural world, but large ..read more

MIRI updates MIRI Communications Manager Gretta Duleba explains MIRI’s current communications strategy. We hope to clearly communicate to policymakers and the general public why there’s an urgent need to shut down frontier AI development, and make the case for installing an “off-switch”. This will not be easy, and there is a lot of work to be done. Some projects we’re currently exploring include a new website, a book, and an online reference resource. Rob Bensinger argues, contra Leopold Aschenbrenner, that the US government should not race to develop artificial superintelligence. “If anyone ..read more

As we explained in our MIRI 2024 Mission and Strategy update, MIRI has pivoted to prioritize policy, communications, and technical governance research over technical alignment research. This follow-up post goes into detail about our communications strategy. The Objective: Shut it Down1 Our objective is to convince major powers to shut down the development of frontier AI systems worldwide before it is too late. We believe that nothing less than this will prevent future misaligned smarter-than-human AI systems from destroying humanity. Persuading governments worldwide to take sufficiently drast ..read more

Update (5-15-2024): I wrote that “it appears that not all of the leading AI labs are honoring the voluntary agreements they made at [AI Safety Summit],” citing a Politico article. However, after seeing more discussion about it (e.g. here), I am now highly uncertain about whether the labs made specific commitments, what those commitments were, and whether commitments were broken. These seem like important questions, so I hope that we can get more clarity. MIRI updates: MIRI is shutting down the Visible Thoughts Project. We originally announced the project in November of 2021 ..read more

The MIRI Newsletter is back in action after a hiatus since July 2022. To recap some of the biggest MIRI developments since then: MIRI released its 2024 Mission and Strategy Update, announcing a major shift in focus: While we’re continuing to support various technical research programs at MIRI, our new top priority is broad public communication and policy change. In short, we’ve become increasingly pessimistic that humanity will be able to solve the alignment problem in time, while we’ve become more hopeful (relatively speaking) about the prospect of intergovernmental agreements to hit the b ..read more

As we announced back in October, I have taken on the senior leadership role at MIRI as its CEO. It’s a big pair of shoes to fill, and an awesome responsibility that I’m honored to take on. There have been several changes at MIRI since our 2020 strategic update, so let’s get into it.1 The short version: We think it’s very unlikely that the AI alignment field will be able to make progress quickly enough to prevent human extinction and the loss of the future’s potential value, that we expect will result from loss of control to smarter-than-human AI systems. However, developments this past year l ..read more

Today, December 6th, 2023, I participated in the U.S. Senate’s eighth bipartisan AI Insight Forum, which focused on the topic of “Risk, Alignment, & Guarding Against Doomsday Scenarios.” I’d like to thank Leader Schumer, and Senators Rounds, Heinrich, and Young, for the invitation to participate in the Forum. One of the central points I made in the Forum discussion was that upcoming general AI systems are different. We can’t just use the same playbook we’ve used for the last fifty years. Participants were asked to submit written statements of up to 5 pages prior to the event. In my statem ..read more

Status: Vague, sorry. The point seems almost tautological to me, and yet also seems like the correct answer to the people going around saying “LLMs turned out to be not very want-y, when are the people who expected ‘agents’ going to update?”, so, here we are. Okay, so you know how AI today isn’t great at certain… let’s say “long-horizon” tasks? Like novel large-scale engineering projects, or writing a long book series with lots of foreshadowing? (Modulo the fact that it can play chess pretty well, which is longer-horizon than some things; this distinction is quantitative rather than qualitativ ..read more

Over the next two days, the UK government is hosting an AI Safety Summit focused on “the safe and responsible development of frontier AI”. They requested that seven companies (Amazon, Anthropic, DeepMind, Inflection, Meta, Microsoft, and OpenAI) “outline their AI Safety Policies across nine areas of AI Safety”. Below, I’ll give my thoughts on the nine areas the UK government described; I’ll note key priorities that I don’t think are addressed by company-side policy at all; and I’ll say a few words (with input from Matthew Gray, whose discussions here I’ve found valuable) about the individual c ..read more