Sci-fi interfaces analyzes the interfaces seen in movies and television show for fun and erudition. Its main author and curator is Chris Noessel. He does this in his “spare time” so two posts a week is a pretty good pace. All screenshots are property of the studios who produced the show and are used under Fair Use laws.
One that has straight-up Nazis in an ongoing alternate-history dimension-hopping series. (The Man in the High Castle.)
One “pretty damned close” (Star Trek Discovery)
Why is this so? Why is strong fascism largely missing in screen sci-fi?
I don’t know the answer to that question for certain, but if you’ve read this blog, you know that that hasn’t stopped me before. Here are my best guesses.
Of course the first thing we should note is that sci-fi isn’t indebted to show fascism. Very arguably, the core of the genre concerns the effects, challenges, and opportunities of technology/science. Sometimes that means swords made out of cauterizing light. Sometimes that means green-skinned aliens. Sometimes that means software hyperevolving and abandoning its users. There’s nothing that says it must care about fascism.
But, it is a lens through which many readers prefer to do their speculative thinking, and fascism changes with technology, so it still feels like a bit of surprise to be missing.
The distinction may not be important to the story
For example, does it matter to the story that it’s fascism rather than, say, despotism? Or tyranny? Or just a bad guy? If not, the writer may not bother working out what kind of evil it is. It may not be worth it.
There may not be enough narrative time
In short formats like film, showing strong fascism takes a lot of narrative time, and must be fit in along with all the other stuff pertinent to your story. Sci-fi in particular has the narrative burden of explaining the new rules implied by its speculative technology, so doesn’t have a lot of room to also include a bunch of stuff about a political movement. If you’re telling a love story about Space Mooks discovering The Cake is a Lie, it may not make sense to go into detail about the government system wrecking things in the background, even if it informs the diegesis. A caustic boss and violent peers may be all you can “afford” to detail.
In longer or serial formats like television, you have more time, so it makes sense to me that that’s where the strongest example of fascism appeared. I note that in Star Trek Discovery we see evidence of T’Kuvma’s fascism only across several episodes rather than all at once.
Background fascism is tough
If you do go to the trouble to depict strong fascism, you then have the problem of perspective: Do you tell WWII from the leaders’ perspectives? Like Mussolini and Farinacci’s? Or from a perspective more similar to your viewership’s, like a layperson? If you tell it from the fascist leader’s perspective (as Star Trek Discovery has), you’re perpetuating the discredited Great Man Theory of historical events (though I suppose most of sci-fi commits this same error), and possibly building up empathy in the wrong place. But to tell it from the layperson’s perspective means you have to convey how and why the society is beginning to burn around them, and that leaves you with a lot of exposition or taking even more time out of your narrative and away from the lead character’s focus. Neither of these options is very satisfying. I imagine it’s a tricky place to write for.
It may not fit the tone
Fascism is a dark thing with its real-world psychological seductions, politics, racism, and ultranationalism. Fascism operates through violence and that almost always warrants a violent response to end it if the society in which it metastasizes can not resolve it through politics. That kind of violence may not fit your age group or the tone you’re going for. No parent wants their young kids watching “Paw Patrol Very Special Episode: The Pups Fight Fascism.”
Additionally, investors may want to tone down any realistic violence as they hope to be part of the next hyper-palatable Star Wars blockbuster franchise.
If a writer pens something that feels too much like Nazi Germany or Fascist Italy, it begins to feel derivative, preachy, or maybe even too on-the-nose to be believable. (If you’d told me three years ago that I’d need to put reviewing sci-fi interfaces on hold to write posts on sci-fi fascism I’m not sure I’d have believed you.)
Audiences who sense they’re watching a morality play instead of an engaging story will turn off, unless, as with V for Vendetta, or even Shadow on the Land, it is obviously the point. And for reasons noted above, those tend to be social fiction or alternate history, not sci-fi.
The fascists have to have their comeuppance
If a story does bother to put all the narrative effort to describing a dictator, and his palingenetic narrative, and how it foments violent ultranationalism amongst his authoritarian loyalists, then something damned well better happen to that dictator over the course of the story, i.e. he is defeated. It would be very depressing for the hero’s journey to play out, but no change in the background fascist government in which it happens. (I am waiting for every last fascist in The Handmaid’s Tale to get what’s coming to them. Under His Eye.) Think of this as Checkov’s Dictator. It can’t just be there and not be used.
Authors hadn’t thought it important
Another possibility is that the authors haven’t been exposed to the dangers of fascism in the real world, (or forgotten about it from history) and so couldn’t imagine why they would want to explore it in speculative ways.
A sea change a-coming
So there are lots of reasons why strong fascism may not have appeared in screen sci-fi. But I don’t see any reason that can’t be overcome with diligent attention (and skill), But sci-fi tends to reflect, amplify, and extend trends in the world around us, so I’ll bet we’re going to see a lot more examination of fascism cropping up in sci-fi over the next years. The green light and production processes being as relatively slow as they are, we probably won’t see a rise in strongly fascist stories until the end of 2018 and beyond.
In the meantime, I’m going to keep an eye on Star Trek Discovery to see where they’re taking that storyline, and of course rewatch V for Vendetta and The Handmaid’s Tale. Not strictly sci-fi, but awesome and on point.
23 AUG 2108 UPDATE: Owing to commenter Mark Connelly’s smart observations, I’ve upped the total to 2. See below.
Equipped with some definitions for fascism, I turned to movies and TV shows that showcased fascism in some way to see what was there. (Reminder: This project focuses on screen sci-fi for reasons.) It’s not a big list, and I’m sure it’s not exhaustive. But I think it was a good list to start with.
HYDRA scum with a double-fist salute. Captain America: The First Avenger. (2011)
Building a list of candidates to consider
In the future it would be awesome to be able to describe some criteria and have an AI read sci-fi scripts or watch the shows to provide results. I’m sure it would surprise us. But we’re not there yet. So first I worked from unaided memory, pulling up top-of-mind examples. Then I worked with aided memory, reviewing the shows I already had in the scifiinterfaces survey. Finally I looked for shows I didn’t know about, soliciting friends and colleagues and finally augmenting with web searches for discussions on the topic and pre-made lists. I wound up with 33 candidate movies and television shows. Then I went one by one and compared them to my five aspects of fascism. That resulted in a lot of whittling down. A lot. At the end I wound up with only…2 (!)
Limits of narrative
We have to admit upfront that the stories we see in TV and movies exist in larger, speculative worlds, and we only see the parts that pertain to the story. (Unlike, say, a world book or fan wiki.) Some, like Infinity Chamber, are built around showing us only the tiniest sliver of the world and leave it up to us to figure the rest out. Over the course of a longer-format show, like television, we might even get to see a great deal of its world, but we won’t ever see everything. We see and hear stuff that happens. That means we might see some aspects of fascism and even a great deal of hinting that it’s the real deal, but we can’t be sure. So, for instance, we never see a charismatic leader responsible for the oppressive bureaucracy in Brazil, but it might just be that the story didn’t take us there.
There are even some shows with actual swastika-wearing Nazis or even Hitler in them, but in most of these we don’t see evidence of all the key aspects of real world fascism. It’s more like the show relies on your knowledge that Nazis are bad, mmkay?
So some groups or societies might be fascist, but we never see enough to say for sure. These got a question mark in the spreadsheet, and are tagged “maybe.”
Actual half-Hitler, half-dinosaur. From the Iron Sky: The Coming Racetrailer.
The “almost” stuff
Describing why a show is almost-but-not-quite can be quite instructive, so let’s discuss the “almost”s. All of these examples might still fit “weak fascism” as discussed in the prior post but that’s little better than calling them “bullies,” which isn’t that useful.
The Prisoner, the late 1960s serial, had some weird recapture balloons knocking people down, but just wasn’t violent enough to count. Violence was a romanticized ideal for Mussolini, and he suggested routine violence was important for (get this) good health. Nazis of course are almost cartoonishly associated with their horrific, at-scale violence. Violence and a fetishization of the military are key to fascists as a means for both their ultranationalism and for pursuing purity and rebirth. It just can’t be fascism without violence.
I am not a free man. The Prisoner (1967)
Similarly, the society in Minority Report seemed authoritarian, but its technologies were carefully depicted as non-violent. Some were psychologically cruel, but bodily, bloodless. So it’s an almost-counts, too.
Fascists are collectivist, meaning they believe that the ingroup of pure people are more important than any individual. If pressed, they’d admit a belief that government should have a centralized power with little accountability, as long as it’s doing its questionable things to the right (wrong) people.
Authoritarian governments are very popular in sci-fi, and very often going hand-in-hand with violence. In this survey, nearly every show was authoritarian. The only way a show would disqualify is if we just didn’t see governmental power in action against its citizens.
The story in 2001: A Space Odyssey, for example, happened within the context of a space exploration agency. If the individual liberties and pluralism were suffering back on Earth while the Discovery One was on its murderous, mind-expanding mission, we just don’t know about it.
Enter aHAL kills Frank, 2001: A Space Odyssey (1968) caption
Going back through one of my earliest posts on the blog, I was reminded of the weird authoritarian state that Korben Dallas lives in, evidenced by the police raid of his apartment block. The built-in warrant reader. The beacons and sirens. The yellow circleseverywhere for placing your hands while police do their policey business. Surely, I thought, this will be fascist.
I am a meat popsicle. The Fifth Element (1997)
But we never get the sense over the course of the movie that there is a political party that is super into being American, or fetishizing national symbols, or believing that their country/people is much much better than all the others and therefore not beholden to the same rules. If this was just, say, Walt-Whitman-type of crush on a country, it would be one thing. But when combined with militaristic violence and a charismatic leader using strong government power claiming to purify the nation, you get fascism.
The AI Samaritan in Person of Interest is wholly totalitarian, but we don’t get the sense that the AI is programmed to think America is better than other countries. It’s just focused on absolute control within.
The Upper City in Metropolis certainly enjoys their class privilege born of the oppression of the Lower City, but we don’t know at all how they feel about other nations.
Mussolini and Hitler held their supporters in a thrall with their public speeches. They sold their narrative. They made people believe they really could achieve some lost state of purity and purge society of its evils. They fomented violence. In turn, their supporters had no problem letting them run roughshod over constraints to their power. There’s a good question as to whether their societies would have turned to fascism if it weren’t for these charismatic, untethered leaders. Then we come to Starship Troopers, often cited as being so gung-go military and ultranationalist that it hurts. But nowhere in the film do we see all that jingoism coming from a political, charismatic leader. And the leader is key to the palingenetic narrative, next.
Rasczak’s Roughnecks get chomped, Starship Troopers (1997)
THX 1138 and Brazil feature states that oppress by bureaucracy. Citizens have no idea what the power structure is that causes their grief.
Gattaca, in contrast, oppresses by every parent’s drive to want the best for their children and the resulting high-pressure meritocracy, needing no leader.
Children of Men violently oppresses because of global hyperscarcity, rather than dictatorial fervor.
That’s a fancy word, isn’t it? It means relating to rebirth or re-creation. I felt certain that when I watched Captain America: The First Avenger, Red Skull would be an open-and-shut case for fascism. But not so. HYDRA is certainly violent, authoritarian, dictatorial, and ultranationalist in their beliefs. But the movie shows that the organization splintered off of the Nazis because Hitler wasn’t ambitious enough. They weren’t there to reclaim a past glory or return their tribe to its former purity or cull a scapegoat.
Red skull. Captain America: The First Avenger.
The palengenetic narrative is a key element to fascism because it is the mechanism by which the dictator gets the authoritarian power they want and convinces their supporters not just to hand it over, but to throw it over and ask what more can they give. They do so because of a fear of imminent collapse invoked by the dictator, and the promise of a return to abundance/purity by purging the scapegoat in their midst. Evidence won’t support these claims, and that’s why it matters that it comes from a dictator. His authority is because he said it.
I have kept a blog about sci-fi interfaces for six years as of this posting. When I began it felt like the world was chugging along fairly well, with occasional needs for pit stops and course corrections, and there was time and space for looking at minutiae.
*the list extends, daily, so future readers, forgive how small and quaint this list must seem.
…nowadays, writing posts to analyze imaginary machines can feel not just trivial, but like an irresponsible misuse of time.
I understand that life entails many things simultaneously, but we’re heading in the U.S.A. towards very important midterm elections, so for a while, I’m going to use this platform that I have to do my part and to combine these concerns, investigating fascism through examples in sci-fi interfaces. Yes, I’ll spill some phosphorus on interfaces along the way but in full disclosure, while there could be, there aren’t any. I’ll discuss why later. Right now I have to SMASH SOME BUGS!
Things that are good to do are often good for multiple reasons. This series of posts is no exception.
Part of my goals will be to sensitize readers to fascism via this lens: What is fascist versus what merely dresses up in its costumes.
Part will be to understand what it means for technology and interfaces to embody and enforce political ideas, because they can and do.
Part of my goals are, as always, to sharpen critical thinking and design skills in myself and the reader.
Part of my tactics are to assure you, now, that the show with the most fascism is not one you would expect. This is a good thing, because fascism merits vigilance and your ability to see through both its disguises and its bad faith (or as H.G. Frankfurt uses the term, its bullshit).
Part of my hope is to encourage you, too, to use whatever platform you have available to do you part to shine a light on and resist it.
It will entail discussing the ethics of design and of using platforms to the best of one’s abilities to do what’s right. I’m almost certainly in over my head. Hopefully I’ll get help from some smart people. I will try and keep a global perspective, but my main focus will be on the rise of Trumpism where I live in the U.S. of A.
If you are a reasonable person who has concerns and disagreements about this content, let’s discuss. If you’re one of the fascists, know that you are not welcome here.
Let’s start with a clarification…no…a discussion of terms. Today “fascist” can be thrown as an insult at actual fascists, at things that look very much like fascists, or mere bullies. So when I’m talking about it of course I need to clarify what I mean.
In its least formal use, when “fascist” is lobbed as an insult, I think it’s usually to describe a person using violence for political ends. Could be citizen, could be police. Could be an established political group or a fringe one. Insult fascism is what you’re likely to hear shouted at a contentious rally, and isn’t very clarifying. It’s missing nuance that distinguishes fascism from mere bullies, or any of the other *isms that can use violence. Capitalism, Communism, Despotism, Tyranny, etc. etc. So we need to look for some more precise meaning.
Academics who study such things would link Fascism’s it’s source, so let’s call this “source” Fascism.
The seminal document, “La dottrina del fascismo” is written in a florid and self-aggrandizing style, so not clarifying. Mussolini named his movement after the Etruscan fasces, a bundle of (weak) rods which are bound together to become strong, supporting an axe handle and blade. Metaphorically, the bundle of sticks are citizenry who bind themselves together and become a tool of violence and thereby authority for a leader. (Note that the metaphor literally asserts that its members are tools.) In this document, Mussolini does lay out a philosophy that can be used as a definition.
When we speak of source fascism, it’s a historical thing, defined by Mussolini and enacted by him and Hitler in WWII. It’s quite specific. In this academic sense, the current administration does not perfectly fit. Trump is personally an individualist. Trump probably doesn’t see violence as key to a health regimen. I doubt he is aware of “La dottrina del fascismo,” much less read it, much less base his twitter palsies on it. He’s not a source fascist. I don’t know that there are any of those anymore. Even if they’re straight up Nazis, they’re of the Neo- variety.
But outside of academic hairsplitting, the term has been genericized. Rather than referring to a specific historical ideology, people use it today more generally to describe a dark pattern that appears in modern politics. So let me talk about what it means more generically.
Trying to move from metaphor to a more formal definition is difficult. Since Mussolini, attempts to define it succinctly have been made by well-known intellectuals and it always comes off as slippery and a bit poetic. See the Wikipedia article for Definitions of Fascism and you’ll see a list of 22 different and earnest attempts by public figures including Umberto Eco, Leon Trotsky, Franklin Roosevelt, and George Orwell.
I’m far from being a political scholar, but it seems to me that the trouble is that fascism is neither derived through reason nor followed by reasonable people, and so it resists anything as reasoned as a observastional definition or consistent ideology. Fascism is a dark thing that rises from limbic, reactionary places and, once roused, squeezes into and out of whatever shapes it can, until reason and the long arc of history exorcises it. Its adherents think they’re mad about one thing but their arguments don’t stand up to examination or evidence. That doesn’t matter to them because they don’t care about arguments. They don’t care about facts. Fascism is a primeval reaction. Like a nation-sized, violent bout of the hangries. (That sounds like I’m being flippant, but I’m earnest about it as a metaphor. Crap. There we are back to metaphors again.)
Fortunately, the diligent editors at Wikipedia have brought all those different definitions into more focus at their Fascism portal. Combining that with the Definitions of Fascism and published interviews with scholars, I’ve compiled a description of five key aspects shared by most of those sources.
Violent: Force is seen as a virtue in and of itself
Over-the-top fetishism of military, pageantry, machismo
A desire for violent suppression of the enemies within
Terrorism against the scapegoat class (see Palingenetic below)
Parliamentarianism (anyone who insists on following protocol)
Dissenters of any stripe
A desire for war and imperialism against the enemies without
Authoritarian: It presumes a strong central government powerGovernment is a Strict Father (Because I said so, by Great Leader, is good enough.)A vague and shifting-as-needed executive power
Laws and enforcement should be harsh
Hostile to diversity, individualism, unconventionality
Pressure to conform to in-group norms in appearance, social role, gender role
Subsumed personal and political freedoms
Ultranationalist: It asserts that everything is subordinate to the exceptional state
Industry, commerce, business exists at the whim of Great Leader
A militarized citizenry: Our followers (true followers, anyway) are heroes who must be willing to die for this cause.
A belief in national exceptionalism: Our country is special, and better, than other countries around the world
Dictatorial: It is headed by a charismatic Great LeaderHis (so far always a dude) legitimacy is derived from appeals to emotion
Seen as having few limits to his power, if any
He peddles an epic rebirth narrative (see Palingenetic, below)
Palingenetic: It invokes a mythic return to past greatnessWe must purge the impurities in our society (or the world) to return ourselves to purity and abundance.A desire to abandon or replace the current political order for a new one
They name a specific class of people as a scapegoat and use lies and propaganda to dehumanize and attack them. (This often camps on racism.)
So when I say “strong” fascism or even just fascism, I mean a combination of those five things. So while Trump isn’t a source fascist, with this list in hand it’s easy to see that in modern usage of the term, Trump, his Trumpettes, and the GoP supporting him are all jack-booted fascists in this generalized sense.
I was surprised and delighted to get an email from Sebastian Sadowski asking if he could produce a visualization for the Untold AI analysis. Because of course. I’ll share that in a post tomorrow, but for now, let me introduce him via an email interview…
1. Hi there. For our readership, introduce yourself, what you do, how you got into it, and how it relates to sci-fi interfaces.
I’m an independent designer for interfaces and data visualizations. The last seven years, I’ve been designing interfaces for various companies and institutions, e.g. to access autonomous vehicles for the Volkswagen Group Future Center, home appliances by Bosch and Siemens or trade-related data for the UN’s International Trade Centre. In recent years, I’ve been dealing more and more with complex data and nowadays mostly work on projects to visualize and access big amounts of complex data. Interfaces in sci-fi movies have always been an inspiration for my projects.
2. Most of the readership doesn’t know how data visualization gets done. Can you walk us through how you get a project, what steps you take, and how you know you’ve done a good job? Who gives you direction, and who uses the output of your work? What makes for a great data visualizer?
Mostly, I get requests from clients who have a big dataset which neither can be opened in Excel nor visualized in it. They are looking for new or custom shapes to visualizing their data and need an interface to make it accessible. I start with exploring the dataset, search for some interesting insights or stories to tell. Next step is to search for inspirational projects which could fit to the dataset and topic —at this point I normally check scifiinterfaces.com if I have an interface or visualization in mind that I’ve seen in a movie. I bundle the ideas and start prototyping with some custom forms and real data, e.g. with D3.js. After that research period, I discuss the results with the client and decide which content we want to show, based on which chart type or which interactive elements we want to include.
The interface is basically a dashboard visualizing patterns exposing irrigation problems to soil variation or pest, bacterial and fungal infections. Besides precise satellite pictures, airborne cameras provide images to analyze healthy/distressed plants, chlorophyll- and nitrogen-levels and more.
The design has been inspired by the work of gmunk who is famous for his work on Tron or Oblivion.
4. What is your background? Are you a designer by training? If someone wanted to get into your line of work, what would they need to study? What skills would they need to develop?
5. What’s your favorite sci-fi interface that you didn’t work on?
The Iron Man interfaces are great, e.g. the holograms and interfaces in Iron Man 2 (see below). They have been an inspiration for a recent side-project where I designed some experimental data visualizations for augmented reality.
Iron Man 2 interface and holograms example - Vimeo
6. What’s your favorite sci-fi interface that you didn’t work on?
Currently, I experiment with augmented reality (AR) and think about how I could visualize data in AR. It’s amazing what we can already achieve with a consumer device such as the iPhone X. If you’re interested in that topic, make sure to read my top 5 learnings for data visualizations in AR.
As of this posting, the Untold AI analysis stands at 11 posts and around 17,000 words. (And there are as yet a few more to come. Probably.) That’s a lot to try and keep in your head. To help you see and reflect on the big picture, I present…a big picture.
This data visualization has five main parts. And while I tried to design them to be understandable from the graphic alone, it’s worth giving a little tour anyway.
On the left are two sci-fi columns connected by Sankey-ish lines. The first lists the sci-fi movies and TV shows in the survey. The first ten are those that adhere to the science. Otherwise, they are not in a particular order. The second column shows the list of takeaways. The takeaways are color-coded and ordered for their severity. The type size reflects how many times that takeaway appears in the survey. The topmost takeaways are those that connect to imperatives. The bottommost are those takeaways that do not. The lines inherit the takeaway color, which enables a close inspection of a show’s node to see whether its takeaways are largely positive or negative.
On the right are two manifestocolumns connected by Sankey-ish lines. The right column shows the manifestos included in the analysis. The left column lists the imperatives found in the manifestos. The manifestos are in alphabetical order. Their node sizes reflect the number of imperatives they contain. The imperatives are color-coded and clustered according to five supercategories, as shown just below the middle of the poster. The topmost imperatives are those that connect to takeaways. The bottommost are those that do not. The lines inherit the color of the imperative, which enables a close inspection of a manifesto’s node to see to which supercategory of imperatives it suggests. The lines connected to each manifesto are divided into two groups, the topmost being those that are connected and the bottommost those that are not. This enables an additional reading of how much a given manifesto’s suggestions are represented in the survey.
The area between the takeaways and imperatives contains connecting lines, showing the mapping between them. These lines fade from the color of the takeaway to the color of the imperative. This area also labels the three kinds of connections. The first are those connections between takeaways and imperatives. The second are those takeaways unconnected to imperatives, which are the “Pure Fiction” takeaways that aren’t of concern to the manifestos. The last are those imperatives unconnected to takeaways, the collection of 29 Untold AI imperatives that are the answer to the question posed at the top of the poster.
Just below the big Sankey columns are the five supercategories of Untold AI. Each has a title, a broad description, and a pie chart. The pie chart highlights the portion of imperatives in that supercategory that aren’t seen in the survey, and the caption for the pie chart posits a reason why sci-fi plays out the way it does against the AI science.
You’ve seen all of this in the posts, but seeing it all together like this encourages a different kind of reflection about it.
Note that it is possible but quite hard to trace the threads leading from, say, a movie to its takeaways to its imperatives to its manifesto, unless you are looking at a very high resolution version of it. One solution to that would be to make the visualization interactive, such that rolling over one node in the diagram would fade away all non-connected nodes and graphs in the visualization, and data brush any related bits below.
A second solution is to print the thing out very large so you can trace these threads with your finger. I’m a big enough nerd that I enjoy poring over this thing in print, so for those who are like me, I’ve made it available via redbubble. I’d recommend the 22×33 if you have good eyesight and can handle small print, or the 31×46 max size otherwise.
Maybe if I find funds or somehow more time and programming expertise I can make that interactive version possible myself.
Some new bits
Sharp-eyed readers may note that there are some new nodes in there from the prior posts! These come from late-breaking entries, late-breaking realizations, and my finally including the manifesto I was party to.
I finally worked the Juvet Agenda in as a manifesto. (Repeating disclosure: I was one of its authors.) It was hard work, but I’m glad I did it, because it turns out it’s the most-connected manifesto of the lot. (Go, team!)
The Juvet Agenda also made me realize that I needed new, related nodes for both takeaways and imperatives: AI will enable or require new models of governance. (It had a fair number of movies, too.) See the detailed graph for the movies and how everything connects.
In this post we look at the imperatives that don’t have matches in AI. Everything is built on a live analysis document, such that new shows and new manifestos can be added later. At the time of publishing, there are 27 of these Untold AI imperatives that sit alongside the 22 imperatives seen in the survey.
What stories about AI aren’t we telling ourselves?
To make these more digestible, I’ve synthesized the imperatives into five groups.
We should build the right AI
We should build the AI right
We must manage the risks involved
We must monitor AIs
We must encourage an accurate cultural narrative
For each group…
I summarize it (as I interpreted things across the manifestos).
I list the imperatives that were seen in the survey and then those absent from the survey
I take a stab at why it might not have gotten any play in screen sci-fi and hopefully some ideas about ways that can be overcome.
Since I suspect this will be of practical interest to writers interested in AI, I’ve provided story ideas using those imperatives.
Where to learn more about the topic.
Let’s unfold Untold AI.
1. We should build the right AI (the thing itself)
Narrow AI must be made ethically, transparently, and equitably, or it stands to be a tool used by evil forces to take advantage of global systems and make things worse. As we work towards General AI, we must ensure that it is verified, valid, secure, and controllable. We must also be certain that its incentives are aligned with human welfare before we allow it to evolve into superintelligence and therefore, out of our control. To hedge our bets, we should seed ASIs that balance each other.
Related imperatives seen in the survey
We must take care to only create beneficial intelligence
We must ensure human welfare
AGI’s goals must be aligned with ours
AI must be free from bias
AI must be verified: Make sure it does what we want it to do
AI must be valid: Make sure it does not do what we don’t want it to do
AI must be controllable: That we can we correct or unplug an AI if needed without retaliation
We should augment, not replace humans
We should design AI to be part of human teams
AI should help humanity solve problems humanity cannot alone
We must develop inductive goals and models, so the AI could look at a few related facts and infer causes, rather than only following established top-down rules to conclusions.
Related imperatives absent from the survey
AI must be secure. It must be inaccessible to malefactors.
AI must provide clear confidences in its decisions. Sure, it’s recommending you return to school to get a doctorate, but it’s important to know if it’s only, like, 16% certain.
AI reasoning must have an explainable/understandable rationale, especially for judicial cases and system failures.
AI must be accountable. Anyone subject to an AI decision must have the right to object and request human review.
We should enable a human-like learning capability in AI.
We must research and build ASIs that balance each other, to avoid an intelligence monopoly.
The AI must be reliable. (All the AI we see is “reliable,” so we don’t see the negatives of unreliable AI.)a
Why don’t these appear in sci-fi AI?
At a high level of abstraction, it appears in sci-if all the time. Any time you see an AI on screen who is helpful to the protagonists, you have encountered an AI that is in one sense good. BB-8 for instance. Good AI. But the reason it’s good is rarely offered. It’s just the way they are. They’re just programmed that way. (There is one scene in Phantom Menace where Amidala offers a ceremonial thanks to R2-D2, so perhaps there are also reward loops.) But how we get there is the interesting bit, and not seen in the survey.
And, at the more detailed level—the level apparent in the imperatives—we don’t see the kinds of things we currently believe will make for good AI: like inductive goals and models. Or an AI offering judicial ruling, and having the accused exonerated by a human court. So when it comes to the details, sci-fi doesn’t illustrate the real reasons a good AI would be good.
Additionally, when AI is the villain of the story (I, Robot, Demon Seed, The Matrices, etc.) it is about having the wrong AI, but it’s often wrong for no reason or a silly reason. It’s inherently evil, say, or displaying human motivations like revenge. Now it’s hard to write an interesting story illustrating the right AI that just works well, but if it’s in the background and has some interesting worldbuilding consequences, that could work as well.
But what if…?
Sherlock Holmes was an inductive AI, and Watson was the comparatively stupid human babysitting it. Twist: Watson discovers that Holmes created AI Moriarty for job security.
A jurist in Human Recourse [sic] discovers that the AI judge from whom she inherits cases has been replaced, because the original AI judge was secretly convicted of a mind crime…against her.
A hacker falls through a literal hole in an ASI’s server, and has a set of Alice-in-Wonderland psychedelic encounters with characters inspired not by logical fallacies, but by AI principles.
Inspired with your own story idea? Tweet it with the hashtag #TheRightAI and tag @scifiinterfaces.
2. We should build the AI right (processes and methods)
We must take care that we are able to go about the building of AI cooperatively, ethically, and effectively. The right people should be in the room throughout to ensure diverse perspectives and equitable results. If we use the wrong people or the wrong tools, it affects our ability to build the “right AI.” Or more to the point, it will result in an AI that is wrong on some critical point.
Related imperatives seen in the survey
We should adopt dual-use patterns from other mature domains
We must study the psychology of AI/uncanny valley
Related imperatives absent from the survey
We must fund AI research
We need effective design tools for new AIs
We must foster research cooperation, discussion
We should develop golden-mean world-model precision
We should encourage innovation (not stifle)
We must develop broad machine ethics dialogue
We should expand the range of stakeholders & domain experts
Why don’t these appear in sci-fi AI?
Building stuff is not very cinemagenic. It takes a long time. It’s repetitive. There are a lot of stops and starts and restarts. It often doesn’t look “right” until just before the end. Design and development, if it ever appears, is relegated to a montage sequence. The closest thing we get in the survey is Person of Interest, and there, it’s only shown in flashback sequences if those sequences have some relevance to the more action-oriented present-time plot. Perhaps this can be shown in the negative, where crappy AI results from doing the opposite of these practices. Or perhaps it really needs a long-form format like television coupled with the right frame story.
But what if…?
An underdog team of ragtag students take a surprising route to creating their competition AI and win against their arrogant longtime rivals.
A young man must adopt a “baby” AI at his bar mitzvah, and raise it to be a virtuous companion for his adulthood. In truth, he is raising himself.
An aspiring artist steals the identity of an AI from his quality assurance job at Three Laws Testing Labs to get a shot at national acclaim.
Pygmalion & Galatea, but not sculpture. (Admittedly this is close to Her.)
Inspired with your own story idea? Tweet it with the hashtag #TheAIRight and tag @scifiinterfaces.
We pursue AI because it carries so much promise to solve problems at a scale humans have never been able to manage themselves. But AIs carry with them risks that can scale as the thing becomes more powerful. We need ways to clearly understand, test, and articulate those risks so we can be proactive about avoiding them.