Recently read this essay, which differentiated between two terms that I had not heard before: Safety Technologies vs. Risk Technologies. I’m going to dig into these terms and their relevance to protocols, sharing my thoughts here along the way.
To start I’ll be reading a paper referenced in the essay, called “Differential technology development: An innovation governance consideration for navigating technology risks”
Initial thoughts before starting:
In section 5.3.1, the authors state “If progress on a specific technology is not led by a single or a handful of coordinating actors, delaying or making technologies safer may be difficult. Coordination through international organisations may have the greatest leverage to delay or shape technologies with global power implications.”
As the scale of risks increase, the inherent tension between safety & concentration on the one hand and autonomy & decentralization on the other becomes more obvious. We’re seeing this with AI today, where efforts to slow development (for reasons of existential risk) are being done through large, powerful institutions. Meanwhile, risks to individuals, like LLMs producing misinformation or inducing antisocial behavioral patterns, are being mitigated in a more bottom-up fashion – anecdotally speaking
Innovation is just part of the puzzle – distributing defensive technology is also important. One advantage of indiscriminate technological acceleration is that everyone is more evenly exposed to consequences. A country with advanced defensive tech (even something as benign as air conditioning) might pursue a more aggressive development agenda than it would if it were more exposed to the consequences, thus deepening inequalities in the global risk environment
Safety protocols should seek to raise floors before ceilings
The authors make a contentious point that I’m sympathetic to – that we can predict the impacts of technological progress. Not only that, but we can change our behavior in advance. Ulrich Beck argued that most of human history was spent dealing with risk linearly, but now we deal with it in unexpected and forward-looking ways. We think about the future much more than we used to, and we are preoccupied with getting there (getting then).
The problem with all of this is the Imperious Immediacy of Interest. We still value the near future over the far future. Longtermism might be shifting the window, but at the level of individual behavior I think we’re still solidly impatient
Chess engines are supposedly good not because of their attacking ability, but because they don’t make mistakes. I think the steel man argument for d/acc is analogous to that strategy… take care of the downside, and the upside will take care of itself.
Paul Virilio’s idea of the Integral Accident * feels different in the Douglas Adams [tech] age:
When you invent the ship, you also invent the shipwreck; when you invent the plane you also invent the plane crash; and when you invent electricity, you invent electrocution… Every technology carries its own negativity, which is invented at the same time as technical progress. – Paul Virillio
Let’s hope that enough accidents happen in the early stages of development when the consequences still fall on the funny / absurd side, so that we learn by the time the stakes get higher.
* I would link directly to Wikipedia, but they recently removed a lot of Paul Virilio’s page
Good framing from Matt Webb as per usual! To me, that tracks with how we characterize integral accidents of LLMs as “hallucinating” or “lying” or other personifying verbs.
Those are also integral accidents for humans, like someone involved in a coverup, or a conspiracy theorist, or a guy who did much Ayahuasca