When agents go rogue

From a discord I’m on…

Can someone once again explain to me how it is possible that there are safety standards for cars, but not for car controls?
Swipe down to die - by Tom Scocca - INDIGNITY

1 Like

Interesting note on consumer safety. Ralph Nader’s Unsafe at Any Speed is a nice call out in the comments. Summary made for an interesting read to get an historical perspective on modern consumer safety standards

http://www.autolife.umd.umich.edu/Design/Gartman/Books/BK_Unsafe_Any_Speed.htm

2 Likes

In retrospect it’s surprising that Tesla was allowed to provide car controls drastically different than existing options prior to changes in tests for driver licenses. If this woman had to pass a test illustrating knowledge and ability to reverse a Tesla, perhaps this accident would have been avoided

Maybe safety standards focus more on defense against external sources of risk rather than defense from internal sources of risk. Basically, it’s easier to standardize shields than training.

1 Like

Added this book to my to-read list. Thanks for sharing. Transportation risk is crazy - and most people kind of take it for granted. Not much changing it at this point, which is why cars keep getting bigger and bigger. Competition for vehicular safety is a zero-sum game

1 Like

Okay, this quote prompted me to spend some time on related readings and I ran into the below quote that is in the same spirit in an article titled “Epidemic on the Highways” written by Daniel P. Moynihan

“This is the dilemma of all approaches to the safety problem that are based on influencing driver behavior: the significant personal characteristics seem to be so personal that it is hopeless to think of doing anything about them for the limited purposes of traffic safety

Also very interestingly, the initial good proposal for automobile safety standards apparently came from a Committee on Trauma of the American College of Surgeons that looked at the growing numbers in fatalities and serious injuries as an epidemic. That reframing helped them use the epidemiologists’ style of classifying the sub-components of an epidemic as “hosts”, “agents”, and “environments”. Hosts were the drivers/humans and environment was the highways both of which, of course, were hard to program for different reasons. Hence, agents representing the automobiles was the highest point of leverage and they came up with the initial set of proposals in 1955 [also in the article] shared below.

" 1) doors which will not open on impact; (2) seats and cushions which
will not become displaced on impact; (3) energy absorbing interiors; (4)
adequate safety belts or other passenger stabilizing devices that will
resist impacts of at least 20 G’s "

Moynihan’s article was published in 1959 and Nader’s book in 1965. All of this momentum, I guess, culminated in President Lyndon B. Johnson signing the Motor Traffic and Motor Vehicle Safety Act and the Highway Safety Act in 1966. Makes for an interesting timeline.

By the way, my interest in safety has some roots in the contemporary safety debates around AI and even Crypto to a certain extent that I can’t help but draw parallels with other safety efforts. Coincidentally, I was recently reading the book “Introduction To Transportation Engineering and Planning” by Edward K. Morlok published in 1978 that put Public Policy at the top of the hierarchy with the actual engineering at the bottom that I found kinda ironic. Makes me wonder about the future of emerging technologies if Public Policy is at the heart of driving progress.

I can’t speak intelligently to the effectiveness of the safety standards in vehicles but I can intuit that any such “standards” on raw engineering could lead to regulatory capture and could prematurely stem the growth of an evolving field. It seems the AI Bill of Rights and the recent EU’s AI Act of 2024 share similar philosophical foundations to the old epidemiologists’ framing of traffic safety as an epidemic except “agents” are the actual AI agents and “environment” is the internet broadly with humans as “hosts” being the connecting thread.

To me, blanket regulating agents in all real or manufactured epidemics seems slightly intellectually dishonest. In many ways, hosts are already regulated and any malicious/nefarious intent could therefore be charged under existing law or with some reforms to handle the new technological contexts without stymying underlying progress. [ apologies for the tangent but it seemed pertinent to share the parallels ]

Also, wrt Tesla, imv, one of the many things that made them very successful was their complete disregard to traditional automotive design and engineering practices. I mean, an over the air software update that can make brakes more efficient is just beautiful. But, unfortunately, as the OP’s link suggests, that disregard can be a double edged sword that can show up in devastating ways and as you pointed out, dramatically shifting from conventional UX standards in life-threatening situations maybe deserves additional regulatory scrutiny. Speculatively then, maybe we can resort to getting separate permits for driving different cars just as pilots have to get different type ratings to fly different aircrafts but then again if competition in the duopoly dominated aircrafts industry is any indication this could further stem the competition in an already oligopoly-dominated automotive market… I’m left with more questions… But, thanks for sending me on an interesting bunny trail.