Healthcare leaders to create standards for AI

Trustworthy & Responsible AI Network (TRAIN), a consortium created to explore and set standards for the safe application of artificial intelligence (AI) in health care.

Healthcare leaders launch Trustworthy & Responsible AI Network



In other places:
The Artificial Intelligence Code of Conduct (AICC)

Thank you for posting this.

Although the acronym is truly admirable, I fear this is a mind destroying proposal. Recommend avoidance until outcome measures of engagement in standards discussion with representatives of AdventHealth, Advocate Health, Boston Children’s Hospital, Cleveland Clinic, Duke Health, Johns Hopkins Medicine, Mass General Brigham, MedStar Health, Mercy, Mount Sinai Health System, Northwestern Medicine, Providence, Sharp HealthCare, University of Texas Southwestern Medical Center, University of Wisconsin School of Medicine and Public Health, Vanderbilt University Medical Center, and Microsoft are published in a peer reviewed journal. :slightly_smiling_face:

Could you elaborate a bit?

While the multi-decade standards effort in healthcare IT has been the foundation for data exchange APIs at the billing, patient health record, and (to a lesser extent) population outcomes level, there is a strong GIGO factor, and data quality issues become more, not less, important when using health records as training data. The neglect of data quality is not due to lack of attention, but due to the nature of empirical observation. There is no such thing as a correct ontology for the world. There are only select phenomenon that become interesting through social elaboration. With the new possibilities for empirical research through sensing technologies, real world observations about health are running much, much faster than standards can catch up.

The purpose of these standards efforts is to capture value within the parameters of the health care industry — ok, that’s expected — but if you care about discovery at the limits of this system the TRAIN discussions may seem pretty sluggish once you get in there. The classic paper by Carl Shapiro and Hal Varian on The Art of Standards Wars is great to read, not just because it has colorful detail about forgotten but interesting battles but because it makes clear, from today’s perspective, how closely linked standards are to efforts to box out competitors. The fractal nature of health data calls for a more decentralized approach, which will also save you from extremely dull panel discussions that you have to attend for political reasons.

1 Like

You might find this book relevant: Regulatory Hacking

Can also please simplify further? I think I’m just a little green and I don’t quite get the following:

  1. I get the GIGO and data quality bit but don’t understand how the lack of a correct ontology matters. Machine learning tech is able to reliably analyze, and now produce, data that is ontologically sensible. I guess this must be possible because there is some sense in the underlying ontology and medicine wouldn’t be exempt from this?

  2. How do new sensing technologies (do you mean things like MRI scans or tech like sequencing?) relate to the ontology? Given the need for FDA approvals, clinical trials and validation before a tech can be used in healthcare one would assume there is plenty of time for standards to catch up. In the case of AI in healthcare, FDA moved relatively quickly. Almost as soon as AI was actually affecting radiology, FDA took action.

  3. Regulatory capture is a well known business angle, no doubt at work here too. The point which I found interesting, especially for this forum, was two fold:
    a) That standards (and protocols) were being used to bring relevant groups together. There is something very alive about the creation of standards.
    b) The consortium is formed for safety and quality, which are classic regulatory goals. Regulation comes after discovery and productization of the new tech. How would the limits of TRAIN affect discovery at all?

  4. Also please help me understand how healthcare data is fractal and how does decentralization address this?

As a meta-question, the whole thing could be viewed as disappointing and regular old regulatory capture perhaps, but how is this mind destroying? If anything, I would argue that if a consortium is being formed, and an individual cares about the outcomes then they should get involved. After all, their standards could become law, and if we wait for a peer reviewed outcomes study then we are 20 years too late.

These are all very good points, just to acknowledge the most important:

  1. The production of sensible outputs by AI is not dependent upon the kinds of standards these consortia are typically concerned with, which have their roots in expert systems.

  2. The FDA document is worth a closer read than I gave it, but on first glance I wondered how they were going to track algorithmic evolution and versioning effects using rule-based documentation, if understood the goal correctly.

  3. I loved Lang’s essay too. It made me ask: what is the most interesting current border between chaos and order?

  4. Fractal wa a poorly chosen word. Let me give this some more thought.

Meta-question: the mind destroying effects of standards negotiations was no doubt an exaggeration. I think Varian’s advice to aspiring engineers (and I may have the attribution wrong) was something like: “join a standards body and then quit a standards body.”

I heard Amazon is providing “ambient listening” AI solutions for hospitals, to automate patient care documentation. Easy to see how standards for safety and usability would play a role there. Medical charts seem like a lynchpin in hospital operations.

This is a diagram by safety researcher Rene Amalberti, maybe a useful frame here. Basically a theory of diminishing returns for safety, driven by media attention. Paradoxically: the safer a system → the less often accidents occur → the better news they make → the more attention they receive.

I think it rings true in a lot of fields. Air travel, for example. From The Demand for Safety and Its Paradoxes

1 Like

I’m paying a legit price for my sarcasm. I’ll try to explain more seriously. I like the chart, but I think “AI for health” activities still belong in the lower left hand corner (ignoring the century-long time scale).

Here’s an example from deep in the weeds of health research:

So there is standard measure of functional health, the Six Minute Walk Test. It’s a pretty useful standard, because it is not very difficult to conduct and has been proven to correlate with severity of diseases affecting physical fitness, such as COPD. But, on the other hand, it is a relatively sparse measure that doesn’t take into account a lot of context. For instance, a person with an ankle injury is not going to walk as far as person with the same COPD severity but no ankle injury. A person with short legs will also not walk as far.

OK, so let’s supplement the standard with a good list of ancillary factors affecting walking distance.

We can go pretty far down this path, but let’s stop here and just say: “We’ve got a rough measure that works okay for a lot of purposes and we’ll cope with any doubts when they arise.” That’s the current state of affairs, and any important measurement standard is going to have its stewards and critics, and this has been going on for a long time.

But there are some new things happening

  1. An explosion of empirical data from new sensing tools. (Electromechanical, optical, biochemical/genomic)
  2. AI

With the ability to take many more measurements and find patterns that can bear on many more questions, we might want to do something like:

  1. Continuously measure physical movement.
  2. Develop classification approaches that bear on specific questions, even individual questions.
  3. Organize real world trials, including n-of-1 and small group trials.

There’s so much potential here, that watching the standards conflicts play out at the level of “who can come up with the most valuable trademarked digital alternative to the 6 minute walk test” is kind of irritating. This is winning the old game. In daylight, when I have a grip on myself, I can’t deny the value of the old game. Everything good we have in medicine (along the bad things, of course) we’ve gotten from the old game. But I don’t think competing to win here is how we get the most out of our technologies.

To just get all the way around the circle: I admit that if we’re going to use AI to predict COPD severity, and we’re going to do it by training the model to accurately derive a six minute walk score from continuous activity data, then we probably need a safety board to make sure it’s done right and version updates don’t break it.

I should take it seriously and stop rolling my eyes, but it’s hard.