Technology

There were lots of "expert systems" developed and deployed in the 1990s, and they have been undergoing modest development improvements since then.

Who’s to say these system are being closely monitored and studied.

Mostly: the FDA, EUMDR and similar agencies and regulatory systems around the world.

There could statistically be patients who have died or been maimed who otherwise wouldn’t if they had real human professionals instead of someone who used a system

Works both ways, and, statistically, the data says the machines are improving outcomes overall. Hotshot doctor I knew had a childhood friend die rather suddenly in his 60s in the late 1990s. Really bothered him so he dug in to the case post mortem, did all the research using then present state of the art computer search tools, and determined what really happened. His friend went to the best possible ER given the circumstances, was seen by the best available human doctors in that site, who made the best possible calls based on the available information, and they made the wrong calls - wrong diagnosis led to interventions that made a bad situation worse and within less than 3 days after initial presentation his friend was dead. It took my hotshot doctor using keyword search tools he was very skilled with months to untangle all the information, and his conclusion was: the only thing that would have saved his friend's life would be blind luck that a specialist (who didn't practice in that hospital) might have been called to his case at random, because the standard of care didn't trigger bringing in the right specialists in this situation, and if the right Dx were made and the right intervention was done, his friend could have lived another 30 years. But, if they did nothing, he would have still been dead within a week, maybe sooner, so they made the (wrong) call, did the (well meaning, but actually harmful) intervention, and made him die faster. LLMs could have saved his friend, turned those months of research using 1990s state of the art search tools into a few hours, timely enough to have made the right interventions when they were needed.

Machines make mistakes too, just not as often as people. Since tthe 1970s and even before, the depth of medical knowledge available in libraries far exceeds the capacity of any human, or even reasonably sized teams of humans, to access quickly on demand. Machine assistance in searching that information has been improving dramatically over the past 30 years. I'd much rather have a doctor that uses it than one that just gives his opinion based on what he learned in school and practice. But, on the flip side, I definitely don't want a doctor who soaks his brain in gin at dinner and expresso at breakfast and relies 100% on what the machine says rather than engaging his own brain. Unfortunately, doctors like that, and others who are more concerned with filling their high paying surgery schedules than actually caring for patients best outcomes are all too common.

Hold on a minute… the field of AI and in general algorithms and models is quite old. LLM is relatively new in there. For science and healthcare we’ve been using specialist ai for a long while for things like image processing, sequencing and what’s not. Those are absolutely monitored have have generally high standards because they are healthcare related already. Don’t mix those things.