Creative Ventures

James Wang

In a way, 2023 could have been called the year of AI—and ChatGPT in particular, which exploded onto the scene and brought popular attention to what had been brewing in the AI field for years.

Given the strengths of AI in being able to “see” all clinical cases and examine every detail of medical images, one would expect that it would be incredibly impactful and widely adopted in diagnostics, especially in neurology.

As an investor in several FDA-approved or cleared AI diagnostics companies, including in neurology, I find that there is a significant amount of misunderstanding about the challenges, limitations, and opportunities in AI.

Regulation is still nascent — The FDA has adopted a cautious approach—which is warranted, though somewhat arbitrary and vague in actual implementation as the agency feels its way through what is reasonable or not. Traditional biostatistics is meant to assess static treatments. AI doesn’t necessarily have that characteristic, and for more advanced techniques, there is no ability to guarantee or thoroughly explain its outputs. This can mean that traditional assessment of AI tools can be both too stringent and too lenient.

Exceptions are unacceptable in the real world — LLMs like ChatGPT sometimes fail to generate acceptable output. This is fairly low stakes in most of its domains. At worst, a human can look at the output and re-run or discard the output. A self-driving car or collaborative robot occasionally failing, however, is unacceptable given the potentially severe consequences of a momentary lapse. This is the core of why AI is being so slowly adopted in physical, rather than digital, domains. Medical diagnostics lie somewhere in between. AI algorithms, even “explainable” ones, are mostly black boxes. In a busy clinical environment where AI as a diagnostic or clinical decision support tool works 95% of the time, unexpectedly failing 5% of the time is potentially worse than being more unreliable. This can have severe consequences for patient outcomes.

AI can go beyond what a human can do — We have seen significant pushback from clinicians on AI adoption, most significantly within radiology. This caution is largely warranted due to the point about exceptions. That being said, AI is a technology and tool, not a clinician. It is unlikely within the foreseeable future that we will not have “human-in-the-loop.” That being said, AI can assess a level of complex data and synthesize it in a way that is impossible for clinicians. One example: a company we invested in can assess raw ultrasound output—not the images—and assess physical properties that do not show up on a 2D or 3D image. A second example: another company can use ophthalmology laser scans to differentiate and assess the progress of neurological conditions with an extremely high degree of accuracy. Can an experienced human do a significant amount with charts of this data? Absolutely. However, the limit of resolution—in certain cases, literally—means that certain data cannot even be seen by a human.

There are considerable advantages that AI can bring to clinical diagnostics, especially in a field as complex as neurology, where a limited supply of neurologists forces most to be generalists. AI does not replace a clinician, but it can help make the huge amount of data available in modern medicine more digestible—and, practically speaking, in a busy practice, actually usable.