(Bloomberg) -- RagaAI has secured funding to develop a tool that aims to diagnose and fix flaws with artificial intelligence systems, responding to an increasing emphasis on safety and reliability during the AI boom.

The startup founded by Gaurav Agarwal, an alumnus of chipmakers Nvidia Corp. and Texas Instruments Inc., has come out of stealth-mode to close a $4.7 million seed round led by pi Ventures. It was joined by firms including Anorak Ventures and TenOneTen Ventures. The Silicon Valley startup uses foundation models to detect and fix problems in AI, such as hallucinations — a well-publicized phenomenon — or failure to factor in critical real-time information.

The 44-year-old founder recounted what he called a near-death experience while test-driving a semi-autonomous vehicle on a California highway. The AI was unable to detect road debris so he had to brake manually to avoid an accident, he said.

“I’ve seen AI failures while at Ola and Nvidia,” Agarwal, also the startup’s CEO, said in a telephone interview. “There’s an urgent need to avoid these types of scenarios, particularly in high-stakes AI use cases such as cancer detection, aircraft maintenance and AI-powered recruitment tools.” 

Read More: Australia Weighs Mandatory Restrictions on High-Risk AI Use

Agarwal, who’s based in Fremont, Calif., created RagaAI ahead of the seminal launch of ChatGPT by OpenAI, which catalyzed the global development of large language models. RagaAI is among a new crop of startups trying to fulfill a need that’s arisen during that frenzy of activity.  

AI could potentially contribute $15.7 trillion to the global economy by 2030, according to a PwC study. But much of the current activity is focused on building models and applications, rather than diagnosing errors or testing them. 

RagaAI’s platform — which Agarwal said his team developed in-house — offers more than 300 tests to help enterprises triage issues and trace their root causes. It can spot things such as poor data labeling and bias, he said. It helps detect hallucinations — false and misleading information presented as fact — and also deliberate attacks or attempts intended to get a model to make mistakes, Agarwal added.

His startup will deploy the capital into research, expanding its team of about 40 engineers mainly in Bangalore. It already works with several large clients in ecommerce, aeronautics and medical imaging, though Agarwal wouldn’t disclose names. The platform has been able to help clients reduce failures by 90%, he said without into details.

Read More: Humans Still Cheaper Than AI in Vast Majority of Jobs, MIT Finds

©2024 Bloomberg L.P.