This Tool Tests AI’s Resilience to ‘Tainted’ Data
The National Institute of Standards and Technology (NIST) is reissue a tool Vulnerability Check artificial intelligence (AI) Models are being “poisoned” by toxic data.
The move comes nine months after President Biden Executive Order on the safe, secure, and trustworthy development of AI, and is a direct response to that order’s requirement that NIST support model testing. NIST also recently released a program to help Americans use AI without becoming victims of synthetic or AI-generated content and promote the development of AI for the benefit of society.
The tool is called DiopterOriginally released two years ago and aimed at helping small and medium-sized businesses and government agencies, the tool allows someone to identify what types of attacks would cause their AI model to perform poorly and quantify the performance degradation to see the conditions that cause the model to fail.
Also: Beware AI ‘model collapse’: How training on synthetic data pollutes the next generation
Why does this matter?
It’s important that organizations take steps to ensure AI programs are safe. NIST is actively encouraging federal agencies to use AI in a variety of systems. AI models are trained on existing data, and if someone intentionally feeds malicious data — such as data that causes the AI to ignore stop signs or speed limits — the consequences could be dire, NIST points out.
Despite all the transformative benefits of AI, NIST Director Laurie E. Locascio said the technology poses risks that are much greater than those associated with other types of software. “These guidance documents and testing platforms will inform software creators of these unique risks and help them develop ways to mitigate them while still supporting innovation,” she noted in the release.
Also: Security Guide Provides Essential First Layer of Data Protection in the AI Gold Rush
NIST says Dioptra can test multiple combinations of attacks, defenses, and modeling architectures to better understand which attacks pose the greatest threat and which solutions might be best.
The tool doesn’t promise to eliminate all risk, but it does claim to help mitigate risk while still supporting innovation. available for download free of charge.
