NVIDIA launches ‘Guardrails’ to help generative AI from giving the wrong information

NVIDIA launches ‘Guardrails’ to help generative AI from giving the wrong information

NVIDIA has announced the launch of NeMo Guardrails, an open-source software to help developers guide generative AI applications to create responses that are accurate, appropriate, on-topic, and secure.

The launch comes as more businesses turn towards deploying large language models (LLMs) to answer customer queries, create text, and write software.

NeMo Guardrails is designed to help users keep this new class of AI-powered applications safe and is a layer of software that sits between the user and the LLM, or other AI tools.

The AI developer can use it to set up three boundaries for AI models - topical, safety, and security guardrails.

Topical guardrails

Topical guardrails are designed to ensure that conversations stay focused on a particular topic and prevent them from veering off into undesired areas.

They serve as a mechanism to detect when a person or a bot engages in conversations that fall outside of the topical range. These topical guardrails can handle the situation and steer the conversations back to the intended topics. For example, if a customer service bot is intended to answer questions about products, it should recognize that a question is outside of the scope and answer accordingly.

Safety guardrails

Safety guardrails ensure that interactions with an LLM do not result in misinformation, toxic responses, or inappropriate content. LLMs are known to make up plausible-sounding answers. Safety guardrails can help detect and enforce policies to deliver appropriate responses.

Other important aspects of safety guardrails are ensuring that the model’s responses are factual and supported by credible sources, preventing humans from hacking the AI systems to provide inappropriate answers and mitigating biases.

Security guardrails

Security guardrails prevent an LLM from executing malicious code or calls to an external application in a way that poses security risks.

LLM applications are an attractive attack surface when they are allowed to access external systems, and they pose significant cybersecurity risks. Security guardrails help provide a robust security model and mitigate against LLM-based attacks as they are discovered.

According to NVIDIA, NeMo Guardrails will work with all LLMs, including ChatGPT. With much of the NeMo framework already available as open-source code on GitHub, enterprises also can get it as a complete and supported package, part of the NVIDIA AI Enterprise software platform.