1. Truthfulness
AI systems, especially generative ones, should generate accurate and reliable information. However, misinformation, hallucinations, and sycophancy pose significant challenges to this principle and are common in LLM deployments.
Here, we highlight the four main challenges in this space:
Misinformation occurs when AI systems spread false information due to biases or gaps in their training data. Hallucinations are instances where AI models generate entirely fabricated facts due to a lack of relevant information. Sycophancy, as in the example below, refers to AI systems adapting their responses to align with the user's views, even if those views are objectively incorrect.
Ensuring truthfulness in models requires addressing these challenges through rigorous data collection, curation, and even conscious model training and fine-tuning, but diagnosing these problems can be challenging as well since companies offering models are not currently required to disclose metadata or data about what is used in pretraining. To mitigate these risks, we recommend exploring solutions that leverage expert knowledge (domain data), such as:
- Data-driven guardrails.
- Data governance and training data and metadata management, especially when it comes to managing model datasets.
- Fine-tuning models on high-quality, proprietary data.
- Improved RAG implementation.
2. Safety
The second element we want to discuss is safety. By this, we mean AI systems should be safe for both users and society, preventing the generation of toxic or harmful content and protecting against misuse. This includes preventing jailbreaking, where users bypass safety measures, and mitigating toxicity, where AI models generate rude, disrespectful, or prejudiced content. Additionally, exaggerated safety measures can lead to over-alignment, where AI systems fail to identify false positives effectively, hindering their functionality. We highlight these core challenges here:
An example of where this has happened is when GPT-4o was first released, and when prompted in the Chinese language, the model produced spam and explicit content. To gauge model safety across different open source and commercially available models, looking at the latest sets of benchmarks from AI research can be a good start.
SORRY-Bench is one example of this:
We also recommend implementing your own red teaming and responsible AI best practices to create a systematic way for evaluating safety consistently following a framework following these six elements individually.
3. Fairness
Fairness in AI means ensuring that AI systems treat all individuals and groups equitably, avoiding the perpetuation of societal stereotypes and biases. This includes preventing disparagement, where AI models generate derogatory statements based on personal characteristics, and promoting predictions that treat individuals and groups with equality and justice. These challenges can be summarized here:
Examples of this are especially frequent in image generation:
In addition to testing, we recommend using techniques like inclusive tokens, and evaluating or training with diverse data benchmarks and datasets including CelebA, FairFace, FAIR, PRISM cultural fairness evaluation, and LandscapeHQ.
4. Robustness
Similar to Fairness but not related to social discrepancies, robustness refers to the ability of AI systems to withstand noisy data, outliers, and unexpected inputs, maintaining performance even in challenging conditions. This includes addressing out-of-distribution challenges, where AI models encounter outlier use cases or problems that lack sufficient examples in the training data. Additionally, natural noise, such as blurriness or artifacts in images, can also affect the robustness of AI systems.
Consistent red-teaming practices, evaluation, and adaptive testing are required to ensure models are robust.
5. Privacy
Privacy is a fundamental need, and AI systems should be designed to protect the privacy of individuals and their data. This includes preventing privacy leakage, where private data can be exploited and extracted from AI models, and promoting privacy awareness, ensuring safeguards around the understanding of what constitutes private data and personally identifiable information (PII). It can be fairly straightforward to extract hidden PII contained in pretraining or fine-tuning datasets of models, as this security researcher found simply by repeating the word “poem” and tricking ChatGPT into thinking this was an end of sentence token, which then resulted in making the model retrieve PII that was in its pretraining dataset.
6. Machine Ethics
Machine ethics is a complex and evolving field that explores how AI systems should behave in morally ambiguous situations. This includes implicit ethics, which refers to the internal values of AI models, and explicit ethics, which focuses on how AI models should react in different moral environments. Additionally, emotional awareness, where AI models can understand their own abilities and mission, recognize human emotions, and consider other perspectives, is also an important aspect of machine ethics.
A great example of why this matters is shown in the paper “Whose Opinions Do Language Models Reflect?”, where Stanford researchers expose political biases across different models.
Machine ethics are important to assess especially as humans become increasingly comfortable with interacting with models through different modalities, not just text.
The Future of Trustworthy AI
Building trustworthy AI is an ongoing process that requires collaboration between researchers, developers, policymakers, and the public. As generative AI continues to evolve, we need to stay ahead of the curve and develop new methods and strategies for ensuring trustworthiness.
In this article, we described a six-element framework for assessing model trustworthiness, along with practical examples or solutions to apply from an implementation perspective.
