[ad_1]
Start AI Mistral has Fired New content moderation API.
The API, the same API that supports moderation in Mistral’s Le Chat chatbot platform, can be tailored to fit specific applications and security standards, Mistral says. It is powered by a fine-grained model (Ministral 8B) that has been trained to classify text in a range of languages, including English, French and German, into one of nine categories: sexual content, hate, discrimination, violence, threats, dangerous and criminal content. Self-harm, health, financial, legal and personal information.
The moderation API can be applied to raw text or chat transcript, Mistral says.
“Over the past few months, we have seen growing enthusiasm across the industry and research community for new AI-based supervision systems, which can help make supervision more scalable and robust across applications,” Mistral wrote in a blog post. “Our content moderation classifier leverages the most relevant policy categories for effective guardrails and offers a practical approach to form safety by addressing form harms such as unqualified tips and personally identifiable information.”
AI-powered supervision systems are useful in theory. But they are also vulnerable to the same biases and technical flaws as other AI systems.
For example, some models trained to detect toxicity find that phrases in African American Vernacular English (AAVE), the informal grammar used by some black Americans, are disproportionately “toxic.” Social media posts about people with disabilities are often rated as more negative or toxic by common public sentiment and toxicity detection models, studies have shown. Found.
Mistral claims its moderation model is highly accurate, but also admits it’s a work in progress. Notably, the company has not compared the performance of its API with other popular moderation APIs, such as Jigsaw’s Perspective API and OpenAI’s moderation API.
“We are working with our customers to build and share scalable, lightweight, and customizable moderation tools, and we will continue to engage with the research community to contribute to safety advances in the broader field,” the company said.
Mistral too Announce API batch today. The company says it can reduce the cost of forms submitted through its API by 25% by processing high-volume requests asynchronously. Anthropic, OpenAI, Google, and others also offer bundling options for their AI APIs.
[ad_2]