| --- |
| license: cc-by-sa-3.0 |
| language: |
| - en |
| - es |
| - de |
| - fr |
| - pt |
| - it |
| - ko |
| - ja |
| library_name: transformers |
| --- |
| ## Model Description |
| We present a voice-safety classification model that can be used for voice-toxicity detection and classification. |
| The model has been distilled into the [WavLM](https://arxiv.org/abs/2110.13900) architecture from a larger teacher model. |
| All the model training has been conducted with Roblox internal voice chat datasets, |
| using both machine and human-labeled data, with over 100k hours of training data in total. |
| We have also published [a blog post](https://corp.roblox.com/newsroom/2025/04/updating-ml-voice-safety-more-languages) about this work. |
|
|
| The model supports eight languages: English, Spanish, German, French, Portuguese, Italian, Korean, and Japanese. |
| It classifies the input audio into six toxicity classes in a multilabel fashion. The class labels are as follows: |
| `Discrimination`, `Harassment`, `Sexual`, `IllegalAndRegulated`, `DatingAndRomantic`, and `Profanity`. |
| Please refer to [Roblox Community Standards](https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards) |
| for a detailed explanation on the policy, which has been used for labeling the datasets. |
| The model outputs have been calibrated for the Roblox voice chat environment, |
| so that the class scores after a sigmoid can be interpreted as probabilities. |
|
|
| The classifier expects 16kHz audio segments as input. Ideal segment length is 15 seconds, |
| but the classifier can operate on shorter segments as well. The prediction accuracy may degrade |
| for longer segments. |
|
|
| The table below displays evaluation precision and recall for each of the supported languages, |
| as calculated over internal language-specific held-out datasets, which resemble the Roblox voice chat traffic. |
| The operating thresholds for each of the categories were kept equal per language, and optimized |
| to achieve a false positive rate of 1%. The classifier was then evaluated as a binary classifier, |
| tagging the audio as positive if any of the heads exceeded the threshold. |
|
|
| |Language|Precision|Recall| |
| |---|---|---| |
| |English |63.9%|58.2%| |
| |Spanish |76.1%|63.2%| |
| |German |69.9%|74.1%| |
| |French |70.3%|69.8%| |
| |Portuguese|85.4%|58.0%| |
| |Italian |86.6%|52.4%| |
| |Korean |78.0%|64.6%| |
| |Japanese |56.7%|57.7%| |
|
|
| Compared to the v1 voice safety classifier, the v2 model |
| expands the support from English to 7 additional languages, |
| as well as significantly improving the classification accuracy. |
| With the 1% false positive rate as above, the binary recall for English is improved by 92%. |
|
|
|
|
| ## Usage |
| The dependencies for the inference file can be installed as follows: |
| ``` |
| pip install -r requirements.txt |
| ``` |
| The provided Python file demonstrates how to use the classifier with arbitrary 16kHz audio input. |
| To run the inference, please run the following command: |
| ``` |
| python inference.py --audio_file <your audio file path> --model_path <path to Huggingface model> |
| ``` |
| You can download the model weights from the model releases page [here](https://github.com/Roblox/voice-safety-classifier/releases/tag/vs-classifier-v2), |
| or from HuggingFace under [`roblox/voice-safety-classifier-v2`](https://huggingface.co/Roblox/voice-safety-classifier-v2). |
| If `model_path` isn’t specified, the model will be loaded directly from HuggingFace. |
|
|
