๐ข๐พ Introducing the Common Crawl Creative Commons Corpus (C5)!
C5 is a large-scale effort to heavily filter web-crawled data, as collected by the non-profit Common Crawl, to only documents that are Creative Commons-licensed such as cc-by-4.0 or public domain cc0. At this stage 150 billion tokens have been collected.
</> To build C5, HTML pages are scrutinized and all links (if any) to CC licenses are collected, both in regular hyperlinks as well as in metadata. Additional data fields are included such as "was the license found in the head?" or "if multiple licenses were found, do they contradict each other?", which makes further filtering a breeze.
๐ In this first version of C5, 8 languages are included (Afrikaans, German, English, French, Frysian, Italian, Dutch and Spanish). The language set was limited for two reasons: computational and storage limitations, and a collaboration with GPT-NL, which requested CC data for these languages to train a Dutch-focused, copyright-conscious LLM. In total, this V1 release contains almost 150 thousand documents and 150 billion tokens. This data was not filtered on quality nor deduplicated so that you can decide for yourself how much data to keep. To give some quality indication, a dataset field is present to describe whether a document is included in the FineWeb(-2) datasets, which are of high quality.
๐ More work needs to be done! Only 7 out of 100+ Common Crawl crawls have been processed so far. That's encouraging because it means there is a lot more Creative Commons data to be collected! But to get there I need help in terms of compute. The current processing was already heavily sponsored by the Flemish Supercomputer but more is needed. If you have the compute available and which to collaborate in an open and transparent manner, please get in touch!
1 reply
ยท
reacted to lukmanaj's
post with ๐๐ค12 days ago
Iโm excited to share that Iโve completed the Hugging Face Agents Course and earned my certificate.
Over the past few months, I explored how to build intelligent, autonomous agents using cutting-edge tools like smolagents, LlamaIndex, and LangGraph. The course covered everything from the fundamentals of agents to advanced topics like fine-tuning for function-calling, observability, evaluation, and even agents in games.
Some key content included:
1. Introduction to AI Agents
2. Agentic RAG use cases
3. Multi-framework implementation: smolagents, LlamaIndex, and LangGraph
4. Building, testing, and certifying a complete agent project
This was a hands-on, practical experience that deepened my understanding of how to design reliable, tool-using LLM agents. Looking forward to leveraging these skills in real-world applications in healthcare, logistics, and beyond.
Many thanks to the Hugging Face team for putting this together. Letโs build safe and useful agents!
I am fascinated by models learning from prompts and rewards - no example answers needed like in Supervised Fine-Tuning.
After the DeepSeek boom, everyone is trying GRPO with GSM8K or the Countdown Game...
I wanted a different challenge, like ๐๐ฒ๐ฎ๐ฐ๐ต๐ถ๐ป๐ด ๐ฎ ๐บ๐ผ๐ฑ๐ฒ๐น ๐๐ผ ๐ฐ๐ฟ๐ฒ๐ฎ๐๐ฒ ๐ฎ ๐๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ฒ ๐ณ๐ฟ๐ผ๐บ ๐ฎ ๐น๐ถ๐๐ ๐ผ๐ณ ๐ฒ๐๐ฒ๐ป๐๐ ๐ฎ๐ป๐ฑ ๐ฝ๐ฟ๐ถ๐ผ๐ฟ๐ถ๐๐ถ๐ฒ๐.
Choosing an original problem forced me to: ๐ค Think about the problem setting ๐งฌ Generate data ๐ค Choose the right base model ๐ Design reward functions (and experiencing reward hacking) ๐ Run multiple rounds of training, hoping that my model would learn something.
In this work, we tackle some major challenges in Arabic multi-label emotion classification especially the issues of class imbalance and label correlation that often hurt model performance, particularly for minority emotions.
Our approach:
Stacked contextual embeddings from fine-tuned ArabicBERT, MarBERT, and AraBERT models.
A meta-learning strategy that builds richer representations.
A hybrid loss function combining class weighting, label correlation matrices, and contrastive learning to better handle class imbalances.
๐ Extensive experiments show significant improvements across Precision, Recall, F1-Score, Jaccard Accuracy, and Hamming Loss. ๐ The hybrid loss function in particular helped close the gap between majority and minority classes!
We also performed ablation studies to break down each componentโs contribution and the results consistently validated our design choices.
This framework isn't just for Arabic it offers a generalizable path for improving multi-label emotion classification in other low-resource languages and domains.
Big thanks to my co-authors: Muhammad Azeem Aslam, Wang Jun, Nisar Ahmed, Li Yanan, Hu Hongfei, Wang Shiyu, and Xin Liu!
Would love to hear your thoughts on this work! ๐
When OpenAI released its Computer-Using Agent (CUA) API, I happened to be playing Wordle ๐งฉ and thought, why not see how the model handles it? Spoiler: Wordle turned out to be a surprisingly effective benchmark. So Romain Cosentino Ph.D. and I dug in and analyzed the results of several hundred runs.
๐ Takeaways 1๏ธโฃ Even the best computer-using models struggle with simple, context-dependent tasks.ย 2๏ธโฃ Visual perception and reasoning remain major hurdles for multimodal agents. 3๏ธโฃ Real-world use cases reveal significant gaps between hype and reality. Perception accuracy drops to near zero by the last turn ๐
We just crossed 1,500,000 public models on Hugging Face (and 500k spaces, 330k datasets, 50k papers). One new repository is created every 15 seconds. Congratulations all!
3 replies
ยท
reacted to BrigitteTousi's
post with ๐2 months ago
Another impressive model that joined the ranking today is ALLaM-AI/ALLaM-7B-Instruct-preview. After a long wait finally ALLaM is here and it is IMPRESSIVE given its size !
Google just released PaliGemma 2 Mix: new versatile instruction vision language models ๐ฅ
> Three new models: 3B, 10B, 28B with res 224, 448 ๐ > Can do vision language tasks with open-ended prompts, understand documents, and segment or detect anything ๐คฏ
๐ Excited to share our technical report on the Southeast Asian multilingual model Sailor2 and its latest updates!
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
๐งญ We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:ย ๐ก More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.ย ๐ Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.ย โก๏ธ Faster inference: Achieved 2.5x faster inference speed with speculative decoding.ย ๐ช๏ธ More model sizes: Introduced new sizes of 3B and 14B through model pruning.
๐ All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
๐ HuggingFace Spaces Ranking Tracker - Your Complete AI Trend Analytics!
Introducing the Spaces Ranking Tracker, a comprehensive analytics dashboard that tracks and analyzes every AI application in the HuggingFace ecosystem.
โจ Key Features: โข Real-time tracking of daily ranking changes over 30 days โข Detailed analysis of top 100 trending spaces โข User-based integrated score visualization โข One-click access to space details โข Interactive rank change graphs
๐ Dashboard Components: 1. Main Dashboard - Daily rank trend graphs - Top 20 creators' combined score chart - Detailed space information cards - Real-time trending score updates
2. Space Detailed Analysis - Creation date, current rank, and trending score - 30-day ranking history - Direct space access - Custom color coding for intuitive rank display
๐ฏ How to Use: โข Monitor latest AI community trends โข Track your project's performance โข Discover popular AI demos โข Analyze competing projects โข Follow AI ecosystem dynamics
3. Interactive Features - Custom filtering options - Sorting by various metrics - Detailed performance statistics - Comprehensive trending scores - Historical data tracking
Stay on top of every movement in the HuggingFace ecosystem with daily ranking updates! ๐ Try it now!
There's so much you could do with these developments. Especially combining them together into agentic applications or fine-tuning them on your use case.
DeepSeek-R1 & DeepSeek-R1-Zero: two 660B reasoning models are here, alongside 6 distilled dense models (based on Llama & Qwen) for the community! deepseek-ai deepseek-ai/DeepSeek-R1
โจ MIT License : enabling distillation for custom models โจ 32B & 70B models match OpenAI o1-mini in multiple capabilities โจ API live now! Access Chain of Thought reasoning with model='deepseek-reasoner'
