bmorphism (Barton Rhodes)

reacted to AdinaY's post with 🚀 5 months ago

Post

2622

UI-TARS 🔥 series of native GUI agent models (2B/7B/72B) released by ByteDance, combining perception, reasoning, grounding, and memory into one system.

Model:

bytedance-research
Paper: UI-TARS: Pioneering Automated GUI Interaction with Native Agents (2501.12326)

reacted to mlabonne's post with 🧠 10 months ago

Post

19297

✂️ Uncensor any LLM with abliteration

I wrote an article about abliteration and how NeuralDaredevil-8B was created. Beyond removing alignment, I believe it's an interesting technique with a lot of potential. It's basically fine-tuning without retraining.

In this article, we see how it works, implement it in Google Colab, and heal the abliterated model to recover the performance drop due to this technique. The final model is an uncensored and high-quality model with the highest MMLU score on the Open LLM Leaderboard (8B category).

https://huggingface.co/blog/mlabonne/abliteration

26 replies

·

reacted to mikonvergence's post with 🧠 10 months ago

Post

2349

𝐍𝐞𝐰 𝐑𝐞𝐥𝐞𝐚𝐬𝐞: 𝐌𝐚𝐣𝐨𝐫 𝐓𝐎𝐌 𝐃𝐢𝐠𝐢𝐭𝐚𝐥 𝐄𝐥𝐞𝐯𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥 𝐄𝐱𝐩𝐚𝐧𝐬𝐢𝐨𝐧 🗺️

Dataset: Major-TOM/Core-DEM

Today with European Space Agency - ESA and Adobe Research, we release a global expansion to Major TOM with GLO-30 DEM data.

You can now instantly access nearly 2M of Major TOM samples with elevation data to build your next AI model for EO. 🌍

🔍 Browse the data in our usual viewer app: Major-TOM/MajorTOM-Core-Viewer

Fantastic work championed by Paul Borne--Pons @NewtNewt 🚀

reacted to m-ric's post with 🔥 10 months ago

Post

2168

🎮 𝗔 𝗻𝗲𝘂𝗿𝗮𝗹 𝗻𝗲𝘁𝘄𝗼𝗿𝗸 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗲𝘀 𝗗𝗢𝗢𝗠: 𝗚𝗼𝗼𝗴𝗹𝗲 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵𝗲𝗿𝘀 𝗼𝗽𝗲𝗻 𝘁𝗵𝗲 𝘄𝗮𝘆 𝗳𝗼𝗿 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆-𝗔𝗜-𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝗱 𝗴𝗮𝗺𝗲𝘀!

Imagine if games were completely live-generated by an AI model : the NPCs and their dialogues, the storyline, and even the game environment. The player’s in-game actions would have a real, lasting impact on the game story.

In a very exciting paper, Google researchers just gave us the first credible glimpse of this future.

➡️ They created GameNGen, the first neural model that can simulate a complex 3D game in real-time. They use it to simulate the classic game DOOM running at over 20 frames per second on a single TPU, with image quality comparable to lossy JPEG compression. And it feels just like the true game!

Here's how they did it:
1. They trained an RL agent to play DOOM and recorded its gameplay sessions.
2. They then used these recordings to train a diffusion model to predict the next frame, based on past frames and player actions.
3. During inference, they use only 4 denoising steps (instead of the usual dozens) to generate each frame quickly.

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
🎮🤔 Human players can barely tell the difference between short clips (3 seconds) of the real game or the simulation
🧠 The model maintains game state (health, ammo, etc.) over long periods despite having only 3 seconds of effective context length
🔄 They use "noise augmentation" during training to prevent quality degradation in long play sessions
🚀 The game runs on one TPU at 20 FPS with 4 denoising steps, or 50 FPS with model distillation (with some quality loss)

The researchers did not open source the code, but I feel like we’ve just seen a part of the future being written!

Their paper (exploding the upvote counter) 👉 Diffusion Models Are Real-Time Game Engines (2408.14837)
In a similar vein, play @Jofthomas 's 'Everchanging Quest' 🎮 Jofthomas/Everchanging-Quest

reacted to Xenova's post with 👍 11 months ago

Post

8152

Introducing Whisper Diarization: Multilingual speech recognition with word-level timestamps and speaker segmentation, running 100% locally in your browser thanks to 🤗 Transformers.js!

Tested on this iconic Letterman interview w/ Grace Hopper from 1983!
- Demo: Xenova/whisper-speaker-diarization
- Source code: Xenova/whisper-speaker-diarization

1 reply

·

reacted to gokaygokay's post with ➕ 12 months ago

Post

5047

Kolors: Effective Training of Diffusion Model for Photorealistic Text-to-Image Synthesis

Kolors is a large-scale text-to-image generation model based on latent diffusion, developed by the Kuaishou Kolors team.

Hugging Face Spaces
- gokaygokay/Kolors

Model Page
- Kwai-Kolors/Kolors

reacted to Xenova's post with 🔥 about 1 year ago

Post

6115

Florence-2, the new vision foundation model by Microsoft, can now run 100% locally in your browser on WebGPU, thanks to Transformers.js! 🤗🤯

It supports tasks like image captioning, optical character recognition, object detection, and many more! 😍 WOW!
- Demo: Xenova/florence2-webgpu
- Models: https://huggingface.co/models?library=transformers.js&other=florence2
- Source code: https://github.com/xenova/transformers.js/tree/v3/examples/florence2-webgpu

reacted to fdaudens's post with 👀 about 1 year ago

Post

1793

80% of fact-checked misinformation claims involve media, with a rise in AI-generated content in 2023, according to a new study, “A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild.” Worth a read for journalists, especially fact-checkers.

TL;DR:
• 📊 135,838 fact checks analyzed
• 📸 80% of these claims involve media
• 🎥 Videos became more common starting in 2022, now more than 60% of fact-checked claims that include media
• 🤖 AI-generated content was rare until Spring of 2023, and then dramatically increased
• 🖼️ Image manipulations don’t require complex operations. Most of the time it’s context manipulations

• Read the paper here: AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild (2405.11697)
• Take a look at the dataset: academic-datasets/AMMeBa

Thanks @davanstrien for spotting it!

reacted to shekkizh's post with 🚀 about 1 year ago

Post

1228

Hi folks,
Tenyx announced its latest model Llama3-TenyxChat-70B, which outperforms a GPT-4 variant on several MT-Bench measurements.

By post-training Llama-3 70B in 15 hours, our model improves reasoning capabilities leveraging the relationship between geometry and LLM task complexity (Take a look at our paper: https://arxiv.org/abs/2312.01648, to be presented at ICML 2024)
Model: tenyx/Llama3-TenyxChat-70B, HuggingFace Space: tenyx/Llama3-TenyxChat-70B

posted an update about 1 year ago

Post

1715

@llmixer is doing g-d's work by orthogonalizing weakness out of free-thinking models, well-done @llmixer

reacted to codelion's post with 👍 about 1 year ago

Post

1820

Meta's new LLama-3 ( meta-llama/Meta-Llama-3-8B-Instruct) is an extremely capable model out of the box for coding related tasks. It is the first model that we have seen that beats GPT-4 on Static-Analysis-Eval - patched-codes/static-analysis-eval.

reacted to abhishek's post with 🚀 about 1 year ago

Post

3076

How to Finetune phi-3 on MacBook Pro
https://huggingface.co/blog/abhishek/phi3-finetune-macbook

reacted to HugoLaurencon's post with ❤️ about 1 year ago

Post

2563

Idefics2 is trained mostly on OBELICS, our open interleaved image-text document dataset.

Training on interleaved data is crucial to reaching high performance on VQA tasks, taking an arbitrary number of images as input, and doing in-context learning.

Dataset: HuggingFaceM4/OBELICS
Nomic visualization: https://atlas.nomic.ai/map/f2fba2aa-3647-4f49-a0f3-9347daeee499/ee4a84bd-f125-4bcc-a683-1b4e231cb10f
Link to OBELICS thread: https://twitter.com/HugoLaurencon/status/1694005892839006301

reacted to clefourrier's post with 🔥 about 1 year ago

Post

2035

Are you looking for the perfect leaderboard/arena for your use case? 👀

There's a new tool for this!
https://huggingface.co/spaces/leaderboards/LeaderboardFinder

Select your modality, language, task... then search! 🔍
Some categories of interest:
- does the leaderboard accept submissions?
- is the test set private or public?
- is it using an automatic metric, human evaluators, or llm as a judge?

The spaces list is build from space metadata, and reloaded every hour.

Enjoy!

reacted to akhaliq's post with ❤️ over 1 year ago

Post

ELLA

Equip Diffusion Models with LLM for Enhanced Semantic Alignment

ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment (2403.05135)

Diffusion models have demonstrated remarkable performance in the domain of text-to-image generation. However, most widely used models still employ CLIP as their text encoder, which constrains their ability to comprehend dense prompts, encompassing multiple objects, detailed attributes, complex relationships, long-text alignment, etc. In this paper, we introduce an Efficient Large Language Model Adapter, termed ELLA, which equips text-to-image diffusion models with powerful Large Language Models (LLM) to enhance text alignment without training of either U-Net or LLM. To seamlessly bridge two pre-trained models, we investigate a range of semantic alignment connector designs and propose a novel module, the Timestep-Aware Semantic Connector (TSC), which dynamically extracts timestep-dependent conditions from LLM. Our approach adapts semantic features at different stages of the denoising process, assisting diffusion models in interpreting lengthy and intricate prompts over sampling timesteps. Additionally, ELLA can be readily incorporated with community models and tools to improve their prompt-following capabilities. To assess text-to-image models in dense prompt following, we introduce Dense Prompt Graph Benchmark (DPG-Bench), a challenging benchmark consisting of 1K dense prompts. Extensive experiments demonstrate the superiority of ELLA in dense prompt following compared to state-of-the-art methods, particularly in multiple object compositions involving diverse attributes and relationships.

reacted to visheratin's post with 🤗 over 1 year ago

Post

Keep stacking cool stuff and getting better results! After I changed the standard vision encoder to SigLIP, NLLB-CLIP got a 10% average performance improvement. And now, I added matryoshka layers (https://arxiv.org/abs/2205.13147) to enable smaller embeddings and got another 6% performance boost! Plus, thanks to MRL, 4.5x smaller embeddings retain 90%+ quality.

The large model is finally SoTA for both image and text multilingual retrieval!

The models are available on the hub:
- visheratin/nllb-siglip-mrl-base
- visheratin/nllb-siglip-mrl-large