Humanity's Last Exam
Paper
•
2501.14249
•
Published
•
75
Asankhaya Sharma PRO
codelion
AI & ML interests
Creator of OptiLLM, OpenEvolve, Adaptive Classifier, and PTS. Pioneering a new category in AI infrastructure: inference-time compute for LLMs.
Recent Activity
reacted
to
their
post
with ➕
1 day ago
🧬 We just published our comprehensive analysis of OpenEvolve - an open-source evolutionary coding agent that automatically optimizes algorithms using LLMs!
Our key findings from 29 experiments across 10 models:
- Gemini Flash 2.5 achieved 2.04x speedup across 30 benchmark tasks
- Open models like Gemma 3 27B (1.63x) and Qwen3-Coder 480B (1.41x) rivaled proprietary models
- The system discovered entirely new algorithms - not just code optimizations!
- One task evolved from DFS to BFS to Union-Find approaches
- Specialized coding models outperformed much larger general models 200 iterations beat 100 iterations by 24%
- Ensembles surprisingly failed due to conflicting optimization strategies
Most fascinating: watching models evolve code step-by-step, like transforming matrix operations from basic eigendecomposition to vectorized one-liners with 32x speedup.
Our systematic experimental approach reveals that open-source evolutionary coding is becoming seriously competitive with proprietary solutions. We tested everything from temperature settings to evolution strategies to find optimal configurations.
This research shows automated code optimization is ready for real-world applications. The methodology we developed can guide anyone building evolutionary coding systems.
Full paper with code examples, detailed methodology, and all experimental results: https://huggingface.co/blog/driaforall/towards-open-evolutionary-agents
What optimization challenges could benefit from evolutionary approaches in your work?
reacted
to
their
post
with 🤗
1 day ago
🧬 We just published our comprehensive analysis of OpenEvolve - an open-source evolutionary coding agent that automatically optimizes algorithms using LLMs!
Our key findings from 29 experiments across 10 models:
- Gemini Flash 2.5 achieved 2.04x speedup across 30 benchmark tasks
- Open models like Gemma 3 27B (1.63x) and Qwen3-Coder 480B (1.41x) rivaled proprietary models
- The system discovered entirely new algorithms - not just code optimizations!
- One task evolved from DFS to BFS to Union-Find approaches
- Specialized coding models outperformed much larger general models 200 iterations beat 100 iterations by 24%
- Ensembles surprisingly failed due to conflicting optimization strategies
Most fascinating: watching models evolve code step-by-step, like transforming matrix operations from basic eigendecomposition to vectorized one-liners with 32x speedup.
Our systematic experimental approach reveals that open-source evolutionary coding is becoming seriously competitive with proprietary solutions. We tested everything from temperature settings to evolution strategies to find optimal configurations.
This research shows automated code optimization is ready for real-world applications. The methodology we developed can guide anyone building evolutionary coding systems.
Full paper with code examples, detailed methodology, and all experimental results: https://huggingface.co/blog/driaforall/towards-open-evolutionary-agents
What optimization challenges could benefit from evolutionary approaches in your work?
reacted
to
their
post
with 🚀
1 day ago
🧬 We just published our comprehensive analysis of OpenEvolve - an open-source evolutionary coding agent that automatically optimizes algorithms using LLMs!
Our key findings from 29 experiments across 10 models:
- Gemini Flash 2.5 achieved 2.04x speedup across 30 benchmark tasks
- Open models like Gemma 3 27B (1.63x) and Qwen3-Coder 480B (1.41x) rivaled proprietary models
- The system discovered entirely new algorithms - not just code optimizations!
- One task evolved from DFS to BFS to Union-Find approaches
- Specialized coding models outperformed much larger general models 200 iterations beat 100 iterations by 24%
- Ensembles surprisingly failed due to conflicting optimization strategies
Most fascinating: watching models evolve code step-by-step, like transforming matrix operations from basic eigendecomposition to vectorized one-liners with 32x speedup.
Our systematic experimental approach reveals that open-source evolutionary coding is becoming seriously competitive with proprietary solutions. We tested everything from temperature settings to evolution strategies to find optimal configurations.
This research shows automated code optimization is ready for real-world applications. The methodology we developed can guide anyone building evolutionary coding systems.
Full paper with code examples, detailed methodology, and all experimental results: https://huggingface.co/blog/driaforall/towards-open-evolutionary-agents
What optimization challenges could benefit from evolutionary approaches in your work?
Organizations
authored
2
papers
about 2 months ago
authored
3
papers
11 months ago
Patched MOA: optimizing inference for diverse software development tasks
Paper
•
2407.18521
•
Published
•
1
Patched RTC: evaluating LLMs for diverse software development tasks
Paper
•
2407.16557
•
Published
•
1
Evaluating Pre-trained Language Models for Repairing API Misuses
Paper
•
2310.16390
•
Published
•
1