pinned
Running
ThumbGate
🛡
Infrastructure firewall for AI coding agents
Igor Ganapolsky
IgorGanapolsky
AI & ML interests
Python PyTorch LLMs, Google ML Kit, Android Nano AI
Recent Activity
updated a model 4 days ago
IgorGanapolsky/thumbgate-pre-action-gates-paper published a model 4 days ago
IgorGanapolsky/thumbgate-pre-action-gates-paper repliedto sergiopaniego's post 4 days ago
we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv"
you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced
and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine
the loop:
- OpenCode owns its tool loop inside an OpenEnv sandbox
- an in-sandbox proxy records the real token ids + logprobs, per turn
- a hidden-test verifier scores the result, and that is the reward
- TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL
blog + runnable example: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-trainingOrganizations
