ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents.
ProofPaper ↗
Key points
- Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies.
- Yet most existing approaches remain text-centric, linearizing spatial layouts and action-state correspondences into language that loses critical geometric structure.
- Recent efforts have begun incorporating visual evidence, but construct and update skills separately from policy optimization, leaving their mutual improvement underexplored.
- An optional cold-start mechanism further accelerates early-stage learning.
Sources (2)
- [1]ViSkill: Reinforcing VLM Agents with Evolving Visual-Native SkillsHugging Face Daily Papers · Oct 8, 12:00 AM
We propose ViSkill, a visual-native skill learning framework that encodes successful interactions as composite visual skill cards directly accessible to VLM agents.
Skill-augmented agents improve sample efficiency by distilling successful trajectories into reusable strategies.
- [2]ViSkill: Reinforcing VLM Agents with Evolving Visual-Native SkillsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:44 PM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026MIMESIS: Learning User Simulators as Training Environments for Interactive Agents
- Oct 7, 2026Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
- Oct 7, 2026On-Policy Distillation Teaches New Skills but Not New Knowledge
- Oct 7, 2026UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Policy
- Oct 6, 2026Teaching a Robot Dog New Tricks: Diverse Quadruped Skills via Combined Reinforcement and Imitation Learning with Adversarial Task Selection
- Oct 6, 2026Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
