Overfitting in Self-Evolving Agent Skills

Agent self-evolution is attracting growing interest as a practical way to improve performance on desired tasks. Much as LLMs learn from training instances, self-evolving agents work through a set of tasks, reflect on their execution trajectories and feedback, and retain lessons to guide future attempts. These lessons are often stored as skills: anything from a agents.md file containing guidance to a hierarchy of artifacts and executable scripts. Skill optimization turns experience into persistent guidance without changing the underlying model’s weights.

But do these skills generalize beyond the examples that shaped them? In our paper, Do Self-Evolving Skills Generalize to Held-Out Tasks?, we study this question using a familiar machine-learning setup: separate training and test sets. We optimize skills on training cases, then evaluate how well those skills help agents complete held-out cases of the same task type.

The distinction matters: better performance on familiar tasks does not necessarily mean better performance on new ones. We find substantial variation—some gains carry over, others shrink, and some evolved skills perform worse than giving the agent no skill at all.

01 Two loops in classic self-evolving skill methods

A typical skill-evolution system couples two loops.

In the task-execution loop, an agent follows a skill, acts in an environment, observes feedback, and continues working. The skill might tell it how to inspect inputs, select tools, perform an analysis, and check its output.

In the skill-refinement loop, a refiner examines the execution trajectories and outcomes, then rewrites the skill. The updated artifact becomes guidance for subsequent executions.

These loops turn experience into persistent instructions without changing the underlying model’s weights. A missed check can become a checklist item; a failed procedure can become a revised workflow.

But the refiner learns from a finite set of examples. Repeated practice may make the skill increasingly effective on those examples while embedding assumptions that fail elsewhere. The learning takes place in the skill, and so can the overfitting.

Five ways to evolve a skill

We reviewed recent papers on self-evolving skills and summarize their methods below to provide an overview of how researchers are approaching this challenge.

SkillOpt: limit the edits, then test the candidate

• Left: trajectory minibatches produce explicit add/delete/replace proposals.
• Middle: proposals are merged, ranked, and clipped to an edit budget. This limits how much one update can change.
• Right: a validation gate accepts a candidate only if its score improves. Rejected edits enter a buffer to inform later proposals.

Within typical self-evolving loops, SkillOpt mainly constrains the skill-rewrite process: an edit budget limits how much each update can change, and a validation gate accepts the candidate only if its score improves. The complete method also includes slower epoch-level consolidation, outside this crop.

SkillGen: count what the skill repairs and what it breaks

• Left: compare failed attempts with nearby successful ones to identify behaviors missing from failures.
• Center: the generator writes a candidate; the verifier compares its outcomes with cached no-skill outcomes on the same verification inputs.
• Verifier → feedback: count repaired failures and newly broken successes, then specify what to keep, remove, add, or emphasize. Select the best verified candidate; deactivate it if the net gain is insufficient.

Within typical self-evolving loops, SkillGen mainly changes how experience is analyzed and candidate skills are verified: contrastive diagnosis identifies behaviors missing from failed attempts, and paired evaluation with and without the skill measures repairs against regressions. The resulting feedback guides the next revision.

Trace2Skill: parallel patches before a shared rewrite

• Left: collect successful and failed executions using the initial skill.
• Middle: independent analysts receive one trace each and the same frozen skill. Each proposes a patch; error analysts can inspect files and investigate the cause.
• Right: merge the patches into one update, consolidating recurring lessons and resolving conflicts before applying edits.

Within typical self-evolving loops, Trace2Skill mainly changes the skill-refinement process: parallel analysts propose patches from individual trajectories against the same frozen skill, then a merger consolidates those patches into one coherent update. This delays rewriting until lessons from multiple attempts have been considered together.

EvoSkill: diagnose a gap, then build a new capability

• Base Agent: the trace shows an incorrect numerical answer, exposing a capability gap.
• Proposer: diagnose that gap and request a reusable data-extraction verification skill.
• Skill-Builder: turn the proposal into a skill folder with instructions, trigger metadata, and optional supporting code.

Within typical self-evolving loops, EvoSkill mainly changes how new capabilities are proposed, built, and selected: a Proposer diagnoses failures, a Builder turns its proposals into skill folders, and held-out validation determines which skill-equipped agent configurations remain in the candidate frontier.

GEPA: preserve candidates that solve different tasks

• Score matrix: P₂ solves Tasks 2 and 3, while P₃ alone solves Task 1.
• Per-task comparison: P₁ also solves Task 2, but adds nothing beyond P₂ in this example. P₃ contributes something P₂ cannot do.
• Filtered pool: preserve complementary candidates such as P₂ and P₃ instead of always selecting one average-score winner to evolve.

Within typical self-evolving loops, GEPA mainly changes which candidate is refined next and how variants are explored: it samples parents from an instance-wise Pareto frontier, uses execution feedback to revise a prompt module, and evaluates the resulting variant before adding it to the pool. This crop highlights candidate selection; mutation and evaluation complete the loop.

Three directions in skill evolution

These methods can be understood as pursuing three overlapping directions within the same evolution loop.

Extract better lessons from experience. What should the refiner learn from execution traces? Trace2Skill analyzes trajectories independently and consolidates recurring patches; SkillGen contrasts failures with nearby successes to identify missing behaviors. Both invest in turning individual attempts into evidence for reusable instructions. ^{[3] [4]}

Make updates more controlled and empirically justified. How far should a rewrite go, and when should it be accepted? SkillOpt limits edit size and gates updates on validation performance. SkillGen measures the candidate’s net effect against no skill, explicitly counting both repairs and regressions. Their emphasis is on preventing plausible revisions from accumulating without evidence that they help. ^{[2] [3]}

Explore alternatives instead of committing to one evolving artifact. Which candidate should be developed next? EvoSkill branches skill-equipped agent configurations and retains a bounded frontier; GEPA uses instance-wise Pareto selection to explore candidates with complementary strengths. They also search over different objects: EvoSkill can create or revise skill folders, while GEPA mutates prompts within a system. ^{[5] [6]}

02 Generalization varies across methods and tasks

In our experiments, skill optimization proceeds through repeated rounds, much like epochs in machine learning. We track rounds 1–20 of evolution, with each round going through all training cases. At round i, we also freeze the skill checkpoint and evaluate its overall success rate on held-out test cases. This lets us observe whether continued optimization improves performance beyond the examples used to refine the skill.

The train/test split is defined within each dataset. We measure generalization to new instances of the same task type—not transfer across datasets or unrelated task types.

Generalization varies across methods and tasks. In Figure 8, training improvements transfer only weakly on SpreadsheetBench, while the gains track more closely on HealthBench. A rising training curve alone does not establish reusable improvement. ^[1]

Test cases may simply be harder, so raw training–test gaps can mislead. We compare the skill’s advantage over No Skill separately on each split, then measure how much of that advantage survives. The summary below shows both incomplete transfer and outright harm on held-out tasks.

These findings call for cautious optimism about deploying self-evolving skills. Evolution can produce useful knowledge, but training benefits often shrink on unseen tasks, and some evolved skills perform worse than no skill. Held-out benefit needs to be demonstrated before deployment. ^[1]

Overfitted Skills: Typical Failure Modes

Skill text gives us a way to examine possible causes.

Hard-coded variables and outputs. A skill can name columns and output paths taken from development tasks. Those details make a procedure concrete, but become brittle if reused when a new task has different inputs or requirements. A reusable procedure needs to explain how to identify the appropriate variables from the current task.

Procedures without conditions. Instructions such as checking data structure or matching output shape can be useful. However, a checklist that never explains when its checks apply leaves the agent to infer their relevance. Repetition and emphasis cannot supply that missing context.

Local fixes turned into global rules. In one SkillOpt example, a conditional requirement to create a summary file becomes mandatory for every task, even when the task does not request one. That edit raises training performance by 5 points and lowers test performance by 5 points. ^[1]

The issue is the scope of the lesson. Feedback identifies a failure in one situation; the revised skill may prescribe behavior across all future situations.

These are illustrative cases rather than a measured breakdown of every failure. They suggest a useful diagnostic question: which parts of this instruction describe a reusable procedure, and which parts assume that the next task resembles a previous one?

03 Meta-Learning for Skill Optimization

Our experiments suggest that the problem is not that skills are too weak to retain useful information. Instead, they can retain too many details of the training cases. When those details become instructions for new tasks, they can constrain rather than help the agent—an instance of overfitting.

This motivates a different approach: rather than only teaching an agent what to remember, teach it how to write a skill. The agent can then use that ability to construct guidance suited to the task at hand. Similar in spirit to meta-learning, GSO optimizes a persistent meta-skill that guides the construction and use of fresh task-specific skills, rather than carrying the same concrete instructions across tasks.

The GSO loop has three steps:

1. Agent rollout. The agent executes tasks using skills generated under the current meta-skill, interacts with the environment, and produces execution trajectories and feedback.
2. Meta-skill evolution. Failed trajectories inform revisions to the guidance for constructing and using skills: interpreting tasks, choosing tools and sources, writing procedures, checking results, diagnosing failures, and attempting bounded repairs. A candidate changes one component and is compared with the current version on validation tasks. For binary outcomes, acceptance requires repairing more than twice as many tasks as the update breaks. ^[1]
3. Skill evolution based on the meta-skill. The updated meta-skill guides the generation of a fresh task-specific skill, grounding its concrete instructions in the current inputs and environment. The agent uses that skill in the next rollout, completing the loop.

04 Results: Some Evolution Helps, Some Hurts

The final results make the practical cost visible: 10 of the 36 existing-method test results fall below No Skill. On five of the six benchmarks, at least one existing approach performs worse than the agent without a skill. No existing approach is best everywhere. ^[1] For a user, an evolved artifact has value only if it improves future execution. A skill that fixes familiar examples while introducing new failures can undermine the reason for retaining experience.

GSO achieves the highest test score across all six settings, with gains of 4.3–22.5 percentage points over the strongest existing comparator in each. The broader lesson is that self-evolving skills need the same scrutiny we apply to other learned systems: separate development from evaluation, measure held-out gains, and check what an update breaks as well as what it repairs.

An agent’s experience becomes useful knowledge when the lessons it retains continue to help on tasks that did not teach them.

Appendix A Benchmarks

Public benchmarks used in GSO and the related skill-evolution papers discussed above. Setup descriptions summarize the tasks and evaluation protocols used in these studies.

Read the paper

References

1. Xihao Piao, Zifeng Wang, and Zheng Chen. Do Self-Evolving Skills Generalize to Held-Out Tasks? 2026. arXiv:2609.39148v2.
2. Yifan Yang et al. SkillOpt: Executive Strategy for Self-Evolving Agent Skills. 2026. arXiv:2605.23904v2. Sections 3.2–3.6.
3. Yuchen Ma et al. SkillGen: Verified Inference-Time Agent Skill Synthesis. 2026. arXiv:2605.10999v1. Sections 3.1–3.3.
4. Jingwei Ni et al. Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills. 2026. arXiv:2603.25158v4. Sections 2.2–2.5.
5. Salaheddin Alzubi et al. EvoSkill: Automated Skill Discovery for Multi-Agent Systems. 2026. arXiv:2603.02766v1. Sections 2.1–2.2.
6. Lakshya A. Agrawal et al. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. ICLR 2026. arXiv:2507.19457v2. Section 3 and Appendix D.1.
7. Zeyao Ma et al. SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation 2024. Paper.
8. Carlos E. Jimenez et al. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? 2023. Paper.
9. OpenAI. Introducing SWE-bench Verified. 2024. Official release of the human-validated subset. Source.
10. Rebecca Soskin Hicks et al. HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats 2026. Paper.
11. Kuangshi Ai et al. SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents 2026. Paper.
12. Mohit Shridhar et al. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning 2020. Paper.
13. The Databricks AI Research Team. Introducing OfficeQA: A benchmark for end-to-end grounded reasoning. 2025. Original benchmark release. Source.
14. Minesh Mathew et al. DocVQA: A Dataset for VQA on Document Images 2020. Paper.
15. Matthew Dunn et al. SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine 2017. Paper.
16. Panupong Pasupat and Percy Liang. Compositional Semantic Parsing on Semi-Structured Tables. ACL–IJCNLP 2015. Introduces WikiTableQuestions. Source.
17. Mathematical Association of America. American Invitational Mathematics Examination (AIME). Official competition source; the 2025 and 2026 question sets are competition editions. Source.
18. Qiying Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale 2025. Paper.
19. Linyang He et al. LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches 2026. Paper.
20. Chaoqun He et al. OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems 2024. Paper.
21. Bofei Gao et al. Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models 2024. Paper.
22. Colin White et al. LiveBench: A Challenging, Contamination-Limited LLM Benchmark 2024. Paper.
23. Zhilin Yang et al. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering 2018. Paper.
24. Yichen Jiang et al. HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification. Findings of EMNLP 2020. Source.
25. Thinh Pham et al. SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models 2025. Paper.
26. Jason Wei et al. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents 2025. Paper.
27. Valentina Pyatkin et al. Generalizing Verifiable Instruction Following 2025. Paper.
28. Li Siyan et al. PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles 2024. Paper.
29. Ruoyao Wang et al. ScienceWorld: Is your Agent Smarter than a 5th Grader? 2022. Paper.
30. Naman Jain et al. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code 2024. Paper.
31. Xiang Deng et al. Mind2Web: Towards a Generalist Agent for the Web 2023. Paper.
32. Zhenting Wang et al. MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers 2025. Paper.
33. Shunyu Yao et al. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains 2024. Paper.
34. Qiao Jin et al. PubMedQA: A Dataset for Biomedical Research Question Answering 2019. Paper.
35. Taicheng Guo et al. What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks. NeurIPS 2023, Datasets and Benchmarks Track. Introduces ChemLLMBench. Source.
36. Zixiang Xu et al. SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments 2025. Paper.

@article{piao2026self,
  title={Do Self-Evolving Skills Generalize to Held-Out Tasks?},
  author={Piao, Xihao and Wang, Zifeng and Chen, Zhen},
  journal={arXiv preprint arXiv:2609.39148},
  year={2026}
}