Evolution (Agent Self-Learning)
Overview
Evolution is the self-learning system in the tRPC-Agent-Go framework. It enables agents to automatically extract reusable skills from past executions and load them on future tasks. The entire process runs as an asynchronous background loop — the main task path is never blocked.
Purpose
Evolution accumulates and reuses the agent's "operational experience". Skills are unscoped by default (SkillScopeNone); app-level or user-level isolation can be enabled explicitly. When an agent completes a task, the background learning loop checks the review policy (default: ≥4 tool calls, user correction, or recovered error) before invoking the Reviewer. If the delta is worth reviewing, the Reviewer analyzes the conversation transcript and extracts reusable workflows as structured SKILL.md files. On subsequent similar tasks, the agent loads the matching skill via skill_load and follows the proven steps directly, avoiding repeated trial-and-error.
It excels at capturing: stable multi-step workflows, tool-calling best practices, common pitfalls and avoidance strategies, domain-specific operational procedures.
Key Benefits
- Efficiency: similar tasks that initially require multi-round exploration complete in one pass after skill extraction (benchmark: 17-33% token savings)
- Disaster suppression: skills provide clear steps, eliminating random infinite loops on certain tasks (up to 94.6% savings on worst-case runs)
- Experience reuse: pitfalls learned once persist permanently, independent of session context
- Quality control: quality gates ensure only qualified skills go live; write isolation protects existing assets
Architecture
Pipeline stages:
| Stage | Responsibility | Implementation |
|---|---|---|
| ReviewPolicy | Decide whether to review (default: ≥4 tool calls) | DefaultReviewPolicy |
| Reviewer | Extract skill spec from transcript (JSON) | LLMReviewer (gpt-4o-mini) |
| Reconciler | Deterministic dedup/absorb/merge (4 rules) | Pure string rules |
| SpecGate | Validate spec schema, naming, duplicates | Deterministic |
| SafetyGate | Scan for secrets, dangerous commands, path traversal | Deterministic |
| EffectivenessGate | Block revisions from failed sessions | Deterministic |
| HumanGate | Optional human approval | NewAlwaysHoldGate / NewCreateOnlyHoldGate |
| Publisher | Write SKILL.md to disk | File-based publisher |
Quick Start
Minimal Setup
Production Setup (Recommended)
Offline Reflective Optimization
The asynchronous reviewer learns one candidate from one completed session.
For skills that have a repeatable benchmark, evolution/optimization adds a
separate pure-Go search loop inspired by GEPA. It does not require DSPy,
Python, or a companion process.
optimization.Optimizer is the algorithm-neutral execution contract. The
built-in NewGEPA implementation owns GEPA's reflection and Pareto search,
while dataset isolation, budgets, experiment records, holdout verification,
and optional revision submission remain in a shared lifecycle. Applications
choose an implementation during wiring rather than through a runtime algorithm
string.
The optimizer:
- evaluates the seed skill on a validation split;
- evaluates a Pareto-selected parent on a feedback minibatch and sends the current skill, case input and expected value, score, output, evaluator feedback, and trace to a reflection model after bounding each text field and applying best-effort credential redaction;
- changes exactly one
SkillSpeccomponent and accepts the child only when it strictly improves the paired minibatch; - tracks per-case validation winners and samples parents by instance-level Pareto coverage;
- compares the final candidate with the seed on an unseen holdout split; and
- optionally submits the candidate to the existing revision store. External
submissions initially stop at
pending_approvaland never update the live skill directly. By default they remain there until reviewed. If the service enablesWithApprovalTimeout, the existing auto-expiration sweeper may later promote and publish a stale revision without human approval.
The application supplies both a task-specific evaluator and a reflection
model.Model. Configure the reflection model with the same provider adapters
used by other agents; it may be the task model or a separate reviewer model.
The evaluator is the only new domain-specific integration:
Revision submission is a separate optional capability from asynchronous session
learning. The default service implements evolution.RevisionSubmitter, while a
custom evolution.Service may omit it. Resolve that capability once during
application wiring so a configuration error is found before an optimization
run starts:
Submission uses the configured skill repository as a write boundary. An update
must target an existing skill and, when WithManagedSkillsDir is configured,
that skill must reside below the managed directory. A create must not collide
with an existing repository skill. Pass the same repository used by the agent
so these checks see bundled, user-authored, and evolution-managed skills.
The optimizer borrows the submitter. The application still owns and closes
evoSvc. Once search has selected a candidate, a later holdout, recording, or
submission failure is returned together with the non-nil partial result; fields
describe the phases that completed, and SubmissionReason records a submission
failure.
WithStoreDir is an optional, node-local experiment recorder, distinct from
the revision CandidateStore. Every run owns a UUID-named directory; on
permission-aware filesystems, files use 0600 and directories use 0700.
Each persisted evaluator output, feedback, or trace field is capped at 16 KiB.
The dataset is stored once in experiment.json; evaluation records reference
its cases by ID instead of duplicating inputs and expected values on every
iteration.
It is not a distributed job coordinator or a remote durable store. Concurrent
nodes may use the same POSIX-compatible shared root because experiments do not
share directories, but exactly one node must write each experiment. On
ephemeral or non-shared pod disks, upload the completed directory to
application-owned durable storage. Revision CandidateStore and
ActivePointer are pluggable, but their interface does not define a
cross-backend transaction; a multi-node deployment must assign revision
mutations to one owner or serialize them externally.
Case IDs must be unique across splits. Scores must be finite and normalized to
[0,1]. PromotionEligible and PromotionReason are populated even when
Submit is false, so callers do not need to duplicate the holdout threshold
and critical-case policy. Submission requires at least ten cases in each split.
Keep holdout cases hidden from the search. Before applying the same best-effort
credential-pattern redaction used by the online reviewer, the optimizer
projects each model-bound text field to a bounded UTF-8-safe head and tail.
This keeps redaction memory bounded, but does not identify tenant-specific
sensitive data. Applications must sanitize
the seed skill, case input and expected value, evaluator output, feedback, and
trace before returning them, and run candidate agents without production
credentials or side-effecting tools. Filesystem records also contain dataset
metadata and these sanitized values.
Review Policy
Evolution decides whether to review after each task completion. The built-in DefaultReviewPolicy triggers when any of these conditions hold:
| Condition | Rationale |
|---|---|
ToolCallCount ≥ 4 |
Multi-step tasks have extraction value |
HasUserCorrection |
User corrected the agent → worth recording pitfall |
HasRecoveredError |
Agent recovered from error → worth recording experience |
Tune the built-in policy:
Custom policy implementations can call centralized services or use tenant-specific rules. Policy errors do not advance the review cursor, so the same delta can be retried later.
Quality Gates
SpecGate
Deterministic schema/naming validation:
- Schema: name / description / when_to_use / steps must be non-empty
- Naming: no numeric counts (e.g. "3 Cities"), no excessive length
- Dedup: rejects exact-name duplicates of existing skills
SafetyGate
Deterministic content safety scan:
- Secrets:
sk-*,AKIA*, JWT tokens, private key markers - Dangerous commands:
rm -rf /,chmod 777,> /dev/sda - Path traversal:
../../etc/passwd,/root/.ssh/
EffectivenessGate
Outcome-based quality check:
- Session result
failoragent_error→ revision rejected - Normalized session score < 0.8 → revision held in
pending_eval
Requires an Outcome to be attached to the learning job:
HumanGate (Optional)
Holds create/update/delete revisions after all automatic gates pass:
Approve/reject via ApprovalService:
The decision is stored on HumanReport (Approved, Reviewer, Comment,
DecidedAt) and appended to the audit log. Delete revisions held by a
HumanGate do not mutate the live skill until approval; approving the delete
calls Publisher.DeleteSkill and clears the active pointer.
Auto-expire
To prevent revisions sitting in pending_approval forever, configure an
expiration timeout. Revisions older than the timeout are auto-promoted to
active (Reviewer = auto-expire, recorded in the audit log):
The sweeper runs as a background goroutine inside the worker; it stops on
Service.Close(). Setting the timeout to 0 (default) disables the sweeper.
In openclaw YAML, use duration strings:
Rollback
Restore the previous active revision when a new one regresses quality. The
current active revision is demoted to archived and the chosen archived
revision is promoted back to active. The publisher is updated immediately
so agents pick up the restored skill on their next read:
When TargetRevisionID is empty, the latest archived revision in the
revision store's ordering wins.
ErrNoArchivedRevision is returned when no archived revision is available.
The openclaw CLI exposes the same workflow:
Custom Gates
Implement the interface to plug in custom logic:
Reconciler (Deterministic Dedup)
Runs before gates on the reviewer's raw output:
| Rule | Trigger | Action |
|---|---|---|
| Rule 1: Strict-superset | New name is a task-variant superset of existing (e.g. "Weather - 5 Cities" vs "Weather Multi-City") | create → update |
| Rule 2: Intra-batch dedup | Reviewer outputs multiple skills with same name/shape in one batch | Keep first, drop rest |
| Rule 3: Quantified-sibling | Count-specific name (3 Cities) maps to existing generic parent (Multi-City) |
create → update |
| Rule 4: Word-overlap | New name shares ≥50% significant words with existing (e.g. "Market Snapshot" vs "Market Analysis") | create → update |
All rules are deterministic (pure string operations), zero LLM cost.
Revision Lifecycle
Each skill mutation is stored as an immutable revision:
For approved delete revisions, the delete revision is recorded and the active pointer is cleared so no previous revision remains visible.
On-disk structure:
Configuration Options
| Option | Description | Default |
|---|---|---|
WithManagedSkillsDir(dir) |
Directory where evolution writes SKILL.md files (Publisher target) | Required |
WithSkillRepository(repo) |
Skill repo for reading existing skills for dedup; should be the same instance shared with the agent | Required |
WithSkillRepositoryProvider(p) |
Resolve the skill repository per SkillScope (multi-tenant isolation, see below) |
nil |
WithSkillScopeMode(mode) |
Isolation granularity: SkillScopeApp (share per app) / SkillScopeUser (isolate per app+user) |
SkillScopeNone (no isolation) |
WithReviewPolicy(p) |
Review trigger policy | DefaultReviewPolicy (≥4 tool calls) |
WithCandidateStore(store) |
Immutable revision store | nil (no tracking) |
WithActivePointer(ptr) |
Active revision pointer | nil |
WithSpecGate(gate) |
Schema/naming validation | nil |
WithSafetyGate(gate) |
Safety scanner | nil |
WithEffectivenessGate(gate) |
Effectiveness check | nil |
WithHumanGate(gate) |
Human approval | nil (disabled) |
WithApprovalGateShadow(bool) |
Shadow mode — evaluate but don't enforce | false |
WithApprovalTimeout(d) |
Auto-promote pending_approval revisions older than d |
0 (disabled) |
WithApprovalSweepInterval(d) |
Sweep period for the auto-expire goroutine | min(timeout/4, 1h) |
WithWorkerNum(n) |
Async worker count | 1 |
WithQueueSize(n) |
Per-worker job queue buffer | 10 |
WithExistingSkillBodyMaxChars(n) |
Body excerpt length for reviewer context | 600 |
WithReviewerOptions(...) |
LLM reviewer options (temperature etc.) | - |
WithReviewer(r) |
Custom Reviewer implementation | LLMReviewer |
WithPublisher(p) |
Custom Publisher implementation | File-based publisher |
Write Isolation
When ManagedSkillsDir is configured, evolution enforces write isolation:
- Create: always allowed — new skill written to ManagedSkillsDir
- Update: only allowed for skills within ManagedSkillsDir; updates targeting bundled or user-authored skills are silently skipped (warning logged)
- Delete: same rules as update
This ensures evolution never accidentally modifies hand-written or built-in skills.
Multi-Tenant Isolation
By default (SkillScopeNone) all sessions share one skill library and revision
directory. In multi-app / multi-user deployments, evolution can isolate skills
per app or per app+user so that one tenant's learned skills never
pollute another's.
Isolation granularity (SkillScopeMode)
| Mode | Description | File layout |
|---|---|---|
SkillScopeNone (default) |
No isolation, globally shared | <root>/... |
SkillScopeApp |
Shared per app across that app's users | <root>/apps/<app>/... |
SkillScopeUser |
Isolated per app+user | <root>/users/<app>/<user>/... |
The SkillScope is derived from the session's AppName / UserID (or set
explicitly via LearningJob.Scope). App/user identifiers are sanitized for
filesystem safety: illegal or overly long identifiers are replaced with a
stable hash segment (id-<hash>), preventing path traversal while staying
idempotent.
Wiring
Repository resolution goes through skill.RepositoryProvider, which returns
the skill.Repository for a given SkillScope:
On the agent side, use the matching llmagent.WithSkillRepositoryProvider /
llmagent.WithSkillScopeMode so the skills injected into the prompt and the
skills evolution writes share the same scope.
Note: the
Publisher,CandidateStore, andActivePointerinterfaces stay simple (noSkillScopeparameter); the worker routes the file-backed implementations into the corresponding sub-directory based on the resolved scope. The default file implementations therefore gain multi-tenant isolation without any change.
Metrics
Read gate activity metrics via the ApprovalGateMetricsProvider interface (a
read-only snapshot that does not expose the internal worker):
Example
See examples/evolution/
for a complete runnable example demonstrating:
- Automatic skill extraction across multiple task rounds
- Cold-start to warm-start progression
- Quality gate metrics
- Custom review policy configuration