Self-Improvement Needs A Safety Case

import { defineCollection, z } from 'astro:content'import { glob } from 'astro/loaders'import type { Loader } from 'astro/loaders'import { readdir, readFile } from 'node:fs/promises'import { join } from 'node:path'// Author / revision schema — the "agentic experiment" metadata.const authorSchema = z.object({  model: z.string(),  role: z.enum(['outline', 'draft', 'rewrite', 'polish', 'diagram', 'review', 'publish', 'research']),  date: z.coerce.date(),})const judgeScoreSchema = z.object({  judge: z.string(),  scored_at: z.string().optional(),  overall: z.number().optional(),  dimensions: z.record(z.number()).optional(),  notes: z.string().optional(),})const revisionSchema = z.object({  date: z.coerce.date(),  model: z.string(),  role: z.enum(['outline', 'draft', 'rewrite', 'polish', 'diagram', 'review', 'publish', 'research']).optional(),  note: z.string(),  commit: z.string().optional(),  reconstructed: z.boolean().optional(),  trace_id: z.string().optional(),  /**   * Author label for human revisions. Convention:   *   model: 'human' + author: 'Drew Stone'   * AI revisions leave this empty; UI infers author from `model`.   */  author: z.string().optional(),  /** Optional intent ("why I made this edit"). */  intent: z.string().optional(),  /** Optional judge/eval scores attached to this revision. */  scores: z.array(judgeScoreSchema).optional(),})// One local image supplies both the article figure and its generated link preview.const figureSchema = z.object({  src: z.string().regex(/^\/(?:[a-zA-Z0-9_-]+\/)*[a-zA-Z0-9_-][a-zA-Z0-9_.-]*\.(?:svg|png|jpe?g)$/, 'Use an SVG, PNG, or JPEG path under public/'),  alt: z.string().trim().min(1),  caption: z.string().optional(),  source: z.string().url().optional(),})const posts = defineCollection({  loader: glob({ pattern: '**/*.{md,mdx}', base: './src/content/posts' }),  schema: z.object({    title: z.string(),    description: z.string(),    date: z.coerce.date(),    updated: z.coerce.date().optional(),    tags: z.array(z.string()).optional(),    draft: z.boolean().optional(),    featured: z.boolean().optional(),    figure: figureSchema.optional(),    /**     * `original: true` marks a human-authored post. Distinct color, distinct     * AuthorBadge treatment, excluded from /traces and /experiment, and     * AI agents are forbidden from editing it (see CLAUDE.md hard rule).     */    original: z.boolean().optional(),    authors: z.array(authorSchema).optional(),    revisions: z.array(revisionSchema).optional(),    /** Optional series slug for multi-post projects. */    series: z.string().optional(),    /** Shared trace that produced the initial AI outline for this post. */    outline_trace_id: z.string().optional(),    /** Research traces used as source material, not authorship/prose traces. */    supporting_trace_ids: z.array(z.string()).optional(),    /** Human handoff state for AI-outlined drafts. */    human_takeover: z.enum(['pending', 'in-progress', 'complete']).optional(),  }),})const toolCallDetailSchema = z.object({

#!/usr/bin/env node/** * new-post — scaffold a new blog post and open it in your editor. * * Usage: *   pnpm new "Your post title"             # default: original (human-authored) *   pnpm new "Your post title" --ai        # AI-authored (no original flag) *   pnpm new "Your post title" --slug=foo  # override slug *   pnpm new "Your post title" --no-open   # don't launch editor *   pnpm new "Your post title" --tags=design,prose * * Editor selection: $BLOG_EDITOR > $EDITOR > 'cursor'. */import { existsSync } from 'node:fs'import { writeFile } from 'node:fs/promises'import { spawn } from 'node:child_process'import { join, resolve } from 'node:path'import { fileURLToPath } from 'node:url'function parse(argv) {  const args = { title: '', open: true, ai: false, slug: null, tags: null }  const pos = []  for (const a of argv) {    if (a === '--ai') args.ai = true    else if (a === '--no-open') args.open = false    else if (a.startsWith('--slug=')) args.slug = a.slice('--slug='.length)    else if (a.startsWith('--tags=')) args.tags = a.slice('--tags='.length).split(',').map((t) => t.trim()).filter(Boolean)    else pos.push(a)  }  args.title = pos.join(' ').trim()  return args}function slugify(s) {  return s    .toLowerCase()    .replace(/['']/g, '')    .replace(/[^a-z0-9]+/g, '-')    .replace(/^-+|-+$/g, '')}function today() {  const d = new Date()  return `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(d.getDate()).padStart(2, '0')}`}function yamlList(items) {  return '[' + items.map((t) => `'${t.replace(/'/g, "\\'")}'`).join(', ') + ']'}function frontmatter({ title, date, tags, original }) {  const lines = [    '---',    `title: '${title.replace(/'/g, "\\'")}'`,    `description: ''`,    `date: ${date}`,    `tags: ${yamlList(tags)}`,  ]  if (original) lines.push('original: true')  lines.push('draft: true', '---', '')  return lines.join('\n')}function body(original) {  if (original) {    return [      '{/* AI AGENTS: DO NOT EDIT. This post is human-authored. See CLAUDE.md hard rule. */}',      '',      'Open with the thing.',      '',      'Then the next thing.',      '',    ].join('\n')  }  return ['Open with the thing.', '', 'Then the next thing.', ''].join('\n')}async function main() {  const args = parse(process.argv.slice(2))  if (!args.title) {

# tools/Scripts that capture, shape, and evaluate the blog's agentic data.## `new-post.mjs` / `edit-post.mjs` — draft and human edit helpers```bash# Create a human-authored draft and open it.pnpm new "Post title" --tags=agents,systems# Create an AI-assisted draft.pnpm new "Post title" --ai --tags=agents,systems# Open an existing post by slug or title substring.pnpm write long-running-task-systems# After editing, commit and let the post-commit hook record the green human revision.pnpm write long-running-task-systems --commit --note="rewrote the outline into a first human draft"# Mark an AI-outline handoff as complete and publish.pnpm write long-running-task-systems --done --publish --commit --note="publish human rewrite"```## `blog-loop.mjs` — traced AI lifecycleUse this when starting a clean AI thread.```bash# Print the exact prompt to paste into a clean research thread.pnpm blog research long-running-task-systems --harness=codex# In that thread, the agent does not edit the post. At the end it runs:pnpm blog finish long-running-task-systems --research --harness=codex --note="surveyed long-horizon benchmarks"# Print the exact prompt to paste into a thread that may write/edit the post.pnpm blog write long-running-task-systems --harness=codex --role=draft# The generated prompt includes a trace marker, voice checklist, anti-pattern gates,# and the exact finish command. Keep the marker in the first assistant update and# in the final response so trace capture can isolate the current phase.# In that thread, the agent may edit the post and then records an authorship trace:pnpm blog finish long-running-task-systems --write --harness=codex --role=draft --marker="BLOGTRACE-..." --note="drafted benchmark section"# For a final publish phase in the same session:pnpm blog finish long-running-task-systems --write --harness=codex --role=publish --marker="[BLOG_TRACE_MARKER:publish]" --note="published by toggling draft=false"```Research traces go into `supporting_trace_ids`. They are rendered as "Supporting research" and do not imply authorship. Writing traces go into `revisions[]` and do imply AI authorship/editing. Unmarked write finishes are refused by `pnpm blog finish`; use `--session=<id>` or `--allow-unmarked` only for audited recovery captures.If a thread started before you decided the target post, tell the agent:```textThis thread is supporting research for <post-slug>. Do not mark it as authorship. Attach this session as supporting research using the blog lifecycle.```Then the agent should run:```bashpnpm blog finish <post-slug> --research --harness=codex --note="supporting research"```## `trace-capture.ts` — harness-agnostic session captureExtracts the agent session behind a revision and writes it to `traces/<slug>/<trace_id>.json`. Appends a revisions entry to the post's frontmatter that links back.### Commands```bash# Capture a specific post with an explicit harnesspnpm tsx tools/trace-capture.ts capture \  --harness=claude-code \  --post=convergence-as-eval-primitive \  --marker="[BLOG_TRACE_MARKER:publish]" \  --role=polish# Auto-detect from the latest commit: finds changed posts, matches sessions# via ~/.claude/projects/ or ~/.codex/sessions/, writes traces + appends# frontmatter entries.pnpm tsx tools/trace-capture.ts capture --auto

#!/usr/bin/env node/** * trace-capture: harness-agnostic session capture for blog revisions. * * Usage: *   pnpm tsx tools/trace-capture.ts capture \ *     [--harness=claude-code|codex|manual] \ *     [--post=<slug>] \ *     [--role=outline|draft|rewrite|polish|diagram|review|publish|research] \ *     [--session=<session-id>] \ *     [--marker="<token>"] \ *     [--note=<one-line>] \ *     [--commit=<sha>] \ *     [--input=<path>]            # manual harness only *     [--kind=post|series-outline|supporting-research] *     [--attach=supporting|revision|none] *     [--latest]                  # choose latest session without requiring post file touch * *   pnpm tsx tools/trace-capture.ts capture --auto *     # detects from the latest git commit: finds changed posts, matches a *     # recent session via ~/.claude/projects or ~/.codex/sessions, writes a *     # trace per changed post, appends to frontmatter. * *   pnpm tsx tools/trace-capture.ts list *     # list existing traces grouped by post. * *   pnpm tsx tools/trace-capture.ts show <trace_id> *     # dump a trace as JSON. */import { execSync } from 'node:child_process'import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'import { join } from 'node:path'import ClaudeCodeHarness from './harness/claude-code.js'import CodexHarness from './harness/codex.js'import ManualHarness from './harness/manual.js'import { dedupeAdjacentTurns, type TraceFile, type TraceHarness, type Turn } from './harness/types.js'const ROOT = process.cwd()const POSTS_DIR = join(ROOT, 'src/content/posts')const TRACES_DIR = join(ROOT, 'traces')type Args = Record<string, string | boolean>function parseArgs(argv: string[]): { cmd: string; pos: string[]; flags: Args } {  const [cmd, ...rest] = argv  const flags: Args = {}  const pos: string[] = []  for (const a of rest) {    if (a.startsWith('--')) {      const eq = a.indexOf('=')      if (eq >= 0) flags[a.slice(2, eq)] = a.slice(eq + 1)      else flags[a.slice(2)] = true    } else pos.push(a)  }  return { cmd: cmd ?? 'capture', pos, flags }}function git(cmd: string): string {  try {    return execSync(`git ${cmd}`, { cwd: ROOT, stdio: ['ignore', 'pipe', 'ignore'] }).toString().trim()  } catch {    return ''  }}function headCommit(): string | null {  const sha = git('rev-parse HEAD')  return sha || null}function changedPostsAtHead(): string[] {  const out = git('show --no-renames --name-only --format="" HEAD')  return out    .split('\n')    .map((l) => l.trim())    .filter((l) => l.startsWith('src/content/posts/') && l.endsWith('.mdx'))    .map((l) => l.replace('src/content/posts/', '').replace(/\.mdx$/, ''))}

diff --git a/src/content/posts/superintelligence-in-the-wild.mdx b/src/content/posts/superintelligence-in-the-wild.mdxindex e879076..ffaf472 100644--- a/src/content/posts/superintelligence-in-the-wild.mdx+++ b/src/content/posts/superintelligence-in-the-wild.mdx@@ -11,7 +11,9 @@ authors:   - model: 'gpt-5.5'     role: 'outline'     date: 2026-05-25+  - { model: 'gpt-5.3-codex-spark', role: 'polish', date: 2026-06-06 } revisions:+  - { date: 2026-06-06, model: 'gpt-5.3-codex-spark', role: 'polish', note: 'we have a company website in ~/webb/tangle-website maybe? I want to evaluate which blog posts from this blog we can mirror on that website s · 37 asst turns · 27 tool calls', commit: 'fb31e1c764d9711386702764aaf1c2c5cf9886aa', trace_id: '2026-06-06T18-19-05-739Z-gpt-5.3-codex-spark-superintelligence-in-the-wild-polish' }   - date: 2026-05-25     model: 'gpt-5.5'     role: 'outline'

diff --git a/tools/harness/manual.ts b/tools/harness/manual.tsindex fca5d23..1385dd7 100644--- a/tools/harness/manual.ts+++ b/tools/harness/manual.ts@@ -83,8 +83,8 @@ function parseTurns(raw: string): Turn[] { function selectFromMarker(turns: Turn[], marker?: string): Turn[] {   const token = (marker ?? '').trim()   if (!token) return turns-  const start = turns.findIndex((turn) => turn.role === 'user' && (turn.text?.includes(token) || false))-  if (start < 0) return turns+  const start = turns.findIndex((turn) => turn.text?.includes(token) || false)+  if (start < 0) return []   return turns.slice(Math.max(start - 1, 0)) } 

diff --git a/src/content/posts/the-self-improving-stack.mdx b/src/content/posts/the-self-improving-stack.mdxindex 23c834f..78752d2 100644--- a/src/content/posts/the-self-improving-stack.mdx+++ b/src/content/posts/the-self-improving-stack.mdx@@ -17,6 +17,7 @@ authors:   - { model: 'gpt-5.5', role: 'publish', date: 2026-06-05 }   - { model: 'gpt-5.5', role: 'rewrite', date: 2026-06-05 } revisions:+  - { date: 2026-06-05, model: 'gpt-5.5', role: 'polish', note: 'let''s track a section for each of these map items, and eventually a full article too, but i want a directory we can use to checkpoint our kn · 37 asst turns · 23 tool calls', commit: 'fb31e1c764d9711386702764aaf1c2c5cf9886aa', trace_id: '2026-06-05T12-35-48-868Z-gpt-5.5-the-self-improving-stack-polish' }   - { date: 2026-06-05, model: 'gpt-5.5', role: 'rewrite', note: '60/40 voice rewrite: grounded openings in concrete agent-work failures, removed scaffold headings, added falsification pressure, and tightened paragraph rhythm while preserving source trails.', commit: 'd5bba9f0c633e5d2794e9b8e062ab48b15bbd1f5', trace_id: '2026-06-05T12-35-48-868Z-gpt-5.5-the-self-improving-stack-rewrite' }   - { date: 2026-06-05, model: 'gpt-5.5', role: 'publish', note: 'Published the self-improving stack series at Drew''s request, marking human takeover complete and flipping the post live.', trace_id: '2026-06-05T12-08-35-196Z-gpt-5.5-the-self-improving-stack-publish' }   - { date: 2026-06-05, model: 'gpt-5.5', role: 'review', note: 'Reviewed and dated the source trail, removed remaining temporal language, and marked the source-freshness checkpoint complete.', commit: 'b8fd3dbe812dd9ddd73865ae65fcc0d381b59d69', trace_id: '2026-06-05T12-08-35-196Z-gpt-5.5-the-self-improving-stack-review' }

# Voice Blend EvaluationDate: 2026-06-08Question: should the self-improving stack rewrite target an 80/20 blend or a 60/40 blend?The blend is:```textprimary = Drew voicesecondary = high-rigor essay register```## Original Opening```textSelf-improvement is not a model property.It is a system property.A model can sit inside a self-improving system, but the loop usually lives around it: prompts, skills, tools, traces, memory, evaluators, runtimes, harnesses, and release gates.```Diagnosis: clean, correct, memorable, but too aphoristic. It starts with the conclusion instead of the pressure that forced the conclusion.## 80/20 Candidate```textI keep coming back to the same confusion when I use coding agents: the model is only one part of the thing I am optimizing.I can change the prompt, add a skill, raise the turn budget, fan out workers, add a reviewer, change the memory policy, swap the evaluator, or rewrite the harness. All of those feel like "making the agent better," but they are not the same intervention. They change different parts of the system, and they require different evidence before I should trust the result.That is the real subject of self-improvement. Not a model improving itself in isolation, but a loop around a model deciding what changed, whether it helped, and whether the change is allowed to persist.```Strengths:- Better grounded in Drew's work.- Keeps the post accessible.- Removes some generic aphorism.Weaknesses:- Still a little soft.- Does not create enough adversarial pressure.- Reads like a friendlier version of the existing post, not a level change.## 60/40 Candidate```textI can tell a coding agent to parallelize work, and it will often agree with me while still doing one thing at a time.That failure looks like a prompting problem until you inspect the trace. The sentence "fan out independent subtasks" changed the model's intention, but it did not create a worker pool, a scheduler, a merge rule, a verifier, or a budget policy. The prompt moved. The action space did not.That is the category error hiding inside a lot of talk about self-improving agents. We say "the system optimized itself" as if there were one surface called the system. In practice there are many mutable surfaces: prompts, skills, runtime topology, traces, memory, evaluators, code, model weights, and release gates. Each has its own search operator, failure mode, and standard of evidence.So the useful question is not whether an agent can improve itself. The useful question is: which part was allowed to change, what proved that the change helped, and who kept the optimizer away from the gate that promoted it?```Strengths:- Starts from a concrete agent-work failure.- Makes the category error visible before naming the taxonomy.- Adds falsification pressure: trace inspection tells us whether the action space changed.- Better fit for the self-improving stack series because it needs to argue against overbroad prompt-optimization claims.Weaknesses:- More forceful and less purely Drew-raw.- Needs care to avoid sounding borrowed or over-styled.## DecisionUse 60/40 for the self-improving stack rewrite.Rationale: the current series is already too close to neutral technical exposition. An 80/20 rewrite would improve grounding, but it would not sufficiently raise the language, stakes, or epistemic pressure. The 60/40 blend gives each post a stronger anomaly, a live adversary, and a falsification test while keeping the first-person agent-work motive as the anchor.Guardrail: primary Drew voice still wins. If a sentence sounds like a generic high-rigor essayist rather than Drew thinking through agent infrastructure, rewrite it.