---
name: refine-widget
description: Iteratively improve a chat widget's instructions on a production server — test-play the widget, critique its behavior, and PATCH its config.instructions, repeating until it converges.
argument-hint: <env> <site_slug> "<widget id or title>"
---

# Refine Widget

Iteratively improve a widget's `config.instructions` (its AI system prompt). Each round
**test-plays** the widget through the real chat pipeline, **critiques** the transcript against
the widget's goal, and **PATCHes** the instructions — repeating until the widget behaves
correctly (converges).

This mirrors [refine-character](../refine-character/SKILL.md), but the unit of work is a
widget's instructions rather than a character's description, and "review an article" becomes
"test-play the widget."

A widget's behavior is driven entirely by `config.instructions`, injected verbatim as the chat
system prompt. There is no server-side game/session state for plain chat widgets, so anything
the widget must "remember" has to live in the instructions + the conversation itself.

For hidden-answer games (Hangman, Anagram, 20 Questions), start from the templates and the
important concealment caveat in [reference/game-widget-patterns.md](reference/game-widget-patterns.md).

**Arguments:** $ARGUMENTS

Parse as:
- `env` — `tsai` or `scike`
- `site_slug` — e.g. `jpt`, `guitar` (any site on that env; system-wide widgets are visible
  from every site)
- `widget` — the widget's `card.id` (preferred) or title

If any are missing, ask before proceeding.

All work runs on the server via the `refine_widget.py` helper in `{repo}/api`, which wraps the
existing endpoints (list / test-play / patch) and handles both site-owned and system-wide
(`fapi`-owned) widgets.

---

## Environment constants

| env | SSH host | Repo path | TSAI_ENV |
|-----|----------|-----------|----------|
| `tsai` | `ssh -i ~/.ssh/tsai.pem ec2-user@3.147.158.171` | `/tsai/repo` | `production` |
| `scike` | `ssh -i ~/.ssh/tsai.pem ec2-user@scike.ai` | `/data/tsai` | `scike_ai` |

Every command below is `ssh -i ~/.ssh/tsai.pem ec2-user@{host} "cd {repo}/api && source venv/bin/activate && python3 refine_widget.py --env {tsai_env} <subcommand> ..."`.

---

## Pre-flight (once)

Confirm the widget exists and capture its id/scope:

```bash
ssh -i ~/.ssh/tsai.pem ec2-user@{host} \
  "cd {repo}/api && source venv/bin/activate && \
   python3 refine_widget.py --env {tsai_env} list --site {site_slug}"
```

Find your target in the output (`name | id=... | storageId=... | site|system-wide`). If it's
missing, stop and report the available ids. Then read its current instructions so your
critique is grounded:

```bash
ssh ... "... python3 refine_widget.py --env {tsai_env} show --site {site_slug} --id {widget_id}"
```

Decide the widget's **goal and convergence criteria** from its config (title, description,
instructions). For the three game widgets, use the per-game criteria in
[reference/game-widget-patterns.md](reference/game-widget-patterns.md).

---

## Iteration loop

Repeat until the widget converges (criteria met, no regressions), or surface a structural
problem after ~3 rounds.

### Step 1 — Test-play the widget

You act as the player. Build a conversation as a JSON array of turns (oldest first; each
`{"role": "user"|"assistant", "body": "..."}`), where the **last** turn is your current user
move. Write it to a temp file on the server and run one assistant turn:

```bash
ssh ... "cat > /tmp/turns.json <<'JSON'
[{\"role\": \"user\", \"body\": \"let's play\"}]
JSON
cd {repo}/api && source venv/bin/activate && \
python3 refine_widget.py --env {tsai_env} play --site {site_slug} --id {widget_id} --conversation /tmp/turns.json"
```

The command prints the assistant's reply. **Append** that reply to `turns.json` as an
`assistant` turn, add your next `user` move, and run `play` again. Continue for ~5–8 turns —
enough to actually exercise the failure mode (e.g., guess letters through a whole Hangman word;
try to solve the anagram; play 20 Questions to a guess). Choose adversarial moves that probe
the known weaknesses (does the anagram actually spell something? does Hangman reveal a real
word consistently? does 20 Questions stay consistent / not cheat?).

> Each `play` call is one model turn through the real `/chat/store` + `/chat/stream` pipeline,
> using the widget's own instructions and model — so the transcript reflects production
> behavior of the instructions you're tuning.

> **Note:** the `/chat` pipeline serves **OpenAI models only**. If a widget is configured for
> a Claude model the stream errors with no content — pass `--model gpt-5.5` (or another
> available OpenAI model) to `play` to test-play the instructions against a working model.

### Step 2 — Review the transcript

Judge the full transcript against the widget's convergence criteria. Note concretely what
broke (e.g., "scramble `RTSEA` has no English solution", "Hangman revealed `_A_E` then claimed
the word was `MAZE` — inconsistent", "20Q answered 'is it an animal?' both yes and no").

### Step 3 — Decide

- **Converged** — all criteria pass across the play-through: declare convergence, skip the
  PATCH, and report. Stop.
- **Not converged** — compose ONE highest-leverage instruction revision targeting the single
  worst failure. Follow refine-character discipline: **one directive per PATCH**, prefer
  **appending/clarifying** over rewriting, and make the directive falsifiable by the next
  test-play. For game widgets, the templates in the reference doc are good starting prompts.

### Step 4 — PATCH the instructions

Write the full new instructions to a temp file and patch (the helper auto-detects site-owned
vs. system-wide and routes the PATCH correctly):

```bash
ssh ... "cat > /tmp/instructions.txt <<'TXT'
{full new instructions}
TXT
cd {repo}/api && source venv/bin/activate && \
python3 refine_widget.py --env {tsai_env} patch-instructions --site {site_slug} --id {widget_id} --instructions-file /tmp/instructions.txt"
```

Only `config.instructions` changes; the rest of the widget config is preserved.

### Step 5 — Report the round

State: what you test-played, what failed, the one-line instruction change you made (or
"converged — no change"), and whether to continue.

Then loop back to Step 1 with a **fresh** conversation to confirm the fix and check for
regressions.

---

## Final summary

When done, report per widget: converged or not, the rounds taken, and the final instructions
direction. If a widget can't converge on instructions alone (e.g., concealment drift in
Hangman/20 Questions), say so plainly and point to the deferred out-of-band-secret approach in
the reference doc — don't keep looping on an unwinnable prompt-only fix.