Human in the loop

Let an agent ask a human, and guarantee it never gets stuck waiting

An agent driving an environment sometimes needs a person: to confirm a deploy, to pick between two plans, to say whether a diff looks right. Fuse gives that a first-class shape. The agent raises a request, a human answers it from the CLI or the API, and the answer goes back to the agent. Every request has a deadline, so whatever the human does (or does not do), the agent always gets an outcome and carries on.

This page covers the core loop and the four request kinds an agent can raise: ask for a decision, review for a verdict on a file or a diff, secret for a credential, and takeover to hand a human the shell or the desktop. Then gates that hold exec, lifecycle and computer actions until a human decides, deciders that answer for the human when they can, the stall and expiry signals Fuse raises on its own, and the access model around all of it: who may answer, how a person is named, the audit log and webhooks.

Concepts

An intervention is one request for a human. It has an id, belongs to an environment, and moves exactly once from pending to a terminal state:

Every terminal intervention carries a decision, so the agent only ever reads decision.outcome (and decision.answer for an approved ask, or decision.reason for anything else). Expiry and cancellation are decided by the actor system.

Invariants

These are what keep an agent from getting stuck:

1. Every intervention has a deadline. The default is 10 minutes, the range is 10s to 24h. There are no unbounded waits.
2. Every pending intervention reaches a terminal state, and a waiting caller always sees it. A read past the deadline expires the intervention on the spot; a sweep on the reconcile loop catches the rest.
3. Destroying an environment cancels its pending interventions.
4. A pending intervention keeps its environment alive. The idle reaper (resources.idle_timeout) skips an environment while a human is deciding.
5. State lives in Postgres, not in the connection. An orchestrator restart or a dropped client loses nothing: the caller polls the same id again and gets the decision.
6. A rejection always carries a reason the agent can read and replan on.

A migrate carries pending interventions to the new environment id. A fork inherits the policy but starts with no pending interventions.

Declaring it

Add a hitl: block to the Fusefile. Declaring a request kind, even as {}, is what allows it; without the block every request is refused.

version: 1
run: ./agent.sh

hitl:
  timeout: 10m        # default deadline, 10s to 24h
  on_timeout: reject  # or allow
  requests:
    ask: {}

on_timeout: allow resolves an expired ask as approve with an empty answer. Use it only when “nobody objected” really is a yes.

The block is enforced by the orchestrator; nothing of it goes into the guest. fuse compile shows it under hitl in the create request, and raw API callers send the same shape as hitl on POST /v1/environments:

{
  "hitl": {
    "timeout_seconds": 600,
    "on_timeout": "reject",
    "requests": { "ask": {} }
  }
}

Asking

The agent raises an ask with a question and, optionally, a fixed set of choices. A per-request timeout can shorten the deadline, never extend it. The SDKs wait for the outcome by default.

in, err := client.Environments.RequestHuman(ctx, envID, fuse.HumanRequest{
	Question: "Deploy build 812 to prod?",
	Choices:  []string{"yes", "no"},
})
if err != nil {
	return err
}
if in.Approved() && in.Decision.Answer == "yes" {
	deploy()
} else {
	log.Printf("not deploying: %s", in.Decision.Reason)
}

Set NoWait: true to get the pending intervention back at once and wait on it later with client.Interventions.Wait(ctx, in.ID). The other SDKs have the same call: requestHuman in TypeScript, request_human in Python and Rust.

Under the hood this is two calls:

# raise: 201 with the pending intervention
curl -X POST "$FUSE/v1/environments/$ENV?action=request" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"kind":"ask","question":"Deploy build 812 to prod?","choices":["yes","no"]}'

# wait: long-polls up to 50s, always 200, loop while state is pending
curl "$FUSE/v1/interventions/$ID?wait=30s" -H "Authorization: Bearer $TOKEN"

An undeclared kind, or an environment created without a hitl: block, is a 400 that says so.

Answering

From the terminal:

fuse hitl list                          # pending, across every environment
fuse hitl list --env fuse-agent --state all
fuse hitl show int_0187...
fuse hitl answer int_0187... yes
fuse hitl reject int_0187... --reason "build 812 failed QA, wait for 813"

From the API, POST /v1/interventions/{id}?action=approve with {"answer": "yes"}, or ?action=reject with {"reason": "..."}. The SDKs expose this as interventions.approve and interventions.reject.

An answer must be one of the ask’s choices when it has any. Deciding an intervention that is already resolved, including one whose deadline passed a moment before the decision arrived, is a 409.

Deciding requires the hitl:decide scope or the master token. A full-access API key can raise a request but cannot answer one, so an agent driving Fuse with its own key cannot approve itself. Give the human a scoped key instead, as below.

Reviews

A review asks a human for a verdict on something the agent produced: a patch, a migration, a config file, a draft. The human approves or rejects it, and their comments go back to the agent either way.

hitl:
  requests:
    review: {}

The agent sends exactly one of:

• path: an absolute, clean guest path. Fuse reads the file at request time, up to 1 MiB, so the human reviews what the agent submitted even if the file changes afterwards. A longer file is cut at 1 MiB and the review is marked truncated. A file Fuse cannot read is a 400.
• content: the text inline, at most 1 MiB.

format is text or diff. Left empty, Fuse guesses: diff when the content starts like a unified diff (diff --git, or --- followed by a +++ line) or the path ends in .diff or .patch, else text. message is the agent’s note to the reviewer.

r, err := client.Environments.RequestReview(ctx, envID, fuse.ReviewRequest{
	Path:    "/work/migrations/0042_add_index.sql",
	Message: "adds the index the slow query log asked for; safe to run online?",
})
if err != nil {
	return err
}
if !r.Approved {
	return fmt.Errorf("review rejected: %s", r.Comments)
}
log.Printf("approved, reviewer says: %s", r.Comments)

TypeScript has the same call as requestReview, Python and Rust as request_review. From the API it is the ordinary request:

curl -X POST "$FUSE/v1/environments/$ENV?action=request" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"kind":"review","path":"/work/migrations/0042_add_index.sql","message":"safe to run online?"}'

The intervention’s payload.review holds what is up for review:

"payload": {
  "review": {
    "path": "/work/migrations/0042_add_index.sql",
    "content": "CREATE INDEX CONCURRENTLY ...",
    "format": "text",
    "message": "safe to run online?"
  }
}

The human reads it with fuse hitl show, which prints the content under the details, and decides with a reason as the comments:

fuse hitl show int_0187...
fuse hitl approve int_0187... --reason "fine, but run it outside business hours"
fuse hitl reject int_0187... --reason "CONCURRENTLY cannot run inside a transaction; split it out"

The comments are decision.reason on both outcomes. A review takes no answer, edited_argv, edited_text or grant. Like every intervention payload, the content is encrypted at rest with TOKEN_ENCRYPTION_KEY.

Gates

A gate holds an action Fuse relays until a human decides, records it, or refuses it. Three surfaces have gates: exec (SDK and CLI exec), lifecycle (environment API actions) and computer (the computer actions a computer-use agent sends).

hitl:
  timeout: 10m
  gates:
    - name: force-push
      on: exec
      match: { program: git, args: [push, --force] }
      grant: ttl:30m          # an approval covers repeats for 30m
      timeout: 2m

    - name: watch-shell
      on: exec
      match: { shell: true }
      mode: notify            # let it through, record it

    - name: no-self-destruct
      on: lifecycle
      match: { action: [destroy, migrate] }
      mode: deny
      reason: this environment is not the agent's to destroy

Gate reference

Every field set in match must match. When several gates match one action, deny beats ask beats notify; each matching notify gate still leaves a record. Start a new gate as notify, read what it would have caught, then tighten it to ask.

Wrappers. env (and its VAR=x arguments), sudo, nice, timeout, xargs, nohup and stdbuf are stripped before matching, so env GIT_TRACE=1 git push --force and sudo git push --force are both git push --force to a gate.

What a held call sees. The API answers 202 with {"intervention_id", "intervention"}. When a human approves, the orchestrator performs the action itself, exactly as approved, and stores the outcome on the intervention’s result: result.exec for an exec, result.computer for a computer action, result.env_id for a fork or migrate, result.snapshot_id for a snapshot, result.error when the approved action failed. GET /v1/interventions/{id}?wait= returns once that result is in. A deny answers 403 with details.reason: hitl_denied, the gate in details.gate, and the gate’s reason as the message.

The SDKs wait on a 202 for you: exec returns the approved command’s result, computer the action’s output and screenshot, fork the new environment, and so on. A rejection raises InterventionRejectedError with the reason; wait: false (or the client option) raises PendingInterventionError with the handle instead of waiting. The CLI prints the intervention it is waiting on:

fuse environment exec vm-1 -- git push --force origin main
# held by hitl gate force-push: waiting on intervention int_...

Approving, editing and granting

fuse hitl approve int_...                                  # run it as asked
fuse hitl approve int_... --edit -- git push --force-with-lease origin main
fuse hitl approve int_... --edit                           # edit argv in $EDITOR
fuse hitl approve int_... --text "staging.example.com"     # retype a held computer type
fuse hitl approve int_... --grant env                      # and don't ask again
fuse hitl reject  int_... --reason "push to a branch instead"

An edited argv is checked against the deny gates again before it is accepted. A grant lets identical repeats through without a prompt: the same gate and the exact same argv (or the same lifecycle action, or the exact same computer action body), for the environment’s lifetime (env) or a duration (ttl:30m). A different argv still prompts. Grants follow a migrate, are not copied to a fork, are dropped on destroy, and are audited (grant_created, and grant_used each time one lets an action through). An edit cannot carry a grant, since the grant would cover the original argv the human did not approve as written.

Computer gates

A computer-use agent drives the desktop one action at a time through POST /v1/environments/{id}/computer. A computer gate holds, records or refuses those actions by their name, and a type action by its text:

hitl:
  gates:
    - name: watch-typing
      on: computer
      match: { action: type }
      mode: notify              # record everything the agent types

    - name: prod-host
      on: computer
      match: { text_contains: [prod.example.com] }   # implies action: type
      timeout: 5m

    - name: no-drag
      on: computer
      match: { action: [left_click_drag, left_mouse_down] }
      mode: deny
      reason: drag and drop is off on this desktop

match.action takes the computer action names: screenshot, cursor_position, mouse_move, left_click, right_click, middle_click, double_click, triple_click, left_click_drag, left_mouse_down, left_mouse_up, type, key, hold_key, scroll, wait and zoom. match.text_contains matches a type action whose text contains any entry, ignoring case. It implies action: type, so it can only be combined with action: [type]; a gate that lists other actions as well is refused, and should be split in two.

ask, notify and deny work as they do for exec. A held action answers 202 with the intervention, and the SDKs wait on it. The human sees the action in payload.computer: its action, the text of a type, and input, the whole body the agent sent.

"payload": {
  "computer": {
    "action": "type",
    "text": "ssh deploy@prod.example.com",
    "input": { "action": "type", "text": "ssh deploy@prod.example.com" }
  }
}

On approval the orchestrator performs the action itself, with the body the agent sent, and stores what the guest answered (output and screenshot) in result.computer. The SDK returns that as the action’s result, so the agent’s loop carries on with a screenshot taken after the approved action, not before it.

The human can retype a held type before it runs:

fuse hitl approve int_0187... --text "ssh deploy@staging.example.com"

From the API that is {"edited_text": "..."} on ?action=approve, and in Go ApproveOptions.EditedText. It applies only to a type, cannot be empty (reject the action instead), and is checked against the deny gates again before it is accepted. Like an edited argv it cannot carry a grant: an explicit grant with it is a 400, and a gate’s default grant drops to once.

Why there are no coordinate rules. A gate cannot say “never click at (812, 430)”. Coordinates name a spot on the screen, not a thing: a window that moves, a banner that pushes the page down or a different resolution puts something else under the same pixel. Action types do not shift, so gates match those. To hold “clicking the Pay button”, gate left_click with a decider: it gets a screenshot of the display with the action, and can tell what is under the pointer.

Deciders

A gate in ask mode can put a decider in front of the human: a webhook Fuse asks first. It answers allow, deny or ask, and only ask reaches a person. Classifiers, model judges and business rules live there, so the human sees only what a rule could not settle.

version: 1
run: ./agent.sh

secrets:
  - decider_secret

hitl:
  gates:
    - name: push
      on: exec
      match: { program: git, args: [push] }
      decide:
        url: https://decider.example.com/fuse
        secret: decider_secret

fuse up --secret decider_secret="$(openssl rand -hex 32)"

decide goes on an ask gate on exec, lifecycle or computer. A notify or deny gate has nothing to decide, and a stall or expiry signal has no action to judge, so decide on any of those is refused. secret names an entry of the top-level secrets: list (in the create request, a key of secrets), and the create is a 400 without it. Its value signs each call the same way as webhook deliveries, and is sealed and kept the same way.

For each held action Fuse sends a POST with this JSON body (hitl.DeciderRequest):

{
  "env_id": "fuse-agent",
  "gate": "push",
  "on": "exec",
  "triggered_by": { "id": "key:ak_123", "source": "key", "key_id": "ak_123" },
  "exec": { "argv": ["git", "push", "origin", "agent/fix-flaky-test"] },
  "deadline": "2026-11-04T15:14:05Z"
}

Unlike a webhook delivery it carries the action itself, since judging it is the point: exec for an exec gate, lifecycle for a lifecycle gate, or computer (the same action, text and input as payload.computer) for a computer gate. A computer call also carries screenshot, a base64 PNG of the display as it is before the action, or nothing when one could not be taken within 5 seconds. deadline is when the intervention would expire if it went to a human.

The decider answers 200 with:

{ "decision": "deny", "reason": "push to main; open a pull request instead" }

Any failure is ask. The call has a fixed 5 second timeout (a computer call first spends up to 5 more taking the screenshot), and it is not retried. A timeout, a transport error, a non-2xx answer, a redirect (which is not followed), a body that is not JSON or names any other decision, and a signing secret the orchestrator no longer holds all hold the action for a human, with the error in payload.decider.error. A broken decider can only send more work to a person, never let an action through. The decider is asked only when no grant already covers the action, and an allow creates no grant.

Writing a decider

Verify the signature against the raw body before anything else, and answer quickly: the agent’s call is waiting. A minimal decider in Python, with the standard library only:

import hashlib
import hmac
import json
import os
import time
from http.server import BaseHTTPRequestHandler, HTTPServer

SECRET = os.environ["DECIDER_SECRET"].encode()


def verify(header: str, body: bytes, tolerance: int = 300) -> bool:
    parts = dict(p.strip().split("=", 1) for p in header.split(",") if "=" in p)
    t, v1 = parts.get("t"), parts.get("v1")
    if not t or not t.isdigit() or not v1 or abs(time.time() - int(t)) > tolerance:
        return False
    want = hmac.new(SECRET, t.encode() + b"." + body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(want, v1)


def decide(req: dict) -> dict:
    argv = (req.get("exec") or {}).get("argv", [])
    if "main" in argv or "--force" in argv:
        return {"decision": "deny", "reason": "push a branch and open a pull request instead"}
    if argv and argv[-1].startswith("agent/"):
        return {"decision": "allow"}
    return {"decision": "ask", "reason": "not an agent/ branch"}


class Decider(BaseHTTPRequestHandler):
    def do_POST(self):
        body = self.rfile.read(int(self.headers.get("Content-Length", 0)))
        if not verify(self.headers.get("Fuse-Signature", ""), body):
            # a non-2xx answer holds the action for a human
            self.send_response(401)
            self.end_headers()
            return
        out = json.dumps(decide(json.loads(body))).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(out)))
        self.end_headers()
        self.wfile.write(out)


HTTPServer(("0.0.0.0", 8080), Decider).serve_forever()

decision is read case-insensitively, reason is optional (an empty one becomes allowed by the decider of gate <name> or denied by the decider of gate <name>), and the answer is read up to 64 KiB.

Allowlist mode

Matching what to stop is a blocklist, and a blocklist misses whatever its author did not think of. For an agent you do not trust, list what may run instead:

hitl:
  exec:
    default: ask        # or deny
    allow: [ls, cat, grep, git, make]

Anything not in allow is held (or refused), and interpreters and shells are not special: python -c ..., bash -c ... and shell: execs are held unless their program is listed. Leave interpreters out of allow: allowing python allows everything. Gates still apply on top, so git can be allowed while git push --force stays gated.

A guardrail, not a boundary

Gates see what Fuse relays: API exec, the environment actions and API computer actions. They do not see what happens inside the guest. A process the agent starts there (a run: agent, a script, a shell it already holds through fuse environment shell) can do anything root can, with no gate involved. So:

• Gate the agent’s API access. When the agent runs outside the VM and only reaches it through exec, gates are enforced on everything it does.
• When the agent runs inside the guest, exec gates do not apply to its own commands, and computer gates do not see input it sends to the display itself (with xdotool, say). Lifecycle gates still do, since those actions only exist on the API.
• Gates match structure (argv), not strings. They cannot see inside a shell string, which is why shell is its own match and why allowlist mode holds shells.
• Deciding needs hitl:decide, which a full-access key does not hold, but the master token does. Exec currently needs the master token, so an agent that execs holds a credential that could also approve its own held execs. Lifecycle actions work with a full-access key; give the agent one of those wherever it does not need exec.
• A deny on destroy also stops you: the environment is then removed by its max_runtime or idle_timeout, which are not gated.

Secrets

Sometimes the agent needs a credential nobody gave it: a token to push a branch, a login for a private package mirror. A secret request carries it from a human into the guest without the value passing through the agent, an API response, Postgres or the logs.

Ask for credentials through secret, never during a takeover. A takeover shell is recorded, and whatever the guest prints, including a token pasted into a command line or typed at a prompt that echoes, ends up in the recording. A secret request writes the value to one declared place and nothing else ever sees it.

Declare every place a value may land. The agent names one of these targets when it asks; it can never pick a path or a variable of its own.

hitl:
  requests:
    secret:
      targets:
        - env: GH_TOKEN
          label: GitHub token with repo scope on acme/api
        - file: /root/.netrc
          label: netrc for the internal package mirror

Each target sets exactly one of env (a variable name) or file (an absolute guest path outside /fuse, which Fuse keeps for its own credentials), and a label. Up to 20 targets are allowed.

The agent raises the request with a target and, optionally, a message:

in, err := client.Environments.RequestHuman(ctx, envID, fuse.HumanRequest{
	Kind:    fuse.HitlKindSecret,
	Target:  "GH_TOKEN",
	Message: "git push to acme/api got a 403; need a token to open the PR",
})
if err != nil {
	return err
}
if !in.Approved() {
	return fmt.Errorf("no token: %s", in.Decision.Reason)
}
// in.Decision.Answer is "done". the value is in the guest, never here.

A target that is not declared is a 400. The agent gets back decision.answer: "done" and nothing more: the value is never stored, logged, returned or put on the event stream, and the intervention and its secret_answered audit row record only who answered and when.

Read the label first. The agent writes the message, so a confused or prompt-injected agent can write anything there, including “paste the AWS root key”. The label comes from the Fusefile, and the intervention’s payload.secret carries it apart from the agent’s text, so show it first:

"payload": {
  "secret": {
    "target": "GH_TOKEN",
    "label": "GitHub token with repo scope on acme/api",
    "message": "git push to acme/api got a 403; need a token to open the PR"
  }
}

The human answers from the CLI, which prints the label first, then the agent’s message, and reads the value without echoing it (or from stdin when it is not a terminal):

fuse hitl secret int_0187...

or from the API with POST /v1/interventions/{id}?action=secret and {"value": "..."}. The SDKs expose this as interventions.answerSecret in TypeScript, AnswerSecret in Go and answer_secret in Python and Rust:

await fuse.interventions.answerSecret(id, value);

Answering needs the hitl:secret scope, which a full-access key never holds (fuse hitl invite fuse-agent --scopes secret mints one), and the same no self-approval and require_actor rules as deciding. Two humans answering the same request at once get one write and one 409.

Where the value lands:

• env: every exec call the agent makes after the answer sees the variable. It is read per call, so a process that was already running, or an attach shell, does not.
• file: the file is created (with its parent directories) with mode 0600 before the value is written, replacing what was there.

Secret requests need TOKEN_ENCRYPTION_KEY on the orchestrator. Without it, creating an environment that declares hitl.requests.secret is a 400, and so is raising or answering a secret request.

Takeover

Sometimes the fastest fix is a person at the keyboard: a process wedged on a lock, a merge the agent keeps getting wrong, a tool that only works interactively. A takeover hands the environment to a human and keeps the agent out until they hand it back.

hitl:
  timeout: 30m          # how long a takeover may hold the environment
  requests:
    takeover:
      via: [terminal]   # the default; add desktop for the live desktop

via lists the ways in: terminal, a recorded shell, and desktop, the recorded live desktop. An empty list means terminal.

A takeover starts one of two ways. The agent asks for one, and its call waits:

in, err := client.Environments.RequestHuman(ctx, envID, fuse.HumanRequest{
	Kind:   fuse.HitlKindTakeover,
	Reason: "migration 0042 hangs on a lock and I cannot find the holder",
})
if err != nil {
	return err
}
log.Printf("human says: %s", in.Decision.Answer)

Or a human starts it. If the agent has a takeover request pending, this claims it (the agent may not claim its own); otherwise Fuse opens one on the human’s behalf:

fuse hitl takeover fuse-agent      # take control and open a recorded shell
fuse hitl takeover fuse-agent --via desktop   # or the recorded desktop
fuse hitl takeover fuse-agent --no-shell   # only take the lock
fuse hitl release fuse-agent --note "killed the stale psql session holding the lock; rerun 0042"

From the API, POST /v1/environments/{id}?action=takeover (body {"via": "desktop", "reason": "..."}, both optional) and ?action=release with {"note": "..."}. The SDKs expose environments.takeover and environments.release. Both need the hitl:takeover scope, which a full-access key never holds: mint one for the human, for example with fuse hitl invite fuse-agent --scopes takeover, or use the master token.

While a human holds the environment, the agent waits. Its exec calls and computer actions get a 423 before they reach the guest:

{
  "error": {
    "code": "locked",
    "message": "environment fuse-agent is locked by takeover intervention int_0187...; wait for it to resolve",
    "details": { "intervention_id": "int_0187...", "kind": "takeover" }
  }
}

The SDKs handle this for you: on a 423 they wait on that intervention until it is no longer pending, then send the call again, so an agent’s exec simply takes longer. Pass the same no-wait option as for a held gate to get a LockedError with the intervention id and kind instead. Raw API callers do the same loop with GET /v1/interventions/{id}?wait=50s.

The deadline bounds the lock. A takeover is an intervention like any other, so it has a deadline: hitl.timeout, or the agent’s shorter timeout_seconds. When it passes the lock lifts and the takeover expires. Set hitl.timeout to the longest takeover you want, since a request can only shorten it.

Release hands the note to the agent. It becomes the takeover’s decision.answer, so write it for the agent: what you changed and what it should do next. An empty note becomes released by <actor>. Anyone with hitl:takeover can release, not only the holder, so a takeover whose holder walked away does not have to wait out its deadline. A second human taking over an environment someone holds gets a 409; the holder taking over again gets the same takeover back.

The recorded shell

fuse hitl takeover opens the shell with GET /v1/interventions/{id}/attach, the attach stream with a recording on it. Only the takeover’s holder (or the master token) may open it, and only the Go SDK and the CLI speak it. Closing the shell ends that recorded session, not the takeover: the agent stays locked out until a release or the deadline.

The first thing the human sees is a banner:

[fuse] this session is recorded (takeover int_0187..., environment fuse-agent, as alice).
[fuse] ask for credentials through a secret request, never type them here.

The session is saved as an asciinema v2 cast: what the guest printed and each window resize, never the keystrokes. A password typed at a prompt that does not echo is not in it; anything the guest echoes back is. The cast is encrypted with TOKEN_ENCRYPTION_KEY (and stored in the clear, with a warning in the orchestrator log, without it), stops growing at 8 MiB, and is linked from a takeover_session audit row by its recording_id. Fetch it with hitl:read:

curl "$FUSE/v1/hitl/audit?env=fuse-agent" -H "Authorization: Bearer $TOKEN" \
  | jq -r '.entries[] | select(.event == "takeover_session") | .recording_id'

curl "$FUSE/v1/hitl/recordings/rec_0187..." -H "Authorization: Bearer $TOKEN" > session.cast
asciinema play session.cast

The SDKs fetch the same thing with interventions.getRecording in TypeScript, Recording in Go and get_recording in Python.

The recorded desktop

A takeover via desktop hands the human the environment’s live desktop, for the cases a shell cannot fix: a login page with a captcha, a dialog the agent keeps misreading, a drag it cannot get right. It needs a top-level desktop: block; a Fusefile (or a create request) that allows via: [desktop] without one is refused.

desktop:
  width: 1280
  height: 800

hitl:
  requests:
    takeover:
      via: [terminal, desktop]

fuse hitl takeover fuse-agent --via desktop takes the lock and opens the same browser viewer as fuse desktop (--no-open prints its URL instead, --listen picks where it listens). Closing the viewer keeps the takeover held, as closing the shell does; release it with fuse hitl release.

The viewer streams from GET /v1/interventions/{id}/desktop with Upgrade: fuse-vnc/1: the computer stream (raw RFB) with a recording on it. It needs hitl:takeover, only the takeover’s holder (or the master token) may open it, and the takeover must have been started with via: desktop. When the human claims an agent’s pending request, the via the human names wins over the one the agent asked for. In Go, Environments.TakeoverDesktop opens it.

While the takeover is held, the agent’s computer actions get the 423 above, and so does the plain GET /v1/environments/{id}/computer/stream, since input sent on it would get past the lock. fuse desktop therefore fails during a takeover; the holder’s viewer is the one way in.

The session is recorded as:

• an input-action log: each change of the pointer buttons, with its coordinates; named keys such as Return, Tab, Escape, the arrows and F1-F12, and chords such as ctrl+c; the number of printable keys typed, never which ones; and the size of each clipboard paste, never its content.
• a screenshot when the session starts and every 10 seconds after.

Like the terminal cast, it never holds what the human typed. The screenshots are what shows the work, so whatever is on the screen when one is taken is in the recording: ask for credentials through a secret request, not by typing them into the desktop. The recording is sealed with TOKEN_ENCRYPTION_KEY when the viewer closes, linked from a takeover_session audit row by recording_id, and stops taking screenshots at 32 MiB (the log is capped at 50,000 entries), marked truncated; the view keeps working.

GET /v1/hitl/recordings/{id} returns a desktop recording as application/json with Fuse-Recording-Kind: desktop; a terminal cast stays application/x-asciicast with Fuse-Recording-Kind: terminal. The JSON looks like this, with t in seconds since the session started:

{
  "version": 1,
  "kind": "desktop",
  "intervention_id": "int_0187...",
  "env_id": "fuse-agent",
  "actor": "alice",
  "started_at": "2026-11-04T15:04:05Z",
  "ended_at": "2026-11-04T15:09:41Z",
  "events": [
    { "t": 3.2, "type": "pointer", "x": 640, "y": 412, "buttons": 1 },
    { "t": 3.3, "type": "pointer", "x": 640, "y": 412, "buttons": 0 },
    { "t": 4.1, "type": "text", "chars": 12 },
    { "t": 5.0, "type": "key", "key": "Return" },
    { "t": 9.7, "type": "clipboard", "bytes": 38 }
  ],
  "screenshots": [
    { "t": 0.0, "png": "iVBORw0KGgo..." }
  ]
}

truncated is set when a cap was hit. input_log_incomplete is set when the viewer sent something the log does not understand, after which no more input was logged; the screenshots carry on.

Signals

Gates fire when the agent does something. Signals fire when Fuse notices something on its own: the agent looks stuck (stall), or the environment is about to be reaped (expiry). They are declared as gates:

hitl:
  gates:
    - name: failing-loop
      on: stall
      match: { failures: 3 }   # three non-zero exec exits in a row

    - name: gone-quiet
      on: stall
      match: { quiet: 15m }    # no exec or computer call for 15m
      mode: notify

    - name: about-to-reap
      on: expiry
      match: { before: 5m }    # 5m before idle_timeout or max_runtime

In the create request the durations are seconds: quiet_seconds and before_seconds. A signal gate is ask (the default) or notify, never deny, takes no grant, and matches only its own fields.

Stall

A gate can set both. In ask mode a stall pauses the agent: its exec and computer calls get the same 423 as a takeover, with kind: "stall", and the SDKs wait the same way. A human acks it with a plain approve or reject; either one lifts the lock and the agent’s waiting call goes through. In notify mode the stall is only recorded, born resolved, and nothing pauses. No stall is raised while the environment is already locked, and the lock lifts at the stall’s deadline (timeout on the gate, else hitl.timeout) like any other.

fuse hitl show int_0187...      # payload.signal: failures and last_exit_code, or quiet_seconds
fuse hitl approve int_0187...   # let the agent carry on

Expiry

before raises an expiry this long ahead of the reaper that is due first: resources.idle_timeout, or max_runtime for an environment running a task. It is 1m to 24h. payload.signal names the reaper and its reap_at.

In ask mode the reaper waits while the expiry is pending. What happens next is up to the human:

• Approve keeps the environment alive. With no duration that is one more full window of the reaper that was due (at most 24h); --extend picks one, 1m to 24h. The decision’s answer reads extended until <time>, and payload.signal.extended_until holds it. The extension survives an orchestrator restart, and the gate fires again ahead of the next reap.
• Reject lets the reap go ahead.
• Nobody answering does too. An expiry’s deadline is capped at the reap time, and it always times out as a reject, whatever on_timeout says.

fuse hitl approve int_0187... --extend 30m

From the API, approve with {"extend_seconds": 1800}; in the SDKs, extendSeconds in TypeScript and ExtendSeconds in Go’s ApproveWith. extend_seconds on anything other than an expiry approval is a 400. A notify expiry only records the warning; the reaper does not wait for it.

Access

Fuse has no users or roles. What a human needs to handle one environment’s requests is a short-lived, narrow credential, so that is what Fuse mints: an API key with scopes.

A key minted without scopes is a full-access key, exactly as every key was before scopes existed. It holds hitl:read, but never hitl:decide, hitl:secret, hitl:takeover or hitl:act-as: those are only ever minted on purpose. That is what keeps an agent’s own key from approving its own requests, and keeps a takeover shell as rare as attach, which is master-only.

A key minted with scopes:

• reaches only the hitl routes (/v1/interventions, /v1/hitl/*, and minting a narrower key). Anything else is a 403 with reason env_scope.
• may be limited to one environment. It then covers that environment and every environment forked from it, never a parent or a sibling, and gets a 404 on any other environment, as if it did not exist. A migrate moves the key to the new environment id.
• must expire, at most 24 hours after it is minted. An expired key is a 401.
• is revoked when its environment is destroyed.
• may be pinned to an actor (see Actors).

The quickest way to mint one is fuse hitl invite. It defaults to --scopes decide and --ttl 1h, and prints the secret once:

fuse hitl invite fuse-agent --scopes decide --ttl 2h --actor alice

Hand the key to Alice; with it she can list, read and decide the requests of fuse-agent and its forks, and nothing else. fuse apikeys create mints the same thing with every option spelled out:

fuse apikeys create --scopes read,decide --env fuse-agent --ttl 2h \
  --actor alice --actor-name "Alice Ng" --label alice-oncall

From the API, POST /v1/api-keys with {"scopes": ["hitl:decide"], "env_id": "fuse-agent", "ttl_seconds": 7200, "actor": {"id": "alice"}}.

Minting cannot escalate. The master token can mint anything. A scoped key can mint a narrower copy of itself: only scopes it holds, only its own environment or one of its forks, expiring no later than it does, and for its own pinned actor (pinning an actor from an unpinned key needs hitl:act-as). Anything wider is a 403 with reason mint_escalation. A full-access key cannot mint at all. Every mint of a scoped key, and its first use, is an audit row.

Actors

A key answers “is this call allowed”. An actor answers “which person is this”: an opaque id (at most 128 bytes, like user_42, alice@corp.com or slack:U0123) and an optional display name. Fuse records it on every intervention and audit row and never checks it. Whoever builds on Fuse maps their own auth onto it.

It travels as two headers on any request:

Fuse-Actor: alice@corp.com
Fuse-Actor-Name: Alice Ng

The server resolves who a request is recorded as from the key and the headers:

There are two common setups. A small team uses fuse hitl invite --actor, so every request on Alice’s key records as alice with no integration at all. A product built on Fuse holds one hitl:act-as key in its backend and, after its own session check, names the logged-in user on each call.

An asserted actor is only as true as the key holder’s own auth. Fuse cannot know whether the backend checked the session; it records the claim and who made it (alice, asserted by ak_123), and nothing more.

Every SDK has a withActor call that returns a copy of the client sending the headers. The original client is unchanged.

const fuse = client.withActor({ id: session.user.id, name: session.user.email });
await fuse.interventions.approve(id, { answer: "yes" });

cl := client.WithActor(fuse.ActorLabel{ID: user.ID, Name: user.Email})
in, err := cl.Interventions.Approve(ctx, id, "yes", "")

from fuse import ActorLabel

c = client.with_actor(ActorLabel(id=user.id, name=user.email))
c.interventions.approve(intervention_id, "yes")

let c = client.with_actor(ActorLabel::new(&user.id).name(&user.email));
c.interventions().approve(&id, "yes", "").await?;

Each SDK also takes an actor client option, so the agent’s side can tag which run raised each request. From the CLI, fuse hitl answer and fuse hitl reject take --actor.

GET /v1/interventions?actor=alice (and the audit log’s actor filter) finds everything an actor raised or decided.

Requiring an actor

By default any key with hitl:decide can decide. Set require_actor to insist that every decision names a person: it must come from a key pinned to an actor, or carry Fuse-Actor from a hitl:act-as key or the master token. Anything else is a 403 with reason actor_required.

hitl:
  require_actor: true
  requests:
    ask: {}

No self-approval

An intervention can never be decided by the key that raised it, or by the actor that raised it, whatever scopes the decider holds. Both are checked, so a person cannot raise a request under one key and approve it with another. Either case is a 403 with reason self_approval. The one exemption is the master token with no Fuse-Actor header: it is the operator’s root credential, not a party to the request, so it may decide a request it raised the same way.

Audit log

Every hitl action appends a row to an append-only audit log in Postgres:

A row records the actor, its source, the key id, the remote address, the time and the intervention id, and for a takeover_session the id of its recording. It never holds the question, the answer or the reason; those stay on the encrypted intervention row, so the audit log is safe to export.

fuse hitl audit                              # every row, oldest first
fuse hitl audit --env fuse-agent --since 24h
fuse hitl audit --actor alice
fuse hitl audit --verify

From the API, GET /v1/hitl/audit?env=&actor=&since= with hitl:read. A key limited to an environment sees only that environment’s rows.

Each row carries prev_hash, the hash of the row before it, so the rows form a chain. Editing, deleting or reordering a row breaks the chain at that point. --verify reads the whole log, recomputes every hash on your machine rather than trusting the server’s view, and reports the first row that no longer matches:

audit chain broken at seq 42: row content does not match its hash (the row was edited)

Because it reads every row, --verify takes no filters and needs a key that is not limited to one environment.

Webhooks

hitl.notify tells a webhook about every audit event on the environment, so a Slack bot or a pager can ping someone the moment a request is raised. Each entry names a URL and a secret from the Fusefile’s secrets: list, whose value signs the deliveries. Up to five webhooks are allowed.

version: 1
run: ./agent.sh

secrets:
  - hitl_webhook_secret

hitl:
  requests:
    ask: {}
  notify:
    - webhook: https://hooks.example.com/fuse
      secret: hitl_webhook_secret

fuse up --secret hitl_webhook_secret="$(openssl rand -hex 32)"

A delivery is a POST whose JSON body is the whole notification:

{
  "event": "raised",
  "intervention_id": "int_0187...",
  "env_id": "fuse-agent",
  "gate": "",
  "on": "ask",
  "deadline": "2026-10-29T15:04:05Z",
  "url": "https://fuse.example.com/v1/interventions/int_0187..."
}

It is content-free by design: the receiver learns that something needs a look and where, never what was asked. The human fetches the content from the API with their own key. url is built from the orchestrator’s ORCH_PUBLIC_URL and is left out when that is unset. Invite events carry no intervention id, kind or deadline. gate names the gate that raised the intervention and is empty for a request.

Each delivery has a Fuse-Signature header:

Fuse-Signature: t=<unix seconds>,v1=<hex HMAC-SHA256 of "<t>.<body>">

The HMAC is keyed by the secret’s value and computed over the timestamp, a dot, and the raw request body. Verify against the raw bytes, not re-encoded JSON, compare in constant time, and reject old timestamps so a captured delivery cannot be replayed:

import crypto from "node:crypto";

// rawBody is the request body as a Buffer, before any json parsing.
export function verifyFuse(secret, header, rawBody, toleranceSec = 300) {
  const parts = Object.fromEntries(header.split(",").map((p) => p.trim().split("=")));
  const { t, v1 } = parts;
  if (!t || !v1) return false;
  if (Math.abs(Date.now() / 1000 - Number(t)) > toleranceSec) return false;
  const want = crypto.createHmac("sha256", secret).update(`${t}.`).update(rawBody).digest("hex");
  return want.length === v1.length && crypto.timingSafeEqual(Buffer.from(want), Buffer.from(v1));
}

import hashlib
import hmac
import time


def verify_fuse(secret: bytes, header: str, raw_body: bytes, tolerance: int = 300) -> bool:
    # raw_body is the request body as received, before any json parsing.
    parts = dict(p.strip().split("=", 1) for p in header.split(",") if "=" in p)
    t, v1 = parts.get("t"), parts.get("v1")
    if not t or not t.isdigit() or not v1 or abs(time.time() - int(t)) > tolerance:
        return False
    want = hmac.new(secret, t.encode() + b"." + raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(want, v1)

A delivery is retried on any transport error or non-2xx answer, with a 10s timeout per try. Redirects are not followed. With restate off it is tried up to 4 times: at once, then after 1s, 4s and 16s, in the orchestrator process, so a restart drops it; the audit log is the record, the webhook is a nudge.

With restate on (FUSE_RESTATE_MODE=self or cloud) and TOKEN_ENCRYPTION_KEY set, each delivery is a restate Webhooks.Deliver invocation and survives an orchestrator restart. It is retried after 1s, then four times longer each try up to once an hour, for 24 hours from the event; a delivery still failing after that is recorded as a hitl_webhook dead letter. The same notification can arrive more than once if the orchestrator dies mid-post, so dedupe on intervention_id and event. Restate journals the delivery, but the endpoint url and signing secret go in sealed with TOKEN_ENCRYPTION_KEY; the journal holds the notification’s ids, gate, deadline and link. Without the key, deliveries stay in process as with restate off.

The signing secrets are sealed with TOKEN_ENCRYPTION_KEY. Without it they are kept in memory only, so deliveries stop after an orchestrator restart rather than the secret being stored in the clear.

Watching

GET /v1/environments/{id}/events emits an intervention event whenever one is raised or resolved:

{"event":"intervention","vm_id":"fuse-agent","state":"running",
 "intervention":{"id":"int_0187...","kind":"ask","state":"pending"}}

It carries ids and states only, never the question or the answer, and the stream drops events under load. Treat it as a hint to re-read the intervention, never as the record.

Where the content lives

The payload (a question, a held argv or computer action, a review’s content), the answer and the reason are encrypted at rest with the orchestrator’s TOKEN_ENCRYPTION_KEY before they reach Postgres. Ids, states and timestamps stay plaintext so the deadline sweep and list filters work without decrypting. Without the key (dev mode) the content is stored in the clear.

Two things are different. A secret’s value is never stored at all, with or without the key: it goes into the guest and is dropped. Takeover recordings, terminal and desktop, are sealed with the same key, and stored in the clear without it. The screenshot sent to a decider is not stored at all.