On this page
  1. The terminal itself
  2. Session kernel
  3. Reaching the agents users bring
  4. Frontend
  5. Attention as the core resource
  6. Poor networks and many devices
  7. High-fidelity input and output

This page lists what cotty is meant to do, in three parts: the terminal itself, the session kernel and the frontend. It is a map. Each capability gets its own page when it is designed. So far that is attention cards.

The terminal itself

cotty has to be worth using with no agent in sight. A person spends the whole day in a terminal, and nothing built on top of a second-rate one gets used. These capabilities are the ground for everything below.

  1. Emulation.
    • Compatible with xterm and the VT terminals. TERM and terminfo work, and an old program reached over ssh behaves as it always has.
    • Complete support for the extensions that programs now rely on. Among them: true color, styled underlines, hyperlinks (OSC 8), the clipboard (OSC 52), the Kitty keyboard protocol, images by the Kitty graphics protocol and by Sixel, synchronized output, desktop notifications (OSC 9, 99 and 777), and the marks a shell puts around a command (OSC 133). The full list will be a specification page.
    • Every extension is a progressive enhancement. The ground is the plain behavior of xterm, and an extension only adds to it. It takes effect when a program turns it on, and cotty answers the queries a program uses to find out what a terminal supports. A program that asks for nothing sees the terminal it has always seen.
    • What cotty adds of its own follows the same rule from the other side: in a terminal that does not know it, it falls back gracefully to plain text.
    • The emulator runs once for each session, inside the session kernel. Every client shows the same screen, and the kernel always knows what is on it. The record, the snapshots for agents and the state sync below all rest on this.
  2. Rendering.
    • Typing never waits for output, and heavy output never stalls the window.
    • Text is set correctly: wide characters, combining marks, emoji, and fallback between fonts.
    • Images and semantic blocks are drawn natively, and text reflows when the window changes size.
  3. Commands as units. The screen knows where each command starts and ends, so a person can:
    • jump from one command to the next, and keep the running command in view;
    • fold the output of a command, or select and copy it whole;
    • see at a glance which command failed;
    • find any command that was run, with its output.
  4. A client for every platform.
    • On the desktop: macOS, Linux and Windows. On phones and tablets: iOS and Android. In a browser: a web page attaches to a session, with nothing to install.
    • The web page is served by the session kernel on the person’s own machine, never by a server of ours. How a device reaches a session is capability 4 of the kernel.
    • Each client is native to its platform, with its input methods, shortcuts and notifications.
    • Windows, tabs and panes are built in, so no separate multiplexer is needed.

The standard for emulation, rendering and clients is the best terminal on each platform. The measures, such as a budget for the delay between a key and its character, are not set yet.

Session kernel

  1. A structured session record.
    • For each command the kernel records the command line, the directory, the start and end time, the exit code, the raw output and the output as finally displayed, and whether a full-screen program ran.
    • The means: OSC 133 marks command boundaries, tcgetpgrp gives the foreground process, /proc gives the directory, and the switch to the alternate screen (1049) reveals a full-screen program.
    • The record can be searched and replayed locally.
  2. A standard interface for agents, over MCP or a socket API.
    • Read command blocks, or a screen snapshot with region information.
    • Send input, wait for a command to end, and get its exit code without polling.
    • cotty is a neutral host. Users bring their own agents.
    • A person and an agent share one session. Each sees the other and can interrupt.
  3. Control that does not depend on the agent. It covers the agents users bring, not only an agent built into cotty. The routes to such an agent are listed below.
    • Approval and policy: commands from an agent pass through rules first.
    • Output redaction: API keys and tokens are masked before output goes to a cloud model.
    • Undo: on btrfs or zfs, a snapshot is taken before a dangerous command.
    • An audit record.
  4. Sessions that persist across devices. A background process holds the session. A laptop, a phone or a web page can detach and attach. A device reaches a session in one of two ways.
    • Over a network the person already has: the machine itself, the local network, their own VPN or an ssh tunnel. This needs no account and no server of ours.
    • Through a relay, from anywhere. The relay is optional. It forwards and does not decrypt: each machine has its own name and its own certificate, the encryption ends on that machine, and the web page is served by that machine and not by the relay. The relay can see when a machine is reachable and how much traffic passes. It cannot see the content, and it cannot change the page.
    • Pairing. A device is paired once, with a code shown on the machine, or by approval on a device that is already paired. A browser signs in to the machine itself, with a passkey. The relay holds nothing that opens a session.
    • A browser that is not trusted gets a view that cannot type, and secrets are redacted before anything is sent to it.
  5. State sync over poor networks. The kernel syncs screen state, not bytes, as mosh does, and extends this to scrollback and structured command blocks.
  6. Local first, any model. Assistive features can use any OpenAI-compatible endpoint, including a self-hosted model. No account is needed, and data can stay inside the local network.

Reaching the agents users bring

A terminal sits between the keyboard and the program. That position alone does not give control over an agent. An agent such as Claude Code or Codex runs each command in a child process of its own and reads the output through pipes, so the terminal sees only the agent’s interface. The position gives control over people, and over agents that run their commands through the terminal’s interface.

There are three routes to an agent the user brings:

Route How What it needs
Execution interface The agent runs commands with the tool that cotty provides, which is capability 2. The commands go through the pty, so they are recorded and pass the rules. The agent’s cooperation.
Hooks cotty connects to the agent’s hooks, such as PreToolUse in Claude Code, and turns each approval into a card. An adapter for each agent.
Launch cotty starts the agent inside a sandbox (bubblewrap, Landlock), and points the address of its model API, such as ANTHROPIC_BASE_URL or OPENAI_BASE_URL, at a local proxy. The proxy redacts and audits requests before they leave the machine. Nothing from the agent.

The launch route comes closest to a guarantee by the terminal layer. Only its first step is verified so far: an agent connects to the address it is given. Reading and redacting requests in the proxy is not verified. See Redirecting an agent’s model endpoint to a local port.

Which route cotty builds first is not decided.

Existing terminals control only their own built-in agents. The comparison is in Six capabilities across terminals.

Frontend

The frontend follows three directions.

Attention as the core resource

This is the center of the frontend.

  • The terminal was designed for one person watching one byte stream. Now one person watches many agents.
  • The frontend is a decision cockpit. Pending requests appear as cards that state what will be done, the impact (diff, files, commands, network) and the risk. One key approves, rejects or modifies, and cards can be handled in batches. A decision can be saved as a rule.
  • It depends on the kernel’s semantic data. The evidence on a card comes from the record, not from the agent’s own account.

See Attention cards.

Poor networks and many devices

Technically this is the hardest part.

  • What a terminal shows is the state of a character grid. mosh proved that syncing state beats forwarding bytes: Ctrl+C takes effect at once, the connection survives a change of network, and typing is echoed locally by prediction.
  • mosh has no scrollback, no structure, and no support for several devices.
  • cotty keeps those three properties, and syncs screen state, structured command blocks and scrollback. The same session opens with a different view on each device: an inbox on a phone, the whole picture on a laptop.
  • The difficulty is in protocol design, correct terminal compatibility, and tuning on real networks.
  • A real case is in-flight Wi-Fi with a round trip of about 940 ms, described in the field notes.

High-fidelity input and output

This advances step by step.

  • Input. Terminal input is lossy key bytes, but what people give an agent is natural language, screenshots, files and speech. cotty provides a real input box: multi-line editing, pasted screenshots and files as attachments, and dictation.
  • Output. Every agent CLI rebuilds a small GUI out of character cells. Layout breaks when the window is resized, and copied text carries box-drawing characters. cotty defines semantic blocks over OSC (markdown, diff, image) that the frontend renders natively: text can be selected, lines wrap with the window, and content can be folded. In a terminal without support, a block falls back to plain text.
  • Adoption. This needs the ecosystem. cotty first connects Claude Code and Codex through hooks, then publishes the protocol. herdr took the same step with OSC 7501.