# Tutorial: Hosted LLM proxy

**Audience:** Operators enabling external LLM providers without putting API keys on the decoy sensor or Python broker.

**Time:** ~45-60 minutes (loopback proof + production runbook review).

**Outcome:** You can enable the `hosted` broker adapter against an allowlisted Go inference proxy, prove keys never touch the broker, and plan a vaulted production deployment.

## What you will learn

1. How the hosted adapter + Go `inference.Proxy` split keeps credentials on the proxy plane only.
2. Loopback proof that the broker POSTs typed slots and never sees provider keys.
3. Production runbook: vault, allowlist, mTLS profile.

## Prerequisites

- CyberHalluciNet sensor built with `--mode=ai` and broker UDS wired (`--broker-uds`).
- Python broker image or venv with `CYBERHALLUCINET_BROKER_ADAPTER=hosted` available.
- For loopback proof: ability to run a local RecordingTransport / stub proxy (see `engine/internal/proxy/inference/` tests).

---

## 1. Enable the hosted adapter (loopback proof)

The broker reconstructs requests from **typed contract fields only**, no free-form prompts, no
embedded API keys (`services/ai-broker/.../adapters/hosted.py`). Default transport is
`DenyTransport` (fail closed).

```bash
export CYBERHALLUCINET_BROKER_MODE=uds
export CYBERHALLUCINET_BROKER_UDS=/ipc/broker.sock
export CYBERHALLUCINET_BROKER_ADAPTER=hosted
export CYBERHALLUCINET_BROKER_HOSTED_ROUTE=route-lab-openai
export CYBERHALLUCINET_BROKER_HOSTED_PROXY_URL=http://127.0.0.1:9090/v1/generate
```

Start the broker, then the sensor:

```bash
sensor --mode=ai --broker-uds=/ipc/broker.sock
```

**Proof checklist (keys off broker):**

1. `grep -r sk- services/ai-broker/` on the broker host → no provider secrets in env or config.
2. Broker logs / packet capture show POST bodies with slot contracts only, no `Authorization: Bearer` from the broker process.
3. The Go proxy (`engine/internal/proxy/inference/proxy.go`) injects credentials from its own route table; broker receives shaped slot responses only.

Use `RecordingTransport` in unit tests (`adapters/hosted.py` + proxy tests) as the reference harness for CI e2e.

---

## 2. Production runbook (vault, allowlist, mTLS)

When moving beyond loopback, treat the inference proxy as a **separate plane** (same pattern as
CTI worker / Ops, never co-locate provider keys on the Internet-facing decoy).

### Vault / credential storage

| Rule | Detail |
|------|--------|
| Keys live on proxy only | Route `Credential` field populated at deploy from vault, not in broker env, not in sensor YAML |
| No hot reload from attacker paths | Proxy reads creds at startup or via Ops redeploy (estate rotation = redeploy, not live push) |
| Audit | Vault access logs tied to route ID + decoy estate owner |

Example: inject `OPENAI_API_KEY` (or provider equivalent) into the proxy container/systemd unit via
your secret manager; broker env lists only `CYBERHALLUCINET_BROKER_HOSTED_PROXY_URL` pointing at
the proxy’s **exact** allowlisted URL.

### Allowlist

`HostedRoutePolicy.assert_allowed` and `inference.Proxy` reject:

- Caller-supplied upstream URLs (exact match on configured `proxy_url` only)
- Userinfo in URLs (`https://key@host` → denied)
- Metadata / link-local targets from broker (proxy adds second line of defense)

Configure one route per provider profile:

```yaml
# proxy config (conceptual: lives with inference proxy deploy, not sensor.yaml)
routes:
  - id: route-prod-openai
    url: https://inference-proxy.internal:8443/v1/generate
    max_body_bytes: 65536
```

Broker env must use the **identical** URL string as `CYBERHALLUCINET_BROKER_HOSTED_PROXY_URL`.

### mTLS profile

For cross-VM broker ↔ proxy links:

| Broker side | Proxy side |
|-------------|------------|
| `CYBERHALLUCINET_BROKER_MODE=mtls` (stub profile for cross-VM) | Server cert + client CA |
| No provider keys | Terminates mTLS, attaches provider creds, forwards allowlisted upstream |

Production checklist:

1. Issue broker client cert (CN = broker instance ID); proxy trusts operator CA only.
2. Proxy listens on private interface / service mesh, not Internet-facing.
3. DenyTransport remains default on any host that has not completed sign-off.
4. Run purple-team / fingerprint gates before exposing generative hosted fills to real attacker traffic.

See also: [Decoy realism & AI fills](../how-to/decoy-realism-ai.md) (adapter matrix),
[plane separation](../architecture/plane-separation.md), [production sign-off](../assurance/production-signoff.md).

---

## Related

- [AI safety controls](ai-safety-controls.md): kill-switch, digest pin, guard model
- [Hosted adapter reference](../how-to/decoy-realism-ai.md#adapter-matrix)