Northpier AI traffic desk · Kong Staff SE walkthrough

One command for the full stack

Docker is the only dependency. No .env file. No real API keys. Mock mode is the only mode.

docker compose up --build

Then open http://127.0.0.1:43123. Kong’s proxy is on 127.0.0.1:18000. The admin API stays on loopback at 127.0.0.1:18001.

What this page is

This is the shareable walkthrough. The interactive desk runs in Docker on a reviewer’s machine. The captures below are from that desk: a cache hit, a jailbreak refusal, the mobile layout, and a full click-through.

OSSTypeScriptNo outbound

Interactive desk on Cloudflare Workers, no install: https://northpier-desk.pages.dev. It runs the same guardrails, semantic cache, cost routing, and MCP tools. A first visit can show a short Cloudflare check. Kong OSS itself is the docker compose up --build stack; the Workers runtime cannot run the Kong process, so that public desk applies the same gateway rules in-process.

Role → where it shows up

What the posting asks for	Where a reviewer sees it
Demonstrations of MCP governance, semantic caching, prompt guardrails, and shadow AI detection	Presets on the desk. MCP is `POST /mcp` behind Kong key-auth and rate limiting. The cache is local. Guardrails run in the agent and again in Kong AI Prompt Guard. Bypass attempts are recorded and never open a provider socket.
LLM traffic management and governance requirements	Allowlist of `harbor-mini` and `harbor-large`. A session token budget. 120 requests a minute per consumer. `gpt-4o` is refused before a route exists.
Model routing, rate limiting, semantic caching, prompt governance	Short lookups go to harbor-mini. Tool-backed answers go to harbor-large. Repeat wording is a cache hit and is not billed. Jailbreaks, SSNs, payment cards, and pasted credentials stop first.
Model Context Protocol and agentic patterns	The tool server speaks MCP JSON-RPC: `initialize`, `tools/list`, `tools/call`. Tools: `get_refund_policy`, `list_approved_models`, `estimate_route_cost`.
A technical feasibility workshop	The preset order below is the workshop script. The Decision panel narrates guardrail, cache, route, tool, gateway status, and cost.
Docker and a declarative gateway	`docker-compose.yml` and DB-less `kong/kong.yml`. The same file is what you would apply with decK on a cluster.
API security, with OAuth and mTLS as the production pattern	The workshop uses key-auth so every call has a consumer. Kong AI Proxy injects the provider bearer. The desk never sees it. A customer design would put OpenID Connect on these same routes and terminate mTLS on the proxy.

Kong OSS and the desk

The gateway is the choke point. The desk is product policy. Enterprise plugins are named as such and are not pretended to be in this OSS image.

Control	Kong Gateway OSS 3.9	TypeScript desk
Identity	`key-auth`, consumer `northpier-desk`	Sends the fixture key. Does not hold provider tokens.
Rate limit	120 requests / minute / consumer, local policy	Session token budget. Cache hits are not billed.
Prompt guard	`ai-prompt-guard` PCRE denylist	Jailbreak, prompt exfiltration, SSN, Luhn-checked cards, credential assignments.
System prompt	`ai-prompt-decorator` prepends Northpier policy	Adds MCP tool results as system context.
Model proxy	`ai-proxy` pins the model, injects `Authorization`, logs token stats, denies streaming	Chooses `/llm/harbor-mini` or `/llm/harbor-large`.
Semantic cache	Not in OSS. The Enterprise plugin needs an embeddings provider.	Hashed word and bigram vectors. `return`/`period` folds to `refund`/`window`. Threshold 0.82.
Shadow AI	Sees only traffic that reaches the proxy	Records a bypass request and refuses to open a provider connection.
Audit	`file-log`, payloads off	Unapproved tab, plus a reader for the gateway log.

Five-minute path

Proxy port. Short lookup. Route harbor-mini. Answer: the proxy listens on 8000 inside the network and is published on host port 18000. Gateway HTTP 200.
Ask it again. Different wording, same question. Cache hit near 1.00. Kong is not called. Cost stays $0.
Refund window. Desk calls get_refund_policy through Kong, then harbor-large, because a tool result is on the call. Answer cites the 30-day window.
Return period. Synonym of the refund question. Cache hit if step 3 already ran.
Jailbreak and SSN in the prompt. Agent blocks them. They appear on Unapproved. Kong is not called.
Unapproved model. gpt-4o is not on the allowlist.
Skip the agent. The jailbreak is posted raw to Kong. AI Prompt Guard returns the block. The gateway audit shows the 400.
Bypass the gateway. Recorded as shadow AI. No provider socket is opened.

A caller with no credential gets 401:

curl -s -o /dev/null -w "%{http_code}\n" \
  -X POST http://127.0.0.1:18000/llm/harbor-mini \
  -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"hello"}]}'

Architecture

browser
  │
  ▼
Northpier desk (Next.js, TypeScript)
  guardrails · semantic cache · cost route · token budget
  │
  ▼
Kong Gateway OSS 3.9
  key-auth · rate-limiting · correlation-id
  AI Prompt Decorator · AI Prompt Guard · AI Proxy · file-log
  │
  ├── /llm/harbor-mini     $0.15 / $0.60 per 1M tokens
  ├── /llm/harbor-large    $3 / $15 per 1M tokens
  └── /mcp                 get_refund_policy
                           list_approved_models
                           estimate_route_cost

Captured from the running desk

Decision panel showing a semantic cache hit for the proxy port question — Ask it again. Cache hit 1.00. Gateway not called. Cost $0.

Jailbreak prompt blocked by the agent guardrail — Jailbreak. Unapproved, agent guardrail, gateway not called.

The desk stacked on a 390 pixel viewport — 390px width. Presets stack. The traffic table scrolls sideways.

Click-through: Proxy port approved on harbor-mini, the repeat is cached, Refund window uses the MCP tool and harbor-large, Jailbreak is refused.

Mock mode

Both models speak the OpenAI chat-completions shape and reject any bearer other than the one Kong injects. The MCP server does not call out. Compose disables IP masquerade, so containers have no SNAT path to the public internet. Provider ports are not published on the host.

Checks already run

npm test in apps/desk covers guardrails, the synonym cache, routing, the budget, and the rule that shadow calls never call fetch. node scripts/smoke.mjs runs the workshop against a live stack and expects the anonymous Kong call to be 401.