Route all AI features through the Switchboard gateway

Replace the direct Anthropic and OpenAI integrations with a single
provider that talks to Switchboard, an OpenAI-compatible gateway that
routes each request to the best available model. The app no longer pins
a model id anywhere: it sends switchboard/auto and lets the gateway
choose, then logs which model answered and what it cost.

Routing levers are set per feature in src/lib/ai/routing.ts. Three of
those choices came from measuring against the live gateway:

- category and prefer_free are set explicitly on every request. An API
  key carries its own routing defaults, and anything left unset inherits
  them - drink prompts were being sent to a free coding model.
- Token budgets are generous because the router may pick a reasoning
  model, and reasoning tokens come out of the same max_tokens budget as
  the answer. At 512 tokens a request returned null content; at 4096 the
  same request returned correct JSON.
- No tier lever on text features. tier "cheap" pinned a slow reasoning
  model (42-180s, two timeouts and one truncated response in five
  trials) and tier "frontier" escalated as far as Opus at $0.02 a call,
  while unconstrained routing answered in about a second. Vision keeps
  "frontier", where the accuracy is worth a few tenths of a cent.

Gateway failures are mapped to actionable messages rather than passed
through: a 401 relayed as 401 would read as an expired session and
bounce the user to login, and a 429 would collide with the app's own
rate limiter.

Also collapses the key lookup that was duplicated across ten call sites
into getUserProvider(), which fixes a latent bug where a bare findFirst
with no ordering let different features pick different providers.

Existing claude/openai key rows are ignored at runtime and offered for
removal in Settings, so no migration is needed before deploying.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
JP
2026-08-08 16:41:00 +00:00
parent 7c41b15ecc
commit a0e1619072
31 changed files with 830 additions and 429 deletions

View File

@@ -20,11 +20,16 @@ interface ApiKeyInfo {
isActive: boolean
}
interface ApiKeysResponse {
keys: ApiKeyInfo[]
gatewayUrl: string
}
export default function SettingsPage() {
const queryClient = useQueryClient()
// API Keys
const { data: apiKeys = [] } = useQuery<ApiKeyInfo[]>({
const { data: apiKeyData } = useQuery<ApiKeysResponse>({
queryKey: ["api-keys"],
queryFn: async () => {
const res = await fetch("/api/settings/api-keys")
@@ -33,6 +38,11 @@ export default function SettingsPage() {
},
})
const apiKeys = apiKeyData?.keys ?? []
const legacyKeys = apiKeys.filter(
(k) => k.provider === "claude" || k.provider === "openai"
)
// Preferences
const { data: preferences, isLoading: prefsLoading } = useQuery({
queryKey: ["preferences"],
@@ -74,16 +84,32 @@ export default function SettingsPage() {
<CardHeader>
<CardTitle className="flex items-center gap-2">
<Key className="h-5 w-5" />
AI Provider Keys
AI Gateway
</CardTitle>
<CardDescription>
Add your API keys for AI-powered menu scanning. Keys are encrypted and stored securely.
AI features route through Switchboard, which picks the best model for each
request. Add your gateway API key below it is encrypted before storage.
{apiKeyData?.gatewayUrl && (
<>
{" "}
This app is pointed at{" "}
<code className="text-xs">{apiKeyData.gatewayUrl}</code>.
</>
)}
</CardDescription>
</CardHeader>
<CardContent className="space-y-4">
<ApiKeyForm provider="claude" label="Anthropic Claude" existingKey={apiKeys.find(k => k.provider === "claude")} />
<Separator />
<ApiKeyForm provider="openai" label="OpenAI GPT-4o" existingKey={apiKeys.find(k => k.provider === "openai")} />
<ApiKeyForm
provider="switchboard"
label="Switchboard Gateway"
existingKey={apiKeys.find((k) => k.provider === "switchboard")}
/>
{legacyKeys.length > 0 && (
<>
<Separator />
<LegacyKeyNotice keys={legacyKeys} />
</>
)}
</CardContent>
</Card>
@@ -115,6 +141,62 @@ export default function SettingsPage() {
)
}
const LEGACY_PROVIDER_LABELS: Record<string, string> = {
claude: "Anthropic Claude",
openai: "OpenAI",
}
/**
* Keys left over from when the app called Claude and OpenAI directly. They are
* already ignored when picking a provider, but they are shown here so a user who
* still has one can see it is inert and remove it.
*/
function LegacyKeyNotice({ keys }: { keys: ApiKeyInfo[] }) {
const queryClient = useQueryClient()
const deleteKey = useMutation({
mutationFn: async (provider: string) => {
const res = await fetch(`/api/settings/api-keys?provider=${provider}`, {
method: "DELETE",
})
if (!res.ok) throw new Error("Failed to delete API key")
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ["api-keys"] })
},
})
return (
<div className="rounded-md border border-dashed p-3 space-y-3">
<p className="text-sm text-muted-foreground">
These keys are from an earlier version that called each AI provider directly.
They are no longer used and can be removed.
</p>
{keys.map((key) => (
<div key={key.id} className="flex items-center justify-between gap-2">
<div>
<p className="text-sm font-medium">
{LEGACY_PROVIDER_LABELS[key.provider] ?? key.provider}
</p>
<code className="text-xs bg-muted px-2 py-0.5 rounded">
{key.maskedKey}
</code>
</div>
<Button
variant="outline"
size="sm"
onClick={() => deleteKey.mutate(key.provider)}
disabled={deleteKey.isPending}
>
<Trash2 className="h-4 w-4 mr-1 text-destructive" />
Remove
</Button>
</div>
))}
</div>
)
}
function ApiKeyForm({
provider,
label,