Route all AI features through the Switchboard gateway
Replace the direct Anthropic and OpenAI integrations with a single provider that talks to Switchboard, an OpenAI-compatible gateway that routes each request to the best available model. The app no longer pins a model id anywhere: it sends switchboard/auto and lets the gateway choose, then logs which model answered and what it cost. Routing levers are set per feature in src/lib/ai/routing.ts. Three of those choices came from measuring against the live gateway: - category and prefer_free are set explicitly on every request. An API key carries its own routing defaults, and anything left unset inherits them - drink prompts were being sent to a free coding model. - Token budgets are generous because the router may pick a reasoning model, and reasoning tokens come out of the same max_tokens budget as the answer. At 512 tokens a request returned null content; at 4096 the same request returned correct JSON. - No tier lever on text features. tier "cheap" pinned a slow reasoning model (42-180s, two timeouts and one truncated response in five trials) and tier "frontier" escalated as far as Opus at $0.02 a call, while unconstrained routing answered in about a second. Vision keeps "frontier", where the accuracy is worth a few tenths of a cent. Gateway failures are mapped to actionable messages rather than passed through: a 401 relayed as 401 would read as an expired session and bounce the user to login, and a 429 would collide with the app's own rate limiter. Also collapses the key lookup that was duplicated across ten call sites into getUserProvider(), which fixes a latent bug where a bare findFirst with no ordering let different features pick different providers. Existing claude/openai key rows are ignored at runtime and offered for removal in Settings, so no migration is needed before deploying. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -20,11 +20,16 @@ interface ApiKeyInfo {
|
||||
isActive: boolean
|
||||
}
|
||||
|
||||
interface ApiKeysResponse {
|
||||
keys: ApiKeyInfo[]
|
||||
gatewayUrl: string
|
||||
}
|
||||
|
||||
export default function SettingsPage() {
|
||||
const queryClient = useQueryClient()
|
||||
|
||||
// API Keys
|
||||
const { data: apiKeys = [] } = useQuery<ApiKeyInfo[]>({
|
||||
const { data: apiKeyData } = useQuery<ApiKeysResponse>({
|
||||
queryKey: ["api-keys"],
|
||||
queryFn: async () => {
|
||||
const res = await fetch("/api/settings/api-keys")
|
||||
@@ -33,6 +38,11 @@ export default function SettingsPage() {
|
||||
},
|
||||
})
|
||||
|
||||
const apiKeys = apiKeyData?.keys ?? []
|
||||
const legacyKeys = apiKeys.filter(
|
||||
(k) => k.provider === "claude" || k.provider === "openai"
|
||||
)
|
||||
|
||||
// Preferences
|
||||
const { data: preferences, isLoading: prefsLoading } = useQuery({
|
||||
queryKey: ["preferences"],
|
||||
@@ -74,16 +84,32 @@ export default function SettingsPage() {
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2">
|
||||
<Key className="h-5 w-5" />
|
||||
AI Provider Keys
|
||||
AI Gateway
|
||||
</CardTitle>
|
||||
<CardDescription>
|
||||
Add your API keys for AI-powered menu scanning. Keys are encrypted and stored securely.
|
||||
AI features route through Switchboard, which picks the best model for each
|
||||
request. Add your gateway API key below — it is encrypted before storage.
|
||||
{apiKeyData?.gatewayUrl && (
|
||||
<>
|
||||
{" "}
|
||||
This app is pointed at{" "}
|
||||
<code className="text-xs">{apiKeyData.gatewayUrl}</code>.
|
||||
</>
|
||||
)}
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<ApiKeyForm provider="claude" label="Anthropic Claude" existingKey={apiKeys.find(k => k.provider === "claude")} />
|
||||
<Separator />
|
||||
<ApiKeyForm provider="openai" label="OpenAI GPT-4o" existingKey={apiKeys.find(k => k.provider === "openai")} />
|
||||
<ApiKeyForm
|
||||
provider="switchboard"
|
||||
label="Switchboard Gateway"
|
||||
existingKey={apiKeys.find((k) => k.provider === "switchboard")}
|
||||
/>
|
||||
{legacyKeys.length > 0 && (
|
||||
<>
|
||||
<Separator />
|
||||
<LegacyKeyNotice keys={legacyKeys} />
|
||||
</>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
@@ -115,6 +141,62 @@ export default function SettingsPage() {
|
||||
)
|
||||
}
|
||||
|
||||
const LEGACY_PROVIDER_LABELS: Record<string, string> = {
|
||||
claude: "Anthropic Claude",
|
||||
openai: "OpenAI",
|
||||
}
|
||||
|
||||
/**
|
||||
* Keys left over from when the app called Claude and OpenAI directly. They are
|
||||
* already ignored when picking a provider, but they are shown here so a user who
|
||||
* still has one can see it is inert and remove it.
|
||||
*/
|
||||
function LegacyKeyNotice({ keys }: { keys: ApiKeyInfo[] }) {
|
||||
const queryClient = useQueryClient()
|
||||
|
||||
const deleteKey = useMutation({
|
||||
mutationFn: async (provider: string) => {
|
||||
const res = await fetch(`/api/settings/api-keys?provider=${provider}`, {
|
||||
method: "DELETE",
|
||||
})
|
||||
if (!res.ok) throw new Error("Failed to delete API key")
|
||||
},
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["api-keys"] })
|
||||
},
|
||||
})
|
||||
|
||||
return (
|
||||
<div className="rounded-md border border-dashed p-3 space-y-3">
|
||||
<p className="text-sm text-muted-foreground">
|
||||
These keys are from an earlier version that called each AI provider directly.
|
||||
They are no longer used and can be removed.
|
||||
</p>
|
||||
{keys.map((key) => (
|
||||
<div key={key.id} className="flex items-center justify-between gap-2">
|
||||
<div>
|
||||
<p className="text-sm font-medium">
|
||||
{LEGACY_PROVIDER_LABELS[key.provider] ?? key.provider}
|
||||
</p>
|
||||
<code className="text-xs bg-muted px-2 py-0.5 rounded">
|
||||
{key.maskedKey}
|
||||
</code>
|
||||
</div>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => deleteKey.mutate(key.provider)}
|
||||
disabled={deleteKey.isPending}
|
||||
>
|
||||
<Trash2 className="h-4 w-4 mr-1 text-destructive" />
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function ApiKeyForm({
|
||||
provider,
|
||||
label,
|
||||
|
||||
Reference in New Issue
Block a user