Route all AI features through the Switchboard gateway

Replace the direct Anthropic and OpenAI integrations with a single
provider that talks to Switchboard, an OpenAI-compatible gateway that
routes each request to the best available model. The app no longer pins
a model id anywhere: it sends switchboard/auto and lets the gateway
choose, then logs which model answered and what it cost.

Routing levers are set per feature in src/lib/ai/routing.ts. Three of
those choices came from measuring against the live gateway:

- category and prefer_free are set explicitly on every request. An API
  key carries its own routing defaults, and anything left unset inherits
  them - drink prompts were being sent to a free coding model.
- Token budgets are generous because the router may pick a reasoning
  model, and reasoning tokens come out of the same max_tokens budget as
  the answer. At 512 tokens a request returned null content; at 4096 the
  same request returned correct JSON.
- No tier lever on text features. tier "cheap" pinned a slow reasoning
  model (42-180s, two timeouts and one truncated response in five
  trials) and tier "frontier" escalated as far as Opus at $0.02 a call,
  while unconstrained routing answered in about a second. Vision keeps
  "frontier", where the accuracy is worth a few tenths of a cent.

Gateway failures are mapped to actionable messages rather than passed
through: a 401 relayed as 401 would read as an expired session and
bounce the user to login, and a 429 would collide with the app's own
rate limiter.

Also collapses the key lookup that was duplicated across ten call sites
into getUserProvider(), which fixes a latent bug where a bare findFirst
with no ordering let different features pick different providers.

Existing claude/openai key rows are ignored at runtime and offered for
removal in Settings, so no migration is needed before deploying.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
JP
2026-08-08 16:41:00 +00:00
parent 7c41b15ecc
commit a0e1619072
31 changed files with 830 additions and 429 deletions

View File

@@ -1,8 +1,9 @@
import { NextResponse } from "next/server"
import { auth } from "@/lib/auth"
import { prisma } from "@/lib/prisma"
import { decrypt } from "@/lib/encryption"
import { createProvider } from "@/lib/ai/provider-factory"
import { getUserProvider } from "@/lib/ai/provider-factory"
import { FEATURE_ROUTING } from "@/lib/ai/routing"
import { aiErrorResponse } from "@/lib/ai/errors"
import { rateLimit } from "@/lib/rate-limit"
import { WHAT_CAN_I_MAKE_PROMPT, buildBarInventoryString } from "@/lib/ai/prompts"
import { fuzzyMatchIngredients, recalculateMissingCount } from "@/lib/ingredient-matcher"
@@ -22,17 +23,6 @@ export async function POST() {
}
try {
const apiKeyRecord = await prisma.userApiKey.findFirst({
where: { userId: session.user.id, isActive: true },
})
if (!apiKeyRecord) {
return NextResponse.json(
{ error: "No AI provider configured. Add an API key in Settings." },
{ status: 400 }
)
}
const barItems = await prisma.barItem.findMany({
where: {
userId: session.user.id,
@@ -51,12 +41,12 @@ export async function POST() {
const inventoryString = buildBarInventoryString(barItems)
const prompt = WHAT_CAN_I_MAKE_PROMPT.replace("{barInventory}", inventoryString)
const apiKey = decrypt(apiKeyRecord.encryptedKey, apiKeyRecord.iv)
const provider = createProvider(apiKeyRecord.provider, apiKey)
const provider = await getUserProvider(session.user.id)
const rawResponse = await provider.sendTextRequest(
prompt,
"What cocktails can I make with my bar inventory?"
"What cocktails can I make with my bar inventory?",
FEATURE_ROUTING.bartenderSuggest
)
// Parse JSON from response
@@ -98,10 +88,6 @@ export async function POST() {
return NextResponse.json({ suggestions })
} catch (error) {
console.error("Bartender suggest error:", error)
return NextResponse.json(
{ error: "Failed to generate suggestions. Please try again." },
{ status: 500 }
)
return aiErrorResponse(error, "Failed to generate suggestions. Please try again.")
}
}