Route all AI features through the Switchboard gateway
Replace the direct Anthropic and OpenAI integrations with a single provider that talks to Switchboard, an OpenAI-compatible gateway that routes each request to the best available model. The app no longer pins a model id anywhere: it sends switchboard/auto and lets the gateway choose, then logs which model answered and what it cost. Routing levers are set per feature in src/lib/ai/routing.ts. Three of those choices came from measuring against the live gateway: - category and prefer_free are set explicitly on every request. An API key carries its own routing defaults, and anything left unset inherits them - drink prompts were being sent to a free coding model. - Token budgets are generous because the router may pick a reasoning model, and reasoning tokens come out of the same max_tokens budget as the answer. At 512 tokens a request returned null content; at 4096 the same request returned correct JSON. - No tier lever on text features. tier "cheap" pinned a slow reasoning model (42-180s, two timeouts and one truncated response in five trials) and tier "frontier" escalated as far as Opus at $0.02 a call, while unconstrained routing answered in about a second. Vision keeps "frontier", where the accuracy is worth a few tenths of a cent. Gateway failures are mapped to actionable messages rather than passed through: a 401 relayed as 401 would read as an expired session and bounce the user to login, and a 429 would collide with the app's own rate limiter. Also collapses the key lookup that was duplicated across ten call sites into getUserProvider(), which fixes a latent bug where a bare findFirst with no ordering let different features pick different providers. Existing claude/openai key rows are ignored at runtime and offered for removal in Settings, so no migration is needed before deploying. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -245,6 +245,10 @@ if [[ "$SKIP_CONFIG" == "false" ]]; then
|
||||
prompt_secret AUTH_SECRET "NextAuth secret" "$(openssl rand -base64 32)"
|
||||
prompt_secret ENC_KEY "Encryption key" "$(openssl rand -hex 32)"
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}AI Gateway:${NC}"
|
||||
prompt_value SWITCHBOARD_URL "Switchboard gateway URL" "http://192.168.2.11:8787/v1"
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}OAuth Providers (optional — press Enter to skip):${NC}"
|
||||
prompt_value GOOGLE_CID "Google Client ID" ""
|
||||
@@ -288,6 +292,9 @@ MINIO_USE_SSL="false"
|
||||
|
||||
# ─── Encryption (for API key storage) ────────────────────
|
||||
ENCRYPTION_KEY="${ENC_KEY}"
|
||||
|
||||
# ─── AI Gateway (Switchboard) ────────────────────────────
|
||||
SWITCHBOARD_BASE_URL="${SWITCHBOARD_URL}"
|
||||
ENVEOF
|
||||
|
||||
chmod 600 "$ENV_FILE"
|
||||
@@ -403,7 +410,7 @@ echo " Update: $DC_CMD pull && $DC_CMD up -d"
|
||||
echo ""
|
||||
echo -e " ${BOLD}${YELLOW}Next steps:${NC}"
|
||||
echo " 1. Open ${APP_URL_CHECK} and create your account"
|
||||
echo " 2. Add your Claude or OpenAI API key in Settings"
|
||||
echo " 2. Add your Switchboard API key in Settings"
|
||||
echo " 3. Set up a reverse proxy (nginx/Caddy) for HTTPS"
|
||||
if [[ -z "${GOOGLE_CID:-}" ]] && [[ -z "${GITHUB_CID:-}" ]]; then
|
||||
echo " 4. (Optional) Add OAuth providers in $ENV_FILE"
|
||||
|
||||
Reference in New Issue
Block a user