Shell command (\!) API keys are cached forever, causing expired token errors with authHeader

[alennartz]

Description

When using authHeader: true with a shell command (!) for apiKey (e.g., Azure AD tokens via az cli), the token is fetched once at startup and never refreshed. Sessions that outlast the token's lifetime (typically ~60 minutes for Azure) fail with expired token errors, requiring a full restart of pi.

Configuration

{
  "providers": {
    "azure-foundry": {
      "baseUrl": "https://example.services.ai.azure.com/anthropic",
      "api": "anthropic-messages",
      "apiKey": "!az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv",
      "authHeader": true,
      "models": [...]
    }
  }
}

Root Cause

Three things combine to make this unfixable without restarting:

1. resolve-config-value.ts caches shell command results in a Map with no TTL. Once cached, the same value is returned for the lifetime of the process:

   const commandResultCache = new Map<string, string | undefined>();
   // ...
   if (commandResultCache.has(commandConfig)) {
       return commandResultCache.get(commandConfig); // stale forever
   }

2. ModelRegistry.refresh() does not call clearConfigValueCache() before reloading models, so even opening /model and re-selecting doesn't help — the cache still returns the old token.

3. authHeader: true resolves the API key and bakes it into the model's static headers.Authorization at parse time in parseModels(). There's no dynamic re-resolution at request time.

Expected Behavior

Users should never see an expired token error during a session. Token refresh should be completely transparent — no manual intervention, no /model re-selection, no restart.

Suggested Fix

Add a cacheTtl field (in seconds) to the provider configuration for ! shell commands:

{
  "providers": {
    "azure-foundry": {
      "apiKey": "!az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv",
      "cacheTtl": 1800,
      ...
    }
  }
}

When cacheTtl is set, the ! command result should be re-executed automatically after the TTL expires. The re-resolution needs to happen at request time — not just at model parse time — so that authHeader headers are always current. This way the token stays fresh transparently and the user never encounters an auth error from an expired token.

Without a cacheTtl, the current cache-forever behavior is fine for static secrets that don't expire.

Docs Note

The models.md documentation says:

The file reloads each time you open /model. Edit during session; no restart needed.

This is misleading for ! commands since the command cache prevents actual re-execution even when the file is re-read. It probably makes sense to also alter the logic/model to clear cache entries and refresh them. This could serve as a minimal fix for this issue, but the devx is not as nice as the full fix proposed above

[Br1an67]

I'd like to work on this.

I dug into the code and confirmed the three-part root cause:

1. resolve-config-value.ts (line ~9): commandResultCache is a plain Map with no TTL —
2. model-registry.ts refresh() (line ~266): clears customProviderApiKeys and resets API/OAuth providers, but never calls clearConfigValueCache() — so even /model re-selection doesn't help
3. model-registry.ts parseModels() / applyProviderConfig() (lines ~489, ~648): authHeader: true resolves the API key once via resolveConfigValue() and bakes it into static headers.Authorization — no dynamic re-resolution at request time

Proposed fix:

• Call clearConfigValueCache() in refresh() as a minimal safety net
• For authHeader providers with cacheTtl, resolve the API key lazily at request time instead of baking it into static headers at parse time

Without cacheTtl, current cache-forever behavior stays for static secrets. Happy to hear thoughts before starting.

[deltoid9818]

I'd like to fix this. Three changes:

1. resolve-config-value.ts: commandResultCache now stores {value, timestamp}. resolveConfigValue() and executeCommand() accept optional cacheTtlMs. When set, cached entries older than the TTL are re-executed. cacheTtlMs=0 means never cache. undefined preserves current cache-forever behavior (backward compat).

2. model-registry.ts: Added cacheTtl (optional number, seconds) to ProviderConfigSchema and ProviderConfigInput. refresh() now calls clearConfigValueCache() before reloading models, so both the command cache and baked-in Authorization headers get refreshed. All authHeader call sites (parseModels, applyProviderConfig, fallbackResolver) pass cacheTtl * 1000 through to resolveConfigValue.

3. Semantics: cacheTtl: undefined = cache forever, cacheTtl: 0 = never cache, cacheTtl: N = cache for N seconds.

6 tests (3 unit for resolve-config-value, 3 integration for model-registry including the full refresh → re-resolve chain). npm run check clean.

[alennartz]

Since I've opened this issue. I've worked around with a custom model provider extension. Specifically for azure foundry, it's a better experience anyway than just a pure TTL the main flaws with the TTL approach I've come to realize are the following.

Real TTL durations are likely an implementation detail of the source. A robust solution would need a way to be told reliably what the TTL is. Jwt are ubiquitous enough that support for them could be added first class but this becomes a bigger change.

Another pitfall is the upstream source of the token might have its own caching layer (as a does) this makes it doubly important to know real expiration because fixed TTL inevitably misses the upstream refresh window eventually and becomes way out of sync with the true expiration of the token it's holding.

As a result of this I'd conclude that this issue deserves one of two things a more in depth solution or simply being closed with a mention that the existing custom model extensibility points are better suited for this scenario.

[badlogic]

Implemented in main.

What changed:

• models.json shell commands for provider apiKey and request headers are now resolved at request time
• authHeader: true now rebuilds Authorization from the freshly resolved value on each request
• /model availability checks no longer execute shell commands
• compaction and branch summarization use the same request-time auth/header resolution path

Deliberately not added:

• no built-in TTL
• no generic caching policy
• no stale-on-error fallback

Reason: pi cannot safely guess the right caching and recovery strategy for arbitrary shell commands. Some commands should never cache, some need a short TTL, some need stale reuse, and some already have caching upstream.

If your command is slow or you want TTL or custom recovery behavior, wrap it in your own script or command that implements the policy you want, then call that from models.json.

Docs were updated to spell that out.