Access Groups Perf Tests

[yuneng-jiang]

Access Group Performance Test

Summary

Unified access groups add a new permission layer that lets keys and teams reach models, MCP servers, and agents through shared groups instead of (or in addition to) native per-key/per-team model lists. This introduces additional DB lookups on the critical auth path: when native permissions don't satisfy a request, the proxy resolves each access_group_id via get_access_object() (one find_unique per group, cache-first with 10-min TTL).

This test measures the latency impact of that resolution across four categories of request to confirm the feature doesn't regress proxy performance.

Test Matrix

Category	Description	Expected overhead
A — Baseline	Key/team with no access groups calls a natively allowed model	None (control group)
B — AGs present, native passes	Key/team has access groups but the model is natively allowed	None — AGs should not be consulted
C — AG fallback success	Model is only reachable via an access group	AG resolution cost (1 cache/DB lookup per group)
D — AG fallback deny	Model not in native permissions or any access group	Full AG resolution cost, then denial (no provider call)

Test Scenarios

ID	Key	Team	Model	Expected	Category
A1	Isolated key, no AGs	—	gpt-4o	200	Baseline
A2	Team key, no AGs	Team, no AGs	gpt-4o	200	Baseline
B1	Isolated key, with AGs	—	gpt-4o	200	AG present, native passes
B2	Team key, no AGs	Team, with AGs	gpt-4o	200	AG present, native passes
C1	Isolated key, with AGs	—	bedrock-claude-sonnet-4	200	AG fallback success
C2	Team key, no AGs	Team, with AGs	bedrock-claude-sonnet-4	200	AG fallback success
D1	Isolated key, with AGs	—	nonexistent-model	4xx	AG fallback deny
D2	Team key, no AGs	Team, with AGs	nonexistent-model	4xx	AG fallback deny

What to compare

• A vs B — confirms zero overhead when access groups are present but unused.
• A vs C — quantifies the AG fallback cost on the success path.
• A vs D — quantifies the AG fallback cost on the denial path (pure proxy, no provider round-trip).
• C vs D — isolates provider latency from auth overhead (D has no provider call).

Test Setup

Access Groups

Name	ID	Models
ag1	56a2374c-1e69-46da-af1e-383ef3568c98	text-embedding-3-small, bedrock-claude-sonnet-4, bedrock-claude-sonnet-4.5, bedrock-claude-opus-4.5
ag2	11471732-763f-4371-8f12-459f9197f6bd	bedrock-nova-premier, bedrock-claude-sonnet-3.5

Models on Proxy

gpt-4o
text-embedding-3-small
bedrock-claude-sonnet-4
bedrock-claude-sonnet-4.5
bedrock-claude-opus-4.5
bedrock-nova-premier
bedrock-claude-sonnet-3.5

Keys

Key	Native models	Team	Access groups
sk-ctGZv4EFE97raVgHIzeZHg	["gpt-4o"]	—	—
sk-e3QqXXZzSU2gmM3lnq-RcQ	["gpt-4o"]	—	ag1, ag2
sk-FvJ0wU_IiFwCwXO50KEPBw	—	ag-team-no-access-groups	—
sk-fNCzqA43eJjx9qCFEwWJ5Q	—	ag-team-with-access-groups	—

Teams

Alias	ID	Native models	Access groups
ag-team-no-access-groups	c2d294d1-1398-46d8-972c-29fbc9428add	["gpt-4o"]	—
ag-team-with-access-groups	e08b4e17-3b60-4c76-a354-f66e4bc9751f	["gpt-4o"]	ag1, ag2

Metrics

For each scenario (50 requests per scenario):

• Latency: p50, p95, p99, mean, stdev (ms)
• Success / failure count
• Cross-scenario delta vs baseline (A1/A2)