Proxy Performance and Local Latency

Adding an intermediary proxy to your application's network path naturally raises questions about latency, memory usage, and performance overhead.

The AgentSecrets proxy is designed from the ground up in Go to minimize request overhead, ensuring that security does not come at the expense of speed.

---

Expected latency overhead

The local proxy introduces a loopback network hop. Because this communication occurs entirely within the host machine over the loopback interface (127.0.0.1), network transit time is sub-millisecond (typically 0.1ms to 0.3ms).

The total latency profile of a proxy request is divided into three components:

1. Local Loopback Transit: The application sends the request to the proxy (~0.2ms).
2. Secret Decryption: The proxy resolves the reference name, fetches the cipher from the OS Keychain, and decrypts it. Because the decrypted keys are securely cached in the proxy daemon's RAM, this step takes < 0.05ms for cached keys, and 5ms to 15ms for uncached keys requiring hardware Keychain/Secure Enclave decryptions.
3. Outbound Transit & TLS: The proxy opens an outbound TLS connection to the remote API.

---

Local vs remote resolution

A common misconception is that the local proxy makes a remote network call to the AgentSecrets cloud servers to resolve secrets on every API request. It does not.

• Offline Decryption: All decryption keys and credentials reside locally. Decryption is performed entirely on your local machine using your cryptographic private key.
• Asynchronous Syncing: The proxy synchronizes domain allowlists, project settings, and credential revocation lists with the cloud backend asynchronously in the background. This synchronization occurs out-of-band and never blocks your application's API request path.
• Connection Reuse: The proxy maintains persistent HTTP keep-alive connections to common API gateways (like api.openai.com and api.stripe.com), avoiding the expensive TLS handshake overhead (usually 50ms - 150ms) on subsequent calls.

---

Benchmarks

The following benchmarks compare the latency of direct API calls against calls routed through the AgentSecrets local proxy. Testing was performed on an AWS c6i.xlarge instance (Ubuntu 22.04 LTS) targeting https://api.stripe.com/v1/balance.

Loading diagram...

---

Optimizing for high-frequency agent calls

For high-throughput applications making hundreds of API calls per minute (such as autonomous swarm agents or high-frequency trading services), you can optimize loopback connection performance:

1Enable Connection Keep-Alive