Langfuse launches AI Gateway

Every AI application generates a constant stream of data: prompts, model calls, costs and evaluation results. Storing and querying that data at scale is exactly what ClickHouse was built for. That’s why Langfuse runs on ClickHouse, and why it joined ClickHouse in January. Langfuse is the open-source AI engineering platform that thousands of teams use to trace, evaluate and improve their AI applications, from prompt management to production monitoring. Today, Langfuse is launching the AI Gateway in public beta, so every model call, including those made by coding agents, lands in one place with who made it and what it cost. Here’s the Langfuse team with the details.

Today, we are launching Langfuse AI Gateway in public beta. We're excited to introduce our next product. Thank you to everyone who shared feedback that this is what you want us to build next to establish a unified way use models, track costs, and govern access to AI.

Why we built Langfuse AI Gateway

Over the last three years, something changed in how companies use AI. It is no longer one chatbot built by one team. It is coding agents on every engineer's laptop, AI assistants in every team, workflow tools adopted bottom-up, and dozens of custom AI apps in production.

It is becoming increasingly hard for companies to answer the questions: who in our organization uses which models, what for, how much does it cost, is it secure, and how can we optimize and govern it?

Managing AI usage within a company is becoming its own discipline:

• Platform teams are asked to make AI adoption safe, simple, and affordable at scale.
• Finance teams need to track AI costs across providers and teams every month.

Langfuse AI Gateway is a central entry point to govern all AI traffic that is easy to host, adopt, and enforce. It can log all requests natively in Langfuse, so you can analyze AI usage and make decisions based on it.

Langfuse is uniquely positioned for this

Langfuse is the leading open source project for AI observability and evals (35k+ GitHub stars, 90B+ observations/month, 90M+ package downloads/month). 21 of the Fortune 50 and leading AI-native businesses like Ramp, Hugging Face, and Canva rely on Langfuse to understand and improve their AI agents. Open source is important here as an AI gateway needs to be scalable, reliable, and self-hosting it air-gapped can solve many security concerns in regulated industries.

Adding a gateway was widely requested by our users. To get a complete picture of your company's AI usage, you need to see every model call. Our 100+ integrations already cover many of the AI agents you build, but much of a company's AI usage happens in tools nobody instruments, like coding agents, internal assistants, and scripts. A gateway sits centrally, where every call passes, so nothing is missed. And unlike a standalone gateway, which can only count tokens and cost, Langfuse turns each call into a full trace you can search, evaluate, and improve. This is why adding a native gateway is a natural next step for Langfuse.

Our gateway goes far beyond cost accounting

Most gateways tell you what AI costs. Langfuse AI Gateway goes further, because every call through it becomes a full Langfuse trace (opt-in).

See your spend. Cost and usage for every call land in one AI spend dashboard, by model and provider. Set spend limits per key and get alerted before anyone runs into a surprise bill.