One model layer across six AI providers, with active guardrails and tracing
A shared model catalogue and gateway that let teams choose and change models without rebuilding their workflows, with keys, budgets, guardrails and tracing handled in one place.
- Partner
- A multinational advisory firm
- Period
- October 2024 to January 2026
- Providers
- 6
- OpenAI, Anthropic, Google, Groq, xAI and Perplexity
- Catalogue
- 1
- Shared by pipelines and agents
- Guardrail checks
- 3
- PII, prompt injection and moderation
The challenge
Model choice changes quickly. Workflows written against one provider's interface become costly to move, and each team handling its own keys and policies leaves no consistent oversight.
What we built
A model configuration and gateway layer used by both pipelines and agents. Teams select a provider and model from one catalogue, requests route through Portkey or Helicone with keys and budgets held in the gateway, guardrail policies attach per agent, and traces are exported to LangSmith.
What was delivered
- One model catalogue across six providers, with deprecated models flagged
- Provider keys and team budgets held in the gateway, not in workflows
- PII detection, prompt injection checks and moderation on every call from an agent with guardrails attached
Partner background
Our partner is a multinational advisory firm whose teams work with large volumes of documents, spreadsheets, databases and email. It wanted one internal platform where those teams could build data pipelines and AI agents themselves, instead of commissioning a new application for each need. The platform had to run inside the partner's Microsoft 365 and Azure environment, keep each team's work separate, and move work from experiment to production through controlled environments. This case study covers one part of that platform; nine case studies describe the whole.
The challenge
Provider lock-in
Each provider has its own SDK, parameters and model names. Code written for one does not move easily to another when prices, quality or availability change.
No single point of control
Without a common layer, keys, usage, budgets and safety policies are handled differently in every workflow.
Safety in one place
Teams sending firm data to external models needed personal data and injected instructions caught before a call reached the model provider, not left to each workflow.
Visibility
Multi-step agents are hard to debug without a record of each model call and its inputs.
Objectives
- Offer every approved model through one catalogue
- Route model traffic through a gateway that holds keys and applies policy
- Keep the gateway route replaceable
- Apply guardrails consistently to the agents that need them
- Trace agent chains for debugging and review
Our role
CharCentric provided technical leadership and architecture within a multidisciplinary engineering team, and contributed directly to implementation. The platform was built over 16 months, from October 2024 to January 2026, as a Python and FastAPI backend on Azure.
Scope and timeline
The gateway and guardrail integration was built between December 2024 and May 2025. The agent model catalogue was extended between June and September 2025.
Approach
Configuration over code
Models are described by provider, model name and a small set of behaviour settings, so changing a model is a configuration change, not a code change. The catalogue is refreshed from the providers' model lists, and models the providers announce as deprecated are flagged before they are removed.
An interface for the gateway
Portkey and Helicone implement the same gateway interface. A factory selects one by configuration, so the gateway can change without touching agents.
Policies outside the code
Guardrail policies are defined in the gateway and attached to an agent by reference. Policy changes do not require a redeploy.
Implementation
Model catalogue
One catalogue shared by pipelines and agents covers OpenAI, Anthropic, Google, Groq, xAI and Perplexity, from GPT-5 and Claude to Gemini, Llama, Grok and Sonar, with reasoning models marked separately.
Keys and budgets
Provider keys, team budgets, rate limits and guardrail policies are configured in Portkey. Teams reference a key and never hold it. Helicone is an alternative route with its own security checks.
Behaviour settings
Temperature and maximum tokens are validated before a model is created. A seed is passed to the providers that support one, for best-effort repeatability.
Guardrails
When attached to an agent, the guardrails component routes every model call from that agent through the configured policy: PII detection and redaction, prompt injection detection and content moderation.
Tracing
Traces are exported through OpenTelemetry to LangSmith, giving a step-by-step record of prompts, responses and timing.
Tools and technologies
| Tool | Purpose |
|---|---|
| Portkey | Gateway, virtual keys and guardrails |
| Helicone | Alternative gateway |
| LangChain | Provider-neutral model interface |
| LangSmith | Tracing |
| OpenTelemetry | Trace export |
| Pydantic | Validated model configuration |
What was delivered
- Providers
- 6
- Catalogue
- 1
- Gateway options
- 2
- Guardrail checks
- 3
- A single model catalogue shared by pipelines and agents
- Gateway routing with a replaceable gateway product and central keys
- Active guardrail policies attached per agent
- Tracing for agent chains
Why it matters
Choosing a model is a decision organizations will revisit many times. A gateway and a shared configuration keep that decision cheap to change and keep oversight in one place.
If your organization is planning a platform of this kind, or needs a specific part of one designed and delivered, we would be glad to discuss it.