← Projects

How I would design an enterprise Data & AI platform

A progressive reference design for an enterprise that wants to move from isolated AI pilots to governed delivery at scale without creating a new portal, platform or integration for every use case.

Platform components

Agent development

Channels

user request, response

Plan and act

every model and tool call, through one path

Integration

AI gateway — controls how AI is used

AI integration — how agents reach systems

inference, retrieval, systems of record

Systems and data

Vector data layer

Model garden

Enterprise systems

Monitoring

Agent development framework

Bought, then finished in-house

One way to build an agent, so the tenth costs less than the first

What it is

A shared scaffold - project layout, typed tool contracts, memory and state handling, a standard planning loop, and a security baseline every agent inherits rather than reimplements.

Why it earns its place

Without it every agent is a bespoke codebase and the marginal cost of the next use case never falls. This is the single largest contributor to the delivery speed-up: the framework is what turns a portfolio from n one-off projects into n instances of one thing.

Bought, then finished in-house — the call

The orchestration runtime is off-the-shelf and not worth writing. The opinionated layer on top of it - our tool contracts, our approval semantics, our residency defaults - is ours, because it encodes constraints no framework author knows about.

LangGraphClaude Agent SDKPydanticPython

AI-assisted development

Out of the box

Coding agents inside the delivery process, not beside it

What it is

Coding agents used by the platform team itself, wired to the repository, the CI logs and the internal service catalogue, and governed the same way any other agent is.

Why it earns its place

The fastest way to build agents is with agents. The discipline is that AI-assisted work goes through the same review and release gates as everything else - otherwise you buy speed at the front of the pipeline and pay it back in defects at the end.

Out of the box — the call

Entirely bought. Nobody should be building a coding agent in-house in 2026; the frontier vendors are moving faster than any internal team can follow. What is in-house is the usage policy and the licence-to-outcome tracking behind it.

Claude CodeCursorGitHub Copilot

Golden eval sets

Built in-house

The release gate - retrieval, tool calls, refusal behaviour

What it is

Versioned, human-reviewed test sets per agent covering retrieval accuracy, tool-call correctness and refusal behaviour, run on every change and required to pass before a release ships.

Why it earns its place

An agent with no eval set cannot be changed safely, so it stops being changed. Evals are what make a platform improvable rather than merely deployed - and in Arabic they are the only signal worth trusting, because the public benchmarks were not usable and native review is the fallback.

Built in-house — the call

The harness is off-the-shelf; the sets are not, and cannot be. A golden set is a written record of what your business considers a correct answer, which is exactly the part no vendor can supply.

RagasPromptfooBraintrustNative Arabic review

Agent harness engineering

Built in-house

Designing the agent's skills, tools, memory and blast radius

What it is

The design practice for an individual agent: which tools it holds, what its memory retains, how its context is assembled, and what it is permitted to write to.

Why it earns its place

Most agent failures are harness failures, not model failures - too many tools, ambiguous instructions, unbounded context, or a write path nobody scoped. Treating the harness as a designed artefact rather than an emergent one is what moves an agent from demo to production.

Built in-house — the call

A practice rather than a product. Anything with a write action uses plan-and-execute rather than a reason-act loop, because a plan is something a person can approve before the fact and an auditor can read after it.

MCPInternal design standard

DevOps and CI/CD

Out of the box

Agents ship on the same pipeline as everything else

What it is

Source control, build, container registry, environment promotion and rollback - with prompts, tool definitions and eval sets versioned as code alongside the application.

Why it earns its place

Prompt changes are code changes with none of the ceremony, which is precisely why they cause incidents. Putting them under the same pipeline is unglamorous and removes an entire class of production surprise.

Out of the box — the call

Bought, and reused from the existing engineering estate rather than stood up separately for AI. A parallel AI pipeline is a governance gap disguised as a convenience.

GitLabGitHub ActionsDockerHelm

Channels and interfaces

Outside the AI platform

Where users already are - the platform does not own these

What it is

Mobile and web apps, chat, the contact centre, WhatsApp, SMS and email. Existing product surfaces that call the platform; they are not built by it.

Why it earns its place

Drawn deliberately as out of scope. A platform that starts building its own front doors ends up competing with the product teams it is supposed to accelerate, and adoption is the number that decides whether a platform works.

Outside the AI platform — the call

Owned by the product and channel teams. The platform's obligation stops at a stable, documented API and starts again at the observability the channel teams need to trust it.

WhatsAppWeb and mobile appsContact centre

Master orchestrator

Bought, then finished in-house

Reads intent, picks the agents, sequences the work

What it is

The entry agent. It interprets a request, decomposes it, routes each step to the specialist agent that owns it, and holds the state of the overall task across turns and across agents.

Why it earns its place

This is what makes the platform compose. Without an orchestrator every new use case needs its own end-to-end agent; with one, a new use case is often just a new route across agents that already exist. It is also the natural place to put a human approval step, because it is the only component that can see the whole plan.

Bought, then finished in-house — the call

Built on a bought runtime. Routing policy is in-house - the mapping from intent to agent is a description of the organisation, and it changes whenever the organisation does.

LangGraphTemporalClaude

Business-specific agents

Built in-house

Agents that know one business - leasing, billing, reservations

What it is

Agents that encode the process and vocabulary of a single business: a leasing agent, a reservation agent, an in-stay services agent, a billing agent, a collections agent.

Why it earns its place

This is where the measured business outcome sits. Every one carries a named business owner and a measured outcome before it counts as delivered - the platform below it exists so that this layer is the only layer a new use case has to add.

Built in-house — the call

Built, always. This layer is the proprietary part and the reason the rest of the platform is worth buying: you want the differentiated work concentrated here and nowhere else.

Agent frameworkBusiness-owned eval sets

Shared agents

Bought, then finished in-house

Built once, reused everywhere - SQL, OCR, KYC, code execution

What it is

Capabilities that are not specific to any business unit: a SQL agent over the certified data products, a document OCR and extraction agent, a KYC agent, a sandboxed code executor.

Why it earns its place

The reuse layer, and the clearest lever on delivery speed. Every business-specific agent that consumes a shared agent instead of rebuilding it is weeks removed from a delivery plan - and one accredited implementation of KYC is a materially smaller risk surface than nineteen.

Bought, then finished in-house — the call

Mixed by capability. OCR and extraction are bought where a vendor is genuinely better; the SQL agent is in-house because it depends entirely on our own semantic layer; code execution is a bought sandbox with in-house policy around it.

MCPVision modelsSandboxed runtimeSemantic layer

CPU compute

Out of the box

Where the agents themselves run - isolated from inference

What it is

The container platform the agent processes run on: quotas, autoscaling, and namespace isolation from both inference workloads and production applications.

Why it earns its place

Agent orchestration is cheap, spiky and bursty; inference is expensive and steady. Sharing one pool between them means a spike in one starves the other, and the failure is silent unless the isolation is explicit.

Out of the box — the call

Standard container platform, reused. The in-house part is the quota policy - who gets capacity when everything is contending at once.

KubernetesDockerRancher

Observe agents

Built in-house

Agents watching agents - intent, outcome, sentiment

What it is

A small set of agents whose subject is the platform itself: intent classification on inbound requests, outcome capture on completed tasks, and sentiment analysis on customer-facing conversations.

Why it earns its place

They convert conversations into structured evidence. Outcome capture in particular is what lets a business owner claim a result without self-grading, because the record is produced by the system rather than by the team being measured.

Built in-house — the call

Built, because the taxonomies are ours. An intent label set is a description of what your customers ask you for, and no vendor ships it.

Small classification modelsStructured outputs

Agent operations

Bought, then finished in-house

Registry, prompt repository, shared memory store

What it is

The three stores that make agents an estate rather than a collection: a registry of every agent with its owner, tools and permissions; a versioned prompt repository; and a shared memory store agents write to and read from.

Why it earns its place

The registry is what stops the tenth team from building the ninth team's agent - no shared directory means no reuse, whatever the framework allows. The memory store is what lets agents improve with use instead of restarting cold on every request.

Bought, then finished in-house — the call

The stores are bought infrastructure. The registry itself is in-house and small, because it has to reconcile against our own identity, ownership and cost-centre model to be worth anything.

PostgreSQLRedisGit

API endpoints

Bought, then finished in-house

One address for every model, so callers never bind to a vendor

What it is

A single OpenAI-compatible surface in front of every model, frontier and self-hosted alike, with per-team keys and quotas.

Why it earns its place

Callers depend on the gateway, not on a vendor. That is what makes swapping a model a configuration change rather than a migration, and it is the reason the routing, cost and privacy controls below are enforceable at all - there is exactly one path to enforce them on.

Bought, then finished in-house — the call

Bought gateway, in-house policy. This sits alongside the general-purpose enterprise integration platform rather than replacing it - that layer was designed for request-response systems and still does that job, while tokens, streaming and model routing are a different problem that wants a purpose-built gateway.

LiteLLMKongOpenRouterMuleSoft

Model routing

Built in-house

Cheap models for cheap tasks, decided centrally

What it is

Policy that steers each request to the right model by task class, sensitivity and cost - frontier models for hard reasoning, small or self-hosted models for classification, extraction and summarisation.

Why it earns its place

Most enterprise AI spend is simple tasks running on frontier models because nobody made the routing decision. Centralising it means the saving applies to every agent at once, including the ones written before the policy existed.

Built in-house — the call

In-house. Routing rules encode our own cost model and residency rules; the honest finding behind them is that chat volume alone does not justify self-hosting - batch document and agentic workloads consume tokens at a different rate and carry the case.

ClaudeAzure OpenAIQwenRouting policy

RBAC and identity

Out of the box

An agent inherits the caller's permissions, never exceeds them

What it is

Identity propagation from the user through the orchestrator to every downstream agent and tool call, with role-based limits on models, tools and data.

Why it earns its place

The failure mode this prevents is an agent running with more authority than the person who asked it - the quickest way to turn a helpful assistant into an uncontrolled data-exfiltration path. Propagating identity rather than using a service account is the whole control.

Out of the box — the call

Bought, and deliberately the existing corporate identity provider rather than anything AI-specific. A second identity system is a second place to forget to revoke access.

Entra IDKeycloakOAuth 2.0

Semantic cache

Out of the box

The same question, asked a thousand times, answered once

What it is

An embedding-similarity cache in front of the models, scoped per tenant and invalidated when the underlying source changes.

Why it earns its place

Enterprise question distributions are extremely top-heavy - a small number of questions account for a large share of traffic. Caching those cuts both cost and latency, and latency is what adoption actually responds to.

Out of the box — the call

Bought as a gateway feature. The in-house part is the invalidation rule, which is the part that decides whether the cache is a saving or a source of confidently stale answers.

RedisEmbedding models

Budget controls

Bought, then finished in-house

Hard caps and chargeback per team, agent and use case

What it is

Spend limits and alerts by team, agent and use case, with consumption attributed back to a cost centre rather than pooled into one platform bill.

Why it earns its place

Unattributed AI spend is a platform-owned cost that nobody defends and no business owner feels. Attribution turns it into a line each owner has to justify, which is a far stronger cost control than any technical measure.

Bought, then finished in-house — the call

The metering is a gateway feature; the mapping from key to cost centre is in-house and has to be maintained against a moving organisation chart.

Gateway meteringFinance chargeback

Privacy and residency

Bought, then finished in-house

Regulated data does not leave the estate, whatever it costs

What it is

Classification and redaction at the gateway, with a hard rule that data bound by residency requirements is only ever served by models hosted inside the estate.

Why it earns its place

Residency is a constraint, not an economic decision, and the only reliable place to enforce it is the single point every request already passes through. Enforced anywhere else it becomes a code-review convention, which is another way of saying it will eventually be missed.

Bought, then finished in-house — the call

Bought detection, in-house policy. The classification taxonomy and the residency rules are regulatory positions we own and have to be able to defend.

PresidioUAE data residencySelf-hosted models

Operational guardrails

Out of the box

Topic, safety and injection controls at one enforcement point

What it is

Input and output checks for prompt injection, off-topic use, unsafe content and jailbreak attempts, applied centrally and logged when they fire.

Why it earns its place

Guardrails written per agent drift apart immediately and are impossible to audit as a set. One enforcement point means a new class of attack is answered once for every agent on the platform, including the ones nobody has looked at in months.

Out of the box — the call

Bought, and layered rather than singular - no one classifier catches everything, and the violation log is more operationally useful than the block itself.

Guardrails AILlama GuardContent filters

MCP gateway

Bought, then finished in-house

One brokered path from agents to enterprise tools

What it is

A broker between agents and MCP servers that handles discovery, authentication, per-tool permissions and audit, so an agent asks for a capability rather than for a system.

Why it earns its place

Without a broker, every agent holds its own credentials to every system and the permission surface is unknowable. With one, the question 'what can this agent reach?' has a single answer that can be produced on demand.

Bought, then finished in-house — the call

Bought broker, in-house permission model. The permission model is the part that has to match our own systems of record, so it is not something that arrives configured.

MCPKong

MCP servers

Bought, then finished in-house

Core systems exposed once, consumed by every agent

What it is

One server per system of record - ERP, CRM, property management, HR, the data platform - exposing a small, well-named set of typed operations rather than a mirror of the underlying API.

Why it earns its place

This is the integration cost that used to be paid once per use case and is now paid once per system. It is also where the delivery multiple comes from in practice: the second and third agent against a system integrate in days because the first one paid for the server.

Bought, then finished in-house — the call

Vendor-supplied where the vendor ships one, in-house otherwise. The in-house ones are deliberately thin - a server that reimplements business logic instead of exposing it becomes a second place for that logic to be wrong.

SAPSalesforceOracleFastAPI

Skills

Built in-house

Reusable know-how - the procedure, not the tool

What it is

Packaged, versioned procedures an agent can load on demand: how a credit note is raised, what a compliant lease amendment contains, which checks a vendor onboarding requires.

Why it earns its place

Tools give an agent capability; skills give it competence. Encoding a procedure once means every agent that loads it is correct in the same way, and correcting the procedure corrects every agent at once rather than one prompt at a time.

Built in-house — the call

Built, and authored with the business rather than for it. A skill is institutional knowledge written down, which makes the bottleneck subject-matter time rather than engineering time.

Versioned skill libraryBusiness SMEs

Tools

Built in-house

Typed, permissioned actions with an audit record

What it is

The individual callable actions - read a record, run a query, raise a request, send a document - each with a typed contract, a permission, and a log entry per invocation.

Why it earns its place

The unit of trust in an agent platform. Anything that writes is scoped narrowly and logged, because when a decision is questioned six months later the tool log is the only account of what the system actually did.

Built in-house — the call

Built against the MCP servers. Read tools are permissive by default and write tools are not - the asymmetry is deliberate and it is what makes the platform deployable in finance at all.

MCPOpenAPIAudit log

Vector store

Out of the box

Retrieval over unstructured content, with permissions preserved

What it is

Embeddings and hybrid retrieval over contracts, policies, manuals and correspondence, with source-system permissions carried into the index.

Why it earns its place

Retrieval quality is the ceiling on answer quality - no model recovers from the wrong chunk. Carrying permissions into the index matters just as much: an index that flattens them turns every search into a potential disclosure.

Out of the box — the call

Bought, and treated as replaceable. Chunking, embedding choice and the evaluation of retrieval are in-house and are where the quality actually comes from.

QdrantpgvectorOpenSearch

Frontier models

Out of the box

Commercial APIs for the hardest reasoning

What it is

Commercial frontier models reached through the gateway, used for long-horizon reasoning, agentic work and anything where quality dominates cost.

Why it earns its place

The capability frontier moves faster than any internal programme can track, and staying on it is a procurement and integration problem rather than a research one. The platform's job is to make adopting the next one cheap.

Out of the box — the call

Bought, with more than one vendor deliberately live at all times. Single-vendor dependence at this layer is a commercial position as much as a technical one.

ClaudeAzure OpenAIGemini

Open-weight models

Bought, then finished in-house

Self-hosted inference for residency and for volume

What it is

Open-weight models served on internal GPUs behind the same gateway interface as the commercial APIs.

Why it earns its place

Two jobs. It is the only way to serve data that cannot leave the estate, and it is the cost floor for high-volume batch work where per-token pricing stops making sense. Both reasons are structural rather than a preference for self-hosting.

Bought, then finished in-house — the call

Open weights, in-house serving. Running inference well is a real operational discipline - batching, quantisation, KV cache and utilisation - and it is the part that decides whether self-hosting is actually cheaper.

QwenLlamavLLM

In-house models

Built in-house

Fine-tuned and classical models where a general model is wrong

What it is

Fine-tuned small models and conventional machine learning for narrow, high-volume, well-specified tasks - classification, forecasting, scoring, extraction against a fixed schema.

Why it earns its place

Plenty of work does not want a language model at all. A gradient-boosted model on tabular data is cheaper, faster and more accurate than a frontier model for most forecasting, and pretending otherwise is how AI programmes end up expensive and unimpressive.

Built in-house — the call

Built. Also the layer with a real ongoing cost - a fine-tune is a maintenance commitment, so the bar for creating one is a task that is genuinely stable and genuinely high-volume.

PyTorchHugging FaceMLflow

GPU compute

Out of the box

Inference capacity with quotas, not one shared server

What it is

Pooled GPU capacity for serving and fine-tuning, partitioned by workload class with explicit quotas and isolation from production application workloads.

Why it earns its place

A single shared inference server is fine until the portfolio grows, at which point contention becomes a material availability risk with no detection mechanism. Sizing and isolating capacity is a precondition for scale rather than an optimisation.

Out of the box — the call

Bought hardware, in-house scheduling. Vendor mix is a deliberate hedge against both supply constraints and single-vendor pricing.

AMD InstinctNVIDIAKubernetes

Enterprise systems and data

Bought, then finished in-house

The lakehouse and the systems of record everything resolves to

What it is

The data foundation and the transactional systems behind it: a lakehouse, certified data products, a master customer identifier, catalogue and lineage, and the ERP, CRM and operational systems that own the truth.

Why it earns its place

The constraint on the whole diagram. Agents inherit the quality of what sits here - without one certified source and a master identifier, an agent produces confident answers that reconcile to nothing, and it will be blamed for a data problem it merely surfaced.

Bought, then finished in-house — the call

Bought platform, in-house data model. The unglamorous finding is that most of the effort is governance rather than technology: catalogue and lineage tools already licensed and switched off do nothing, and turning them on requires named data owners more than it requires software.

DatabricksDelta LakeSAPOracle

AI tracing

Out of the box

Every step of every run, reconstructable after the fact

What it is

Distributed tracing across the full run - prompt, retrieval, each tool call, each model hop - retained and searchable.

Why it earns its place

When an agent gets something wrong, the question is which step failed, and without traces the answer is guesswork. This is the difference between fixing a problem and rewriting the prompt until the symptom disappears.

Out of the box — the call

Bought, on OpenTelemetry semantics so the platform is not locked to one backend. Instrumentation is in the framework, so agents are traced by construction rather than by remembering to.

LangfuseOpenTelemetryGrafana

Continuous evaluation

Bought, then finished in-house

Sampled production traffic, scored, with alerts on decline

What it is

Live traffic sampled and scored automatically - model-as-judge for open-ended output, deterministic checks where the answer is checkable - with alerts when a score trends down.

Why it earns its place

Agents degrade quietly. Sources change, a model is updated underneath you, users ask new things. Pre-release evals cannot see any of that, so continuous scoring is what turns quality from an assumption into an observed quantity.

Bought, then finished in-house — the call

Bought scoring, in-house rubrics. The judge itself is evaluated against human labels periodically, because an unvalidated judge is a metric that can be wrong in a comfortable direction.

LLM-as-judgeLangfuseRagas

Agent scorecards

Built in-house

One page per agent - quality, cost, usage, owner

What it is

A standing per-agent record of accuracy, cost per task, volume, latency and adoption, attributed to a named owner and a business unit.

Why it earns its place

Makes the portfolio comparable. It is also how underperforming agents get retired, which matters more than it sounds - a portfolio that only ever adds becomes a maintenance burden that consumes the capacity meant for new work.

Built in-house — the call

In-house, because it joins across cost, quality and business outcome and no single vendor holds all three.

GrafanaInternal reporting

Cost metering

Bought, then finished in-house

Token-level cost attributed to an agent and a cost centre

What it is

Per-request token and infrastructure cost, attributed to agent, use case, business unit and cost centre, reported alongside the outcome the spend bought.

Why it earns its place

Cost per outcome is the only number that survives a budget conversation. Total spend invites a cut; cost per completed task with a measured result invites a decision, and it is what lets the platform be defended on its own terms.

Bought, then finished in-house — the call

Gateway metering joined to internal finance mappings. The joining is the work, and it has to be maintained.

Gateway meteringCost model

Guardrail violations

Built in-house

What was blocked, where, and whether it was right to block

What it is

A log of every guardrail firing - injection attempts, blocked topics, refusals, residency denials - reviewed for both misses and false positives.

Why it earns its place

The false-positive rate matters as much as the block rate. Over-blocking is the fastest way to lose the users a platform was built for, and the only way to see it is to review what was refused rather than only what got through.

Built in-house — the call

The detectors are bought; the review process is in-house and periodic, because the judgement of whether a refusal was correct is a business call.

Violation logPeriodic review

Logging and alerting

Out of the box

Conventional operations, held to the same standard

What it is

Availability, latency, error rates, saturation and on-call paging for the platform's own services - gateway, orchestrator, inference, retrieval.

Why it earns its place

An AI platform is still a production system, and most incidents are ordinary ones: a full disk, an expired certificate, a saturated GPU. Treating it as exotic is how basic operational discipline gets skipped on the most visible system in the estate.

Out of the box — the call

Bought and reused from the existing operations estate. There is no reason for AI to have its own paging stack.

PrometheusGrafanaOpenTelemetry
The reference architecture. Everything an agent does — reach a model, call a tool, touch a system of record — passes through the integration layer, which is what makes routing, cost, residency and safety enforceable at one point rather than argued in every code review.

The mistake I would avoid is treating the diagram above as a shopping list. Installing every box before a real workflow exists produces an expensive platform with no evidence that its abstractions are the right ones. Building every use case independently creates the opposite problem: quick pilots that cannot share controls, integrations or operating knowledge.

I would build the platform progressively. Each stage should remove a constraint that has appeared in delivery. If a component does not yet remove repeated work, reduce risk or expose useful evidence, it can wait.

Stage 0: define the work before defining the stack

Every initiative starts with a short contract:

  • the business outcome and its owner;
  • the user, decision or process being changed;
  • the systems and data involved;
  • the consequence of a wrong answer or action;
  • the baseline cost, time and quality;
  • the unit in which value and operating cost will be measured.

This is also the first governance gate. The Map function in the NIST AI Risk Management Framework exists to establish context, stakeholders and system boundaries before risk is measured or managed. Sometimes this step concludes that the right answer is a rule, a search index, conventional machine learning, or deleting part of the process. That is a successful decision, not a failed AI proposal.

I would risk-tier the work according to what the system can do. Reading a public document is not the same as recommending a financial decision; drafting a transaction is not the same as executing it. Higher consequence means stronger evaluation, narrower permissions, explicit approval and more production evidence. It should not automatically mean another committee.

Stage 1: prove one narrow workflow end to end

The first production use case should be valuable but bounded. It should have a named user group, a reachable source of truth, a measurable baseline and a failure mode the team can inspect. I would borrow mature capabilities wherever possible: a hosted model, an existing identity provider, the current application runtime and the observability stack already operated by the enterprise.

The aim is not merely to prove that a model can answer. It is to prove the whole service:

  1. the user can reach it in the place where work already happens;
  2. the right data can be retrieved under the user’s own permissions;
  3. the output meets a defined quality bar;
  4. failures abstain, escalate or fall back safely;
  5. usage, quality, latency and cost can be reconstructed;
  6. someone owns the service after launch.

At this stage I would prefer an extension inside the existing ERP, CRM, finance or productivity system when the workflow remains inside that system. A separate portal is not evidence of an AI strategy. It is another product to secure, support, teach, upgrade and eventually retire.

Stage 2: standardise the paths that have started to repeat

Once a second and third workload repeat the same plumbing, I would establish the first platform contracts.

One path to models. An AI gateway provides approved model access, identity, rate limits, policy enforcement, routing, logging and cost attribution. The application asks for a capability rather than binding itself to a provider. Model changes can then be evaluated and configured instead of rewritten. McKinsey’s guidance on scaling GenAI similarly recommends open modular architecture and a central gateway for approved access, policy, attribution and logging.

One identity chain. The user’s identity and permissions should survive the whole request: interface, orchestrator, retrieval and system action. Shared service credentials are easier in the first demo and become ungovernable at scale.

One telemetry envelope. Every model call, retrieval, tool invocation and approval should carry a trace identifier, use-case owner, environment, model, latency, cost and outcome. Sensitive content does not need to be copied into every log, but the path must be reconstructable.

One evaluation contract. Each product owns a versioned set of representative tasks, expected behavior and unacceptable failures. Retrieval quality, answer quality, tool arguments, refusal behavior and business outcome are different measures; one aggregate score hides the part that broke.

Stage 3: make integrations reusable products

The most valuable platform assets are often not model abstractions. They are governed ways to reach enterprise systems.

I would integrate a system of record once, behind a stable contract, then let multiple products consume it. Read tools and write tools should be distinct. Writes should be typed, narrowly scoped and idempotent where possible. The tool should enforce what the caller is allowed to do; a prompt instruction is not an authorization boundary.

The platform should know:

  • who owns the integration;
  • which operations and data classes it exposes;
  • how identity and consent are propagated;
  • what is logged and retained;
  • how versions and breaking changes are handled;
  • how retries, timeouts and partial failure behave.

This is where compounding begins. The first workflow pays for a trustworthy connection; later workflows reuse it. Point-to-point connectors do the reverse: every use case creates another dependency that must be rediscovered during an incident.

Stage 4: improve data in the path of value

I would not pause delivery for an enterprise-wide data perfection programme. Nor would I pretend AI can repair an undefined source of truth.

The practical sequence is use-case-led. Certify the data products needed by the next valuable workflows: ownership, semantics, quality expectations, lineage and access policy. Resolve shared identities where cross-system journeys need them. Add retrieval only where unstructured knowledge is part of the decision, and evaluate retrieval separately from generation.

Agents inherit the quality of the records and rules they can reach. When two systems disagree, the platform needs an explicit authority or reconciliation rule. Otherwise the model will confidently express an ambiguity the enterprise has not resolved.

Stage 5: turn controls into a delivery system

As the portfolio grows, manual review becomes the bottleneck it was meant to prevent. I would convert repeatable governance claims into executable gates:

  • approved data classes and model routes;
  • threat modelling for untrusted content and tool access;
  • golden-set evaluation and regression thresholds;
  • prompt-injection and data-egress tests;
  • permission and cross-tenant isolation tests;
  • human approval for consequential actions;
  • rollback, incident ownership and evidence retention;
  • cost and latency budgets by use case.

Governance remains accountable to people, but evidence is produced by the delivery path. NIST calls governance cross-cutting; it is not a final stage after development. The AWS Well-Architected Generative AI Lens also treats quality, traceability, excessive agency, reliability and cost across scoping, development, deployment and continuous improvement.

Stage 6: federate delivery without federating the controls

At scale, a central team cannot build every domain workflow and should not try. I would split responsibility deliberately:

  • a platform team owns shared model access, identity patterns, integration contracts, evaluation infrastructure, observability and paved-road templates;
  • domain product teams own the problem, workflow, data meaning, user adoption, golden examples and business outcome;
  • security, risk, legal and data owners define policy and escalation paths that the delivery system can enforce and evidence.

This is a federated product model, not unrestricted self-service. Teams can move quickly because the compliant path is the easiest path, while exceptions are visible and deliberate.

Models are a portfolio, not a platform identity

I would keep several model options behind the gateway: frontier commercial models, smaller efficient models and - where residency, economics or control justify the operational burden - self-hosted open-weight models.

Routing should be driven by an evaluation and policy matrix:

  • quality on the actual task and language;
  • data classification and residency;
  • latency and availability;
  • context and tool requirements;
  • cost at expected volume;
  • provider concentration and exit cost.

The cheapest model per token is not necessarily the cheapest workflow. A weaker model that retries, calls more tools or sends more work to a human can cost more per successful outcome. The emerging work of the Linux Foundation’s Tokenomics Foundation is useful here: connect token telemetry and total cost to the value produced.

Agents require a stronger action boundary

An agent should not receive broad access because it may need a capability later. Give each agent the minimum tools and data required for the current task. Treat untrusted content as data, not as instruction. Separate planning from execution when an action is consequential, so a person or policy engine can inspect the proposed operation before it happens.

For every action I would want to answer:

  1. Which identity requested it?
  2. Which data and instructions shaped it?
  3. Which policy allowed it?
  4. What exactly changed?
  5. Can it be reversed or compensated?
  6. What evidence remains?

That is the difference between an impressive demonstration and an operable enterprise service.

What “perfect” means

The perfect platform is not the one with the most boxes. It is the one that makes the next safe, useful product materially cheaper to deliver and operate.

I would judge it by declining time to the first production workflow, reuse of integrations and controls, evaluation coverage, adoption, successful outcomes, incident rate, cost per outcome and the speed at which a model or vendor can be changed. I would also track what has been removed. A platform that only adds components accumulates its own legacy.

The architecture should therefore remain smaller than the portfolio it enables: buy mature commodity capability, build the judgment and workflow specific to the enterprise, and introduce a new surface only when the user or process truly needs one. I expand that decision in what to build, what to buy, and where AI should live.

AI gateway · governed integrations · evaluation and observability · model portfolio · workflow orchestration