What to build, what to buy, and where enterprise AI should live
· Enterprise AI, Architecture, Build vs buy
“Build or buy?” is too broad to be useful. An enterprise AI product is a stack of decisions: model, infrastructure, orchestration, integration, evaluation, policy, workflow and interface. The correct answer can be “buy” at the model layer, “configure” at the platform layer and “build” at the workflow layer in the same product.
My default is simple:
Buy what represents the market’s accumulated capability. Build what encodes the enterprise’s judgment, process and advantage.
That is only half of the decision. The other half is placement. A team can make the correct decision to build and still create unnecessary debt by putting the result in a new portal when it belonged inside the ERP, CRM or finance system where the work already happens.
The first test: is the capability already a product?
I would buy when a capability is world-class, requires little organization-specific customization, can run as an independent service, and has a healthy market that will improve it faster than an internal team can.
Typical examples include:
- foundation and language models;
- OCR and document-layout models;
- object detection and speech models;
- coding harnesses and developer copilots;
- commodity inference infrastructure;
- tracing backends, secret stores and common security tooling.
Buying here is not a concession. It is focus. Training a general model or maintaining a coding harness does not make most enterprises more distinctive. It transfers scarce engineering capacity away from the workflows, integration and evaluation that only the enterprise can define.
A bought capability still needs an exit. I want a stable interface, exportable data, observable behavior and an evaluation suite owned by the enterprise. If a provider change requires rewriting the product, the team did not buy a commodity; it accepted an architecture dependency without pricing the exit.
Buy, configure, or build is a layered decision
Integration
The middle is where most real decisions land.
Buy and configure when the underlying capability is mature but the enterprise must control access, residency, retention, routing or acceptable behavior. A hosted model behind an enterprise gateway is the obvious example: the model is bought; approved use, policy and evaluation are owned.
Buy and integrate when the capability is good but must participate in a larger process. Keep the adapter thin. The vendor should remain replaceable and the business process should not be expressed in vendor-specific configuration that only one consultancy can understand.
Build when the value comes from changing how operations work, introducing a new process, applying organization-specific judgment, or creating an experience that spans systems. This is where internal product and engineering teams add the most value.
McKinsey’s guidance on scaling GenAI reaches a similar conclusion: use provider capabilities by default, except where a capability directly carries proprietary advantage, and make the architecture open and modular enough to reuse and replace components.
The second test: where should the capability live?
I use the smallest-surface rule: put the capability in the smallest existing product boundary that can own the workflow properly.
- Does the work and its data stay inside one enterprise system? Build inside that system ERP, CRM, finance, HR or service-management agent
- Does a defined process cross systems, approvals or exceptions? Build a governed workflow State, hand-offs and recovery are the product
- Is there a new cross-system user experience or operating model? Build a focused portal Only now has a new interface earned its maintenance cost
- Does the capability need no new human interaction? Build a service, not a screen API, event consumer, scheduled job or background agent
Inside the enterprise system
If an accounts-payable agent reads invoices, checks purchase orders and drafts an ERP transaction, I would first place it inside the ERP experience. The system already has users, roles, records, audit history and operational support. The agent should extend those controls rather than reproduce them.
The same applies to CRM sales assistance, finance analysis, service-management triage and HR case support. If integration beyond that product is not required, a separate application adds sign-on, permissions, training, support and data synchronization without adding user value.
As a workflow
When the value is the process between systems, the workflow should be explicit. Examples include onboarding, claims, procurement exceptions and contract approval. These need durable state, timeouts, approvals, escalation and compensation when one step fails.
An agent may classify, extract, recommend and prepare actions inside that workflow. It should not make the process an invisible conversation. Operators need to see where work is, why it stopped and what can safely happen next.
As a portal
A portal is justified when the enterprise is introducing a genuinely new way for a user to work across systems: a unified operations cockpit, a new customer journey or a research environment that combines governed data and tools.
Even then, I would keep it narrow. A portal should own a coherent job, not become the place where every orphaned AI feature is placed. Once it becomes a generic landing page, each new tile adds another support path while adoption fragments.
As a service with no interface
Some of the highest-value capabilities should be invisible: document classification, anomaly detection, routing, reconciliation and event-driven exception handling. If nobody needs to interact with the model directly, do not build a chat box to prove AI is present.
Software surface area is an operating cost
When I choose to build, I try to minimize the number of portals, runtimes, databases, queues, policy engines and connectors. Every component creates at least six obligations:
- ownership and on-call response;
- identity, secrets and patching;
- observability and cost allocation;
- data retention and recovery;
- upgrades and compatibility;
- eventual migration or retirement.
“Composable” is useful until the components outnumber the team that understands them. I would add a component only when it removes more repeated work or risk than it adds in operational obligation.
This is why I prefer one governed model path, reusable system contracts and the enterprise’s existing identity and observability services. The goal is not a visually impressive architecture. It is a small platform that makes a large portfolio manageable.
The decision changes as the AI journey matures
- 1 Prove One workflow, bought defaults, named outcome and owner
- 2 Productize Identity, evaluation, fallback, telemetry and unit cost
- 3 Standardize Gateway, reusable integrations and paved-road templates
- 4 Federate Domain delivery with shared policy and evidence
- 5 Optimize Route models, remove duplication and retire weak products
Prove
Start with bought defaults and existing systems. The gate is evidence of a valuable, bounded workflow - not architecture completeness. Record the baseline and define the unacceptable failure.
Productize
Before expansion, establish user identity, permission propagation, evaluation, fallback behavior, incident ownership and telemetry. This is where a demo becomes a service.
Standardize
When teams repeat model access, integrations or evaluation plumbing, make those capabilities shared. Do not standardize imagined repetition. The platform earns each abstraction through real delivery.
Federate
Domain teams can now own workflows while a platform team owns common contracts and controls. The NIST AI RMF is useful here because Govern, Map, Measure and Manage span the lifecycle; governance is not a sign-off added at the end.
Optimize and retire
Route workloads by measured quality, risk, latency and total cost. Replace providers when the evaluation evidence supports it. Consolidate duplicate portals and integrations. Retire products that do not reach their users or outcomes.
Cost discipline: price the outcome, not the token
Token price matters, but it is not the unit economics of an AI workflow. I would calculate:
model calls + retrieval + tools + infrastructure + retries + human review + support, divided by successful business outcomes.
That reveals failures hidden by a cheap model. A lower-cost model may call more tools, retry more often or send more cases to people. A self-hosted model may look cheap at high utilization and expensive once idle capacity, engineering and support are included.
Cost attribution should exist from the first production release: owner, use case, environment, model and outcome on every trace. Budgets should trigger design decisions - smaller models, caching, deterministic preprocessing, batch work, shorter context - not only alerts after spend has happened.
The Linux Foundation’s Tokenomics Foundation is developing vendor-neutral practices that connect token telemetry, total cost and AI value. The important shift is from “how many tokens did we buy?” to “what useful output did the consumption produce?”
The review I would run before approving delivery
Buy
- Is the capability genuinely mature and independently deployable?
- Does it need little organization-specific customization?
- Can we measure it against our own tasks and risk tolerance?
- Are data use, residency, retention and service boundaries acceptable?
- Can we replace it without rewriting the product?
- Is its full operating cost lower than owning the capability?
Build
- Does this encode proprietary process, judgment or experience?
- Is there a named product owner and a durable team?
- Can we build it from fewer components?
- Are we reusing identity, integrations, observability and evaluation?
- What will be retired or avoided because this exists?
Place
- Can it live inside the system where work already happens?
- Is a workflow more honest than a conversational interface?
- Does a new portal own a coherent cross-system job?
- Could the capability operate as a service without another screen?
The practical position
I buy models and mature horizontal capabilities. I build business workflow, domain controls, integration contracts, evaluation and the user experience when those are specific to the enterprise. I place the result inside an existing system whenever that system can own the work, use a workflow when the process crosses boundaries, and create a portal only when a new cross-system experience has earned one.
The discipline is not ideological. It is economic. Every internal build should justify why the market cannot supply it; every purchase should preserve the enterprise’s judgment and exit; every new interface should justify the years of maintenance that follow its launch.
That is how I would keep enterprise AI ambitious without allowing the software estate to grow faster than the value it creates.