On-Premise vs Client Cloud vs Managed Cloud: Where Your AI System Should Run

Every AI system deployment answers the same question, and it usually gets treated as a matter of technical taste: where does this thing run? Taste has nothing to do with it. The answer decides who holds the root credentials, who is awake when the system breaks at 3am, and who pays for the capacity nobody is using at 4pm on a Sunday.

There are three answers in practice. This post covers what each one is, what it costs you, and the four questions that select between them.

The three shapes

On-premise. Your hardware, in your building or your colo cage, racked and maintained by your people. You own the machines, the depreciation schedule, and every layer above the concrete.

Client cloud. Your AWS, Azure, or GCP account. We design and build the system and deploy it into infrastructure you already own, under your existing organization, your existing guardrails, and your existing bill.

Managed cloud. A dedicated environment we run for you, pinned to your region, with our team on call for it. You get the system as a working thing rather than as a set of components to operate.

One line collapses a decision people treat as a genuine three-way split: client cloud and managed cloud are the same architecture with a different name on the invoice and a different phone ringing at 3am. The same services, the same region, the same isolation model. On-premise is the only one of the three that is architecturally a different animal.

Residency is table stakes. The network path is the difference.

The most common reason a team asks for on-premise is data residency, and it is the one reason that does not survive contact with the facts.

All three options run in-region. AWS has Canadian regions in Montreal and Calgary, UK capacity in London, and US regions in every corner of the country. Pin the deployment to ca-central-1 and the data sits in Canada at rest, in transit between services, and in every log and backup, exactly as it would if the server were in your basement. Whatever residency clause you are working against, whether that is PIPEDA, PHIPA, UK GDPR, or a contractual commitment to your own customers, all three shapes satisfy it the same way.

Residency selects nothing. Anyone telling you on-premise is the only compliant option is selling hardware.

What differs between the three is how traffic reaches the system, and that is a gradient rather than a binary:

• Managed cloud is reached over the public internet by default. The traffic is encrypted and stays in-region, but it crosses public network segments to get there. That can be tightened: PrivateLink, a site-to-site VPN, or Direct Connect all move the traffic onto private paths, at which point it never touches the public internet either.
• Client cloud sits inside your VPC, behind the private network you already run. Your staff reach it the way they reach any other internal service, without leaving your network.
• On-premise can be sealed completely. No egress, no ingress, no internet at any point. This is the one property on-premise has that neither cloud option can replicate at any price.

If your requirement is a true air gap, that is a real requirement, and only one of these three meets it. But the seal has a price, and it is steeper than most teams expect. An air-gapped system gets no automatic updates, no remote support session when something misbehaves, and no telemetry your vendor can act on. New model weights arrive on physical media and get installed by hand.

You have cut exactly the loop that lets an AI system improve after launch. That is worth doing when the air gap is a mandate. It is expensive when it is a preference nobody ever pressure-tested.

On-premise

Start with where it genuinely wins, because it does: classified environments under a formal air-gap mandate, isolated OT and industrial control networks, and the rare case of a client sitting on a large GPU fleet already bought and already running near its utilization ceiling. In those situations on-premise is correct and the rest of this section does not apply to you.

Everywhere else, here is what you take on.

Capacity becomes a three-year bet you place before you know the workload. You size the cluster during procurement, based on an estimate of demand for a system that is not built yet. GPU generations turn over faster than the depreciation schedule you bought them on, so the bet is not only early, it ages badly.

You buy for peak and run at average. Cloud capacity flexes with demand and can drop to nothing overnight. Owned hardware costs full price whether it is saturated or idle, and most AI workloads are extremely spiky: quiet all night, hammered for two hours after the morning standup.

Everything underneath gets rebuilt by hand. Identity, secrets management, encrypted backups, patching, log aggregation, metrics, alerting, certificate rotation. On cloud these are line items. On-premise they are projects, staffed by the platform engineers you already cannot hire enough of, and their salaries dwarf the hardware savings that motivated the move.

Hardware failure is a purchase order, not an API call. A dead GPU on cloud is a replaced instance in minutes. A dead GPU in your rack is a support contract, a lead time, and someone driving to the data centre.

"In our building" is not the same as "audited." A cloud region arrives with SOC 2, ISO 27001, and the provider's control attestations already in place, and your auditor already knows how to read them. On-premise, you inherit every one of those controls yourself and evidence them yourself, on your own schedule, with your own people.

The model layer moves quarterly. The hardware does not. The useful frontier for open-weight models has shifted several times in the period covered by a single hardware refresh cycle. Systems that can adopt a better model in an afternoon compound; systems that need a capital request to do it do not.

Client cloud

Building into your own cloud account is a legitimate choice and we do it regularly.

What you get. Data never leaves an account you own. Your committed spend and enterprise discounts apply, so the marginal infrastructure cost is often lower than a standalone environment. Your security guardrails, your org-wide service control policies, and your logging pipelines already cover the workload on day one. Your audit boundary does not move. And procurement has no new vendor to review, which for some organizations is worth more than everything else on this list combined.

What it costs. Our delivery speed becomes your change-management speed. Access provisioning that takes an afternoon in a managed environment takes three weeks when it routes through your identity team's queue. On-call splits across two organizations, which means every incident starts with a conversation about whose problem it is. And when the build is done, your platform team inherits an AI system to operate alongside everything else they already own.

None of that is a reason not to do it. It is a reason to plan for it, and to write the weeks spent waiting on access into the timeline instead of discovering them halfway through.

Managed cloud

What you get. The fastest path to something running, because there is no access queue between the design and the deployment. One accountable party when something breaks, with no triage about whose layer failed. The production environment matches exactly what we tested against, because we built both.

Upgrades land without a change window, which matters more than it sounds when the model layer moves as fast as it does. In-region residency and the provider's certifications come with the region, not with a project to obtain them. And you pay for what you use rather than for what you might need at peak.

What it costs. It is a new vendor for procurement to assess, with the review cycle that implies. Data is processed in an account we control, which gets answered concretely rather than rhetorically: a data processing agreement, region pinning written into the contract, and encryption under keys you hold and can revoke.

Exit needs a plan too, so here is ours, stated plainly. Handover of the whole environment into your own account is a supported path with a defined process, not a renegotiation. A managed environment you cannot leave is a hostage situation, and we do not sell those.

AWS migration credits

XY Space works closely with AWS, and one practical consequence is funding. We can secure migration credits applied against the assessment and migration work that moves a workload onto managed cloud, which means the first stretch of the engagement is partly funded by AWS rather than out of your budget.

The amount depends on the size and shape of the workload, and we confirm it before you commit to anything rather than after. The reason it is worth raising early is that "we would move, but the migration cost is hard to justify this year" is the most common reason systems stay on hardware that stopped making sense two refresh cycles ago. Frequently, that objection is already solved.

Four questions that select

Who must hold the root credentials? If the answer is a hard "us, with no exceptions," you are choosing between on-premise and client cloud, and client cloud is almost always the better version of that answer.

Do you need an air gap, or do you need private connectivity? These get used interchangeably and they are not the same. Private connectivity, meaning no traffic on the public internet, is available on both cloud options. A true air gap is available only on-premise. Most teams who ask for the first are describing the second, and once that is clear the decision usually resolves itself.

Do you already have a platform team on call 24/7? Not a team that could be on call. One that already is, with a rotation, a runbook, and someone who answers at 3am. If not, on-premise and client cloud both assume one exists without ever saying so.

Is your utilization floor high enough to beat metered pricing? Owned hardware wins when the machines run hot nearly all the time. If your workload is spiky, and most are, you are paying peak prices for average usage, every hour of every day.

What we recommend

Managed cloud, by default. It is the fastest to stand up, the cheapest to run at realistic utilization, the easiest to keep current as the model layer moves, and it satisfies every residency requirement the other options satisfy. Where governance requires the workload to live in your own account, we build into client cloud and plan the timeline honestly around your access and change processes.

On-premise we would steer you away from, with one exception: if you are under a genuine air-gap mandate, that is a real constraint, and it is the one thing cloud cannot give you at any price. Residency, sovereignty, control, audit posture: everything else that sends teams toward their own hardware is answerable in-region, and answerable without buying a GPU cluster you will be depreciating long after the models you bought it for have been superseded.

If you are weighing this decision now, the useful next step is not a vendor conversation. It is sitting down with whoever owns your infrastructure and whoever owns your compliance posture, and running the five gates below. The answer usually falls out before anyone quotes you anything.

The quiz: try to rule out managed cloud

Five gates. Start at managed cloud and attempt to fail out of it. The questions are written to be hard to answer yes to honestly, because in our experience most of the reasons teams give for the other two options do not survive being asked precisely.

If nothing ruled it out, you landed on managed cloud, which is where most teams land and the answer we would have given you before you took the quiz.