Choosing where AI runs was once a server-location question. It is now a governance question. Inference runs across a full spectrum, from public SaaS through private cloud to air-gapped and on-device, so the old “where can we physically put this?” no longer binds. What is scarce is jurisdiction.
The tell is a US provider storing your data in an Australian region, and the CLOUD Act is why. Where AI runs is a Governance-First Deployment Decision, and “what leaves the machine” becomes a set of channels you can inspect against the Australian Sovereignty Test. It sits at the heart of the local-first shift in on-device AI.
Why Is Deployment Choice Really a Governance Decision Rather Than an Infrastructure One?
The runtime is the last thing you should choose. Infrastructure runs anywhere cheaply, so the binding constraint is whose law and audit governs the data. Compliance, residency, auditability, and liability decide the runtime. Picking a platform first, then retrofitting controls, is the default failure mode, because controls bolted on after the fact routinely fail an APP 8 or IRAP review.
Residency and sovereignty are different things. Residency answers where data physically lives; sovereignty answers whose laws and courts govern it. A deployment can satisfy one while failing the other. The CLOUD Act makes this concrete: it operates on the provider, so a US provider must produce data it holds or controls regardless of where it sits. An Australian or Frankfurt region moves the data without moving the jurisdiction. GDPR Article 48 adds to the bind, since third-country court orders are only valid through mutual legal assistance treaties.
Under Australian Privacy Principle 8 and section 16C of the Privacy Act 1988, you are accountable for an overseas recipient’s handling of personal information, so a foreign model provider’s breach becomes your breach. For government work, ISM and PSPF expectations apply and are assessed through IRAP. Those three checks are the Australian Sovereignty Test, the reason governance leads, and they sit at the centre of the broader local-first picture.
A Data Processing Agreement is necessary but insufficient, because a contract cannot override a statute.
How Do You Evaluate Which Workloads Should Run Locally Versus in the Cloud, and Is It a Build or Buy Decision?
Once jurisdiction selects the architecture, the next question is which workloads sit where. Placement is a per-workload assessment, run separately for each workload. A placement assessment weighs data sensitivity, latency, control, and cost, then maps the workload to an environment. The binary “cloud versus on-prem” question stops being useful once you see the AI Deployment Spectrum: public SaaS, then private cloud in a region, then on-premises, then air-gapped, then on-device, each step shrinking the data perimeter. This is where hybrid cloud-edge inference splits data, model, and control plane.
The Sequential Gate Model applies three gates in order: first regulatory trust, a binary pass/fail; second token-volume economics, the break-even and GPU utilisation question; third team capability, whether you have the MLOps and SRE people. You do not build a cost model before compliance settles.
That leads to build versus buy. Build means self-hosting open-weight models like Llama, Mistral, or Qwen on infrastructure you own. Buy means a managed API or platform. The number most teams omit is the MLOps staffing to run self-hosted inference. Self-hosting costs three to five times the GPU price once power, cooling, and operations count, and the advantage only tips at high, steady volume. The full economics sit in the true cost of local versus cloud.
Two checks flip the verdict most often: do you have the people to run inference and on-call cover, and who controls governance in a cloud-hosted versus on-prem multi-agent platform? If either is unclear, no vendor calculator will answer it; these are governance questions, answered before pricing. On-device, the far end of that spectrum, raises a different problem: the trust boundary moves to the endpoint.
What Is the Endpoint Trust Boundary, and How Do You Assess Trust and Audit Requirements for On-Device Agents?
The Endpoint Trust Boundary is the line beyond which an agent and its data sit outside your physical and logical control. When an agent runs on a user’s device, it lives inside that user’s trust boundary, where it inherits local credentials, files, cached tokens, and sessions, with little oversight.
Moving agents on-device buys privacy but gives up the central logging that produces audit evidence: what was processed, by which components, across which jurisdictions. You need that evidence, and a distributed endpoint loses it. The attack surface also grows, because an agent is already authenticated as the user, so a compromise reaches everything the user can reach.
Sandboxing alone is not enough. Shadow AI, unmanaged agents outside governance, is widespread. Prompt injection is a semantic attack that runs through the model’s own reasoning, and no hardware isolation prevents it. The Model Context Protocol surface lets servers claim permissions without attestation.
Agents are best treated as untrusted principals under Zero Trust and least privilege, with scoped tool calls. Confidential Computing and remote attestation are maturing into a way to close the malicious-operator threat: the operator of the endpoint, who can read memory or tamper with the runtime. Microsoft’s observe-secure-govern pattern (inventory, enforcement, lifecycle policy) is the reference for local-first agents.
What Actually Leaves the Machine, and What Should You Look For in a Local-First Vendor’s Telemetry and Data Egress?
If an agent runs locally, the next question is what still leaves the machine. Data Egress is never a single flow; it is a set of channels, each with its own exit: telemetry and usage metadata, prompts and completions, embeddings and vector stores, and interaction logs. Embeddings are easy to underestimate, but research shows hidden states can be inverted to recover substantial parts of the source text, so a vector store synced to a hosted index can carry your documents across a boundary.
Perplexity Computer is the worked example. Its Privacy Gate is a pre-egress check that decides what may cross from a protected local file to a model. One independent tester’s field report, on the DGX Spark, found it checks plain-text and code files only, while PDFs and Office documents pass through unchecked, and the app keeps several persistent backend connections with mandatory crash reporting. Local-first is a posture you verify channel by channel.
That example shows the label says less than the documentation. Start with the vendor’s data-flow documentation rather than the marketing claim. The documentation worth reading surfaces four things: the data-flow diagram, the telemetry defaults, the foreign endpoints, and the permission blast radius of the harness. At the boundary, DLP, a policy engine, and guardrails are what stop sensitive data crossing the perimeter.
You need a runtime data-trajectory record: which prompt, which model, which endpoint, which jurisdiction, backed by an AI-BOM and a ROPA. If a vendor cannot show you the data trajectory, treat that as a finding.
Where AI runs resolves at the trust boundary and the egress channel. Once jurisdiction selects the architecture and each workload is gated trust-first, what remains is a spectrum of graduated exposure and a set of named channels, both readable from the vendor’s data-flow documentation. The Australian Sovereignty Test settles whatever is still borderline.
Frequently Asked Questions
Is it true that an Australian data centre keeps my data out of reach of US law?
No. Residency and sovereignty are different things. A US-owned provider storing your data in an Australian region has changed geography, not jurisdiction: the CLOUD Act still compels that provider to produce data it holds or controls. Only when the provider sits outside US jurisdiction, and the architecture keeps data and control under Australian law, does residency become sovereignty.
Does the CLOUD Act still apply if my data never physically touches the United States?
Yes. The CLOUD Act operates on the provider, not the server. If a US company holds or controls the data, it can be compelled to produce it regardless of whether the records sit inside or outside the US. An EU or Australian region therefore changes where the bytes rest, not whose courts can reach them. Check ownership and control, not map coordinates.
How do I run the Australian Sovereignty Test on a single workload?
Work through four questions in order. Where does the data physically reside (residency)? Whose law and courts can compel the provider (sovereignty)? Can you produce APP 8 and section 16C evidence, and IRAP or ISM/PSPF alignment for government work? Who is liable if it goes wrong? If any answer is silent about jurisdiction, treat the workload as unresolved rather than safe.
Does “local-first” or “on-device” mean nothing leaves my machine?
No. Local-first shrinks the perimeter but does not close it. Telemetry and usage metadata, prompts and completions, embeddings and vector stores, and interaction logs are distinct egress channels, each with its own exit. A local-first product can still send baseline device-health telemetry by default. Treat “local-first” as a starting posture, then inspect every channel before you trust it.
Is on-premises always more sovereign than the cloud?
Not automatically. A rack you own still runs software, support access, and management planes that may sit under a foreign provider’s control, and remote support can reintroduce the same jurisdiction problem. Sovereignty depends on who can compel access to the data and control plane, not merely where the hardware sits. A well-structured sovereign cloud can outrank a poorly governed on-premises deployment.
What is a Data Processing Agreement, and can it override the CLOUD Act?
A DPA is the contract that sets how a processor handles your data, including cross-border disclosure terms. It is necessary but not sufficient: a contract cannot override a statute. A US provider can agree to Australian-style terms and still be compelled under the CLOUD Act. Use the DPA to record obligations, then verify the legal jurisdiction separately with the Australian Sovereignty Test.
What is prompt injection, and can it move data off a local device?
Prompt injection is a semantic attack that hides malicious instructions in content an agent reads, such as a web page, email, or document. Hardware isolation and sandboxing cannot prevent it, because the attack runs through the model’s own reasoning. If the compromised agent has tool access, it can call an external endpoint and carry data out. Scope tool calls and watch the MCP surface closely.
Do I need an IRAP assessment to use AI for government work?
It depends on the classification and the workload. IRAP assessment and ISM/PSPF alignment are expected for government workloads handling sensitive data, and they sit inside the Australian Sovereignty Test rather than beside it. A low-sensitivity, on-device workload may not require the same assurance as a system holding protected information. Match the assessment to the data, and record the reasoning.
How often should I revisit where each AI workload runs?
Treat placement as a repeatable assessment, not a one-off decision. Re-run it when anything material changes: a new data classification, a vendor acquisition, a region move, a new agent with tool access, or a shift in token-volume economics. Review agents at provisioning, monitoring, and decommissioning too. Governance first means the trigger for re-deciding is a change in obligation, not a change in cost alone.
Can a “we never train on your data” promise make egress safe?
No. A no-training promise addresses one use of your data, not whether it leaves the machine. Prompts, completions, embeddings, and interaction logs can still cross the boundary for inference, logging, or analytics even when training is excluded. Read the data-flow documentation and check the telemetry defaults. A training promise is a single clause, not a complete egress posture.
What is runtime data-trajectory verification, and why does a regulator ask for it?
Runtime data-trajectory verification is the ability to show, after the fact, exactly what data moved where: which prompt, which model, which endpoint, and which jurisdiction. It pairs with an AI-BOM and a ROPA to give a regulator evidence rather than assurance. A vendor that cannot produce a data trajectory cannot prove its egress posture, so treat “show me the data trajectory” as a buying test.
Can embeddings or vector stores leak the source text they were built from?
Yes. Embeddings and vector stores are an egress channel in their own right, because inverted or reconstructed embeddings can recover substantial parts of the source text. If a local-first product syncs vectors to a hosted index, that index can sit under foreign jurisdiction. Ask where vectors are stored, whether they leave the device, and whether they can be inverted before you treat them as metadata.
