Private AI: What It Really Is, How It Works, and How to Choose

Private AI is not a friendlier setting on a public chatbot. It is a boundary you can point to: the model runs on a machine you control or on a host built so operators are not supposed to see your prompts. Most pages stop at “we don’t train on your data.” That is one question. It is not the word.

This guide covers what private AI is, how a private AI model and private AI inference actually work, the two routes and the privacy levels inside the hosted route, and why ‘we don’t train on your data’ alone is too thin.

At a glance

• Private AI means the model, your data, and the answering step stay inside a named boundary.
• The machine that runs the model must see the text for a moment. The question is who owns that machine and whether anyone else can look.
• Two routes: run it yourself, or use a host. Hosts are not equal.
• Turning training off in ChatGPT is the bare minimum, not the destination.

What is private AI?

Private AI is any system where the model, your documents, and the step that produces the answer stay inside a security boundary you can name. Prompts do not go to a public API as a matter of course. They are not used to train someone else’s model, because they do not arrive there – or because the host’s design makes that use impossible to hide behind a toggle.

Public AI is the opposite. You send text to an external provider. The provider can read it while the model runs. It may log it, retain it, or train on it.

Bring the model to the data. Do not send the data to the model.

Private AI: the model, your data, and the answering step stay inside a boundary you can point to. The machine that runs the model has to see the text while it works. That window is called data in use. Encryption in transit and encryption at rest do not cover it. What differs is who owns the machine, and whether anyone else can read the text or train on it.

That is the architectural meaning. Two other uses of the same phrase float around. Researchers also say private AI for cryptographic methods – homomorphic encryption, secure multi-party computation, federated learning – that try to compute on data nobody else sees as plaintext. Those are real. They are not how most people chat with a large model in 2026. Marketers say private AI for a hosted bot with the training switch flipped off. Treat that as a claim. It is not a perimeter.

Private AI is also not the same as “AI privacy settings.” Settings live on someone else’s service. Architecture is where the GPU sits.

Two routes for running private AI

1. You run it. On a laptop, a phone, a private AI server, or an air-gapped box. The tokens do not leave unless you open a hole – a public port, a plugin, an API key.
2. A host runs it. You send the prompt. You are trusting their design: a contract, a no-retention pipeline, zero-access storage of old chats, or hardware that isolates the process.

Route 2 is not one product. It is several different promises.

The spectrum of private AI

If you cannot point to the machine that saw the prompt, you have a promise, not architectural private AI.

How private AI works

A lay version first. You load a model. You ask a question. The model answers. Privacy is decided in the middle, while the question is in memory on whatever computer is doing the work.

What is a private AI model?

A private AI model is usually one of three things, and people blur them:

• Open weights you download and hold – Llama, Mistral, Gemma, Qwen, Phi-class models
• A vendor model deployed in your private cloud or data centre
• A hosted model you never possess

Downloading and holding the model weights does not, by itself, protect your prompts. Once you serve the model through a public endpoint, the model may be publicly accessible, even if you own the files.

What is private AI inference?

In AI, inference is when a trained model processes new input and produces an output, such as an answer, prediction, classification, or generated image. 

Private AI inference is the answering step happening without exposing plaintext outside the boundary you chose.

On a public API, plaintext exists on the provider’s GPU (graphics processing unit). A no-retention host still sees it for that moment. Zero-access storage encrypts the history after the fact; the clerk still reads the letter in order to reply. A trusted execution environment (TEE) tries to keep that moment inside a hardware enclave the operator cannot inspect – but you need to check attestation, not only a sentence on a pricing page.

What is a private AI server?

A private AI server is the machine you (or your team) administer where that answering step runs: an on-premise box, a VPC, or an air-gapped network. It is a deployment, not a brand name. In 2026 the usual stack is a runtime such as Ollama or vLLM, often with Open WebUI in front, bound to a private network.

A usable 7–8B quantised model fits in about 16 GB of RAM. If you bind the same service to the public internet, and you launched a public API by accident.

Private AI vs ChatGPT

Private AI vs ChatGPT is not open source versus OpenAI. It is whether a third party sees the tokens.

Consumer ChatGPT sends the prompt off the device. The operator can read it during inference. Training may be on or off by tier. Retention still exists for abuse review and legal process. Enterprise and API tiers often default to no-training. They still leave the prompt at the provider unless you have a contracted isolation or confidential-compute story.

Private AI software and platforms

Private AI software is the app layer: LM Studio, Jan, Ollama, Open WebUI. A private AI platform is what enterprises mean: data, access control, training or fine-tuning, and inference inside one perimeter. But do not shop them as the same shelf.

Don’t look for the “best” private AI. Look for the one whose protection matches how sensitive the text you are about to send actually is.

Brave Leo is private relative to ChatGPT: reverse proxy, no account required, chats not retained on Brave’s side and not used for training. Default Leo is still remote inference. Lumo encrypts stored history so Proton cannot reopen it; the LLM server still decrypts the live prompt to answer. Venice’s default private mode is zero-retention hosting; stronger modes add hardware isolation. None of those three is the same as Ollama on localhost.

Private AI for business

Private AI for business is the same definition with a procurement problem. Finance, health, legal, and government teams cannot treat a consumer chatbot as a filing cabinet for customer records or unpublished work.

The scale of that exposure is already documented. With 45% of enterprise employees now using generative AI at work, 77% of those users paste data into chatbots, and 82% of those pastes come from unmanaged personal accounts that IT never sees, according to LayerX’s Enterprise AI and SaaS Data Security Report 2025. The same logic reaches the inbox those tools read from: if the mail itself already sits with an ad-scanning provider, no downstream AI policy can un-expose it.

A private AI system to run entirely on premises keeps weights, documents, and inference in your building or data centre. A private AI platform may instead sit in a private cloud you administer. A locked-down API is the easy option and the least private of the three.

If the official tool is slower than ChatGPT, staff will quietly default to ChatGPT – and that quiet workaround is exactly what Shadow AI is.

Why private AI matters more than “we don’t train”

Most private-AI roundups stop at training. Training is one question. Access and memory are the others.

Your AI assistant is building a version of you

ChatGPT-class tools launched as boxes you typed into. They became assistants with memory that lasts across sessions. That memory hardens into a profile: health, work, money, relationships, the way you write when you are tired.

And the raw material for that profile is getting more sensitive, not less. The share of corporate data entered into AI tools that is classified as sensitive has more than tripled in two years – from 10.7% in 2023 to 34.8% in 2025, per Cyberhaven’s 2025 AI Adoption and Risk Report.

A personal AI agent goes further. It does not only chat. It acts; with files, a browser, sometimes credentials. Someone holds that profile and those handles. Private AI is the decision about who.

“Private” is a policy for some, an architecture for others

A line in a privacy policy can be rewritten. A local process cannot. Hardware isolation plus attestation is supposed to be checkable; a slogan is not.

That is the two-route split again. Route 1 is physics. Route 2 is a stack of claims. Verify the level, do not trust the label.

Private is not the same as anonymous

Even when a host cannot read the prompt, usage still has a shape: when you ask, how often, from which device and browser, in whose writing style. Prompts also identify you because people tell assistants who they are.

Privacy, in the old sense, is: they can see who you are but not what you do. Anonymity is the reverse. Public AI often fails both. The failure is not hypothetical: in mid-2025, roughly 4,500 shared ChatGPT conversations – some containing names, résumés and sensitive personal details – were indexed by Google after an opt-in “make discoverable” toggle, until OpenAI disabled the feature that August. Some hosted tools fix anonymity – no account, a proxy, unlinkable billing – and still read the live prompt. Local inference hides the prompt and does not hide a seized laptop.

Metadata and fingerprinting do not vanish because the landing page said “private.”

When a personal AI agent can reach your inbox

The newest agents do not stop at the chat box. They draft replies, read threads, touch calendars, and hold the tokens to do it. The privacy of the assistant is then the privacy of every account it can open.

If that mailbox is already scanned for ads or training, an agent widens a hole that was already there. The cost of that hole is measurable: in IBM’s Cost of a Data Breach 2025 study, one in five organisations reported a breach involving Shadow AI. Organisations with high levels of Shadow AI incurred average breach costs roughly $670,000 higher than those with little or no Shadow AI, while 97% of organisations that experienced an AI-related security incident lacked proper AI access controls.

Encrypted email does not automatically protect a message after an agent has decrypted it and is authorised to send its contents to an external model. Encryption protects the message while it remains within the encrypted system; it cannot control what a trusted application does after it has legitimate access to the plaintext.

Mailfence is not a private AI product. It is a place for the mail and files those agents should not be piping into a public GPU: end-to-end encryption you control, no ads, Belgian hosting, no covert play. The assistant and the inbox are one boundary. Treat them that way.

How to choose private AI

Start with one question: how sensitive is the text you’re about to send? Then match the tool to the answer.

• Personal notes and everyday writing on your own laptop → run a local app like Jan, LM Studio, or Ollama. Nothing ever leaves the machine.
• AI for your whole household or a small team, always available → set up your own private AI server.
• Quick questions asked from a browser sidebar, with a provider that doesn’t keep your conversations → Brave Leo AI.
• A chat history the provider itself cannot reopen → Lumo.
• Cloud AI, but with hardware proof your data is sealed off while it runs → Venice AI.
• Data that must not leave your building for legal or contractual reasons → an on-premises setup.
• The strongest possible answer, and truly nothing personal at stake → ChatGPT. Just be clear about which level that is.

Interested in setting up LLMs locally? Harish explains it here, as well as how your privacy benefits:

And whatever you pick, run through this checklist before you hit send:

1. Would I hand this text to a stranger at that company?
2. Is “training off” actually the protection I need or only the first step?
3. Could this run on my own machine instead?
4. Did I paste it in myself or did an extension or plugin send it for me?
5. Can I point to the exact machine that will see my words?

Key takeaways

Final thoughts

Private AI is a perimeter. Company-hosted tools can be more private than ChatGPT and still not be local. Local tools can still leak through a plugin, an open port, or a seized disk.

Who can see the prompt, and who owns the memory: those are the questions. Everything else is a brochure.

If the assistant will ever touch mail or files, start with a mailbox that is not already someone else’s corpus. Create a Mailfence account.

Privacy is a right, not a feature. That includes the paragraph you were about to paste.

Glossary

• Attestation – a cryptographic proof that a confidential-computing environment really is what it claims to be, so you can verify the isolation instead of trusting a line on a pricing page.
• Boundary – the named perimeter this guide keeps pointing to: the device, server, or account inside which your data and the answering step stay.
• Confidential computing – hardware that processes data inside an isolated enclave, so that even the operator of the machine cannot read it while it runs.
• Data in use – the window when information is decrypted in memory so it can be processed. Encryption in transit and encryption at rest do not cover this gap; private AI is largely about who controls it.
• Federated learning – training that happens across many devices without pooling raw data. Your phone trains its own copy of the model and sends back only small updates, not your messages.
• Homomorphic encryption – a form of encryption that lets a machine compute directly on ciphertext, so it can produce an answer without ever seeing the plaintext. Powerful in theory, but still too slow for everyday chatbot-scale AI.
• Inference – the step where a trained model turns your prompt into an answer. It’s the moment your text is actually being processed, and where privacy is won or lost.
• Shadow AI – unsanctioned use of public AI tools for work, outside company policy or oversight. It is the main route by which sensitive data ends up in models an organisation never approved.
• Secure multi-party computation – a family of protocols that lets several parties compute a shared result while each keeps their own inputs secret. No one holds the whole picture; only the agreed output is revealed.
• TEE (Trusted Execution Environment) – a hardware-isolated region of a processor where code and data are shielded from the rest of the machine, including its operator, while being processed. It is the mechanism most confidential computing relies on.
• Weights – the trained parameters that make up the model itself. “Open weights” (Llama, Mistral, Gemma) are ones you can download and run on your own machine.
• Zero-access – storage encrypted so that the provider holds no key to read it. Your saved history stays on their servers but cannot be reopened by them.