Private LLM in the UAE: Which GPU, How Much Memory, What It Costs

Which GPU a private LLM needs, from RTX PRO 6000 and H100 to B200, MI355X and Rubin: memory by model size, UAE rental prices and what a pilot costs.

By Samvel Kocharyan11 min read

A single GPU server in a UAE data centre rack, with a laptop beside it showing an assistant answering from a company's own documents

The brief usually arrives fully formed. Fifty to a hundred people, a shared drive full of specifications, contracts and spreadsheets, a GPU server already shortlisted in a Dubai data centre, and one firm condition: no prompt and no page goes to an outside AI provider.

That is a private LLM, and in 2026 it is no longer a science project. The hard question is not which model. It is how much GPU memory you need, and that question has an answer that outlives any one graphics card.

How much VRAM does a private LLM need?

GPU memory holds two things: the model's weights, and the working memory of every conversation in progress. NVIDIA's own inference guide gives the arithmetic for both.

The weights are parameters times bytes per parameter: 2 bytes at FP16, 1 at FP8, about half a byte at 4-bit. That is the whole formula:

Model size4-bitFP8FP16
8B (small assistant, classification)about 4 GBabout 8 GBabout 16 GB
32B (strong general model)about 16 GBabout 32 GBabout 64 GB
70B (the usual company assistant)about 35 GBabout 70 GBabout 140 GB
120B (large open-weight, often mixture-of-experts)about 60 GBabout 120 GBabout 240 GB

The second part is the one that surprises people. Every token a conversation holds in context needs its own slice of memory, the KV cache. For a 70B model like Llama 3.1 at BF16 that is about 320 KB per token, so a single 32,000-token conversation, roughly a long contract plus the questions about it, reserves about 10 GB. Twenty people doing that at once need more memory for the conversations than for the model.

That is why the rest of this guide sorts GPUs by memory, not by name or year. When a new card launches, check its memory and it drops straight into the right row.

GPUs for a private LLM, by memory: previous, current and next generation

Memory classPrevious generationCurrentAnnounced or rampingWhat fits
24 to 32 GBGeForce RTX 4090, 24 GBRTX PRO 4000 Blackwell, 24 GB; RTX PRO 4500 Blackwell, 32 GB; GeForce RTX 5090, 32 GB8B models comfortably, 32B at 4-bit for a few users. Pilots and prototypes.
40 to 48 GBA100 40 GB; RTX 6000 Ada, 48 GBL40S, 48 GB; RTX PRO 5000 Blackwell, 48 GB32B at FP8, or 70B at 4-bit with little room for users.
72 to 96 GBA100 80 GBH100, 80 GB (94 GB NVL); RTX PRO 5000 Blackwell, 72 GB; RTX PRO 6000 Blackwell, 96 GB70B at 4-bit with memory left for concurrent conversations. The usual starting point for a company assistant; load-test the user count.
128 to 192 GBH200, 141 GB; B200, 180 GB (192 GB raw); AMD Instinct MI300X, 192 GB; Intel Gaudi 3, 128 GB; DGX Spark, 128 GB unified, a desk-side box for testingRubin CPX, 128 GB, expected end of 202670B at FP8 with more room for long conversations, or 120B-class models at 4-bit and FP8.
256 GB and upAMD Instinct MI325X, 256 GB; B300 (Blackwell Ultra), 288 GB; AMD Instinct MI355X, 288 GBNVIDIA Rubin, up to 288 GB of HBM4; AMD Instinct MI400 series, up to 432 GB of HBM4; Rubin Ultra and Feynman on NVIDIA's roadmap for 2027 and 2028Full-precision large models and heavy traffic. Usually an 8-GPU server and a data-centre contract, not a single rental.

Two honest notes on reading that table. Memory is not the only spec: data-centre cards such as the H100, H200 and B200 use HBM, roughly twice as fast or more than the GDDR memory on workstation cards, and they link eight GPUs into one machine. That matters for big models and heavy traffic. For one assistant serving one company, memory decides first and speed second.

And "announced" means announced. New generations reach UAE rental catalogues months after launch, so plan on what you can rent today.

H100 vs A100 vs RTX PRO 6000 vs B200: which one for a company assistant?

These are the comparisons buyers actually type into a search box, so here is the short version:

  • A100 vs H100. The A100 (40 or 80 GB) is the previous generation, still everywhere, and still capable for a 4-bit 70B model on the 80 GB version. The H100 is faster on the same memory and supports FP8, which lets a 70B model run at higher quality on 80 to 94 GB. If both are on offer at a similar price, take the H100.
  • RTX PRO 6000 Blackwell vs H100. The RTX PRO 6000 has more memory, 96 GB against 80 or 94 GB, and rents for less. The H100 has faster memory and scales across GPUs. For a single-server private assistant, the RTX PRO 6000 is the value pick. For a platform that will grow to several GPUs, start on the H100 family.
  • H100 vs H200 vs B200. The H200 is an H100 with 141 GB of faster memory, which is exactly what large models and long documents need. The B200 moves to 180 GB and a newer architecture. Both are usually rented as 8-GPU servers, so size them against a load test rather than headcount.
  • AMD Instinct (MI300X to MI355X). Generally more memory per GPU than the NVIDIA card each one competes with, 192 to 288 GB, and supported by vLLM. Worth a quote when the model is very large. Check that every tool in your stack runs on it before you commit.

Private LLM cost UAE: the server bill

UAE hosting providers list dedicated GPU servers by the month. One public price list, for servers in a UAE data centre, as of November 2026:

Market benchmark ranges
ItemRange
RTX PRO 2000 Blackwell, 16 GB

Small models, testing

AED 4,999 a month
RTX PRO 4000 Blackwell, 24 GB

8B models, light use

AED 5,499 a month
RTX PRO 4500 Blackwell, 32 GB

32B at 4-bit for a few users

AED 5,999 a month
RTX PRO 5000 Blackwell, 48 GB

32B models with room for users

AED 6,799 a month
L40S, 48 GB

Data-centre card, same memory class

AED 6,999 a month
RTX PRO 6000 Blackwell, 96 GB

70B models at 4-bit, room for conversations

AED 9,999 a month
Market benchmark ranges, not our quote. Source: https://www.serverbasket.ae/p/gpu-dedicated-server

The hyperscale clouds bill the bigger cards by the hour. Their November 2026 list prices in UAE regions: one L40S for USD 2.283 an hour on AWS me-central-1, one H100 NVL for USD 9.98 an hour on Azure UAE North, and eight H200s for USD 75.96 an hour on AWS. Left running around the clock, a single H100 at that list price comes to about USD 7,300 a month. Compare like with like before choosing: the same GPU memory, CPU, RAM and storage, and the hours the assistant actually needs to be up. A list price is not a capacity promise, so confirm the card is actually available in the region before you plan around it.

Owning the hardware instead of renting is possible through colocation, but it moves the problem from a monthly bill to procurement, spares and a failed card at 2 a.m. For a first deployment, renting is the honest default.

Should you wait for the next GPU generation?

Every year brings a card with more memory: Blackwell Ultra and the MI355X at 288 GB, Rubin with HBM4, the MI400 series at up to 432 GB, and more on the roadmaps after that. It is tempting to wait for the next one.

Do not, unless the model you need fits nothing you can rent today. The expensive, slow part of a private LLM is the system around the model: permissions, document cleanup, integration, evaluation. Most of that carries over when you move to a newer card; the move itself is a scoped migration with driver checks, a fresh load test and a rerun of your evaluation set. Renting, rather than buying, keeps that a planned project instead of a write-off.

The build bill: what a pilot really costs

The server sits idle until someone builds the system around it. Market estimates for that work are wide and come from vendors, so treat them as a range, not a quote. One puts a well-scoped private deployment at USD 40,000 to 100,000 upfront, with a RAG pilot in 4 to 8 weeks. Another says most SMB deployments reach a pilot in 6 to 12 weeks for under USD 80,000.

Where that money goes, roughly in order of how often it is underestimated:

  1. Permissions. A shared index that lets a junior analyst ask about the board minutes is one of the most serious failures. Every answer has to respect who is asking.
  2. Document cleanup. Five versions of the same specification, three of them wrong. The assistant will quote whichever it finds first.
  3. Microsoft 365 integration. Connecting through Microsoft Graph with the narrowest permission that works. Sites.Selected gives an app no SharePoint access at all until an administrator grants each site, which is the right default for confidential files.
  4. Evaluation. A set of real questions with correct answers, scored before launch and after every change, so "is it accurate?" has a number.
  5. Serving and operations. The model behind vLLM in Docker, monitoring, backups of the index, and a plan for when the GPU fails.

The model itself is close to free: open-weight families such as Qwen, Llama, DeepSeek and gpt-oss are downloaded, not licensed per seat. Read the licence of the exact version you pick, because they differ between releases.

Is private cheaper than an API?

Usually not, at first. One vendor estimate puts the break-even at about 2 million tokens a day. Measure your own volume before you use cost as the argument.

Cost is rarely the question. The companies that choose private do it because their documents are their business: engineering drawings, tender pricing, client contracts. For them the value is that no outside AI provider receives their prompts or documents, and that the logs sit on a server under their own contract. Be precise about the boundary: the data centre still hosts the machine, and files that live in SharePoint are still processed by Microsoft 365. Agree in writing which components, logs and backups stay inside it.

35%

Countries Gartner expects to be locked into region-specific AI platforms by 2027, up from 5 percent

https://www.gartner.com/en/newsroom/press-releases/2026-01-29-gartner-predicts-35-percent-of-countries-will-be-locked-into-region-specific-ai-platforms-by-2027

When you should not build one

If the goal is faster emails and meeting notes inside Office, the assistant built into Microsoft 365 will get you most of the way, faster and cheaper, and we will tell you so. A private LLM earns its cost when the assistant has to work on confidential material, inside your own permissions, in a place you control. Not as a trophy for the IT roadmap.

Checklist before you ask for a quote

  • How many people will use it, and how many at the busiest moment?
  • How long are the documents they will ask about: a page, or a 200 page contract?
  • Which document sources: SharePoint sites, file shares, email, a document system?
  • Who may see what, and is that written down anywhere today?
  • Which languages: English only, or Arabic and Russian too?
  • Do you have an internal team to run the server, or do you need it run for you?

Bring those six answers and a pilot can be scoped in one conversation. Ours starts with the AI Opportunity Audit, where we size the GPU against your real documents and pick the model on your own questions before you sign for a server.

Frequently asked questions

How much GPU memory does a 70B model need?

About 35 GB for the weights at 4-bit precision, 70 GB at FP8 and 140 GB at FP16, plus memory for every active conversation. Add the weights, the runtime and the context of every live conversation: a 96 GB card fits a 4-bit 70B model with memory to spare, FP8 fits on 94 to 96 GB with less headroom, and larger cards such as the H200 or B200 come in as that total grows. A load test on your own documents settles it.

Which GPU do we need for a private LLM for 50 to 100 staff?

For most document assistants, one card with 80 to 96 GB, such as an H100 or an RTX PRO 6000 Blackwell, running a 4-bit or FP8 model between 30 and 70 billion parameters. Confirm with a load test on your own documents and your busiest hour before signing.

H100 or RTX PRO 6000 for LLM inference?

The H100 has faster HBM memory and scales across several GPUs in one server, which matters for big models and heavy traffic. The RTX PRO 6000 Blackwell has more memory, 96 GB against 80 or 94 GB, at a lower rental price, which suits a single-server private assistant.

Should we wait for the next generation of GPUs?

Usually not. Each generation adds memory and speed, but a rented server can be swapped when a better card reaches UAE data centres, and the software around the model carries over. Waiting only makes sense if the model you need does not fit any card you can rent today.

How much does a private LLM cost in the UAE?

Two bills. The server: dedicated GPU servers in UAE data centres are listed from AED 4,999 to AED 9,999 a month, and hyperscale GPUs are billed by the hour. The build: vendor estimates for a pilot range from under USD 80,000 to USD 100,000, depending on integrations and data quality.

Is a private LLM cheaper than using an AI API?

Only at volume. Below a few million tokens a day, API pricing usually wins on cost alone. Companies choose a private model mainly for confidentiality and control: no prompt or document goes to an outside AI provider, and the logs stay on your server. The data centre that hosts the machine, and Microsoft 365 for files kept in SharePoint, remain inside the boundary you agree in writing.

Can a private LLM work with SharePoint, Teams and our Office files?

Yes. It connects through Microsoft Graph, indexes PDF, Word and Excel files with citations, and answers inside Teams. The app should see only the sites an administrator grants, and each answer should respect what the person asking is allowed to read.

Does UAE law require us to self-host our AI?

Not as a general rule. UAE data protection law regulates how personal data leaves the country rather than banning foreign providers, and free zones and regulated sectors such as health and banking add their own rules. Self-hosting is usually a confidentiality decision, so check your sector regulator first.