On this page
- Key facts
- Which is faster for inference, the RTX PRO 6000 or the H100?
- How much does an RTX PRO 6000 cost per hour compared with an H100?
- Which models fit on one RTX PRO 6000 or one H100?
- When do you need an H100 instead of an RTX PRO 6000?
- Can you rent an RTX PRO 6000 or an H100 in Canada?
- What this comparison does not cover
- FAQ
- Sources
One NVIDIA RTX PRO 6000 Blackwell rents for $3.363 an hour on AWS and holds 96 gigabytes (GB). One H100 rents for $6.88 and holds 80 GB. Those are AWS's Virginia prices on 11 November 2026. The H100 reads its memory about twice as fast. It also links to other H100s over NVLink, so it wins once a model spans several cards.
If your model fits on one card, rent the Blackwell card. You get more memory for half the price.
If your model needs four or more cards working as one, rent H100s. Their links between cards pay for themselves there.
If inference has to stay in Canada, the H100 is the one listed here, usually on a one- to three-year commitment. The Blackwell card means owning the hardware.
Key facts
| RTX PRO 6000 Blackwell Server Edition | H100 SXM | |
|---|---|---|
| GPU memory | 96 GB GDDR7 | 80 GB HBM3 |
| Memory bandwidth | 1,597 GB/s | 3.35 TB/s |
| Smallest FP number format with hardware support | FP4 | FP8 |
| Link between cards | PCIe Gen 5 | NVLink, 900 GB/s |
| Power | Up to 600 W | Up to 700 W |
| AWS, one card, us-east-1 | g7e.2xlarge, $3.363/h | p5.4xlarge, $6.88/h |
| AWS, Montreal (ca-central-1) | Not offered | Eight cards only, p5.48xlarge, $63.296/h |
GPU memory is the memory on the graphics card itself, measured in GB. The model's weights and the working memory for each conversation have to fit in it. Specs come from NVIDIA's RTX PRO 6000 Server Edition page and H100 page. Prices come from AWS's price list, read on 11 November 2026.
Which is faster for inference, the RTX PRO 6000 or the H100?
For one model on one card serving many users, they are close. Once a model spans several cards, the H100 is faster. The difference comes from two numbers: how fast each card reads its own memory, and how fast cards talk to each other.
Memory bandwidth sets the speed of a single reply. To write each new token (about three quarters of a word), the card reads the model's active weights from memory. The H100 SXM, the server version, reads 3.35 terabytes a second. The Blackwell card reads 1,597 GB a second. So the H100 has about 2.1 times the headroom per reply. With many requests in flight, each read serves many replies. Then raw compute matters as much.
Links between cards set the speed of a split model. NVIDIA lists 900 GB/s NVLink on the H100 SXM. The Blackwell card's page lists PCIe Gen 5, the standard slot connection, and no NVLink. A model split across four or eight cards passes data between them for every token, so the slower link becomes the limit.
One public benchmark shows both effects. CloudRift, a GPU rental firm, published it on 27 November 2025. It served models with vLLM, an open-source serving engine. It sent 256 requests at once, each 1,000 tokens in and 1,000 out:
| Setup in CloudRift's test | RTX PRO 6000 | H100 |
|---|---|---|
| GLM-4.5-Air, 4-bit, one card | 3,140 tokens/s | 2,987 tokens/s |
| Qwen3-Coder-480B, 4-bit, four cards | Baseline | 31% more |
| GLM-4.6, 8-bit, eight cards | Baseline | Nearly 3 times |
The numbers are CloudRift's, a year old, on synthetic prompts, from a firm that rents out both cards. Its own advice matches the table. Use the Blackwell card for smaller models. Keep H100s for large models that need cards tightly coupled.
How much does an RTX PRO 6000 cost per hour compared with an H100?
On AWS on demand, one Blackwell card costs about half as much as one H100: $3.363 against $6.88 an hour in Virginia. Per gigabyte of GPU memory, that is about 3.5 cents an hour against 8.6 cents.
| Where | What you rent | GPUs | On-demand price | Per GPU-hour |
|---|---|---|---|---|
| AWS us-east-1 | g7e.2xlarge | 1 RTX PRO 6000 | $3.363/h | $3.36 |
| AWS us-east-1 | p5.4xlarge | 1 H100 | $6.88/h | $6.88 |
| AWS us-east-1 | p5.48xlarge | 8 H100 | $55.04/h | $6.88 |
| AWS ca-central-1 (Montreal) | p5.48xlarge | 8 H100 | $63.296/h | $7.91 |
| Azure Canada Central | NC40ads H100 v5 | 1 H100 NVL, 94 GB | $8.59/h | $8.59 |
AWS prices are from AWS's price list, read on 11 November 2026, for Linux on demand. The Azure price is from Microsoft's retail prices API, read the same day, pay as you go. All prices are in US dollars. Azure's H100 is the NVL version, which fits a standard slot and holds 94 GB, per NVIDIA's H100 page.
The cost that matters is per finished job, not per hour. In our own benchmark, every self-hosted model ran on one RTX PRO 6000. DecideBench v2 priced each self-hosted model by the time it held that card: an AWS g7e.2xlarge at $3.36 an hour, with 4 requests in flight. For example, pplx-decider 27B finished 450 decisions in 186.1 seconds. That came to $386.34 per million decisions. We did not run the same test on an H100, so we have no H100 number to set beside it.
Which models fit on one RTX PRO 6000 or one H100?
A model fits on one card when its weights plus working memory fit in 96 GB or 80 GB. Weights take about 2 bytes per parameter at 16-bit, 1 byte at 8-bit and half a byte at 4-bit. Working memory grows with the length and number of conversations.
Three examples, from model cards and our runs:
- pplx-decider 27B fits on both. Perplexity's card asks for about 52 GiB free for the weights plus working room. It ran on our single Blackwell card.
- Very long inputs change the answer. Perplexity's card reports that pplx-decider needs about 107 GiB at 243,000 tokens, which fits neither GPU.
- A 70-billion-parameter model at 8-bit fits the Blackwell card more comfortably. Its weights take about 70 GB. That leaves around 26 GB of working room on 96 GB and around 10 GB on 80 GB. That gap decides how many conversations you can run at once.
- Kimi K3.1 Pro fits on neither, even eight at a time. Its download is 1.56 TB, per our Kimi K3.1 Pro comparison. Eight H100s hold 640 GB and eight Blackwell cards hold 768 GB. A model that size needs a bigger server or several of them.
The Blackwell card also supports FP4, a 4-bit number format, in hardware, per AWS's G7e page. Smaller weights take less memory and run faster. NVIDIA lists 4 petaFLOPS of FP4 for it and no FP4 figure for the H100. Quality at 4 bits depends on the model, so check it on your own task before you rely on it.
When do you need an H100 instead of an RTX PRO 6000?
You need an H100 when one model has to span several cards, or when the speed of each single reply matters more than the hourly price. Below that line, the cheaper card does the job for less.
Pick the H100 when:
- The model needs four or more cards. NVLink keeps split models fast. CloudRift measured the H100 at nearly three times the Blackwell card on an eight-card model.
- One user's reply speed is the product. At low concurrency, memory bandwidth sets the pace, and the H100 has about twice as much.
- You need it in Canada. The H100 is the card here that the big clouds list, though capacity usually comes with a one- to three-year commitment.
Pick the Blackwell card when the model fits in 96 GB, when you batch many requests, or when you want FP4. That covers most decision models, embedding models and mid-size chat models. If you are choosing between older data-centre cards, our H100 vs A100 for LLM inference post covers that pair. If you want the card on a desk instead of in a rack, read DGX Spark vs RTX PRO 6000.
Can you rent an RTX PRO 6000 or an H100 in Canada?
You can rent an H100 in Canada from all three big clouds. You cannot rent a Blackwell card in Canada from any of them, as of 11 November 2026.
H100 in Canada:
- AWS Montreal offers only the eight-card p5.48xlarge, at $63.296 an hour. The single-card p5.4xlarge is not offered there, per AWS's instance listing read on 11 November 2026.
- Azure Canada Central rents a single H100 for $8.59 an hour, as above.
- Google Cloud Toronto lists A3 Edge (H100) machines in one zone, per its GPU regions page.
GPU quota starts at zero on all three. You request it, and approval for large GPU sizes can take time. Listed is not the same as available. In our experience deploying for Canadian clients, H100 and H200 capacity in Canadian regions is hard to get on demand, and the providers usually ask for a one- to three-year commitment before they allocate it.
RTX PRO 6000 in Canada: AWS offers its RTX PRO 6000 machines in Virginia but not in Montreal. Its price list shows no GPU machines in its Calgary region. Azure lists none in its Canadian regions, and Google lists none in Montreal or Toronto. Marketplace sites advertise RTX PRO 6000 servers in Canada, but we could not verify a Canadian operator's own listing, so we name none.
Without a Canadian rental, you have two routes. Buy a server with the card and host it in your office or a Canadian data centre. The Server Edition is a passively cooled data-centre card, so it goes in a rack server, not a desktop. Or rent the nearest card that is offered in Canada: an H100 for large models, or an L4 (24 GB) in Montreal at $0.894 an hour for small ones. Our Canadian GPU rental guide lists every option with prices.
Owning the card means you own the power, cooling, spares and upkeep. One server is also one point of failure. Our comparison of on-premise, client cloud and managed cloud sets out those trade-offs, and the Canadian residency guide covers what keeps inference in the country.
XY Space deploys open-weight models on dedicated GPUs in the client's own Canadian cloud account or on-premises, sizes the hardware to the model, and builds the workflow on top. Your people run the work and the model takes part. See how we deploy.
If you need a model running with inference kept in Canada, bring one job to a free 30-minute call.
What this comparison does not cover
We ran our own models on the Blackwell card only, so every H100 speed figure here is NVIDIA's spec or CloudRift's test. We did not test the workstation editions of the RTX PRO 6000, the H200, or reserved and spot prices, which can sit far below on-demand rates. Prices move often, so check the provider's page on the day you buy.
FAQ
Is the RTX PRO 6000 better than the H100?
For a model that fits on one card, often yes on value. It has 96 GB against 80 GB and costs about half as much an hour on AWS. The H100 reads memory about twice as fast and links to other H100s over NVLink. That makes it the better card for models split across four or more GPUs, and for the fastest single replies.
Does the RTX PRO 6000 have NVLink?
NVIDIA's Server Edition page lists PCIe Gen 5 and does not list NVLink. Several cards in one server still work together over PCIe, as AWS's eight-card g7e.48xlarge does. They pass data between them more slowly than H100s over NVLink, which is why CloudRift measured the H100 nearly three times faster on an eight-card model.
How much does an RTX PRO 6000 cost per hour?
On AWS, a g7e.2xlarge with one Blackwell card costs $3.363 an hour on demand in us-east-1, per AWS's price list read on 11 November 2026. A single H100 (p5.4xlarge) costs $6.88. AWS does not offer the g7e family in its Montreal region, so the Canadian price is not available from AWS.
Can an RTX PRO 6000 run a 70B model?
Yes, at 8-bit or 4-bit. At 8-bit, a 70-billion-parameter model's weights take about 70 GB, leaving about 26 GB of the card's 96 GB for working memory. At 4-bit they take about 35 GB, which leaves room for many more conversations at once. At 16-bit the weights take about 140 GB, which needs two cards.
Is the H100 worth twice the price of the RTX PRO 6000?
Only for work that uses what the extra money buys: faster memory and NVLink. If your model fits on one card and you serve many requests at once, the Blackwell card gives similar throughput for half the hourly price. If your model spans several cards, the H100 usually costs less per finished job.
Sources
- NVIDIA, RTX PRO 6000 Blackwell Server Edition, read 11 Nov 2026. Memory, bandwidth, FP4 and FP8 throughput, power, MIG, form factor.
- NVIDIA, H100 Tensor Core GPU, read 11 Nov 2026. H100 SXM and H100 NVL memory, bandwidth, NVLink, power.
- AWS, Amazon EC2 G7e instances, read 11 Nov 2026. Sizes, GPU model, FP4 support.
- AWS, Amazon EC2 P5 instances, read 11 Nov 2026. p5.4xlarge (one H100) and p5.48xlarge (eight).
- AWS, EC2 On-Demand pricing and price list API, read 11 Nov 2026. g7e.2xlarge, p5.4xlarge, p5.48xlarge in us-east-1 and ca-central-1; instance offerings by region.
- Microsoft, NCads H100 v5 series, and Azure retail prices API, read 11 Nov 2026.
- Google Cloud, GPU regions and zones, read 11 Nov 2026.
- CloudRift, Benchmarking RTX PRO 6000 vs datacenter GPUs, Dmitry Trifonov, 27 Nov 2025.
- XY Space, DecideBench v2, 9 Nov 2026, and What is pplx-decider.
Written by
Cho Yin Yong
Principal AI Solutions Engineer, XY Space
Principal AI Solutions Engineer at XY Space. University of Toronto lecturer for five years, co-author of two patents, winner of two competitive AI awards, and nine years of regulated engineering leadership.
More from Cho Yin YongShare this article
Work with us
We build the systems these posts describe, and we'll tell you in the first call whether yours is worth building.
Start a project