On this page
The L40S is cheaper to rent and alone has fast 8-bit (FP8) maths. The A100 reads memory up to 2.4 times faster and links up to eight cards for big models. On 11 November 2026, AWS listed one L40S at $1.861 an hour in Virginia. The same list offers eight A100s in Montreal at $25.252 an hour, and no L40S.
If you serve a model under about 30 billion parameters and price per hour decides, pick the L40S.
If you serve a bigger model across several cards, or generation speed per user matters most, pick the A100.
If prompts must be processed in Canada and you want to rent from a big cloud, pick the A100. The big three don't list the L40S in a Canadian region.
Key facts
The L40S and the A100 are both NVIDIA graphics cards built for data centres. GPU memory is the memory on the card itself, counted in gigabytes (GB). The whole model has to fit in it, plus working space for each conversation.
| NVIDIA L40S | NVIDIA A100 | |
|---|---|---|
| GPU memory | 48 GB GDDR6 | 40 GB HBM2 or 80 GB HBM2e |
| Memory bandwidth | 864 GB/s | 1,555 GB/s (40 GB), 1,935 to 2,039 GB/s (80 GB) |
| FP8 maths | Yes, 733 TFLOPS dense | No |
| FP16 maths | 362 TFLOPS dense | 312 TFLOPS dense |
| Card-to-card link | PCIe only, no NVLink | NVLink, 600 GB/s |
| Power | 350 W | 250 to 400 W |
| Cheapest AWS hour, 11 Nov 2026 | g6e.xlarge, 1 card, $1.861 (us-east-1) | p4d.24xlarge, 8 cards, $21.958 (us-east-1) |
| On AWS in Montreal | No | Yes, $25.252 for 8 cards |
Specs are from NVIDIA's L40S page and A100 datasheet. Prices are on-demand, from AWS's price list as we read it on 11 November 2026.
Which is faster for LLM inference, the L40S or the A100?
For writing out answers one word-piece at a time, the A100 is faster, because that step is limited by how fast the card can read its own memory. The L40S has more raw maths, which helps only where maths is the limit: long prompts read in one go, and big batches.
Each new token, about three quarters of a word, means reading the model's weights from memory again. So memory bandwidth, the speed at which a card reads its own memory, sets the ceiling on generation speed for one conversation.
By NVIDIA's figures, the 40 GB A100 reads memory 1.8 times as fast as the L40S. The fastest 80 GB version reads 2.4 times as fast. That ratio is a ceiling, not a measured speed-up. Real speed depends on the serving software, the batch size and the prompt length.
The L40S answers back with 8-bit floating point (FP8), a number format that stores each weight in one byte instead of two. It belongs to NVIDIA's Ada Lovelace generation and does its maths in FP8 directly. NVIDIA lists 733 trillion operations a second for it at FP8, against 312 trillion at 16-bit for the A100.
The A100 can load an FP8 model, but it can't do FP8 maths. The documentation for vLLM, a popular serving program, says FP8 maths needs Ada Lovelace or a newer generation. Older cards like the A100 store the weights in one byte, then unpack them to 16-bit to compute. Both cards get the memory saving. Only the L40S gets the faster maths.
We did not run an inference benchmark on either card for this post. We also found no recent outside test with a setup we could check, so this post gives no speed figure.
How much does each cost per hour?
One L40S rents for $1.861 an hour on AWS in Virginia. AWS doesn't rent a single A100: its A100 instances come as eight-card machines, and the p4d costs $21.958 an hour in Virginia, or $25.252 in Montreal. All three prices come from AWS's price list as we read it on 11 November 2026.
| Instance | Cards | GPU memory | Region | On-demand per hour | Per card |
|---|---|---|---|---|---|
| g6e.xlarge | 1x L40S | 48 GB | us-east-1 | $1.861 | $1.86 |
| g6e.12xlarge | 4x L40S | 192 GB | us-east-1 | $10.49 | $2.62 |
| p4d.24xlarge | 8x A100 40 GB | 320 GB | us-east-1 | $21.958 | $2.74 |
| p4d.24xlarge | 8x A100 40 GB | 320 GB | ca-central-1 | $25.252 | $3.16 |
The per-card column is the hourly price divided by the card count. It includes the CPUs, memory and network bundled with each instance, which differ a lot between these sizes. The g6e.12xlarge price is from our token-based vs scale-to-zero vs scale-to-one post, which works through when a dedicated card beats paying per token.
The entry price is the real gap. Testing a model on one L40S costs under two dollars an hour. Testing it on AWS's A100 machines means paying for all eight cards. Azure does rent a single 80 GB card, the Standard_NC24ads_A100_v4, per its NC A100 v4 size page. We did not read Azure's prices this run, so check the Azure calculator for that size in your region.
Montreal costs about 15% more than Virginia for the same p4d instance. The research behind our pillar post on GPUs you can rent in Canada found Montreal running 11% to 15% above us-east-1 across AWS's GPU families.
Which models fit on each?
At FP8, one L40S holds a model of up to about 30 billion parameters with room left to work. One A100 80 GB holds about 60 billion at 8-bit. Anything bigger needs several cards, and that is where NVLink makes the A100 the better pool.
The arithmetic is simple. At 16-bit each parameter takes two bytes, at 8-bit one byte, at 4-bit half a byte. A 30-billion-parameter model is 30 GB at FP8. On a 48 GB card that leaves about 18 GB for the context of conversations in flight. AWS's G6e page says the instances serve models "with up to 13B parameters", which matches 16-bit weights on one card.
| Model size | 16-bit weights | 8-bit weights | One L40S (48 GB) | One A100 40 GB | One A100 80 GB |
|---|---|---|---|---|---|
| 8B | 16 GB | 8 GB | Yes | Yes | Yes |
| 13B | 26 GB | 13 GB | Yes | Yes | Yes |
| 30B | 60 GB | 30 GB | At 8-bit | At 8-bit, tight | At 8-bit |
| 70B | 140 GB | 70 GB | Needs 2 to 4 cards | Needs 2 to 4 cards | At 8-bit, tight |
The weights columns are bytes per parameter times parameter count. They leave out the working memory each conversation needs, which grows with input length.
Past one card the two part ways. The L40S has no NVLink, so cards in a g6e.12xlarge talk over PCIe at 64 GB/s each way. The A100s in a p4d talk through NVSwitch at 600 GB/s, per AWS's P4 page. Splitting one model across cards sends data between them on every token, so the faster link matters most for 70B-class models and up.
Neither box holds today's frontier open models in full. DeepSeek-V4.1-Flash is about 510 GB on Hugging Face, per our DeepSeek V4.1 Flash vs V4 Pro comparison. That is more than the 320 GB in a p4d or the 384 GB in the largest g6e. Models that size belong on H100-class hardware, which we cover in H100 vs A100 for LLM inference.
What did we run on an L40S?
We trained Yev, our open decision model, on one L40S. The shipped run took 4.71 hours, using LoRA, a small set of extra weights trained on top of a frozen 4-billion-parameter base model. The card processed about 1,663 tokens a second during training, and 48 GB was enough. All six runs took 25.3 hours in total. The details are in how to build a decision model.
The Yev run is a training number for a small model, not an inference benchmark, and it says nothing about the A100. It does show what a single L40S is good for: a model of a few billion parameters, trained on one card in under five hours.
Can you rent an L40S or A100 in Canada?
You can rent an A100 in Canada from AWS and Azure. You can't rent an L40S in Canada from AWS, Azure or Google Cloud, as of 11 November 2026.
- A100 on AWS in Montreal. Eight 40 GB cards in one p4d machine, $25.252 an hour on demand.
- A100 on Azure in Canada Central. One to four 80 GB cards per machine. Microsoft says it now adds new capacity only for its H100 machines, so A100 capacity may be harder to get.
- L40S on AWS. Not offered in Montreal, and AWS's price list shows no GPU machines at all in Calgary (ca-west-1).
- L40S from a Canadian provider. ISAIC, a University of Alberta initiative in Edmonton, lists a dedicated one-L40S virtual machine at $858 CAD a month on its pricing page, read 11 November 2026. It is a monthly rental, not by the hour.
- L40S on Azure and Google. Neither lists it in Canada. Google's GPU locations page has no A100 in Canada either.
GPU quota on all three starts at zero for new accounts, so request it before you plan around a card.
If you want the L40S's price with inference in Canada, there are two routes. Rent the nearest card that is rentable in Canada: the L4 is the cheaper step down, covered in L4 vs L40S. Or own the hardware: an L40S server in a Canadian office or data centre keeps every prompt in the country, and you take on power and upkeep.
XY Space deploys open-weight models on dedicated GPUs in the client's own Canadian cloud account or on-premises, sizes the hardware to the model, and builds the workflow on top. Read how we deploy, the Canadian residency guide, and our comparison of on-premise vs client cloud vs managed cloud. If you need a model running with inference kept in Canada, bring one job to a free 30-minute call.
What this comparison leaves out
- No inference benchmark of our own. The speed section rests on NVIDIA's specs and how generation works, not on a test we ran.
- Azure and Google prices. We confirmed what Azure and Google list in Canada, not what they charge.
- Calgary. We read AWS's price list for ca-west-1, not its live catalogue, since the region isn't turned on for our account. The price list shows no GPU machines there.
- Reserved pricing. All prices are on-demand. Savings Plans and capacity blocks change the maths for steady workloads.
FAQ
Is the L40S better than the A100 for inference?
For models that fit on one card, often yes on price, since one L40S rents for $1.861 an hour on AWS in Virginia and runs FP8 natively. The A100 is better when generation speed per conversation matters, because it reads memory 1.8 to 2.4 times faster, and when a model spans several cards, because NVLink links them at 600 GB/s.
Does the A100 support FP8?
Not for maths. NVIDIA's datasheet for the card lists no FP8 format. Serving software like vLLM can still load an FP8 model on it: it stores the weights in one byte to save memory, then unpacks them to 16-bit to compute. The L40S, H100 and newer cards do the FP8 maths directly.
Can an L40S run a 70B model?
Not on one card. At 8-bit, a 70-billion-parameter model needs about 70 GB, more than the card holds. Four L40S cards in a g6e.12xlarge give 192 GB, enough at FP8 with room for conversations. That box cost $10.49 an hour on demand in us-east-1, per AWS's price list.
Why does AWS only rent A100s eight at a time?
AWS's A100 instances, the p4d.24xlarge and p4de.24xlarge, each come as one machine with eight cards joined by NVSwitch, per its P4 page. AWS built them for training and for large models split across cards. For a single card, Azure's Standard_NC24ads_A100_v4 gives one 80 GB A100, including in Canada Central.
Is the L40S available in Canada?
Not from AWS, Azure or Google Cloud as of 11 November 2026. AWS's g6e family isn't offered in Montreal, Azure lists no L40S size in Canada Central, and Google lists none in Toronto or Montreal. To keep L40S inference in Canada, own the hardware in a Canadian site, or rent an A100, H100 or L4 instead.
Sources
- NVIDIA, L40S GPU product page, read 11 Nov 2026
- NVIDIA, A100 product page and A100 40GB/80GB datasheet (PDF), read 11 Nov 2026
- AWS, Amazon EC2 G6e instances, P4 instances and On-Demand pricing; price list and region offerings read 11 Nov 2026
- Microsoft Learn, NC_A100_v4 size series, updated 25 Sep 2026
- Google Cloud, GPU regions and zones, read 11 Nov 2026
- vLLM, FP8 W8A8 documentation, read 11 Nov 2026
- XY Space, How to build a decision model, 14 Oct 2026
Written by
Cho Yin Yong
Principal AI Solutions Engineer, XY Space
Principal AI Solutions Engineer at XY Space. University of Toronto lecturer for five years, co-author of two patents, winner of two competitive AI awards, and nine years of regulated engineering leadership.
More from Cho Yin YongShare this article
Work with us
We build the systems these posts describe, and we'll tell you in the first call whether yours is worth building.
Start a project