AI ComparisonsNov 15, 20269 min read

L40S vs A100: The Cheaper Card Montreal Doesn't Rent

L40S vs A100: the L40S is cheaper per hour and runs FP8; the A100 reads memory 2.4x faster. On 11 Nov 2026 big clouds rented only the A100 in Canada.

On this page
  1. Key facts
  2. Which is faster for LLM inference, the L40S or the A100?
  3. How much does each cost per hour?
  4. Which models fit on each?
  5. What did we run on an L40S?
  6. Can you rent an L40S or A100 in Canada?
  7. What this comparison leaves out
  8. FAQ
  9. Sources

The L40S is cheaper to rent and alone has fast 8-bit (FP8) maths. The A100 reads memory up to 2.4 times faster and links up to eight cards for big models. On 11 November 2026, AWS listed one L40S at $1.861 an hour in Virginia. The same list offers eight A100s in Montreal at $25.252 an hour, and no L40S.

If you serve a model under about 30 billion parameters and price per hour decides, pick the L40S.

If you serve a bigger model across several cards, or generation speed per user matters most, pick the A100.

If prompts must be processed in Canada and you want to rent from a big cloud, pick the A100. The big three don't list the L40S in a Canadian region.

Key facts

The L40S and the A100 are both NVIDIA graphics cards built for data centres. GPU memory is the memory on the card itself, counted in gigabytes (GB). The whole model has to fit in it, plus working space for each conversation.

NVIDIA L40SNVIDIA A100
GPU memory48 GB GDDR640 GB HBM2 or 80 GB HBM2e
Memory bandwidth864 GB/s1,555 GB/s (40 GB), 1,935 to 2,039 GB/s (80 GB)
FP8 mathsYes, 733 TFLOPS denseNo
FP16 maths362 TFLOPS dense312 TFLOPS dense
Card-to-card linkPCIe only, no NVLinkNVLink, 600 GB/s
Power350 W250 to 400 W
Cheapest AWS hour, 11 Nov 2026g6e.xlarge, 1 card, $1.861 (us-east-1)p4d.24xlarge, 8 cards, $21.958 (us-east-1)
On AWS in MontrealNoYes, $25.252 for 8 cards

Specs are from NVIDIA's L40S page and A100 datasheet. Prices are on-demand, from AWS's price list as we read it on 11 November 2026.

Which is faster for LLM inference, the L40S or the A100?

For writing out answers one word-piece at a time, the A100 is faster, because that step is limited by how fast the card can read its own memory. The L40S has more raw maths, which helps only where maths is the limit: long prompts read in one go, and big batches.

Each new token, about three quarters of a word, means reading the model's weights from memory again. So memory bandwidth, the speed at which a card reads its own memory, sets the ceiling on generation speed for one conversation.

By NVIDIA's figures, the 40 GB A100 reads memory 1.8 times as fast as the L40S. The fastest 80 GB version reads 2.4 times as fast. That ratio is a ceiling, not a measured speed-up. Real speed depends on the serving software, the batch size and the prompt length.

The L40S answers back with 8-bit floating point (FP8), a number format that stores each weight in one byte instead of two. It belongs to NVIDIA's Ada Lovelace generation and does its maths in FP8 directly. NVIDIA lists 733 trillion operations a second for it at FP8, against 312 trillion at 16-bit for the A100.

The A100 can load an FP8 model, but it can't do FP8 maths. The documentation for vLLM, a popular serving program, says FP8 maths needs Ada Lovelace or a newer generation. Older cards like the A100 store the weights in one byte, then unpack them to 16-bit to compute. Both cards get the memory saving. Only the L40S gets the faster maths.

We did not run an inference benchmark on either card for this post. We also found no recent outside test with a setup we could check, so this post gives no speed figure.

How much does each cost per hour?

One L40S rents for $1.861 an hour on AWS in Virginia. AWS doesn't rent a single A100: its A100 instances come as eight-card machines, and the p4d costs $21.958 an hour in Virginia, or $25.252 in Montreal. All three prices come from AWS's price list as we read it on 11 November 2026.

InstanceCardsGPU memoryRegionOn-demand per hourPer card
g6e.xlarge1x L40S48 GBus-east-1$1.861$1.86
g6e.12xlarge4x L40S192 GBus-east-1$10.49$2.62
p4d.24xlarge8x A100 40 GB320 GBus-east-1$21.958$2.74
p4d.24xlarge8x A100 40 GB320 GBca-central-1$25.252$3.16

The per-card column is the hourly price divided by the card count. It includes the CPUs, memory and network bundled with each instance, which differ a lot between these sizes. The g6e.12xlarge price is from our token-based vs scale-to-zero vs scale-to-one post, which works through when a dedicated card beats paying per token.

The entry price is the real gap. Testing a model on one L40S costs under two dollars an hour. Testing it on AWS's A100 machines means paying for all eight cards. Azure does rent a single 80 GB card, the Standard_NC24ads_A100_v4, per its NC A100 v4 size page. We did not read Azure's prices this run, so check the Azure calculator for that size in your region.

Montreal costs about 15% more than Virginia for the same p4d instance. The research behind our pillar post on GPUs you can rent in Canada found Montreal running 11% to 15% above us-east-1 across AWS's GPU families.

Which models fit on each?

At FP8, one L40S holds a model of up to about 30 billion parameters with room left to work. One A100 80 GB holds about 60 billion at 8-bit. Anything bigger needs several cards, and that is where NVLink makes the A100 the better pool.

The arithmetic is simple. At 16-bit each parameter takes two bytes, at 8-bit one byte, at 4-bit half a byte. A 30-billion-parameter model is 30 GB at FP8. On a 48 GB card that leaves about 18 GB for the context of conversations in flight. AWS's G6e page says the instances serve models "with up to 13B parameters", which matches 16-bit weights on one card.

Model size16-bit weights8-bit weightsOne L40S (48 GB)One A100 40 GBOne A100 80 GB
8B16 GB8 GBYesYesYes
13B26 GB13 GBYesYesYes
30B60 GB30 GBAt 8-bitAt 8-bit, tightAt 8-bit
70B140 GB70 GBNeeds 2 to 4 cardsNeeds 2 to 4 cardsAt 8-bit, tight

The weights columns are bytes per parameter times parameter count. They leave out the working memory each conversation needs, which grows with input length.

Past one card the two part ways. The L40S has no NVLink, so cards in a g6e.12xlarge talk over PCIe at 64 GB/s each way. The A100s in a p4d talk through NVSwitch at 600 GB/s, per AWS's P4 page. Splitting one model across cards sends data between them on every token, so the faster link matters most for 70B-class models and up.

Neither box holds today's frontier open models in full. DeepSeek-V4.1-Flash is about 510 GB on Hugging Face, per our DeepSeek V4.1 Flash vs V4 Pro comparison. That is more than the 320 GB in a p4d or the 384 GB in the largest g6e. Models that size belong on H100-class hardware, which we cover in H100 vs A100 for LLM inference.

What did we run on an L40S?

We trained Yev, our open decision model, on one L40S. The shipped run took 4.71 hours, using LoRA, a small set of extra weights trained on top of a frozen 4-billion-parameter base model. The card processed about 1,663 tokens a second during training, and 48 GB was enough. All six runs took 25.3 hours in total. The details are in how to build a decision model.

The Yev run is a training number for a small model, not an inference benchmark, and it says nothing about the A100. It does show what a single L40S is good for: a model of a few billion parameters, trained on one card in under five hours.

Can you rent an L40S or A100 in Canada?

You can rent an A100 in Canada from AWS and Azure. You can't rent an L40S in Canada from AWS, Azure or Google Cloud, as of 11 November 2026.

  • A100 on AWS in Montreal. Eight 40 GB cards in one p4d machine, $25.252 an hour on demand.
  • A100 on Azure in Canada Central. One to four 80 GB cards per machine. Microsoft says it now adds new capacity only for its H100 machines, so A100 capacity may be harder to get.
  • L40S on AWS. Not offered in Montreal, and AWS's price list shows no GPU machines at all in Calgary (ca-west-1).
  • L40S from a Canadian provider. ISAIC, a University of Alberta initiative in Edmonton, lists a dedicated one-L40S virtual machine at $858 CAD a month on its pricing page, read 11 November 2026. It is a monthly rental, not by the hour.
  • L40S on Azure and Google. Neither lists it in Canada. Google's GPU locations page has no A100 in Canada either.

GPU quota on all three starts at zero for new accounts, so request it before you plan around a card.

If you want the L40S's price with inference in Canada, there are two routes. Rent the nearest card that is rentable in Canada: the L4 is the cheaper step down, covered in L4 vs L40S. Or own the hardware: an L40S server in a Canadian office or data centre keeps every prompt in the country, and you take on power and upkeep.

XY Space deploys open-weight models on dedicated GPUs in the client's own Canadian cloud account or on-premises, sizes the hardware to the model, and builds the workflow on top. Read how we deploy, the Canadian residency guide, and our comparison of on-premise vs client cloud vs managed cloud. If you need a model running with inference kept in Canada, bring one job to a free 30-minute call.

What this comparison leaves out

  • No inference benchmark of our own. The speed section rests on NVIDIA's specs and how generation works, not on a test we ran.
  • Azure and Google prices. We confirmed what Azure and Google list in Canada, not what they charge.
  • Calgary. We read AWS's price list for ca-west-1, not its live catalogue, since the region isn't turned on for our account. The price list shows no GPU machines there.
  • Reserved pricing. All prices are on-demand. Savings Plans and capacity blocks change the maths for steady workloads.

FAQ

Is the L40S better than the A100 for inference?

For models that fit on one card, often yes on price, since one L40S rents for $1.861 an hour on AWS in Virginia and runs FP8 natively. The A100 is better when generation speed per conversation matters, because it reads memory 1.8 to 2.4 times faster, and when a model spans several cards, because NVLink links them at 600 GB/s.

Does the A100 support FP8?

Not for maths. NVIDIA's datasheet for the card lists no FP8 format. Serving software like vLLM can still load an FP8 model on it: it stores the weights in one byte to save memory, then unpacks them to 16-bit to compute. The L40S, H100 and newer cards do the FP8 maths directly.

Can an L40S run a 70B model?

Not on one card. At 8-bit, a 70-billion-parameter model needs about 70 GB, more than the card holds. Four L40S cards in a g6e.12xlarge give 192 GB, enough at FP8 with room for conversations. That box cost $10.49 an hour on demand in us-east-1, per AWS's price list.

Why does AWS only rent A100s eight at a time?

AWS's A100 instances, the p4d.24xlarge and p4de.24xlarge, each come as one machine with eight cards joined by NVSwitch, per its P4 page. AWS built them for training and for large models split across cards. For a single card, Azure's Standard_NC24ads_A100_v4 gives one 80 GB A100, including in Canada Central.

Is the L40S available in Canada?

Not from AWS, Azure or Google Cloud as of 11 November 2026. AWS's g6e family isn't offered in Montreal, Azure lists no L40S size in Canada Central, and Google lists none in Toronto or Montreal. To keep L40S inference in Canada, own the hardware in a Canadian site, or rent an A100, H100 or L4 instead.

Sources

Written by

Cho Yin Yong

Principal AI Solutions Engineer, XY Space

Principal AI Solutions Engineer at XY Space. University of Toronto lecturer for five years, co-author of two patents, winner of two competitive AI awards, and nine years of regulated engineering leadership.

More from Cho Yin Yong

Share this article

Work with us

We build the systems these posts describe, and we'll tell you in the first call whether yours is worth building.

Start a project
Work with us

Book a call.We'll come back with specifics.

Start with the map of your organization, or with the one job that hurts. Measured in hours and money, and everything we build stays yours.

Loading form…