For AI infrastructure procurement, give suppliers the same workload brief and ask them to return a configuration, its sizing assumptions and a test plan. That makes the technical proposal easier to compare with the commercial offer. For a regional rollout, keep a separate delivery and support schedule for each site.
Define the AI infrastructure workload in the request for quotation
AI infrastructure for inference and training needs different resources. State which you are buying. For inference, name the model and revision, weight format, expected input and output lengths, peak simultaneous requests and acceptable waiting time. A count of registered users cannot substitute for concurrent demand.For training or fine-tuning, add the method, sequence length, batch size, optimiser and checkpointing approach. Ask the supplier to explain which assumptions drive its memory and networking choices. If an AI infrastructure requirement changes later, both sides should know what needs to be sized again.
Start AI infrastructure memory sizing with weight storage
For a dense model, a rough weight-only calculation is:Weight storage in bytes = parameter count × bits per parameter ÷ 8.
| Dense model size | 16-bit weights | 8-bit weights | 4-bit weights |
| 8 billion parameters | 16 GB | 8 GB | 4 GB |
| 32 billion parameters | 64 GB | 32 GB | 16 GB |
| 70 billion parameters | 140 GB | 70 GB | 35 GB |
These are theoretical decimal GB, not recommended GPU capacities. Quantisation metadata, runtime buffers and the key-value cache take additional space. The cache stores attention data for active requests; longer contexts and more simultaneous requests can increase its memory demand. Loading the weights successfully proves little about peak service demand.
AI infrastructure for training needs a separate budget for gradients, optimiser state and activations. Ask for the calculation per GPU, including the proposed parallelism and memory headroom. Do not accept a combined GPU-memory total as evidence that a workload will fit.
Check AI infrastructure connections as well as capacity
Two 48 GB GPUs do not automatically provide a single 96 GB memory pool. Software must support splitting the model, and the connections between GPUs affect the cost of doing so. Independent model replicas have different communication needs from a model split across devices.The AI infrastructure quotation should identify the GPU topology, PCIe generation and electrical lane width, CPU affinity and any NVLink or NVSwitch equipment actually included. For multiple servers, ask separately about the inter-node fabric. A fast client network does not establish that distributed jobs will run well.
In AI infrastructure planning, an empty slot is also not proof that another GPU can be added. Confirm qualified cards, available lanes, power delivery and cooling in the proposed chassis.
Use a worked AI infrastructure brief without mistaking it for a benchmark
Consider an internal service running one dense 8B model with BF16 weights. Assume typical requests contain 2,000 input tokens and generate 500 output tokens, with a maximum combined sequence length of 8,192 tokens. The initial requirement is four simultaneous requests. There is no training workload or high-availability requirement in this example.The following is an illustrative AI infrastructure RFQ starting point. It is not a customer deployment, a measured result or a recommendation that this system meets the requirement.
| Item | Illustrative starting point | What the supplier must validate |
| GPU | One NVIDIA L40S with 48 GB memory | Exact model, runtime, cache budget and service performance |
| CPU and host memory | One server CPU with 32 physical cores and 256 GB ECC RAM | CPU choice, memory population, loading and preprocessing needs |
| Boot storage | Two 960 GB enterprise SSDs in RAID 1 | Supported controller and recovery procedure |
| Model and data storage | Two 3.84 TB enterprise NVMe drives in RAID 1 | Capacity, endurance and workload performance |
| Network | Dual-port 25 GbE data NIC and a dedicated management port | Switch compatibility, optics, cables and port availability |
| Server platform | OEM-qualified chassis with redundant power supplies | Passive GPU cooling, rack dimensions and complete-system power |
NVIDIA specifies a PCIe Gen4 x16 interface, passive cooling and a maximum GPU power of 350 W for the L40S; it does not support NVLink. Those characteristics help explain the AI infrastructure design constraints. They do not make it the best current purchase for every inference service. RAID 1 provides mirroring, not a substitute for backup.
Before approving the AI infrastructure order, name the exact model revision and agree performance targets. Measure time to first token, token generation rate and total throughput under the intended request mix, with a defined percentile such as p95 where appropriate. Specify whether queueing is included. Four concurrent requests remain a requirement until the test demonstrates acceptable service.
Give facilities complete AI infrastructure system figures
For AI infrastructure, GPU power is not server input power. CPUs, memory, drives, fans and conversion losses also contribute. Request the OEM's planning figures, connector requirements and expected behaviour after losing one power feed. Confirm that the surviving feed can carry the required load under the agreed redundancy design.For a separate arithmetic example, assume four servers draw up to 1.5 kW each and rack networking adds 0.4 kW:
4 × 1.5 kW + 0.4 kW = 6.4 kW.
The inputs are hypothetical and are not power ratings for the sample system above. AI infrastructure facilities planning still needs to account for other equipment, headroom and cooling. Obtain rack depth, loaded weight and delivery-access requirements before shipment. A liquid-cooled proposal also needs defined facility connections, commissioning tasks and maintenance ownership.
Compare delivered and supported AI infrastructure cost
For AI infrastructure, request separate prices for equipment, software, installation, networking, freight, taxes and support. Use the same currency, tax basis and support term when comparing offers. Record the installation country and named delivery destination; regional shipping and service coverage should not be inferred from a supplier's headquarters.For AI infrastructure energy estimates, multiply average whole-server kW by operating hours and the applicable electricity tariff. Do not use maximum GPU power as the average server draw. List cooling or colocation charges separately and explain any assumptions.
Ask for AI infrastructure dispatch, site-arrival and ready-for-service dates. Check when the lead-time clock starts, which components are allocated and who approves substitutions. Where expansion is likely, obtain a priced option with the chassis, power and network changes it requires.
Finally, walk through an AI infrastructure hardware fault at each installation site. Who diagnoses it, where are parts held, who fits them and who verifies service recovery? A response target is not a repair deadline. Confirm service hours, local access arrangements and the handover between suppliers and your own team.
advange can discuss AI infrastructure planning and operations for your data centre against this brief. Use the brief to define your AI infrastructure requirements, then agree the actual scope, local coverage and acceptance work before placing an order.
References
NVIDIA L40S specifications
NVIDIA GenAI performance measurement documentation
