Cisco AI PODs 2026: Validated Designs for NVIDIA H100/H200 on UCS X-Series + Nexus 9000

The rapid evolution of AI workloads, particularly large language models (LLMs), demands high-throughput, low-latency infrastructure. Cisco's AI PODs address this need with validated designs that integrate NVIDIA's latest H100/H200 GPUs into a UCS X-Series compute and Nexus 9000 networking backbone. This isn't a marketing brochure; it's a field-engineer's perspective on what works, what breaks, and what the real trade-offs are when deploying these multi-million dollar stacks. We'll dissect the three primary POD variants—Inferencing, Retrieval-Augmented Generation (RAG), and Generative AI—and provide architectural insights, configuration specifics, and operational considerations.

Cisco AI POD Architecture Overview: UCS X-Series + NVIDIA GPUs

Cisco's AI PODs are built around the UCS X-Series modular system, specifically the UCS X9508 chassis populated with UCS X440p compute nodes. These nodes support NVIDIA H100 or H200 GPUs in both SXM and PCIe form factors, depending on the compute density and NVLink requirements. For instance, a single X440p node can house up to 8x H100 SXM5 GPUs, providing 640 GB of HBM3 memory and 900 GB/s inter-GPU bandwidth via NVLink. The decision between SXM and PCIe is critical: SXM offers superior NVLink bandwidth within the node, essential for tightly coupled training workloads, while PCIe variants offer more flexibility for scaling out lower-density inferencing or RAG scenarios at a potentially lower per-GPU cost. The UCS platform's modularity allows for on-demand scaling of CPU, GPU, and storage resources, managed centrally via Intersight. Brownfield deployments often struggle with existing power and cooling densities; a typical 8-GPU X440p node can draw upwards of 10kW per node, necessitating careful planning for power distribution units (PDUs) and row-based cooling.

Networking forms the foundation, with Nexus 9000 series switches delivering 400GbE and 800GbE connectivity. The Nexus 9336C-FX2 (36x 400GbE) or 93600CD-GX (56x 400GbE, 8x 800GbE) act as leaf switches, connecting directly to UCS X-Fabric modules that front-end the X440p nodes. For spine layer connectivity, the Nexus 9364D-GX2A, with its 64x 400GbE ports and option for 800GbE uplinks, provides the necessary fabric capacity. The choice of optics is non-trivial; QSFP-DD 400G DR4 for up to 500m single-mode fiber, or FR4 for up to 2km, are common, each impacting CAPEX and ongoing power consumption. Airflow management in spine-and-leaf designs with high-density components must be meticulously planned to avoid hot spots, especially with front-to-back chassis designs common in data centers where hot aisle containment might already be in place.

Networking Deep Dive: RoCEv2, PFC, and ECN Tuning with Nexus 9000

RDMA over Converged Ethernet (RoCEv2) is a non-negotiable component for high-performance AI workloads, enabling direct memory access between GPUs without CPU intervention. This requires a lossless Ethernet fabric achieved through Priority Flow Control (PFC) and Explicit Congestion Notification (ECN). Cisco Validated Design (CVD) for AI/ML (e.g., CVD: AI/ML Networking with Nexus 9000, Section 4.2.1) mandates strict QoS configurations. A common field scenario involves misconfigured PFC leading to head-of-line blocking for other traffic classes, essentially negating the benefits of RoCEv2 and impacting storage or management traffic. Careful mapping of RoCEv2 traffic (typically DSCP 48/AF41) to a dedicated lossless queue is paramount.

The following NX-OS configuration snippet demonstrates the critical components for enabling RoCEv2 with PFC and ECN on a Nexus 9000 series leaf switch. This is applied globally and then per-interface. Note the 'no-drop' queue configuration and ECN thresholds. Incorrect ECN parameters can lead to premature congestion marking, decreasing throughput, or, conversely, buffer exhaustion if thresholds are too high. Monitoring buffer usage via show queuing interface and PFC statistics via show interface priority-flow-control is crucial during day-2 operations.

! Global QoS configuration for RoCEv2
class-map type qos match-any CM_AI_ROCEV2
  match dscp 48
class-map type qos match-any CM_DEFAULTS
  match qos-group 0

policy-map type qos PM_AI_ROCEV2_PFC
  class CM_AI_ROCEV2
    set qos-group 3
    set cos 3
  class CM_DEFAULTS
    set qos-group 0
    set cos 0

interface Ethernet1/1-36
  service-policy type qos input PM_AI_ROCEV2_PFC

! Global queuing policy with PFC and ECN
class-map type queuing CM_INGRESS_ROCEV2
  match qos-group 3
class-map type queuing CM_INGRESS_DEFAULTS
  match qos-group 0

policy-map type queuing PM_INGRESS_ROCEV2
  class type queuing CM_INGRESS_ROCEV2
    bandwidth percent 50
    priority level 1
    flow-control send on
    flow-control receive on
    no-drop
    ecn-enable
    choke-dscp-transmit-threshold 20000 30000
    choke-dscp-drop-threshold 30000 40000
  class type queuing CM_INGRESS_DEFAULTS
    bandwidth percent 50
    set cos 0
    ecn-enable
    choke-dscp-transmit-threshold 1000 2000
    choke-dscp-drop-threshold 2000 3000

interface Ethernet1/1-36
  service-policy type queuing input PM_INGRESS_ROCEV2

! Interface-specific PFC enable
interface Ethernet1/1
  priority-flow-control mode on
  priority-flow-control auto
  priority-flow-control ets enable
  priority-flow-control no-drop qos-group 3
  mtu 9216

service-policy type queuing output PM_EGRESS_ROCEV2
  class type queuing CM_EGRESS_ROCEV2
    priority level 1
    bandwidth percent 50
    ecn-enable
    choke-dscp-transmit-threshold 20000 30000
    choke-dscp-drop-threshold 30000 40000
  class type queuing CM_EGRESS_DEFAULTS
    bandwidth percent 50
    ecn-enable
    choke-dscp-transmit-threshold 1000 2000
    choke-dscp-drop-threshold 2000 3000

interface Ethernet1/1-36
  service-policy type queuing output PM_EGRESS_ROCEV2

The default behavior of some NICs or operating systems might send RoCEv2 traffic with DSCP 0. Overriding this with set dscp 48 or ensuring applications explicitly mark traffic is essential. The priority-flow-control auto command helps synchronize PFC settings with connected devices, but often requires a manual priority-flow-control mode on and specifying the no-drop queue to ensure consistent behavior across the fabric. A common misstep is forgetting to configure large MTUs (Jumbo Frames) across all devices, including host NICs, leading to packet fragmentation and significant performance degradation for RDMA traffic. Monitoring PFC counters for discards is a critical indicator of congestion or misconfiguration.

Storage Options: NetApp ONTAP & Pure FlashBlade for AI PODs

AI workloads require high-performance, scalable storage. Cisco's AI POD CVDs (e.g., AI Storage Architectures for LLMs, Section 5.1) typically involve two primary approaches: NetApp ONTAP or Pure Storage FlashBlade //E. Both offer NVMe-oF or high-performance NFS/S3 protocols relevant for AI. NetApp ONTAP with pNFS or NVM-eOF provides shared file system access directly over the Ethernet fabric, bypassing traditional bottlenecked storage networks. For a 256-GPU POD, hundreds of terabytes of high-IOPS storage are needed. A well-configured NetApp AFF A900 with all-NVMe drives is a common choice, delivering multi-hundred GB/s throughput and millions of IOPS.

# Ansible Playbook for NetApp ONTAP NFS Export (excerpt)
- name: Create NFS export for AI data
  netapp.ontap.na_ontap_volume:
    state: present
    name: ai_data_volume
    vserver: svm_ai_prod
    aggregate_name: aggr1_allflash
    size: 200t
    size_unit: tb
    junction_path: /ai_data
    space_guarantee: none
    export_policy: ai_export_policy
    snapshot_policy: daily_ai
    security_style: unix
    snapshot_reserve: 0
  tags: volume_provisioning

- name: Ensure AI export policy exists
  netapp.ontap.na_ontap_export_policy:
    state: present
    name: ai_export_policy
    vserver: svm_ai_prod
  tags: export_policy_provisioning

- name: Allow specific clients read/write access
  netapp.ontap.na_ontap_export_rule:
    state: present
    protocol: nfs
    vserver: svm_ai_prod
    policy_name: ai_export_policy
    rule_index: 10
    client_match: ['10.10.100.0/24', '10.10.101.0/24']
    rorule: ['any']
    rwrule: ['sys']
    superuser: ['sys']
    anon: 0
  tags: export_policy_rules

Pure Storage FlashBlade //E, on the other hand, offers an object-storage-first approach with S3 compatibility, suitable for unstructured data, model checkpoints, and large datasets common in AI. Its architecture is optimized for massive parallelism and sequential reads/writes. GPUDirect Storage (GDS) is a key feature, allowing GPUs to directly access storage, bypassing CPU and host memory for substantial latency reduction and throughput improvements. This requires specific kernel modules and driver support on the compute nodes. The choice between NFS and S3 often comes down to application and framework compatibility. TensorFlow and PyTorch can integrate with both, but specific data pipelines might be optimized for one over the other. Performance monitoring must include storage-side metrics (latency, IOPS, throughput) correlated with compute node GPU utilization to identify bottlenecks that could stem from either the network or the storage solution.

Kubernetes and NVIDIA AI Enterprise Integration

Managing a large-scale AI infrastructure without orchestration is a recipe for operational chaos. Kubernetes, specifically Red Hat OpenShift AI (formerly OpenShift Data Science), integrated with the NVIDIA AI Enterprise suite, forms the control plane. OpenShift AI 2.14, with its GPU Operator, simplifies the deployment and management of NVIDIA drivers, Container Runtimes (e.g., Containerd with NVIDIA Runtime), and other necessary components on Kubernetes. The Cisco Nexus Dashboard Fabric Controller (NDFC), formerly DCNM, is used to manage the VXLAN-EVPN underlay network, providing network automation and visibility for the Kubernetes cluster overlay. This integration is critical for ensuring consistent network policies and performance across the physical and virtual network layers.

NVIDIA AI Enterprise provides the full stack of AI software, including NIM microservices for optimized inference, NeMo for large model development, and Triton Inference Server for high-performance inferencing. Deploying these components as Kubernetes workloads, leveraging specific node selectors for GPU types and quantities, is the standard practice. For instance, a Llama 3.1 70B model might require two H100 GPUs for efficient inferencing, and the Kubernetes scheduler ensures appropriate placement. Intersight Workload Optimizer (IWO) plays a vital role here, analyzing performance metrics from UCS, storage, and even Kubernetes, to recommend optimal GPU placement and resource allocation. This prevents 'dark' GPUs or underutilized CPU cores by dynamically shifting workloads or suggesting infrastructure changes.

# Triton Inference Server deployment manifest (excerpt)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: triton-server-llama3-70b
  labels:
    app: triton-server
spec:
  replicas: 2
  selector:
    matchLabels:
      app: triton-server
  template:
    metadata:
      labels:
        app: triton-server
    spec:
      nodeSelector:
        nvidia.com/gpu.count: "2"
        nvidia.com/gpu.product: "H100-SXM5-80GB"
      tolerations:
        - key: "nvidia.com/gpu"
          operator: "Exists"
          effect: "NoSchedule"
      containers:
        - name: triton
          image: nvcr.io/nvidia/tritonserver:23.10-py3
          command: ["tritonserver"]
          args: ["--model-repository=/models", "--backend-config=python,repo-path=/models/llama3_70b/fastllm/model.py"]
          ports:
            - containerPort: 8000
              name: http
            - containerPort: 8001
              name: grpc
            - containerPort: 8002
              name: metrics
          resources:
            limits:
              nvidia.com/gpu: 2
              memory: "128Gi"
            requests:
              nvidia.com/gpu: 2
              memory: "64Gi"
          volumeMounts:
            - name: model-volume
              mountPath: /models
      volumes:
        - name: model-volume
          persistentVolumeClaim:
            claimName: triton-model-pvc-llama3

AI POD Variants: Inferencing, RAG, and Generative AI

Cisco defines distinct AI POD configurations optimized for different workload types, each with specific sizing and hardware biases (CVD AI PODs: Workload Optimization Strategies, Section 3.1).

• Inferencing PODs: These focus on high-throughput, low-latency execution of pre-trained models. They typically utilize PCIe-based GPUs (e.g., H100 PCIe) which are more cost-effective for horizontal scaling. CPU-to-GPU ratios might be higher as some inference tasks are still CPU-bound for pre/post-processing. Networking emphasizes east-west bandwidth for load-balancing inferencing requests and fast access to data stores for input/output. A common design is 32-GPU PODs, distributed across multiple X440p nodes, prioritizing high-density 100/200GbE per server or 400GbE to fewer uplinks.
• Retrieval-Augmented Generation (RAG) PODs: These are hybrid, requiring both strong inferencing capabilities and efficient access to vector databases or traditional knowledge bases. H100 SXM5 GPUs are becoming prevalent due to their larger HBM capacity for context windows. Storage performance is even more critical here, as RAG involves frequent lookups and retrieval of indexed data. Network design needs to balance RoCEv2 for inter-GPU communication with standard TCP/IP for database interactions. A 64-GPU RAG POD might utilize a mix of X440p nodes, some optimized for GPU density, others for storage I/O, leveraging the modularity of UCS X-Series.
• Generative AI PODs: These are the most demanding, primarily focused on training and fine-tuning large language models. They require maximum GPU-to-GPU bandwidth, high HBM capacity, and incredibly fast, lossless networking. H100/H200 SXM5/SXM6 GPUs are standard, with multiple GPUs interconnected via NVLink within a node, and nodes connected via 400GbE/800GbE RoCEv2 fabric. The topology is often a full-mesh or closely connected fat-tree to minimize hop count and maximize collective communication performance. A 128-GPU or 256-GPU Generative AI POD would typically involve multiple UCS X9508 chassis, often in a single logical cluster, with dual-homed 400GbE or 800GbE uplinks from each X-Fabric module to the leaf layer.

Sizing and TCO Examples: 32, 128, 256-GPU PODs

Here's a comparison of typical configurations and estimated list prices for different AI POD sizes. These are illustrative and highly dependent on specific SKUs, discounting, and additional software licenses. Real-world procurement often involves extensive bundling and negotiations.

POD Size	Primary Use Case	UCS Compute (Estimate)	NVIDIA GPUs (Estimate)	Nexus Networking (Estimate)	Storage (Estimate)	Estimated List Price (USD)	Key Considerations
32-GPU	Inferencing, small-scale RAG	4x X9508 Chassis, 8x X440p (4x H100 PCIe each)	32x H100 PCIe (80GB)	2x N9K-9336C-FX2 (leaf), 1x N9K-9364D-GX2A (spine)	1x NetApp AFF A250 (100TB usable) or Pure //C60 (100TB usable)	$1.5M - $2.5M	Lower density, cost-effective for inferencing. Focus on high ingress/egress bandwidth.
128-GPU	RAG, Fine-tuning, Mid-scale Training	8x X9508 Chassis, 16x X440p (8x H100 SXM5 each)	128x H100 SXM5 (80GB)	4x N9K-93600CD-GX (leaf), 2x N9K-9364D-GX2A (spine)	2x NetApp AFF A900 (500TB usable) or Pure FlashBlade //E (500TB usable)	$8M - $12M	Balanced compute/storage. High RoCEv2 traffic, significant power/cooling needs.
256-GPU	Generative AI Training, Large-scale RAG	16x X9508 Chassis, 32x X440p (8x H100 SXM5 each)	256x H100 SXM5 (80GB)	8x N9K-93600CD-GX (leaf), 4x N9K-9364D-GX2A (spine)	4x NetApp AFF A900 (1PB usable) or Pure FlashBlade //E (1PB usable)	$18M - $25M+	Extreme demands on fabric (400G/800G), latency-sensitive inter-GPU comms. Dedicated cooling rows are often required.

A full 256-GPU POD built with H100 SXM5s represents a substantial capital expenditure. Compared to NVIDIA's DGX SuperPOD, Cisco's approach offers more flexibility in component selection and a unified management plane (Intersight) across compute and sometimes storage. HPE Cray systems, while offering ex-scale capabilities, tend to be significantly higher in initial CAPEX and often involve more specialized systems integration. The TCO analysis must consider not only hardware costs but also software licenses (NVIDIA AI Enterprise, OpenShift AI, Intersight), power consumption (hundreds of kW for larger PODs), and ongoing operational costs for cooling and maintenance.

Field Engineering: Cabling, Optics, and Airflow

Deployment is rarely a clean room scenario. Cabling for 400GbE / 800GbE fabrics is an immediate challenge. A 256-GPU POD involves hundreds of QSFP-DD cables. Each UCS X440p node can have up to 8x 400GbE uplinks via two X-Fabric modules. Multiplied by 32 nodes, this is 256x 400GbE cables to the leaf layer. Managing dense fiber optic cabling (MPO/MTP connectors, trunk cables) within racks and between racks requires meticulous planning and often dedicated cable trays. Incorrect cable management leads to airflow blockages and difficult troubleshooting. Optics selection (DR4, FR4, SR8) significantly impacts cable length and cost; standardizing on a minimal set for simplified sparing is recommended.

Airflow and power are paramount. Each H100 SXM5 GPU module can consume up to 700W. An 8-GPU X440p server can easily exceed 10kW. A 256-GPU POD might draw over 300kW, requiring multiple high-density PDUs (e.g., 3-phase 400V, 60A per PDU) and advanced cooling solutions like in-row cooling or liquid cooling at the chip or chassis level. Brownfield upgrades where existing data centers were designed for 5-7kW/rack often require significant infrastructure modifications. Monitoring temperature and power consumption at the rack, chassis, and component level is critical for preventing thermal runaway and unplanned downtime. Consider power redundancy (A/B feeds) for all critical POD components, from switches to compute and storage.

Day-2 Operations, Failure Modes, and Troubleshooting

The complexity of AI PODs makes day-2 operations challenging. Common failure modes include: RoCEv2 performance degradation due to misconfigured PFC/ECN, intermittent link flaps on 400GbE optics, GPU memory errors under sustained load, and storage bottlenecks during checkpointing or data loading. Troubleshooting requires cross-domain expertise. For instance, low GPU utilization could stem from network congestion (show queuing interface, show interface priority-flow-control on Nexus), storage latency (NetApp Active IQ, Pure1 analytics), or Kubernetes scheduling issues (kubectl describe pod, GPU Operator logs). Intersight Workload Optimizer helps correlate these events, providing a unified view across compute, network, and storage. Implementing robust monitoring, logging, and alerting systems from day one is non-negotiable. Continuous integration/continuous deployment (CI/CD) pipelines for Kubernetes manifests and infrastructure as code (IaC) for network/storage configurations are essential for managing change and preventing configuration drift.

Verdict

For organizations prioritizing a unified compute and network management platform with strong integration into the NVIDIA AI Enterprise ecosystem, Cisco's AI PODs leveraging UCS X-Series and Nexus 9000 are a compelling solution. They offer validated designs that address the extreme demands of AI workloads. The primary winners here are organizations with existing Cisco UCS and Nexus investments, seeking to leverage a familiar operational model for their AI infrastructure. The Inferencing PODs are strong for high-volume, lower-latency inference pipelines, particularly if PCIe GPUs offer a better cost/performance profile. RAG and Generative AI PODs demand the high-bandwidth SVX-based X440p nodes with SXM GPUs and the full 400GbE/800GbE RoCEv2 fabric; these are best suited for organizations needing to develop and train frontier models.

However, the complexity and CAPEX are significant. For organizations without a strong Cisco footprint, or those with highly customized AI stacks, alternative solutions like an open-source bare-metal Kubernetes deployment with commodity networking might offer a lower initial entry barrier, albeit with higher operational overhead. NVIDIA's DGX SuperPOD retains market share for turnkey, fully integrated AI platforms, but at a higher premium. The Cisco AI POD approach represents a well-engineered convergent solution that balances performance, scalability, and manageability, making it a powerful contender for enterprise AI initiatives in 2026 and beyond.

Related reading

• Optimizing RoCEv2 for HPC/AI Workloads
• Kubernetes GPU Scheduling Deep Dive with KubeFlow
• 400G and 800G Datacenter Fabrics: A 2026 Outlook
• NVMe-oF for AI/ML Workloads: Performance and Deployment
• Fortinet FortiGate 7.6 NGFW Design for AI Infrastructure Security