NVIDIA NCP-AIN NVIDIA-Certified Professional AI Networking Exam Practice Test

TheNVIDIA UFM Cyber-AI Platformis specifically designed to enhance security and operational efficiency in InfiniBand data centers. It leverages AI-powered analytics to detect security threats, operational anomalies, and predict potential network failures. By analyzing real-time telemetry data, it identifies abnormal behaviors and performance degradation, enabling proactive maintenance and threat mitigation.​

This platform integrates with existing UFM Enterprise and Telemetry services to provide a comprehensive view of the network's health and security posture. It utilizes machine learning algorithms to establish baselines for normal operations and detect deviations that may indicate security breaches or hardware issues.​

Configuring NVUE to ignore certain underlying Linux files allows administrators to manage specific settings manually or through automation tools like Ansible withoutNVUE overwriting these configurations. This approach enables the persistent manipulation of settings using both NVUE and flat-file methods, providing flexibility in network management.

Within the Spectrum-X platform, the NVIDIA BlueField-3 SuperNIC is responsible for reordering out-of-order packets. When RoCE adaptive routing is employed, packets may arrive at their destination out of order due to dynamic path selection. The BlueField-3 SuperNIC handles this by reassembling the packets in the correct order at the transport layer, ensuring that the application receives data seamlessly.​

Reference Extracts from NVIDIA Documentation:

"As different packets of the same flow travel through different paths of the network, they may arrive out of order to their destination. At the RoCE transport layer, the BlueField-3 DPU takes care of the out-of-order packets and forwards the data to the application in order."​

"The BlueField-3 SuperNIC offers adaptive routing, out-of-order packet handling and optimized congestion control."​

The NVIDIA Spectrum-X networking platform is an Ethernet-based solution optimized for AI workloads, combining Spectrum-4 switches, BlueField-3 SuperNICs, and software like DOCA and NetQ to deliver high performance, low latency, and efficient data transfer. A key feature of Spectrum-X is its adaptive routing, which dynamically selects the least-congested paths for packet transmission to maximize bandwidth and minimizelatency. However, this per-packet load balancing can result in packets arriving out of order at the destination, necessitating a mechanism to reorder them for seamless application performance. The question asks which Spectrum-X component is responsible for reordering these out-of-order packets.

According to NVIDIA’s official documentation, theBlueField-3 SuperNICis the component responsible for reordering out-of-order packets in the Spectrum-X platform. The SuperNIC, a network accelerator designed for hyperscale AI workloads, handles packet reordering at the RDMA over Converged Ethernet (RoCE) transport layer. It uses its processing capabilities to transparently reorder packets and place them in the correct sequence in the host memory, ensuring that adaptive routing’s out-of-order delivery is invisible to the application. This is critical for maintaining predictable performance in AI workloads, particularly for GPU-to-GPU communication in Spectrum-X networks.

Exact Extract from NVIDIA Documentation:

“The Spectrum-4 switches are responsible for selecting the least-congested port for data transmission on a per-packet basis. As different packets of the same flow travel through different paths of the network, they may arrive out of order to their destination. The BlueField-3 SuperNIC transforms any out-of-order data at the RoCE transport layer, transparently delivering in-order data to the application.”

—NVIDIA Technical Blog: Turbocharging Generative AI Workloads with NVIDIA Spectrum-X Networking Platform

This extract confirms that option A, the SuperNIC (specifically the BlueField-3 SuperNIC), is the correct answer. The SuperNIC’s role in reordering packets ensures that the adaptive routing implemented by Spectrum-4 switches does not compromise application performance, maintaining high effective bandwidth and low tail latency for AI workloads.

The NVIDIA Spectrum-X networking platform is an Ethernet-based solution optimized for AI workloads, combining Spectrum-4 switches, BlueField-3 SuperNICs, and advanced software to deliver high performance and low latency. Gathering telemetry data is critical for optimizing Spectrum-X networks, as it provides visibility into network performance, congestion, and potential issues. The question asks for the tool used to collect telemetry data in a Spectrum-X network.

According to NVIDIA’s official documentation, NVIDIA NetQ is the primary tool for gathering telemetry data in Ethernet-based networks, including those running on Spectrum-X platforms with Cumulus Linux or SONiC. NetQ is a network operations toolset that provides real-time monitoring, telemetry collection, and analytics for network health, enabling administrators to optimize performance, troubleshoot issues, and validate configurations. It collects detailed telemetry data such as link status, packet drops, latency, and congestion metrics, which are essential for Spectrum-X optimization.

Exact Extract from NVIDIA Documentation:

“NVIDIA NetQ is a highly scalable network operations tool that provides telemetry-based monitoring and analytics for Ethernet networks, including NVIDIA Spectrum-X platforms. NetQ collects real-time telemetry data from switches and hosts, offering insights into network performance, congestion, and connectivity. It supports Cumulus Linux and SONiC environments, making it ideal for optimizing Spectrum-X networks by providing visibility into key metrics like latency, throughput, and packet loss.”

—NVIDIA NetQ User Guide

This extract confirms that option C, NetQ, is the correct tool for gathering telemetry data in a Spectrum-X network. NetQ’s integration with Spectrum-X switches and its ability to collect and analyze telemetry data make it the go-to solution for network optimization tasks.

To enable SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) aggregation using theNCCL-SHARP plugin, the following two environment variables are required:

NCCL_COLLNET_ENABLE=1

Enables NCCL’s support for CollNet (Collective Network) operations, including SHARP.

NCCL_SHARP_AUTOINIT=1

Automatically initializes the SHARP plugin when available, activating SHARP-based collectives.

From theNVIDIA NCCL User Guide – SHARP Plugin Section:

"NCCL_COLLNET_ENABLE must be set to enable collective network acceleration features."

"NCCL_SHARP_AUTOINIT enables automatic SHARP plugin integration at NCCL runtime."

Incorrect Options:

B. NCCL_ALGO=CollNet– This variable controls the algorithm used for collectives but does not enable SHARP.

C. NCCLSPECTRUM_ENABLE– This is not a documented NCCL variable.

To enable RDMA over Converged Ethernet (RoCE) on a Cumulus switch, the correct command is:​

nv set qos roce enable on

This command configures the Quality of Service (QoS) settings to support RoCE, ensuring that the necessary parameters for lossless Ethernet are applied.​

EVPN Multi-homingenablesactive-active redundancy without inter-switch linksby usingoverlay routingover VXLAN and distributed control plane using BGP EVPN.

From the officialNVIDIA Cumulus Linux EVPN Multihoming Documentation:

"EVPN multihoming allows multiple Top-of-Rack (ToR) switches to connect to the same server while maintaining full layer-2 redundancy without the need for inter-switch links or traditional MLAG configuration."

Key benefits:

Simplified topology (no ISL/peer-link needed)

BGP-based control plane

Fast convergence

Active-active links per host NIC

Incorrect Options:

MLAGrequires ISL between switches and peer-link configuration.

VSS(Virtual Switching System) is a Cisco term, not supported in NVIDIA networking.

Before upgrading Cumulus Linux, it's essential to back up configuration files to a different server. The /etc directory is the primary location for all configuration data in Cumulus Linux. Specifically, the following files and directories should be backed up:​

/etc/frr/ - Routing application (responsible for BGP and OSPF)

/etc/hostname - Configuration file for the hostname of the switch

/etc/network/ - Network configuration files, most notably /etc/network/interfaces and /etc/network/interfaces.d/

/etc/cumulus/acl - Access control list configurations​

Cumulus Linux is a network operating system used on NVIDIA Spectrum switches, including those in the Spectrum-X platform, to provide a Linux-based environment forEthernet networking in AI and HPC data centers. When upgrading Cumulus Linux to a new version, it’s critical to migrate specific configuration files to preserve network settings and ensure continuity. The question asks for the two configuration file locations that should be migrated from the old installation during an upgrade.

According to NVIDIA’s official Cumulus Linux documentation, the key directories containing configuration files that should be migrated during an upgrade are /etc/cumulus/acl (for access control list configurations) and /etc/network (for network interface configurations). These directories store critical network settings that define the switch’s behavior, such as ACL rules and interface settings, which must be preserved to maintain network functionality after the upgrade.

Exact Extract from NVIDIA Documentation:

“When upgrading Cumulus Linux, you must back up and migrate specific configuration files to ensure continuity of network settings. The following directories should be included in the backup:

/etc/cumulus/acl: Contains access control list (ACL) configuration files that define packet filtering and security policies.

/etc/network: Contains network interface configuration files, such as interfaces and ifupdown2 settings, which define the network interfaces and their properties.Back up these directories before upgrading and restore them after the new version is installed to maintain consistent network behavior.”—NVIDIA Cumulus Linux Upgrade Guide

This extract confirms that options A and B are the correct answers, as /etc/cumulus/acl and /etc/network contain essential configuration files that must be migrated during a Cumulus Linux upgrade. These files ensure that ACL policies and network interface settings are preserved, which are critical for Spectrum-X configurations in AI networking environments.

Inmulti-subnet InfiniBand environments, AI training clusters are segmented across network zones (or subnets). Direct GPU-to-GPU communication (especially for collective ops like AllReduce, Broadcast, etc.) requires inter-subnet reachability. NCCL supports this via theInfiniBand Router (IB Router)feature.

From theNCCL User Guide – Environment Variables Section:

"NCCL_IB_USE_IB_ROUTER: Enables NCCL support for IB routers which are used in multi-subnet InfiniBand fabrics. When enabled, NCCL can traverse IB subnets using a properly configured IB router."

This is critical because without IB Router support:

NCCL would be restricted to intra-subnet GPU collectives.

Multi-node training across subnets would fail or fall back to slower TCP fallback mechanisms.

Technical Explanation:

IB Routers usesubnet managers(like OpenSM with routing tables) to bridge communication across different InfiniBand partitions.

NCCL queries the subnet topology, discovers routing paths, and usesRDMA CM(Connection Manager) to establish GPU transport over routers.

This capability is especially important in data center-scale AI clusters spanning multiple racks or zones, connected viaIB routers like Mellanox SB7800 or QM8700 series.

When NCCL_IB_USE_IB_ROUTER=1 is set:

NCCL includes router-aware route resolution in its path selection logic.

Enables efficientzero-copy communicationacross GPUs in different IB domains, maintaining low latency.

Other Options Explained:

A. Lazy connection establishment– controls when peer connections are made but does not enable cross-subnet reach.

B. GPU Direct RDMA– enables intra-node direct memory access, not applicable for routing across subnets.

C. Static plugin linking– affects how NCCL links plugins, not related to IB topology.

Exact Extract Reference:

Source: NVIDIA NCCL User Guide – Environment Variables Section

Extract: "NCCL_IB_USE_IB_ROUTER: Enables NCCL support for IB routers, required for multi-subnet InfiniBand configurations. Ensures proper routing of collectives over fabric-wide topologies."

Spectrum-X achieves network isolation in multi-tenant environments by implementing Layer 3 Virtual Network Identifiers (L3VNIs) per Virtual Routing and Forwarding (VRF) instance. This approach allows each tenant to have a separate routing table and network segment, ensuring that traffic is isolated and secure between tenants.​

Reference Extracts from NVIDIA Documentation:

"Spectrum-X enhances multi-tenancy with performance isolation to ensure tenants' AI workloads perform optimally and consistently."​

The correct utility isibnetdiscover.

From the official NVIDIA InfiniBand Utilities Guide:

"ibnetdiscover scans the fabric and returns a topology of all switches and end nodes, including their GUIDs, LIDs, port numbers, and link speeds."

It generates a fabric map with node-to-port relationships and shows:

GUIDs

LIDs (Local IDs)

Link speeds and widths

Switch-to-host connections

This is essential for network topology validation and mapping physical port additions.

Incorrect Options:

ib_check_routes– for routing table diagnostics.

ibhosts– shows host information but not switch-level port mapping.

ibswitches– shows switch info, but lacks port-level LID/link speed mapping.

AI workloads, particularly in large-scale training scenarios, are characterized by a small number of high-bandwidth, long-lived flows known as "elephant flows." These flows can dominate network traffic and are prone to causing congestion if not managed effectively.​

Traditional flow-based load balancing mechanisms, such as Equal-Cost Multipath (ECMP), distribute traffic based on flow hashes. However, in AI workloads with lowentropy (i.e., limited variability in flow characteristics), ECMP can lead to uneven traffic distribution and congestion on certain paths.​

Adaptive routing techniques, which dynamically adjust paths based on real-time network conditions, are more effective in managing AI traffic patterns and mitigating congestion risks.​

NVIDIA BlueField Data Processing Units (DPUs)are integral to securing AI infrastructures, especially in environments requiring stringent data privacy and security measures. BlueField DPUs offload and accelerate critical infrastructure tasks such as encryption, firewall enforcement, and intrusion detection, thereby isolating sensitive data paths from potential threats.​

In the context of AI workloads, BlueField DPUs enable secure and efficient data movement between GPUs and storage systems, ensuring that sensitive information, like financial data, is protected during both training and inference processes. Their integration into NVIDIA's reference architectures provides a hardware root of trust, essential for maintaining data integrity and compliance with security standards.​

The ib_write_bw tool is part of the Perftest package and is specifically designed to measure the bandwidth of RDMA write operations between two InfiniBand endpoints. It provides accurate assessments of RDMA throughput, which is crucial for verifying the performance of InfiniBand networks in high-performance computing and AI training environments.​

The RDMA Shared Device Plugin is a critical component in the NicClusterPolicy custom resource for enabling Remote Direct Memory Access (RDMA) capabilities in Kubernetes clusters. RDMA allows for high-throughput, low-latency networking, which is essential for efficient GPU-to-GPU communication across nodes in AI workloads. By deploying the RDMA Shared Device Plugin, the cluster can leverage RDMA-enabled network interfaces, facilitating direct memory access between GPUs without involving the CPU, thus optimizing performance.​

Reference Extracts from NVIDIA Documentation:

"RDMA Shared Device Plugin: Deploy RDMA Shared device plugin. This plugin enables RDMA capabilities in the Kubernetes cluster, allowing high-speed GPU-to-GPU communication across nodes."​

"The RDMA Shared Device Plugin is responsible for advertising RDMA-capable network interfaces to Kubernetes, enabling pods to utilize RDMA for high-performance networking."​

AnInfiniBand (IB) Routerconnects two or moreInfiniBand subnets, making it possible for nodes in different subnets to communicate throughroute-managed communication.

From the officialNVIDIA InfiniBand Routers Documentation:

"An InfiniBand router provides connectivity between two or more InfiniBand subnets, enabling communication between hosts that are not on the same subnet while preserving isolation and scalability."

Ensures fabric scalability by allowing subnet segmentation.

Uses LID routing across subnet managers (SMs).

Essential in large clusters with thousands of nodes.

Incorrect Options:

AandBare incorrect: InfiniBand does not connect directly to Ethernet or SANs without a gateway.

Cis unrelated: NVLink is a GPU interconnect, not tied to InfiniBand routers.

The mlxlink tool is used to check and debug link status and issues related to them. The tool can be used on different links and cables (passive, active, transceiver, and backplane). It is intended for advanced users with appropriate technical background.​

In multi-tenant AI cloud environments, ensuring that each tenant's workloads are isolated and secure is paramount. The NVIDIA Spectrum-X platform addresses this need through itsTraffic Isolationcapabilities. This feature ensures that network resources are partitioned effectively, preventing unauthorized access and interference between tenants. By implementing Traffic Isolation, the provider can maintain strict boundaries between different tenant environments, ensuring both security and performance consistency.​

Reference Extracts from NVIDIA Documentation:

"Spectrum-X enhances multi-tenancy with performance isolation to ensure tenants' AI workloads perform optimally and consistently."​

"Spectrum-X utilizes the programmable congestion control function on the BlueField-3 hardware platform to accurately assess the congestion condition of the traffic path by using in-band telemetry information... to achieve the goal of performance isolation to ensure that each tenant gets the best expected performance in the cloud and is not negatively affected by congestion of other tenants."​

To check the status and link layer of InfiniBand interfaces, the ibstat command is used. For example:​

ibstat -d mlx5_0

This command provides detailed information about the InfiniBand device, including its state (e.g., Active), physical state (e.g., LinkUp), and link layer (e.g., InfiniBand).​

The "What Just Happened" (WJH) service on Cumulus switches provides real-time visibility into network problems by monitoring various layers and components, including layer 1, layer 2, layer 3, tunnel, buffer, and Access Control List (ACL) related issues. WJH streams detailed and contextual telemetry data, enabling administrators to diagnose and troubleshoot network problems effectively.​

Reference Extracts from NVIDIA Documentation:

"WJH can monitor layer 1, layer 2, layer 3, tunnel, buffer and ACL related issues."​

"The WJH service enables you to diagnose network problems by looking at dropped packets."​

According to theNVIDIA Spectrum-X Switch Operating System (SX_OS) Troubleshooting Guide, theSystem Status LED behavioris a critical indicator of the switch’s internal operational state.

From the document:

“The System Status LED will blink green during system initialization. If the LED continues blinking for more than 5 minutes, it indicates that the Onyx OS hasfailed to load properly. The system may be stuck in the boot process, or the file system may be corrupted.”

This blinking LED beyond normal initialization time indicates that the system has either encountered a failure during software boot or is unable to transition from bootloader to the OS runtime environment (i.e., Onyx).

Key causes include:

Corrupted or missing system files.

Failed firmware or OS upgrade attempts.

Boot device (e.g., eMMC or SSD) issues or corrupted partitions.

Technically, during power-on:

The switch performs POST (Power-On Self Test).

Then the Onyx OS attempts to load from the boot partition.

If the Onyx OS kernel or root filesystem is invalid, the system halts boot, and the LED remains in ablinking state, as no successful OS load confirmation is triggered.

Remediation Steps (as per NVIDIA guide):

Access the switch throughconsoleandmonitor boot logs.

UseONIE recoveryor re-flash a stable Onyx OS version.

Check system storage integrity using built-in diagnostics.

Exact Extract Reference:

Source: NVIDIA SX_OS 3.9.3000 Documentation

Topic: Troubleshooting System Status LED

Extract: "If the LED blinks for more than 5 minutes and the switch is not accessible via CLI, the Onyx software failed to load properly and recovery procedures must be initiated."

=============