ECN
📝 Short definition
Network mechanism (RFC 3168) that marks packets when congestion is imminent, letting the sender slow down before packets are dropped.
📖 Detailed definition
When a switch detects congestion (filling buffers), it marks packets with an ECN-CE bit. The receiver sends a CNP (Congestion Notification Packet) to the sender, which reduces injection rate. That avoids packet loss without relying on PFC alone.
On RoCE networks, ECN is combined with PFC via DCQCN (Data Center Quantized Congestion Notification, from Microsoft and NVIDIA) for finer congestion control.
💡 Why it matters for on-prem AI
Complements PFC by managing congestion proactively rather than reactively. A production-ready RoCE network should enable PFC + ECN/DCQCN on all gear.
⚠️ Common pitfalls
- ECN alone is not enough: without PFC to absorb bursts, packets can still be lost.
- DCQCN parameters (Kmin, Kmax, timer thresholds) must be tuned for link speed and cluster size — defaults are not tuned for AI workloads.
🔗 See also
Priority Flow Control — Ethernet per-priority pause mechanism for a lossless network required by RoCE.Protocol bringing RDMA benefits to standard Ethernet, without a dedicated InfiniBand fabric, at the cost of configuring a lossless network (PFC, ECN).Network technique that reads and writes remote memory without going through the CPU, with low latency; it comes in InfiniBand and RoCE implementations.