Configure and optimize ECS networking for production

Scenario

When deploying a high-throughput, latency-sensitive application on Alibaba Cloud, developers must bridge infrastructure-level networking with OS-level kernel tuning. This workflow guides you through provisioning ECS network interfaces and security groups, then optimizing the Alibaba Cloud Linux stack to minimize TCP overhead and maximize packet throughput.

Integration steps

1. Attach a secondary ENI to your ECS instance via the ECS API: aliyun ecs AttachNetworkInterface --InstanceId i-xxx --NetworkInterfaceId eni-xxx --DeviceIndex 1
2. Configure Security Group rules to allow high-throughput traffic: aliyun ecs AuthorizeSecurityGroup --SecurityGroupId sg-xxx --IpProtocol tcp --PortRange 8000/9000 --SourceCidrIp 10.0.0.0/8
3. Bind an EIP for public ingress/egress: aliyun vpc AssociateEipAddress --AllocationId eip-xxx --InstanceId i-xxx --Mode NAT
4. Reduce TCP TIME-WAIT accumulation on the Alinux host: sudo sysctl -w net.ipv4.tcp_tw_reuse=1 net.ipv4.tcp_max_tw_buckets=200000 net.ipv4.tcp_fin_timeout=15
5. Enable SMC acceleration for intra-VPC traffic: sudo modprobe smc && sudo sysctl -w net.smc.smc_enabled=1 net.smc.smc_rdma_enabled=1
6. Configure XPS (Transmit Packet Steering) for the secondary NIC (eth1): sudo sh -c 'echo 0-3 > /sys/class/net/eth1/queues/tx-0/xps_cpus'
7. Prevent NetworkManager from overriding NIC configs: sudo nmcli dev set eth1 managed no && sudo systemctl restart NetworkManager
8. Persist kernel parameters across reboots: Add the sysctl values to /etc/sysctl.d/99-network-tuning.conf and run sudo sysctl --system.

Architecture

Alibaba Cloud ECS manages the virtualized network boundary: VSwitch routing, ENI attachment, EIP NAT translation, and stateful Security Group packet filtering. Once traffic passes the SG, it enters the Alibaba Cloud Linux kernel, where the tuned TCP stack handles connection multiplexing, SMC bypasses traditional socket overhead via shared memory, and XPS maps transmit queues to specific CPU cores. This layered approach ensures infrastructure routing remains decoupled from host-level packet processing.

Prerequisites

• Active Alibaba Cloud account with ECS and VPC permissions
• Running ECS instance with Alibaba Cloud Linux 3.x
• Secondary ENI created in the same VSwitch as the instance
• aliyun CLI configured with valid AccessKey/SecretKey
• Root or sudo access to the target instance

Common pitfalls

• NetworkManager interference: Leaving eth1 managed causes NetworkManager to overwrite interface configs and reset XPS bindings on reboot.
• SG rule misalignment: SMC uses RDMA/TCP port 11911 by default; blocking it in the security group silently disables acceleration.
• TCP tuning overreach: Setting tcp_max_tw_buckets too low or tcp_tw_reuse=1 without net.ipv4.tcp_timestamps=1 can cause connection drops under high concurrency.
• Instance type limitations: SMC requires instance families with RDMA-capable NICs (e.g., ecs.g7se); enabling it on unsupported hardware yields fallback to standard TCP.

Typical questions

• set up ECS networking and optimize performance
• configure security groups and tune TCP on ECS
• deploy high-performance ECS server
• ECS网络配置和性能优化
• reduce latency on my ECS instance
• ECS生产环境网络调优
• optimize ECS for high connection throughput
• fix slow network on ECS after setup