Deploy optimized ECS with tuned networking

Scenario

This workflow is required when deploying latency-sensitive, high-throughput microservices or data-processing pipelines on Alibaba Cloud. Developers combine ECS infrastructure provisioning with Alibaba Cloud Linux (Alinux) kernel-level network tuning to maximize packet throughput, minimize TCP connection overhead, and leverage hardware-accelerated inter-process communication for production workloads.

Integration steps

1. Provision Alinux ECS Instance: Launch an instance using the Alibaba Cloud CLI.

aliyun ecs RunInstances --ImageId aliyun_3_x64_20G_alibase_*.vhd --InstanceType ecs.g7.xlarge --VpcId vpc-xxx --VSwitchId vsw-xxx --SecurityGroupId sg-xxx

2. Attach Secondary ENI & Configure Security Group: Bind a dedicated NIC for workload traffic and open high-port ranges.

aliyun ecs AttachNetworkInterface --InstanceId i-xxx --NetworkInterfaceId eni-xxx aliyun ecs AuthorizeSecurityGroup --SecurityGroupId sg-xxx --IpProtocol tcp --PortRange 8000/9000

3. Assign Elastic IP (EIP): Allocate and bind a public IP for external ingress.

aliyun vpc AllocateEipAddress --Bandwidth 100 aliyun vpc AssociateEipAddress --EipId eip-xxx --InstanceId i-xxx

4. Exclude Secondary NIC from NetworkManager: Prevent OS-level routing conflicts by creating /etc/NetworkManager/conf.d/99-unmanaged-eni.conf:

[device-unmanaged]\nmatch-device=interface-name:eth1 systemctl restart NetworkManager

5. Reduce TCP TIME-WAIT: Tune kernel parameters in /etc/sysctl.conf:

net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_max_tw_buckets = 10000 Apply: sysctl -p

6. Enable SMC Acceleration: Load the Shared Memory Communication module for intra-VPC traffic:

modprobe smc echo "smc" >> /etc/modules-load.d/smc.conf

7. Configure XPS (Transmit Packet Steering): Bind TX queues to specific vCPUs on the primary NIC:

echo f > /sys/class/net/eth0/queues/tx-0/xps_cpus echo f > /sys/class/net/eth0/queues/tx-1/xps_cpus

8. Persist & Verify: Add sysctl and XPS bindings to a systemd startup service. Validate with ss -s and ethtool -S eth0.

Architecture

The ECS control plane manages the virtualized network boundary: VPC routing, ENI attachment, Security Group filtering, and EIP NAT translation. Inbound traffic traverses the EIP → Security Group → ENI pipeline. Alinux takes over at the guest OS layer, where the tuned network stack (TCP parameters, SMC kernel module, XPS CPU affinity) processes packets directly in kernel space, bypassing standard scheduler bottlenecks before delivering them to the application socket.

Prerequisites

• Alibaba Cloud CLI (aliyun) authenticated with RAM credentials
• Pre-configured VPC, VSwitch, and baseline Security Group
• Valid Alinux 3.x image ID and compute-optimized instance type (e.g., ecs.g7)
• Root/SSH access to the provisioned instance
• Sufficient ENI quota and EIP bandwidth allocation

Common pitfalls

• NetworkManager overriding configs: Failing to unmanage secondary ENIs causes IP conflicts and routing table corruption.
• Aggressive TIME-WAIT tuning: Enabling tcp_tw_reuse=1 without verifying application connection pooling can trigger stale connection reuse and data corruption.
• SMC module incompatibility: SMC-R requires specific instance families (e.g., ecs.g7se) and matching VPC configurations; loading it on unsupported types silently falls back to TCP.
• XPS CPU mask mismatch: Writing incorrect hex masks to xps_cpus without matching the instance’s vCPU topology causes packet drops and CPU starvation.
• Security Group blocking SMC: SMC uses specific RDMA/SMC ports; missing SG rules will force fallback to standard TCP, negating performance gains.

Typical questions

• set up ECS with optimized networking
• deploy production ECS and tune network performance
• ECS部署并优化网络性能
• configure ECS network and enable SMC acceleration
• 创建ECS实例并优化TCP性能
• high-performance ECS instance setup
• ECS网络配置与性能调优
• deploy and tune ECS for low latency