建站
咨詢
售后
建站咨詢
新聞中心
這里有您想知道的互聯(lián)網(wǎng)營銷解決方案
聊聊云網(wǎng)絡(luò)性能測試
聊聊云網(wǎng)絡(luò)性能測試
作者：扎波特的網(wǎng)線鉗 2021-06-02 10:00:30
云計算 有幾個云上的小伙伴想測測VPC網(wǎng)絡(luò)性能，于是寫了一些dpdk代碼在阿里云上做了一個實驗，也適用于其它云。
創(chuàng)新互聯(lián)公司專業(yè)為企業(yè)提供荷塘網(wǎng)站建設(shè)、荷塘做網(wǎng)站、荷塘網(wǎng)站設(shè)計、荷塘網(wǎng)站制作等企業(yè)網(wǎng)站建設(shè)、網(wǎng)頁設(shè)計與制作、荷塘企業(yè)網(wǎng)站模板建站服務(wù)，10余年荷塘做網(wǎng)站經(jīng)驗，不只是建網(wǎng)站，更提供有價值的思路和整體網(wǎng)絡(luò)服務(wù)。
本文轉(zhuǎn)載自微信公眾號「zartbot」，作者扎波特的網(wǎng)線鉗。轉(zhuǎn)載本文請聯(lián)系zartbot公眾號。
有幾個云上的小伙伴想測測VPC網(wǎng)絡(luò)性能，于是寫了一些dpdk代碼在阿里云上做了一個實驗，也適用于其它云
安裝相關(guān)的庫
使用root登錄,更新一下源
- #備份原有的配置文件
- mkdir /etc/yum.repos.d/bak
- mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/
- #使用阿里云的源覆蓋
- wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo
- yum install -y https://mirrors.aliyun.com/epel/epel-release-latest-8.noarch.rpm
- sed -i 's|^#baseurl=https://download.fedoraproject.org/pub|baseurl=https://mirrors.aliyun.com|' /etc/yum.repos.d/epel*
- sed -i 's|^metalink|#metalink|' /etc/yum.repos.d/epel*
- sudo dnf config-manager --set-enabled PowerTools
- yum makecache
- yum update
- yum groupinstall "Development tools"
- yum install gcc-gfortran kernel-modules-extra tcl tk tcsh terminator tmux kernel-rpm-macros elfutils-libelf-devel libnl3-devel meson createrepo numactl-devel
- pip3 install pyelftools
啟用iommu
- sudo vi /etc/default/grub
- //在 GRUB_CMDLINE_LINUX 行添加"intel_iommu=on iommu=pt"
- //保存退出
然后更新grub并重啟系統(tǒng)
- sudo grub2-mkconfig -o /boot/grub2/grub.cfg
- sudo grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg
- sudo reboot
安裝DPDK
CentOS上需要添加/usr/local路徑, 主要是LD_LIBRARY_PATH PATH 和 PKG_CONFIG_PATH 以及sudo的path
- sudo vi /etc/ld.so.conf.d/dpdk.conf
- >>添加如下path
- /usr/local/lib64
- >>退出
- sudo ldconfig
- vim ~/.bashrc
- >>添加如下path
- export PATH=/usr/local/bin:$PATH
- export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:${PKG_CONFIG_PATH}
- 保存后source
- source ~/.bashrc
- sudo vim /etc/sudoers
- >>將secure_path添加/usr/local/bin
- Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
然后解壓dpdk，并編譯安裝
- wget http://fast.dpdk.org/rel/dpdk-21.05.tar.xz
- tar xf dpdk-21.05.tar.xz
- cd dpdk-21.05
- meson build -D examples=all
- cd build
- ninja
- sudo ninja install
- sudo ldconfig
設(shè)置Hugepage和bind接口
- dpdk-hugepages.py --setup 4G
- modprobe vfio-pci
- dpdk-devbind.py -s
- Network devices using kernel driver
- ===================================
- 0000:00:05.0 'Virtio network device 1000' if=eth0 drv=virtio-pci unused=vfio-pci *Active*
- 0000:00:06.0 'Virtio network device 1000' if=eth1 drv=virtio-pci unused=vfio-pci *Active*
注意虛擬機環(huán)境需要noniommu_mode
- ifconfig eth1 down
- echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
- dpdk-devbind.py -b vfio-pci 0000:00:06.0
驗證
- dpdk-devbind.py -s
- Network devices using DPDK-compatible driver
- ============================================
- 0000:00:06.0 'Virtio network device 1000' drv=vfio-pci unused=
- Network devices using kernel driver
- ===================================
- 0000:00:05.0 'Virtio network device 1000' if=eth0 drv=virtio-pci unused=vfio-pci *Active*
檢查接口支持情況
下載代碼
- cd ~
- wget https://github.com/zartbot/learn_dpdk/archive/refs/heads/main.zip
- unzip main.zip
- cd learn_dpdk-main/
編譯
- cd 01_port_init/devinfo/
- make clean;make
檢查接口支持情況
- ./build/devinfo
- EAL: Detected 24 lcore(s)
- EAL: Detected 1 NUMA nodes
- EAL: Detected shared linkage of DPDK
- EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
- EAL: Selected IOVA mode 'PA'
- EAL: No available 1048576 kB hugepages reported
- EAL: VFIO support initialized
- EAL: Invalid NUMA socket, default to 0
- EAL: Probe PCI driver: net_virtio (1af4:1000) device: 0000:00:05.0 (socket 0)
- eth_virtio_pci_init(): Failed to init PCI device
- EAL: Requested device 0000:00:05.0 cannot be used
- EAL: Invalid NUMA socket, default to 0
- EAL: Probe PCI driver: net_virtio (1af4:1000) device: 0000:00:06.0 (socket 0)
- EAL: Using IOMMU type 8 (No-IOMMU)
- TELEMETRY: No legacy callbacks, legacy socket not created
- *****************************************
- number of available port: 1
- =========================================
- port: 0 Driver:net_virtio
- Link down
- MAC address: 00:16:3E:25:3F:0A
- PCIe:0000:00:06.0
- Max RX Queue: 12 Desc: 65535
- Max TX Queue: 12 Desc: 65535
- Offload Capability:
- DEV_RX_OFFLOAD_VLAN_STRIP
- DEV_RX_OFFLOAD_UDP_CKSUM
- DEV_RX_OFFLOAD_TCP_CKSUM
- DEV_RX_OFFLOAD_TCP_LRO
- DEV_RX_OFFLOAD_JUMBO_FRAME
- -----------------------------------------
- DEV_TX_OFFLOAD_VLAN_INSERT
- DEV_TX_OFFLOAD_UDP_CKSUM
- DEV_TX_OFFLOAD_TCP_CKSUM
- DEV_TX_OFFLOAD_TCP_TSO
- DEV_TX_OFFLOAD_MULTI_SEGS
- =========================================
測速
- cd ~/learn_dpdk-main/02_send_recv/traffic_gen/
修改send_pkt.c的源目的地址，注意目的MAC在阿里云上要為eeff.ffff.ffff
- //init mac
- struct rte_ether_addr s_addr = {{0x00, 0x16, 0x3e, 0x25, 0x0b, 0xe3}};
- struct rte_ether_addr d_addr = {{0xee, 0xff, 0xff, 0xff, 0xff, 0xff}};
- //init IP header
- rte_be32_t s_ip_addr = string_to_ip("10.66.1.220");
- rte_be32_t d_ip_addr = string_to_ip("10.66.1.219");
由于接口支持有限，修改 common.h
- #define NUM_RX_QUEUE 1
- #define NUM_TX_QUEUE 1
- static const struct rte_eth_conf port_conf_default = {
- .rxmode = {
- .max_rx_pkt_len = RTE_ETHER_MAX_LEN,
- .mq_mode = ETH_MQ_RX_NONE,
- },
- .txmode = {
- .mq_mode = ETH_MQ_TX_NONE,
- }
- };
修改portinit.c 關(guān)閉RX-CHECKSUM OFFLOAD, 注釋掉下面這段：
- if (dev_info.rx_offload_capa & DEV_RX_OFFLOAD_CHECKSUM)
- {
- printf("port[%u] support RX cheksum offload.\n", port);
- port_conf.rxmode.offloads |= DEV_RX_OFFLOAD_CHECKSUM;
- }
最后測速大概3.3Mpps左右，接近官方售賣時的4Mpps
- [root@iZuf64vmgrtj12kczyslhdZ traffic_gen]# ./build/run
- EAL: Detected 24 lcore(s)
- EAL: Detected 1 NUMA nodes
- EAL: Detected shared linkage of DPDK
- EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
- EAL: Selected IOVA mode 'PA'
- EAL: No available 1048576 kB hugepages reported
- EAL: VFIO support initialized
- EAL: Invalid NUMA socket, default to 0
- EAL: Probe PCI driver: net_virtio (1af4:1000) device: 0000:00:05.0 (socket 0)
- eth_virtio_pci_init(): Failed to init PCI device
- EAL: Requested device 0000:00:05.0 cannot be used
- EAL: Invalid NUMA socket, default to 0
- EAL: Probe PCI driver: net_virtio (1af4:1000) device: 0000:00:06.0 (socket 0)
- EAL: Using IOMMU type 8 (No-IOMMU)
- TELEMETRY: No legacy callbacks, legacy socket not created
- initializing port 0...
- port[0] support TX UDP checksum offload.
- port[0] support TX TCP checksum offload.
- Port[0] MAC: 00:16:3e:25:0b:e3
- Core 1 doing RX dequeue.
- Core 2 doing packet enqueue.
- RX-Queue[0] PPS: 3280464
- RX-Queue[0] PPS: 3277792
- RX-Queue[0] PPS: 3303116
- RX-Queue[0] PPS: 3307443
- RX-Queue[0] PPS: 3296451
- RX-Queue[0] PPS: 3294396
- RX-Queue[0] PPS: 3297737
- RX-Queue[0] PPS: 3290069
- RX-Queue[0] PPS: 3279720
- RX-Queue[0] PPS: 3285987
- RX-Queue[0] PPS: 3279424
然后把common.h 中收發(fā)都改為4個線程
- #define NUM_RX_QUEUE 1
- #define NUM_TX_QUEUE 1
測試結(jié)果和官方售賣的4Mpps一致了。
- RX-Queue[0] PPS: 578918
- RX-Queue[1] PPS: 866823
- RX-Queue[2] PPS: 2288950
- RX-Queue[3] PPS: 865335
CPU Info
- [root@iZuf64vmgrtj12kczyslhdZ traffic_gen]# cat /proc/cpuinfo | grep Xeon
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
- model name : Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
分享題目：聊聊云網(wǎng)絡(luò)性能測試
網(wǎng)站路徑：http://www.5511xx.com/article/cdddegp.html
