ovs默认安全组流表分析

今天老大说openstack的默认安全组会使ovs产生大量流表，需要优化。So，开始分析一波吧 —- 基于 Q 版 qs。

简介

由于 Open vSwitch 无法直接与 Linux iptables 进行交互，因此为了实现安全组，之前采用了在 br-int 与实例之间增加一个 Linux 网桥来实现基于 iptables 的安全组功能。通常情况下，实例与基础物理网络之间的附加组件越多，性能与可扩展性越低。为了缓解这一问题，Neutron OpenvSwitch Agent 中实现了基于 OVS 流表的安全组驱动，当然这个驱动是可选的。

使用条件

OVS 原生防火前要求 Linux 内核与用户控件支持 conntrack，OVS 版本要求 >= 2.5.0，内核满足如下要求之一即可：

内核版本大于等于 4.3，支持 conntrack；或：
内核版本大于等于 3.3，小于 4.3 ，不支持 conntrack，需要构建 OVS 模块

验证

刚推好的干净环境，1compute+1control，查看 control 上 br-int 和 br-ex 的流表

br-int

# ovs-ofctl dump-flows br-int
cookie=0x99088c4baff699c5, duration=16812.012s, table=0, n_packets=100745, n_bytes=5440626, priority=2,in_port="int-br-ex" actions=drop
cookie=0x99088c4baff699c5, duration=16812.591s, table=0, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,60)
cookie=0x99088c4baff699c5, duration=16812.599s, table=23, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x99088c4baff699c5, duration=16812.575s, table=24, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x99088c4baff699c5, duration=16812.581s, table=60, n_packets=0, n_bytes=0, priority=3 actions=NORMAL

关于正常的二层自学习、转发和一些异常包的 drop 处理。0.0 为什么从 “int-br-ex” 进来的包全部drop，还没有建立会产生相关流量的网络环境？！待跟进……

br-ex

1
2
3

# ovs-ofctl dump-flows br-ex
cookie=0x1d4bdcc47e84eff4, duration=18576.503s, table=0, n_packets=0, n_bytes=0, priority=2,in_port="phy-br-ex" actions=drop
cookie=0x1d4bdcc47e84eff4, duration=18576.548s, table=0, n_packets=302732, n_bytes=39256436, priority=0 actions=NORMAL

同样一些基本的处理，和疑问。

br-tun
顺便看看，暂不讨论

# ovs-ofctl dump-flows br-tun
 cookie=0xafcd0599d35d5ef8, duration=19329.565s, table=0, n_packets=0, n_bytes=0, priority=1,in_port="patch-int" actions=resubmit(,2)
 cookie=0xafcd0599d35d5ef8, duration=19319.066s, table=0, n_packets=0, n_bytes=0, priority=1,in_port="vxlan-ac10000a" actions=resubmit(,4)
 cookie=0xafcd0599d35d5ef8, duration=19329.562s, table=0, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xafcd0599d35d5ef8, duration=19329.559s, table=2, n_packets=0, n_bytes=0, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)
 cookie=0xafcd0599d35d5ef8, duration=19329.553s, table=2, n_packets=0, n_bytes=0, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)
 cookie=0xafcd0599d35d5ef8, duration=19329.550s, table=3, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xafcd0599d35d5ef8, duration=19329.547s, table=4, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xafcd0599d35d5ef8, duration=19329.545s, table=6, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0xafcd0599d35d5ef8, duration=19329.541s, table=10, n_packets=0, n_bytes=0, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xafcd0599d35d5ef8,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:OXM_OF_IN_PORT[]),output:"patch-int"
 cookie=0xafcd0599d35d5ef8, duration=19329.534s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)
 cookie=0xafcd0599d35d5ef8, duration=19329.531s, table=22, n_packets=0, n_bytes=0, priority=0 actions=drop

下图的逻辑基本全了
ovs_br-tun_flow

next, nova boot 一个虚机：

（应该使用devstack，自动完成初始化）

上传 cirros 镜像
# wget -P http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
# openstack image create "cirros" --file cirros --disk-format qcow2 --container-format bare --public
创建 flavor 和 network
# openstack flavor create --vcpu 1 --disk 30 --ram 1024 small
# neutron net-create ... && neutron subnet-create ...
启动虚机
# nova boot --flavor small --image cirros --nic net-name=private1 client1

OK…我应该在compute上作对比的…br-ex没有改变，以下是启动虚机后br-int的流表：

cookie=0xa5efb0c04a407ee7, duration=1387.488s, table=0, n_packets=0, n_bytes=0, priority=10,icmp6,in_port="qvof6079278-43",icmp_type=136 actions=resubmit(,24)
cookie=0xa5efb0c04a407ee7, duration=1387.474s, table=0, n_packets=81, n_bytes=3402, priority=10,arp,in_port="qvof6079278-43" actions=resubmit(,24)
cookie=0xa5efb0c04a407ee7, duration=24153.973s, table=0, n_packets=144749, n_bytes=8299518, priority=2,in_port="int-br-ex" actions=drop
cookie=0xa5efb0c04a407ee7, duration=1387.504s, table=0, n_packets=8, n_bytes=1144, priority=9,in_port="qvof6079278-43" actions=resubmit(,25)
cookie=0xa5efb0c04a407ee7, duration=24245.788s, table=0, n_packets=2, n_bytes=727, priority=0 actions=resubmit(,60)
cookie=0xa5efb0c04a407ee7, duration=24245.792s, table=23, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0xa5efb0c04a407ee7, duration=1387.498s, table=24, n_packets=0, n_bytes=0, priority=2,icmp6,in_port="qvof6079278-43",icmp_type=136,nd_target=fe80::f816:3eff:fe14:a50c actions=resubmit(,60)
cookie=0xa5efb0c04a407ee7, duration=1387.481s, table=24, n_packets=81, n_bytes=3402, priority=2,arp,in_port="qvof6079278-43",arp_spa=192.168.1.10 actions=resubmit(,25)
cookie=0xa5efb0c04a407ee7, duration=24245.778s, table=24, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0xa5efb0c04a407ee7, duration=1387.515s, table=25, n_packets=89, n_bytes=4546, priority=2,in_port="qvof6079278-43",dl_src=fa:16:3e:14:a5:0c actions=resubmit(,60)
cookie=0xa5efb0c04a407ee7, duration=24245.783s, table=60, n_packets=91, n_bytes=5273, priority=3 actions=NORMAL

看见了”qvoxxxxx”，这就说明还是启用了 linux bridge 做 security group,接着看一下果然存在一个linux bridge。
嗯…现在需要修改配置文件，使ovs去实现安全组。

在 /etc/neutron/plugins/ml2 目录下修改 openvswitch_agent.ini 和 ml2_conf.ini 文件

[securitygroup]
firewall_driver = openvswitch
重启agent
# systemctl status neutron-openvswitch-agent
再次启动一个虚机
# nova boot --flavor small --image cirros --nic net-name=private1 client2

结果还是没有实现ovs接替安全组的功能，排查发现Q版默认使用docker管理服务，修改 /var/lib/config-data/puppet-generated/neutron/etc/neutron/plugins/ml2/openvswitch_agent.ini 文件，重启ovs-agent容器,验证发现顺利启用native ovs firewall driver。
启动虚机，查看br-int流表变化情况(以下为增加的流)：

+table=60 priority=100,in_port=10 actions=load:0xa->NXM_NX_REG5[],load:0x3->NXM_NX_REG6[],resubmit(,71)
+table=60 priority=90,dl_vlan=3,dl_dst=fa:16:3e:61:ac:5e actions=load:0xa->NXM_NX_REG5[],load:0x3->NXM_NX_REG6[],pop_vlan,resubmit(,81)
+table=71 priority=95,icmp6,reg5=0xa,in_port=10,icmp_type=130 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xa,in_port=10,icmp_type=133 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xa,in_port=10,icmp_type=135 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xa,in_port=10,icmp_type=136 actions=resubmit(,91)
+table=71 priority=70,icmp6,reg5=0xa,in_port=10,icmp_type=134 actions=resubmit(,93)
+table=71 priority=95,arp,reg5=0xa,in_port=10,dl_src=fa:16:3e:61:ac:5e,arp_spa=192.168.1.6 actions=resubmit(,91)
+table=71 priority=80,udp,reg5=0xa,in_port=10,tp_src=68,tp_dst=67 actions=resubmit(,73)
+table=71 priority=80,udp6,reg5=0xa,in_port=10,tp_src=546,tp_dst=547 actions=resubmit(,73)
+table=71 priority=70,udp,reg5=0xa,in_port=10,tp_src=67,tp_dst=68 actions=resubmit(,93)
+table=71 priority=70,udp6,reg5=0xa,in_port=10,tp_src=547,tp_dst=546 actions=resubmit(,93)
+table=71 priority=65,ct_state=-trk,ip,reg5=0xa,in_port=10,dl_src=fa:16:3e:61:ac:5e,nw_src=192.168.1.6 actions=ct(table=72,zone=NXM_NX_REG6[0..15])
+table=71 priority=65,ct_state=-trk,ipv6,reg5=0xa,in_port=10,dl_src=fa:16:3e:61:ac:5e,ipv6_src=fe80::f816:3eff:fe61:ac5e actions=ct(table=72,zone=NXM_NX_REG6[0..15])
+table=71 priority=10,ct_state=-trk,reg5=0xa,in_port=10 actions=resubmit(,93)
+table=72 priority=74,ct_state=+est-rel-rpl,ipv6,reg5=0xa actions=resubmit(,73)
+table=72 priority=74,ct_state=+est-rel-rpl,ip,reg5=0xa actions=resubmit(,73)
+table=72 priority=74,ct_state=+new-est,ipv6,reg5=0xa actions=resubmit(,73)
+table=72 priority=74,ct_state=+new-est,ip,reg5=0xa actions=resubmit(,73)
+table=72 priority=50,ct_mark=0x1,reg5=0xa actions=resubmit(,93)
+table=72 priority=50,ct_state=+est-rel+rpl,ct_zone=3,ct_mark=0,reg5=0xa actions=resubmit(,91)
+table=72 priority=50,ct_state=-new-est+rel-inv,ct_zone=3,ct_mark=0,reg5=0xa actions=resubmit(,91)
+table=72 priority=40,ct_state=-est,reg5=0xa actions=resubmit(,93)
+table=72 priority=40,ct_state=+est,ip,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=72 priority=40,ct_state=+est,ipv6,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=73 priority=100,reg6=0x3,dl_dst=fa:16:3e:61:ac:5e actions=load:0xa->NXM_NX_REG5[],resubmit(,81)
+table=73 priority=90,ct_state=+new-est,ip,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,91)
+table=73 priority=90,ct_state=+new-est,ipv6,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,91)
+table=73 priority=80,reg5=0xa actions=resubmit(,91)
+table=81 priority=100,arp,reg5=0xa actions=output:10,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xa,icmp_type=130 actions=output:10,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xa,icmp_type=135 actions=output:10,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xa,icmp_type=136 actions=output:10,resubmit(,92)
+table=81 priority=95,udp,reg5=0xa,tp_src=67,tp_dst=68 actions=output:10,resubmit(,92)
+table=81 priority=95,udp6,reg5=0xa,tp_src=547,tp_dst=546 actions=output:10,resubmit(,92)
+table=81 priority=90,ct_state=-trk,ip,reg5=0xa actions=ct(table=82,zone=NXM_NX_REG6[0..15])
+table=81 priority=90,ct_state=-trk,ipv6,reg5=0xa actions=ct(table=82,zone=NXM_NX_REG6[0..15])
+table=81 priority=80,ct_state=+trk,reg5=0xa actions=resubmit(,82)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg6=0x3,nw_src=192.168.1.6 actions=conjunction(8,1/2)
+table=82 priority=70,ct_state=+new-est,ip,reg6=0x3,nw_src=192.168.1.6 actions=conjunction(9,1/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ipv6,reg5=0xa actions=conjunction(16,2/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg5=0xa actions=conjunction(8,2/2)
+table=82 priority=70,ct_state=+new-est,ipv6,reg5=0xa actions=conjunction(17,2/2)
+table=82 priority=70,ct_state=+new-est,ip,reg5=0xa actions=conjunction(9,2/2)
+table=82 priority=70,conj_id=16,ct_state=+est-rel-rpl,ipv6,reg5=0xa actions=load:0x10->NXM_NX_REG7[],output:10,resubmit(,92)
+table=82 priority=70,conj_id=8,ct_state=+est-rel-rpl,ip,reg5=0xa actions=load:0x8->NXM_NX_REG7[],output:10,resubmit(,92)
+table=82 priority=70,conj_id=17,ct_state=+new-est,ipv6,reg5=0xa actions=load:0x11->NXM_NX_REG7[],ct(commit,zone=NXM_NX_REG6[0..15]),output:10,resubmit(,92)
+table=82 priority=70,conj_id=9,ct_state=+new-est,ip,reg5=0xa actions=load:0x9->NXM_NX_REG7[],ct(commit,zone=NXM_NX_REG6[0..15]),output:10,resubmit(,92)
+table=82 priority=50,ct_mark=0x1,reg5=0xa actions=resubmit(,93)
+table=82 priority=50,ct_state=+est-rel+rpl,ct_zone=3,ct_mark=0,reg5=0xa actions=output:10,resubmit(,92)
+table=82 priority=50,ct_state=-new-est+rel-inv,ct_zone=3,ct_mark=0,reg5=0xa actions=output:10,resubmit(,92)
+table=82 priority=40,ct_state=-est,reg5=0xa actions=resubmit(,93)
+table=82 priority=40,ct_state=+est,ip,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=82 priority=40,ct_state=+est,ipv6,reg5=0xa actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))

再启动一个虚机，看看流表增加了什么：

+table=60 priority=100,in_port=12 actions=load:0xc->NXM_NX_REG5[],load:0x5->NXM_NX_REG6[],resubmit(,71)
+table=60 priority=90,dl_vlan=5,dl_dst=fa:16:3e:9a:3f:5f actions=load:0xc->NXM_NX_REG5[],load:0x5->NXM_NX_REG6[],pop_vlan,resubmit(,81)
+table=71 priority=95,icmp6,reg5=0xc,in_port=12,icmp_type=130 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xc,in_port=12,icmp_type=133 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xc,in_port=12,icmp_type=135 actions=resubmit(,91)
+table=71 priority=95,icmp6,reg5=0xc,in_port=12,icmp_type=136 actions=resubmit(,91)
+table=71 priority=70,icmp6,reg5=0xc,in_port=12,icmp_type=134 actions=resubmit(,93)
+table=71 priority=95,arp,reg5=0xc,in_port=12,dl_src=fa:16:3e:9a:3f:5f,arp_spa=192.168.2.8 actions=resubmit(,91)
+table=71 priority=80,udp,reg5=0xc,in_port=12,tp_src=68,tp_dst=67 actions=resubmit(,73)
+table=71 priority=80,udp6,reg5=0xc,in_port=12,tp_src=546,tp_dst=547 actions=resubmit(,73)
+table=71 priority=70,udp,reg5=0xc,in_port=12,tp_src=67,tp_dst=68 actions=resubmit(,93)
+table=71 priority=70,udp6,reg5=0xc,in_port=12,tp_src=547,tp_dst=546 actions=resubmit(,93)
+table=71 priority=65,ct_state=-trk,ip,reg5=0xc,in_port=12,dl_src=fa:16:3e:9a:3f:5f,nw_src=192.168.2.8 actions=ct(table=72,zone=NXM_NX_REG6[0..15])
+table=71 priority=65,ct_state=-trk,ipv6,reg5=0xc,in_port=12,dl_src=fa:16:3e:9a:3f:5f,ipv6_src=fe80::f816:3eff:fe9a:3f5f actions=ct(table=72,zone=NXM_NX_REG6[0..15])
+table=71 priority=10,ct_state=-trk,reg5=0xc,in_port=12 actions=resubmit(,93)
+table=72 priority=74,ct_state=+est-rel-rpl,ipv6,reg5=0xc actions=resubmit(,73)
+table=72 priority=74,ct_state=+est-rel-rpl,ip,reg5=0xc actions=resubmit(,73)
+table=72 priority=74,ct_state=+new-est,ipv6,reg5=0xc actions=resubmit(,73)
+table=72 priority=74,ct_state=+new-est,ip,reg5=0xc actions=resubmit(,73)
+table=72 priority=50,ct_mark=0x1,reg5=0xc actions=resubmit(,93)
+table=72 priority=50,ct_state=+est-rel+rpl,ct_zone=5,ct_mark=0,reg5=0xc actions=resubmit(,91)
+table=72 priority=50,ct_state=-new-est+rel-inv,ct_zone=5,ct_mark=0,reg5=0xc actions=resubmit(,91)
+table=72 priority=40,ct_state=-est,reg5=0xc actions=resubmit(,93)
+table=72 priority=40,ct_state=+est,ip,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=72 priority=40,ct_state=+est,ipv6,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=73 priority=100,reg6=0x5,dl_dst=fa:16:3e:9a:3f:5f actions=load:0xc->NXM_NX_REG5[],resubmit(,81)
+table=73 priority=90,ct_state=+new-est,ip,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,91)
+table=73 priority=90,ct_state=+new-est,ipv6,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15]),resubmit(,91)
+table=73 priority=80,reg5=0xc actions=resubmit(,91)
+table=81 priority=100,arp,reg5=0xc actions=output:12,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xc,icmp_type=130 actions=output:12,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xc,icmp_type=135 actions=output:12,resubmit(,92)
+table=81 priority=100,icmp6,reg5=0xc,icmp_type=136 actions=output:12,resubmit(,92)
+table=81 priority=95,udp,reg5=0xc,tp_src=67,tp_dst=68 actions=output:12,resubmit(,92)
+table=81 priority=95,udp6,reg5=0xc,tp_src=547,tp_dst=546 actions=output:12,resubmit(,92)
+table=81 priority=90,ct_state=-trk,ip,reg5=0xc actions=ct(table=82,zone=NXM_NX_REG6[0..15])
+table=81 priority=90,ct_state=-trk,ipv6,reg5=0xc actions=ct(table=82,zone=NXM_NX_REG6[0..15])
+table=81 priority=80,ct_state=+trk,reg5=0xc actions=resubmit(,82)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg6=0x4,nw_src=192.168.2.8 actions=conjunction(16,1/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg6=0x5,nw_src=192.168.2.8 actions=conjunction(16,1/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg6=0x5,nw_src=192.168.1.11 actions=conjunction(16,1/2)
+table=82 priority=70,ct_state=+new-est,ip,reg6=0x4,nw_src=192.168.2.8 actions=conjunction(17,1/2)
+table=82 priority=70,ct_state=+new-est,ip,reg6=0x5,nw_src=192.168.2.8 actions=conjunction(17,1/2)
+table=82 priority=70,ct_state=+new-est,ip,reg6=0x5,nw_src=192.168.1.11 actions=conjunction(17,1/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ipv6,reg5=0xc actions=conjunction(8,2/2)
+table=82 priority=70,ct_state=+est-rel-rpl,ip,reg5=0xc actions=conjunction(16,2/2)
+table=82 priority=70,ct_state=+new-est,ipv6,reg5=0xc actions=conjunction(9,2/2)
+table=82 priority=70,ct_state=+new-est,ip,reg5=0xc actions=conjunction(17,2/2)
+table=82 priority=70,conj_id=8,ct_state=+est-rel-rpl,ipv6,reg5=0xc actions=load:0x8->NXM_NX_REG7[],output:12,resubmit(,92)
+table=82 priority=70,conj_id=16,ct_state=+est-rel-rpl,ip,reg5=0xc actions=load:0x10->NXM_NX_REG7[],output:12,resubmit(,92)
+table=82 priority=70,conj_id=9,ct_state=+new-est,ipv6,reg5=0xc actions=load:0x9->NXM_NX_REG7[],ct(commit,zone=NXM_NX_REG6[0..15]),output:12,resubmit(,92)
+table=82 priority=70,conj_id=17,ct_state=+new-est,ip,reg5=0xc actions=load:0x11->NXM_NX_REG7[],ct(commit,zone=NXM_NX_REG6[0..15]),output:12,resubmit(,92)
+table=82 priority=50,ct_mark=0x1,reg5=0xc actions=resubmit(,93)
+table=82 priority=50,ct_state=+est-rel+rpl,ct_zone=5,ct_mark=0,reg5=0xc actions=output:12,resubmit(,92)
+table=82 priority=50,ct_state=-new-est+rel-inv,ct_zone=5,ct_mark=0,reg5=0xc actions=output:12,resubmit(,92)
+table=82 priority=40,ct_state=-est,reg5=0xc actions=resubmit(,93)
+table=82 priority=40,ct_state=+est,ip,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
+table=82 priority=40,ct_state=+est,ipv6,reg5=0xc actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))

再跑一个N版的devstack继续进行研究。

以下为默认的流表情况(只关注br-int)：

cookie=0x9730ef020b7d4f04, duration=63695.441s, table=0, n_packets=939, n_bytes=39690, priority=3,in_port="int-br-ex",vlan_tci=0x0000/0x1fff actions=mod_vlan_vid:2,NORMAL
 cookie=0x9730ef020b7d4f04, duration=63701.394s, table=0, n_packets=7, n_bytes=738, priority=2,in_port="int-br-ex" actions=drop
 cookie=0x9730ef020b7d4f04, duration=63701.637s, table=0, n_packets=1148, n_bytes=130485, priority=0 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=63701.639s, table=23, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=63701.635s, table=24, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=63700.652s, table=71, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=89.943s, table=72, n_packets=0, n_bytes=0, priority=50,ct_state=+inv+trk actions=drop
 cookie=0x9730ef020b7d4f04, duration=63700.645s, table=72, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=63700.638s, table=73, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=63700.631s, table=81, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=89.943s, table=82, n_packets=0, n_bytes=0, priority=50,ct_state=+inv+trk actions=drop
 cookie=0x9730ef020b7d4f04, duration=63700.624s, table=82, n_packets=0, n_bytes=0, priority=0 actions=drop

启动一台虚机,产生了大量流表：

cookie=0x9730ef020b7d4f04, duration=64481.177s, table=0, n_packets=7, n_bytes=738, priority=2,in_port="int-br-ex" actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.529s, table=0, n_packets=102, n_bytes=10111, priority=100,in_port="tapf27e3d10-13" actions=load:0xb->NXM_NX_REG5[],load:0x1->NXM_NX_REG6[],resubmit(,71)
 cookie=0x9730ef020b7d4f04, duration=593.529s, table=0, n_packets=66, n_bytes=7041, priority=90,dl_dst=fa:16:3e:33:99:6c actions=load:0xb->NXM_NX_REG5[],load:0x1->NXM_NX_REG6[],resubmit(,81)
 cookie=0x9730ef020b7d4f04, duration=64475.224s, table=0, n_packets=939, n_bytes=39690, priority=3,in_port="int-br-ex",vlan_tci=0x0000/0x1fff actions=mod_vlan_vid:2,NORMAL
 cookie=0x9730ef020b7d4f04, duration=64481.420s, table=0, n_packets=1167, n_bytes=132309, priority=0 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=64481.422s, table=23, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=64481.418s, table=24, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=0, n_bytes=0, priority=95,icmp6,reg5=0xb,in_port="tapf27e3d10-13",icmp_type=130 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=0, n_bytes=0, priority=95,icmp6,reg5=0xb,in_port="tapf27e3d10-13",icmp_type=134 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=2, n_bytes=156, priority=95,icmp6,reg5=0xb,in_port="tapf27e3d10-13",icmp_type=135 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=0, n_bytes=0, priority=95,icmp6,reg5=0xb,in_port="tapf27e3d10-13",icmp_type=136 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=71, n_packets=0, n_bytes=0, priority=70,icmp6,reg5=0xb,in_port="tapf27e3d10-13",icmp_type=134 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=3, n_bytes=126, priority=95,arp,reg5=0xb,in_port="tapf27e3d10-13",dl_src=fa:16:3e:33:99:6c,arp_spa=192.168.1.7 actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=2, n_bytes=676, priority=80,udp,reg5=0xb,in_port="tapf27e3d10-13",tp_src=68,tp_dst=67 actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=71, n_packets=0, n_bytes=0, priority=80,udp6,reg5=0xb,in_port="tapf27e3d10-13",tp_src=546,tp_dst=547 actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=71, n_packets=0, n_bytes=0, priority=70,udp,reg5=0xb,in_port="tapf27e3d10-13",tp_src=67,tp_dst=68 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=71, n_packets=0, n_bytes=0, priority=70,udp6,reg5=0xb,in_port="tapf27e3d10-13",tp_src=547,tp_dst=546 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=92, n_bytes=8903, priority=65,ct_state=-trk,ip,reg5=0xb,in_port="tapf27e3d10-13",dl_src=fa:16:3e:33:99:6c,nw_src=192.168.1.7 actions=ct(table=72,zone=NXM_NX_REG6[0..15])
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=2, n_bytes=160, priority=65,ct_state=-trk,ipv6,reg5=0xb,in_port="tapf27e3d10-13",dl_src=fa:16:3e:33:99:6c,ipv6_src=fe80::f816:3eff:fe33:996c actions=ct(table=72,zone=NXM_NX_REG6[0..15])
 cookie=0x9730ef020b7d4f04, duration=593.528s, table=71, n_packets=0, n_bytes=0, priority=65,ct_state=-trk,ipv6,reg5=0xb,in_port="tapf27e3d10-13",dl_src=fa:16:3e:33:99:6c,ipv6_src=fd7e:d8a3:64c9:0:f816:3eff:fe33:996c actions=ct(table=72,zone=NXM_NX_REG6[0..15])
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=71, n_packets=1, n_bytes=90, priority=10,ct_state=-trk,reg5=0xb,in_port="tapf27e3d10-13" actions=drop
 cookie=0x9730ef020b7d4f04, duration=64480.435s, table=71, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=75, n_bytes=7610, priority=70,ct_state=+est-rel-rpl,ip,reg5=0xb,dl_src=fa:16:3e:33:99:6c actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=70,ct_state=+est-rel-rpl,ipv6,reg5=0xb,dl_src=fa:16:3e:33:99:6c actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=17, n_bytes=1293, priority=70,ct_state=+new-est,ip,reg5=0xb,dl_src=fa:16:3e:33:99:6c actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=2, n_bytes=160, priority=70,ct_state=+new-est,ipv6,reg5=0xb,dl_src=fa:16:3e:33:99:6c actions=resubmit(,73)
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=50,ct_state=+inv+trk actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=50,ct_mark=0x1,reg5=0xb actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=50,ct_state=+est-rel+rpl,ct_zone=1,ct_mark=0,reg5=0xb actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=50,ct_state=-new-est+rel-inv,ct_zone=1,ct_mark=0,reg5=0xb actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=40,ct_state=-est,reg5=0xb actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=40,ct_state=+est,ip,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
 cookie=0x9730ef020b7d4f04, duration=593.525s, table=72, n_packets=0, n_bytes=0, priority=40,ct_state=+est,ipv6,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
 cookie=0x9730ef020b7d4f04, duration=64480.428s, table=72, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=73, n_packets=0, n_bytes=0, priority=100,dl_dst=fa:16:3e:33:99:6c actions=load:0xb->NXM_NX_REG5[],resubmit(,81)
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=73, n_packets=17, n_bytes=1293, priority=90,ct_state=+new-est,ip,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15]),NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=73, n_packets=2, n_bytes=160, priority=90,ct_state=+new-est,ipv6,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15]),NORMAL
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=73, n_packets=77, n_bytes=8286, priority=80,reg5=0xb actions=NORMAL
 cookie=0x9730ef020b7d4f04, duration=64480.421s, table=73, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=81, n_packets=3, n_bytes=126, priority=100,arp,reg5=0xb,dl_dst=fa:16:3e:33:99:6c actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=81, n_packets=0, n_bytes=0, priority=100,icmp6,reg5=0xb,dl_dst=fa:16:3e:33:99:6c,icmp_type=130 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=81, n_packets=0, n_bytes=0, priority=100,icmp6,reg5=0xb,dl_dst=fa:16:3e:33:99:6c,icmp_type=134 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.527s, table=81, n_packets=0, n_bytes=0, priority=100,icmp6,reg5=0xb,dl_dst=fa:16:3e:33:99:6c,icmp_type=135 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=0, n_bytes=0, priority=100,icmp6,reg5=0xb,dl_dst=fa:16:3e:33:99:6c,icmp_type=136 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=2, n_bytes=758, priority=95,udp,reg5=0xb,tp_src=67,tp_dst=68 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=0, n_bytes=0, priority=95,udp6,reg5=0xb,tp_src=547,tp_dst=546 actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=61, n_bytes=6157, priority=90,ct_state=-trk,ip,reg5=0xb actions=ct(table=82,zone=NXM_NX_REG6[0..15])
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=0, n_bytes=0, priority=90,ct_state=-trk,ipv6,reg5=0xb actions=ct(table=82,zone=NXM_NX_REG6[0..15])
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=81, n_packets=0, n_bytes=0, priority=80,ct_state=+trk,reg5=0xb,dl_dst=fa:16:3e:33:99:6c actions=resubmit(,82)
 cookie=0x9730ef020b7d4f04, duration=64480.414s, table=81, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=50,ct_state=+inv+trk actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=50,ct_mark=0x1,reg5=0xb actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=61, n_bytes=6157, priority=50,ct_state=+est-rel+rpl,ct_zone=1,ct_mark=0,reg5=0xb,dl_dst=fa:16:3e:33:99:6c actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=50,ct_state=-new-est+rel-inv,ct_zone=1,ct_mark=0,reg5=0xb,dl_dst=fa:16:3e:33:99:6c actions=strip_vlan,output:"tapf27e3d10-13"
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=40,ct_state=-est,reg5=0xb actions=drop
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=40,ct_state=+est,ip,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
 cookie=0x9730ef020b7d4f04, duration=593.526s, table=82, n_packets=0, n_bytes=0, priority=40,ct_state=+est,ipv6,reg5=0xb actions=ct(commit,zone=NXM_NX_REG6[0..15],exec(load:0x1->NXM_NX_CT_MARK[]))
 cookie=0x9730ef020b7d4f04, duration=64480.407s, table=82, n_packets=0, n_bytes=0, priority=0 actions=drop

分析

需要先了解一下802.1常见字段的定义：

ovs里面的常见字段
| 字段名称 | 说明 |
| - | - |
| in_port=port | 传递数据包的端口的 OpenFlow 端口编号 |
| dl_vlan=vlan | 数据包的 VLAN Tag 值，范围是 0-4095，0xffff 代表不包含 VLAN Tag 的数据包 |
| dl_src=
dl_dst=
| 数据包的 VLAN Tag 值，范围是 0-4095，0xffff 代表不包含 VLAN Tag 的数据包 |
| dl_type=ethertype | 匹配以太网协议类型，其中：dl_type=0x0800 代表 IPv4 协议; dl_type=0x086dd 代表 IPv6 协议; dl_type=0x0806 代表 ARP 协议 |
| nw_src=ip[/netmask] nw_dst=ip[/netmask] | 当 dl_typ=0x0800 时，匹配源或者目标的 IPv4 地址，可以使 IP 地址或者域名 |
| nw_proto=proto | 和 dl_type 字段协同使用。当 dl_type=0x0800 时，匹配 IP 协议编号;当 dl_type=0x086dd 代表 IPv6 协议编号|
| table=number | 指定要使用的流表的编号，范围是 0-254。在不指定的情况下，默认值为 0。通过使用流表编号，可以创建或者修改多个 Table 中的 Flow |
|reg
=value[/mask] | 交换机中的寄存器的值。当一个数据包进入交换机时，所有的寄存器都被清零，用户可以通过 Action 的指令修改寄存器中的值 |
dl: datalink ; nw: network
对于 add−flow，add−flows 和 mod−flows 这三个命令，还需要指定要执行的动作：actions=[target][,target…]
一个流规则中可能有多个动作，按照指定的先后顺序执行。
常见的操作有：
- output:port: 输出数据包到指定的端口。port 是指端口的 OpenFlow 端口编号
- mod_vlan_vid: 修改数据包中的 VLAN tag
- strip_vlan: 移除数据包中的 VLAN tag
- mod_dl_src/ mod_dl_dest: 修改源或者目标的 MAC 地址信息
- mod_nw_src/mod_nw_dst: 修改源或者目标的 IPv4 地址信息
- resubmit:port: 替换流表的 in_port 字段，并重新进行匹配
- load:value−>dst[start..end]: 写数据到指定的字段
  |name | age|
  |—| —|
  |hello|www|
关于TIC、VNI等字段：
{:height=”50%” width=”50%”}
1. TPID (Tag Protocol Identifier，也就是EtherType)
  IEEE定义的新的类型，表明这是一个加了802.1Q标签的帧。TPID包含了一个固定的值0x8100。
2. TCI (Tag Control Information)
  包括用户优先级(User Priority)、规范格式指示器(Canonical Format Indicator)和 VLAN ID。
  - User Priority：该字段为3-bit，用于定义用户优先级，总共有8个(2的3次方)优先级别。IEEE 802.1P 为3比特的用户优先级位定义了操作。最高优先级为7，应用于关键性网络流量，如路由选择信息协议（RIP）和开放最短路径优先（OSPF）协议的路由表更新。优先级6和5主要用于延迟敏感（delay-sensitive）应用程序，如交互式视频和语音。优先级4到1主要用于受控负载（controlled-load）应用程序，如流式多媒体（streaming multimedia）和关键性业务流量（business-critical traffic）－例如，SAP 数据－以及“loss eligible”流量。优先级0是缺省值，并在没有设置其它优先级值的情况下自动启用。
  - CFI：CFI值为0说明是规范格式，1为非规范格式。它被用在令牌环/源路由FDDI介质访问方法中来指示封装帧中所带地址的比特次序信息。
  - VID：该字段为12-bit， VLAN ID 是对 VLAN 的识别字段，在标准 802.1Q 中常被使用。支持4096(2的12次方) VLAN 的识别。在4096可能的VID 中，VID＝0 用于识别帧优先级。 4095(FFF)作为预留值，所以 VLAN 配置的最大可能值为4094。所以有效的VLAN ID范围一般为1-4094。

Tables:

Table 0
Port-based ACLs
Table 1
Ingress VLAN processing
Table 2
VLAN-based ACLs
Table 3
Ingress L2 processing, MAC learning
Table 4
L3 forwarding for IPv4
Table 5
L3 forwarding for IPv6
Table 6
Virtual IP processing, e.g. for router IP addresses implemented by Faucet
Table 7
Egress L2 processing
Table 8
Flooding