简介

VPN的英文全称是“Virtual Private Network”，翻译过来就是“虚拟专用网络”。顾名思义，虚拟专用网络我们可以把它理解成是虚拟出来的企业内部专线。它可以通过特殊的加密的通讯协议在连接在Internet上的位于不同地方的两个或多个企业内部网之间建立一条专有的通讯线路，就好比是架设了一条专线一样，但是它并不需要真正的去铺设光缆之类的物理线路。一句话，VPN的核心就是在利用公共网络建立虚拟私有网。

VPNaas（VPN as a service）是Openstack的Havana版本中，neutron增加的一个新的功能，将VPN功能引入到了neutron中，虽然在Havana版中只支持IPSec协议的VPN，功能还比较弱，但这个关键特性已经有了，后续会不断加强该特性，其长期目标是使其功能非常丰富，支持多种支持静态和动态路由的隧道安全协议，但从短期来看，希望仅由静态态路由完成基于开源IPsec的基本实现。

VPN的本质和分类

vpn比较容易误解的是，某网络区域不可访问的时候使用vpn就能顺利解决，这种情况其实vpn到vpn设备或网关就已终结，由他们进行隧道协议的封装和解封装后完成数据转发。
vpn本身的建立是在网络可达性之上的，其一般完成两件事：1.通过封装对中间设备透明，识别内部对应层次的原有报文语义；2.通过加密使数据不会暴露在外部传输网络中。

网络虚拟化最基础的技术莫过于分层，要实现分层有两种手段，一个是映射，另一个是封装。vpn技术就是基于封装的技术。

  • 映射：主要思路是转发时替换报文语义，如何替换将需要设备进行查询。
  • 封装：把需要的报文语义添加到网包中，处理的时候一层层的解封装即可，尽量对设备透明。

第二层隧道协议

PPP、PPTP、L2TP、L2F等都属于第二层隧道协议，修改帧报头将用户数据封装在PPP帧中通过互联网发送。对于非以太网，ATM和帧中继也是两种最为流行的vpn隧道。第二层隧道的优点是能将大二层直接打通，独立于它传输第三层的数据流。

第三层隧道协议

如果递送的报头位于第三层就是第三层vpn，如gre、mpls、ipsec等。

IP-over-IP

经常会有人将像GRE这种隧道协议和IP-over-IP混淆。IP-over-IP一般具有两层IP报头，内部报头和外部报头，但无论是内部报头还是外部报头都是标准的IP报头，等于还是IP协议。但GRE这种隧道会有自己的新的报头，这样就相当于一种新协议，所以严格地讲我们叫它隧道协议，而不是隧道。

常见 VPN 实现介绍

GRE

建立vpn很方便，且留有专门的virtual tunnel interface可以直接完成IGP的私网收敛，但没有安全机制，即数据没有进行加密，将内部报文直接暴露在外部传输网络中。

MPLS

在传输数据之前就先用L3层的路由机制将L2层的标签在途经的每个路由器上都事先算好了，并且这种标签是自动配置的。

SSL

这种vpn的最大好处在于，仅需要一个单独的TCP或UDP端口便可以轻易穿越大多数防火墙进行数据传送。openVPN是在Linux系统上最好的实现，其使在广域网内网的两台主机可以直接进行通信，即host-to-host。

IPSec（重点介绍）

目前neutron唯一实现的 vpn driver，但也已不再活跃。

IPSec就封装建立隧道而言还是很鸡肋的，匹配感兴趣流较麻烦，多路由节点配置，IGP无法收敛，所以一般采用和gre结合的方式，gre完成隧道建立，ipsec进行加密处理，结合方式有两种：

  • IPSEC Over GRE：IPSEC在里，GRE在外。先把需要加密的数据包封装成IPSEC包，然后再扔到GRE隧道里。作法是把IPSEC的加密图作用在Tunnel口上的，即在Tunnel口上监控（访问控制列表监控本地ip网段-源i和远端ip网段-目的地），是否有需要加密的数据流，有则先加密封装为IPSEC包，然后封装成GRE包进入隧道（这里显而易见的是，GRE隧道始终无论如何都是存在的，即GRE隧道的建立过程并没有被加密），同时，未在访问控制列表里的数据流将以不加密的状态直接走GRE隧道，即存在有些数据可能被不安全地传递的状况。

    1
    2
    3
    4
    - 访问控制列表，针对两个网段的数据流，如：
      ip access-list extended vpn12
      permit ip 10.1.1.0 0.0.0.255 10.2.2.0 0.0.0.255
    - 加密处放在Tunnel口

  • GRE Over IPSEC：GRE Over IPSEC是指，先把数据分装成GRE包，然后再分装成IPSEC包。做法是在物理接口上监控，是否有需要加密的GRE流量（访问控制列表针对GRE两端的设备ip），所有的这两个端点的GRE数据流将被加密分装为IPSEC包再进行传递，这样保证的是所有的数据包都会被加密，包括隧道的建立和路由的建立和传递。

    1
    2
    3
    4
    - 访问列表，针对两个路由器之间的GRE流，如：
      ip access-list extended vpn12
      permit gre host 172.16.11.2 host 172.16.22.2
    - 加密图作用在物理口。

另外还有一个概念是传输模式和隧道模式，加密方式也分为AH和ESP。

  • 传输模式：一层报头，传输过程中IPSec源端点不会修改IP报中的目标IP地址，只是对IP协议的数据部分(payload)进行了加密。
  • 隧道模式：内外两层报头，传输过程中IPSec源端点会修改IP报中的目标IP地址，对整个IP数据包进行加密。

显然隧道模式不符合我们的应用场景。

对于加密方式，由于南北流量需要经过端口复用的SNAT或者一对一的fip，会产生冲突。

  • AH：AH封装的校验从IP头开始，如果NAT将IP的头部改动，AH的校验就会失败，因此我们得出结论，AH是无法与NAT共存的。
  • ESP：
    • 传输模式，虽然不适合场景，此处也说明一下。对于NAT来说，ESP封装比AH的优势在于，无论是加密还是完整性的校验，IP头部都没有被包括进去。但是还是有新的问题，对于ESP的传输模式，NAT 无法更新上层校验和。因为TCP 和 UDP 报头包含一个校验和，它整合了源和目标 IP 地址和端口号的值，而源和目的IP地址和端口号在做NAT时会发生改变。当 NAT 改变了某个包的 IP 地址和（或）端口号时，它通常要更新 TCP 或 UDP 校验和。当 TCP 或 UDP 校验和使用了 ESP 来加密时，它就无法更新这个校验和。由于地址或端口已经被 NAT 更改，目的地的校验和检验就会失败。虽然 UDP 校验和是可选的，但是 TCP 校验和却是必需的。所以ESP的传输模式也不支持NAT穿越。
    • 隧道模式。ESP隧道模式将整个原始的IP包整个进行了加密，且在ESP的头部外面新加了一层IP头部，所以NAT如果只改变最前面的新的IP地址对后面受到保护的部分是不会有影响的。因此，IPsec只有采用ESP的隧道模式来封装数据时才能与NAT共存。但只能对非PAT得情况，即因为ESP协议号50，直接构建在IP层之上，没有类似于UDP/TCP端口号的概念，也就没有NAT复用标识，只能采用fip，q-router的qg桥上的默认snat失效，所以需要采用nat-t，通过借用UDP的方式实现端口复用。详细可参看https://technet.microsoft.com/zh-cn/library/bb878090.aspx。

模块架构

模块架构

数据模型

VPNServices Resource

Attribute Type DefaultValue Vaildation Constraint Notes
id uuid-str generated N/A 用于VPNService对象的UUID
tenant_id uuid-str None Valid tenant_id vpn服务的租户的UUID
name string None N/A VPN服务的名称
description string None N/A VPN服务的描述
status string N/A N/A 指示ipsec vpnservice当前是否可用。可能的值包括：ACTIVE DOWN BUILD ERROR
admin_state_up bool TRUE true/false vpnservice的管理状态。 如果为false（down），则端口不转发数据包
subnet_id uuid N/A Valid subnet id 租户需要vpn服务的子网ID
router_id uuid N/A Valid router id vpn服务所插入的路由器ID

IKEPolicies Resource

Attribute Type DefaultValue Vaildation Constraint Notes
id uuid-str generated N/A 用于IKEPolicy的UUID
tenant_id uuid-str None Valid tenant_id 用于vpn服务的所有者的UUID
name string None N/A 友好的ikepolicy名称
description string None N/A ikepolicy的描述
auth_algorithm string sha1 N/A 身份验证散列算法“sha1”
encryption_algorithm string aes-128 N/A 加密算法3des，aes-128，aes-256，aes-192等
phase1_negotiation_mode string Main Mode N/A IKE模式主模式
pfs string Group5 N/A 完美的正向保密（Group2，Group5，Group14）
ike_version string v1 N/A v1或v2版本
lifetime
units string seconds “seconds” SA单元的生命周期,以’秒’为单位
value integer 3600 for seconds Integer 以秒为单位的生命周期值（值> = 60）

IPsecPolicies Resource

Attribute Type DefaultValue Vaildation Constraint Notes
id uuid-str generated N/A 用于IPsecPolicy的UUID
tenant_id uuid-str None Valid tenant_id 用于vpn服务的所有者的UUID
name string None N/A 友好的IPsecPolicy名称
description string None N/A policy的描述
transform_protocol string ESP N/A 使用Tranform协议，例如ESP或AH或AH-ESP
encapsulation_mode string tunnel N/A 封装模式为隧道模式或传输模式
auth_algorithm string sha1 N/A 身份验证散列算法“sha1”
encryption_algorithm string aes-128 N/A 加密算法3des，aes-128，aes-256，aes-192等
pfs string Group5 N/A 完美的正向保密（Group2，Group5，Group14）
lifetime
units string seconds “seconds” SA单元的生命周期,以’秒’为单位
value integer 3600 for seconds Integer 以秒为单位的生命周期值（值> = 60）

ipsec-site-connection Resource

Attribute Type DefaultValue Vaildation Constraint Notes
id uuid-str generated N/A 用于vpn连接的UUID
tenant_id uuid-str None valid tenant_id 用于vpn服务的所有者的UUID
name string None N/A ipsec-site-connection的名称
description string None N/A ipsec站点连接的描述
peer_address ipaddress(v4 or v6) N/A valid ip address (v4 or v6) 对等VPN网关公共地址或FQDN
peer_id string N/A N/A 对等标识符（可以是名称，字符串或FQDN）
peer_cidrs list[string] N/A valid cidr 对等私有cidr
route_mode string static static 静态的
mtu integer 1500 Integer mtu - 地址分段的最大传输单元（值> = 68）
auth_mode string psk psk/certs 身份验证模式，PSK或证书
psk string N/A NO 预共享密钥,可以是任何字符串
initiator string bi-directional “bi-directional / response-only” 这个VPN是否只能响应连接或者可以启动
admin_state_up bool True “true / false” VPN连接的管理状态。 如果为false（down），vpn连接不转发数据包
status string N/A N/A 指示vpn连接当前是否可用。 可能的值包括：ACTIVE，DOWN，BUILD，ERROR
ikepolicy_id uuid N/A uuid of ikepolicy ikepolicy的uuid ID
ipsecpolicy_id uuid N/A uuid of ipsecpolicy ipsecpolicy的uuid ID
vpnservice_id uuid N/A uuid of vpnservice vpnservice的服务ID
dpd
action string hold “hold / clear / disabled /restart /restart_by_peer” DPD操作控制使用死对等检测协议。 ( clear, hold, restart, disabled, restart-by-peer)
interval integer 30 > 0 DPD延迟的秒数
timeout integer 120 > 0 & > dpd_interval DPD超时的秒数

object diagram

目前，VPNaaS中的设计和对象模型允许每个路由器和每个子网创建1个服务对象。 但是每个服务可以有多个VPN连接对象。具体对象图如下：
object_diagram

API 命令

本部分介绍支持VPNaaS高级服务的CLI命令。详细内容见API参考文档: VPNaaS 2.0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
vpn-service-create      Create a VPNService
vpn-service-delete      Delete a given VPNService
vpn-service-list        List all VPNService for a given tenant.
vpn-service-show        Show detailed information of a given VPNService.
vpn-service-update      Update a given VPNservice.

vpn-ikepolicy-create       Create an IKEPolicy
vpn-ikepolicy-delete       Delete a given IKE Policy.
vpn-ikepolicy-list         List IKEPolicies that belong to a given tenant.
vpn-ikepolicy-show         Show detailed information of a given IKEPolicy.
vpn-ikepolicy-update       Update a given IKE Policy.

vpn-ipsecpolicy-create     Create an IPsec policy
vpn-ipsecpolicy-delete     Delete a given IPsec Policy
vpn-ipsecpolicy-list       List IPsecPolicies that belong to a given tenant    connection.
vpn-ipsecpolicy-show       Show detailed information of a given IPsec Policy
vpn-ipsecpolicy-update     Update a given IPsec Policy.

ipsec-site-connection-create  Create a ipsec-site-connection
ipsec-site-connection-delete  Delete a given ipsec-site-connection.
ipsec-site-connection-list    List ipsec-site-connections that belong to a given tenant.
ipsec-site-connection-show    Show information of a given ipsec-site-connection.
ipsec-site-connection-update  Update a given ipsec-site-connection.

vpn-endpoint-group-create     Create a VPN endpoint groups.
vpn-endpoint-group-delete     Removes a VPN endpoint group.
vpn-endpoint-group-list       Lists VPN endpoint groups.
vpn-endpoint-group-show       Shows details for a VPN endpoint group.
vpn-endpoint-group-update     Updates settings for a VPN endpoint group.

service-provider-list         List service providers.

部署

Devstack快速部署

使用devstack在单节点上快速安装VPNaaS服务，需要在local.conf文件中添加以下内容：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[[local|localrc]]

enable_plugin neutron-vpnaas https://git.openstack.org/openstack/neutron-vpnaas

disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
# Optional, to enable tempest configuration as part of devstack
enable_service tempest

# IPSec driver to use. Optional, defaults to strongswan.For example, install libreswan for CentOS/RHEL 7
# IPSEC_PACKAGE="libreswan"

执行stack.sh脚本文件，并且成功完成后，云主机环境中会发生如下变化：

  • /etc/neutron/neutron.conf文件中service_plugins后面会增加vpnaas字段：

    1
    2
    [DEFAULT]
    service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,vpnaas

  • /etc/neutron/目录下会为你配置好neutron_vpnaas.conf文件，内容如下：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    [service_providers]
    service_provider = VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
    # if you want use libreswan，configuration example is as follows
    #service_provider = VPN:libreswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
    # From neutron.vpnaas
    #
    # Defines providers for advanced services using the format:
    # <service_type>:<name>:<driver>[:default] (multi valued)
    #service_provider =

  • /etc/neutron/l3_agent.ini中会增加[AGENT]和[vpnagent]项：

    1
    2
    3
    4
    5
    6
    7
    8
    [AGENT]
    extensions = vpnaas
    root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
    root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
    [vpnagent]
    vpn_device_driver = neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver
    # when use libreswan
    # vpn_device_driver = neutron_vpnaas.services.vpn.device_drivers.libreswan_ipsec.LibreSwanDriver

  • /etc/neutron/rootwrap.d目录下会增加vpnaas.filters权限管理文件，内容如下：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    # neutron-rootwrap command filters for nodes on which neutron is
    # expected to control network
    # for libreswan, just change "strongswan" to "libreswan"
    # This file should be owned by (and only-writable by) the root user
    # format seems to be
    # cmd-name: filter-name, raw-command, user, args
    [Filters]
    cp: RegExpFilter, cp, root, cp, -a, .*, .*/strongswan.d
    ip: IpFilter, ip, root
    ip_exec: IpNetnsExecFilter, ip, root
    ipsec: CommandFilter, ipsec, root
    rm: RegExpFilter, rm, root, rm, -rf, (.*/strongswan.d|.*/ipsec/[0-9a-z-]+)
    rm_file: RegExpFilter, rm, root, rm, -f, .*/ipsec.secrets
    strongswan: CommandFilter, strongswan, root
    neutron_netns_wrapper: CommandFilter, neutron-vpn-netns-wrapper, root
    neutron_netns_wrapper_local: CommandFilter, /usr/local/bin/neutron-vpn-netns-wrapper, root
    chown: RegExpFilter, chown, root, chown, --from=.*, root.root, .*/(ipsec.secrets|ipsec/[0-9a-z-]+/log)

  • 数据库中会增加vpnservice，vpn_endpoint_groups，vpn_endpoints等与VPN服务相关的表:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    mysql> show tables;
    +-----------------------------------------+
    | Tables_in_neutron |
    +-----------------------------------------+
    | address_scopes |
    | agents |
    | alembic_version |
       ...........
    | vpn_endpoint_groups |
    | vpn_endpoints |
    | vpnservices |
    +-----------------------------------------+

使用软件包安装部署

手动安装好OpenStack环境后，按以下步骤为环境添加VPN功能

  1. 下载vpnaas软件包，以及实现ipsec的软件包，strongswan，openswan，libreswan等等，注意：目前CentOS不支持strongswan，使用libreswan即可。

    • Ubuntu下：

      1
      2
      apt-get install strongswan
      pip install neutron-vpnaas

    • CentOS下：

      1
      2
      yum install libreswan
      pip install neutron-vpnaas

  2. 修改配置文件

    • 修改控制节点上neutron.conf文件，在service_plugins增加对VPNaaS的支持，修改如下所示：

      1
      2
      3
      ***/etc/neutron/neutron.conf***
      [DEFAULT]
      service_plugins = ......,vpnaas

    • 在/etc/neutron目录下添加neutron_vpnaas.conf文件，添加[service_providers]

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      ***/etc/neutron/neutron_vpnaas.conf***
      [service_providers]
      service_provider = VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
      #
      # if you want use libreswan，configuration example is as follows
      #service_provider = VPN:libreswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
      # From neutron.vpnaas
      #
      # Defines providers for advanced services using the format:
      # <service_type>:<name>:<driver>[:default] (multi valued)
      #service_provider =

    • 在/etc/neutron/l3_agent.ini文件中额外添加以下内容：

      1
      2
      3
      4
      5
      6
      7
      8
      9
      ***/etc/neutron/l3_agnet.ini***
      [AGENT]
      extensions = vpnaas
      root_helper_daemon = sudo /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
      root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
      [vpnagent]
      vpn_device_driver = neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver
      # when use libreswan
      # vpn_device_driver = neutron_vpnaas.services.vpn.device_drivers.libreswan_ipsec.LibreSwanDriver

    • /etc/neutron/rootwrap.conf内容如下:

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      ***/etc/neutron/rootwrap.conf***
      # Configuration for neutron-rootwrap
      # This file should be owned by (and only-writeable by) the root user
      [DEFAULT]
      # List of directories to load filter definitions from (separated by ',').
      # These directories MUST all be only writeable by root !
      filters_path=/etc/neutron/rootwrap.d
      # List of directories to search executables in, in case filters do not
      # explicitely specify a full path (separated by ',')
      # If not specified, defaults to system PATH environment variable.
      # These directories MUST all be only writeable by root !
      exec_dirs=/sbin,/usr/sbin,/bin,/usr/bin,/usr/local/bin,/usr/local/sbin,/usr/local/bin
      # Enable logging to syslog
      # Default value is False
      use_syslog=False
      # Which syslog facility to use.
      # Valid values include auth, authpriv, syslog, local0, local1...
      # Default value is 'syslog'
      syslog_log_facility=syslog
      # Which messages to log.
      # INFO means log all usage
      # ERROR means only log unsuccessful attempts
      syslog_log_level=ERROR
      [xenapi]
      # XenAPI configuration is only required by the L2 agent if it is to
      # target a XenServer/XCP compute host's dom0.
      xenapi_connection_url=<None>
      xenapi_connection_username=root
      xenapi_connection_password=<None>

    • 在/etc/neutron/rootwrap.d目录下添加权限管理文件vpnaas.filters，默认安装时该目录下没有该文件，所以必须手动将该文件放入，不然运行时会报没有操作权限的错误。

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      ***vpnaas.filters***
      # neutron-rootwrap command filters for nodes on which neutron is
      # expected to control network
      # for libreswan, just change "strongswan" to "libreswan"
      # This file should be owned by (and only-writable by) the root user
      # format seems to be
      # cmd-name: filter-name, raw-command, user, args
      [Filters]
      cp: RegExpFilter, cp, root, cp, -a, .*, .*/strongswan.d
      ip: IpFilter, ip, root
      ip_exec: IpNetnsExecFilter, ip, root
      ipsec: CommandFilter, ipsec, root
      rm: RegExpFilter, rm, root, rm, -rf, (.*/strongswan.d|.*/ipsec/[0-9a-z-]+)
      rm_file: RegExpFilter, rm, root, rm, -f, .*/ipsec.secrets
      strongswan: CommandFilter, strongswan, root
      neutron_netns_wrapper: CommandFilter, neutron-vpn-netns-wrapper, root
      neutron_netns_wrapper_local: CommandFilter, /usr/local/bin/neutron-vpn-netns-wrapper, root
      chown: RegExpFilter, chown, root, chown, --from=.*, root.root, .*/(ipsec.secrets|ipsec/[0-9a-z-]+/log)

  3. 创建和更新数据库中所需要的VPN服务相关的表，更换ipsec driver后，也需要执行以下命令来更新数据库。

    1
    # neutron-db-manage --subproject neutron-vpnaas upgrade head

  4. 重启控制节点上的neutron-server和网络节点上的neutron-l3-agent，各项服务必须正常运行

    1
    # sudo systemctl restart "devstack@q-*"

查看服务状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
stack@vpn-test2:~$ sudo systemctl list-units | grep devstack
devstack@c-api.service                                                      loaded active running   Devstack devstack@c-api.service
devstack@c-sch.service                                                      loaded active running   Devstack devstack@c-sch.service
devstack@c-vol.service                                                      loaded active running   Devstack devstack@c-vol.service
devstack@dstat.service                                                      loaded active running   Devstack devstack@dstat.service
devstack@etcd.service                                                       loaded active running   Devstack devstack@etcd.service
devstack@g-api.service                                                      loaded active running   Devstack devstack@g-api.service
devstack@g-reg.service                                                      loaded active running   Devstack devstack@g-reg.service
devstack@keystone.service                                                   loaded active running   Devstack devstack@keystone.service
devstack@n-api-meta.service                                                 loaded active running   Devstack devstack@n-api-meta.service
devstack@n-api.service                                                      loaded active running   Devstack devstack@n-api.service
devstack@n-cauth.service                                                    loaded active running   Devstack devstack@n-cauth.service
devstack@n-cond-cell1.service                                               loaded active running   Devstack devstack@n-cond-cell1.service
devstack@n-cpu.service                                                      loaded active running   Devstack devstack@n-cpu.service
devstack@n-novnc-cell1.service                                              loaded active running   Devstack devstack@n-novnc-cell1.service
devstack@n-sch.service                                                      loaded active running   Devstack devstack@n-sch.service
devstack@n-super-cond.service                                               loaded active running   Devstack devstack@n-super-cond.service
devstack@placement-api.service                                              loaded active running   Devstack devstack@placement-api.service
devstack@q-agt.service                                                      loaded active running   Devstack devstack@q-agt.service
devstack@q-dhcp.service                                                     loaded active running   Devstack devstack@q-dhcp.service
devstack@q-l3.service                                                       loaded active running   Devstack devstack@q-l3.service
devstack@q-meta.service                                                     loaded active running   Devstack devstack@q-meta.service
devstack@q-svc.service                                                      loaded active running   Devstack devstack@q-svc.service
system-devstack.slice                                                       loaded active active    system-devstack.slice

注意：当采用默认devstack快速部署后，想更换ipsec_driver(如;strongswan→libreswan），同样可以按照以上步骤及配置进行更换，也必须执行neutron-db-manage来更新数据库内容，否则重启neutron服务时会出错。

测试

使用两个Devstack节点进行测试

使用两个由公共“public”网络连接的DevStack节点来测试VPNaaS。 第二个节点可以使用与第一个节点相同的公共网络进行设置，但它将使用不同的网关IP（即路由器IP）。 在本次测试中，我们假设有两个DevStack节点（East和West），每个节点都在物理机上运行（如果需要，可以对多个虚拟机执行相同的操作）。 （注意：也可以在一个节点上使用两个虚拟路由器创建类似拓扑）

示例拓扑：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
(10.1.0.0/24 - DevStack East)
              |
              |  10.1.0.1
     [Neutron Router]
              |  172.24.4.226
              |
              |  172.24.4.225
     [Internet GW]
              |
              |
     [Internet GW]
              | 172.24.4.232
              |
              | 172.24.4.233
     [Neutron Router]
              |  10.2.0.1
              |
     (10.2.0.0/24 DevStack West)

专用物理端口可用于通过物理交换机互连的“public”网络连接（例如：eth2）。 需要将端口添加到每个DevStack节点上的OVS桥（例如：sudo ovs-vsctl add-port br-ex eth2）。

DevStack配置

对于East节点，可以将以下字段添加到local.conf文件，这将自动为你配置一个10.1.0.0/24的私有网络和172.24.4.0/24的公共网络。

1
2
3
4
5
6
PUBLIC_SUBNET_NAME=yoursubnet
PRIVATE_SUBNET_NAME=mysubnet
FIXED_RANGE=10.1.0.0/24
NETWORK_GATEWAY=10.1.0.1
PUBLIC_NETWORK_GATEWAY=172.24.4.225
Q_FLOATING_ALLOCATION_POOL=start=172.24.4.226,end=172.24.4.231

对于West节点，添加以下内容，以使用不同的本地网络和公共的网关IP。

1
2
3
4
5
6
PUBLIC_SUBNET_NAME=yoursubnet
PRIVATE_SUBNET_NAME=mysubnet
FIXED_RANGE=10.2.0.0/24
NETWORK_GATEWAY=10.2.0.1
PUBLIC_NETWORK_GATEWAY=172.24.4.232
Q_FLOATING_ALLOCATION_POOL=start=172.24.4.233,end=172.24.4.238

VPNaaS配置

使用在East和West节点上运行的DevStack并确认连接（即确保可以到另一个节点上路由器/ GW 能够ping通），然后执行以下VPNaaS CLI命令。

  • East节点
    1
    2
    3
    neutron vpn-ikepolicy-create ikepolicy1
    neutron vpn-ipsecpolicy-create ipsecpolicy1
    neutron vpn-service-create --name myvpn --description "My vpn service" router1 mysubnet
1
neutron ipsec-site-connection-create --name vpnconnection1 --vpnservice-id myvpn --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address 172.24.4.233 --peer-id 172.24.4.233 --peer-cidr 10.2.0.0/24 --psk secret
  • West节点
    1
    2
    3
    neutron vpn-ikepolicy-create ikepolicy1
    neutron vpn-ipsecpolicy-create ipsecpolicy1
    neutron vpn-service-create --name myvpn --description "My vpn service" router1 mysubnet
1
neutron ipsec-site-connection-create --name vpnconnection1 --vpnservice-id myvpn --ikepolicy-id ikepolicy1 --ipsecpolicy-id ipsecpolicy1 --peer-address 172.24.4.226 --peer-id 172.24.4.226 --peer-cidr 10.1.0.0/24 --psk secret

注：

  • peer-address 和peer-id 对写成对方router的对外网关ip地址
  • peer-cidr写成对方路由保护的内网，即想要建立ipsec-site-connection连接的子网
  • psk可以自定义，只要双方一致即可
验证

可以在每个节点上启动VM，然后从VM ping到远端路由器的公共IP。 使用运行在其中一个节点上的tcpdump，可以看到加密数据包（ESP）。

使用单个Devstack节点进行测试

这里的想法是使用DevStack部署一个OpenStack云，两个路由器（一个自动创建），两个私有网络（自动创建一个）10.1.0.0 / 24和10.2.0.0/24，每个私有网络中有一个虚拟机， 并使用公共网络（172.24.4.0/24）在两个私有网络之间建立VPN连接。

本次示例，local.conf文件内容如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
[[local|localrc]]
GIT_BASE=https://github.com
DEST=/opt/stack

disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_plugin neutron-vpnaas https://git.openstack.org/openstack/neutron-vpnaas

FIXED_RANGE=10.1.0.0/24
FIXED_NETWORK_SIZE=256
NETWORK_GATEWAY=10.1.0.1
PRIVATE_SUBNET_NAME=privateA

PUBLIC_SUBNET_NAME=public-subnet
FLOATING_RANGE=172.24.4.0/24
PUBLIC_NETWORK_GATEWAY=172.24.4.10
Q_FLOATING_ALLOCATION_POOL="start=172.24.4.11,end=172.24.4.29"

LIBVIRT_TYPE=qemu

IMAGE_URLS="http://cloud-images.ubuntu.com/releases/14.04.1/release/ubuntu-14.04-server-cloudimg-amd64.tar.gz,http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-uec.tar.gz"

SCREEN_LOGDIR=/opt/stack/screen-logs
SYSLOG=True
LOGFILE=~/devstack/stack.sh.log

ADMIN_PASSWORD=password
MYSQL_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=tokentoken

Q_USE_DEBUG_COMMAND=True

# RECLONE=No
RECLONE=yes
OFFLINE=False

执行 ./stack.sh，确保执行成功。

环境初始化

stack.sh完成后，将拥有一个专用网络（10.1.0.0/24）和一个路由器（router1）。 为了准备建立VPN连接，需要创建第二个网络，子网和路由器，并在每个专用网络中启动一个VM。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Create second net, subnet, router
  source ~/devstack/openrc admin demo
  neutron net-create privateB
  neutron subnet-create --name subB privateB 10.2.0.0/24 --gateway 10.2.0.1
  neutron router-create router2
  neutron router-interface-add router2 subB
  neutron router-gateway-set router2 public

  # Start up a VM in the privateA subnet.
  PRIVATE_NET=`neutron net-list | grep 'private ' | cut -f 2 -d' '`
  nova boot --flavor 1 --image cirros-0.3.3-x86_64-uec --nic net-id=$PRIVATE_NET peter

  # Start up a VM in the privateB subnet
  PRIVATE_NETB=`neutron net-list | grep privateB | cut -f 2 -d' '`
  nova boot --flavor 1 --image cirros-0.3.3-x86_64-uec --nic net-id=$PRIVATE_NETB paul

IPSec站点到站点连接创建

以下命令将创建IPSec连接

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Create VPN connections
   neutron vpn-ikepolicy-create ikepolicy
   neutron vpn-ipsecpolicy-create ipsecpolicy
   neutron vpn-service-create --name myvpn --description "My vpn service" router1 privateA
    
   neutron ipsec-site-connection-create --name vpnconnection1 --vpnservice-id myvpn \
   --ikepolicy-id ikepolicy --ipsecpolicy-id ipsecpolicy --peer-address 172.24.4.13 \
   --peer-id 172.24.4.13 --peer-cidr 10.2.0.0/24 --psk secret
    
   neutron vpn-service-create --name myvpnB --description "My vpn serviceB" router2 subB
    
   neutron ipsec-site-connection-create --name vpnconnection2 --vpnservice-id myvpnB \
   --ikepolicy-id ikepolicy --ipsecpolicy-id ipsecpolicy --peer-address 172.24.4.11 \
   --peer-id 172.24.4.11 --peer-cidr 10.1.0.0/24 --psk secret

验证

此时（一旦连接变为Active状态 - 可能需要30秒左右），应该能够从privateA网络中的虚拟机ping到privateB网络中的虚拟机。 如果您使用来自其中一个路由器名称空间的qg-＃##接口进行tcpdump，您将看到加密数据包。 如果删除其中一个连接，您将看到ping失败（如果所有连接都正确）。

*! MLS 和 Endpoint group

MLS（Multi local subnets）即多个本地子网。在Mitaka版本中，除了当前的多个对等CIDR之外，IPSec站点间连接还将支持多个本地子网。 创建VPN服务时，多个本地子网功能由未指定本地子网触发。 通过在VPN服务创建中提供子网，可以在单个本地子网中保持向后兼容性。

为了支持多个本地子网，在Liberty版本已经提供了一种叫Endpoint groups的新功能，称为“端点组”。 每个端点组将定义一个或多个特定类型的端点，并可用于为IPSec连接指定本地和对等端点。 端点组将“连接的内容”与VPN服务的“如何连接”分开，并且将来可用于不同flavor的VPN。 一个例子：

1
2
3
4
# Create VPN connections
  neutron vpn-ikepolicy-create ikepolicy
  neutron vpn-ipsecpolicy-create ipsecpolicy
  neutron vpn-service-create --name myvpnC --description "My vpn service" router1

要准备IPSec站点到站点，可以为本地子网创建端点组，并为对等CIDR创建端点组，如下所示：

1
2
neutron vpn-endpoint-group-create --name my-locals --type subnet --value privateA --value privateA2
neutron vpn-endpoint-group-create --name my-peers --type cidr --value 10.2.0.0/24 --value 20.2.0.0/24

其中privateA和privateA2是两个本地（专用）子网，10.2.0.0/24和20.2.0.0/24是两个CIDR，代表连接将使用的对等（专用）子网。 然后，在创建IPSec站点到站点连接时，将指定这些端点组ID，而不是peer-cidrs属性：

1
2
3
neutron ipsec-site-connection-create --name vpnconnection3 --vpnservice-id myvpnC \
   --ikepolicy-id ikepolicy --ipsecpolicy-id ipsecpolicy --peer-address 172.24.4.11 \
   --peer-id 172.24.4.11 --local-ep-group my-locals --peer-ep-group my-peers --psk secret

VPN test for CentOS

测试环境

云主机环境：CentOS7.4
Devstack版本：master
Openstack版本：master(Rocky，Queens)
ipsec_driver类型：libreswan

1
2
3
4
5
6
7
8
9
10
11
12
[stack@vpnaas devstack]$ neutron service-provider-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------+-------------+---------+
| service_type  | name        | default |
+---------------+-------------+---------+
| L3_ROUTER_NAT | single_node | False   |
| L3_ROUTER_NAT | ha          | False   |
| L3_ROUTER_NAT | dvrha       | False   |
| VPN           | libreswan   | True    |
| L3_ROUTER_NAT | dvr         | False   |
| VPN           | libreswan   | True    |
+---------------+-------------+---------+

环境配置

Devstack目录下local.conf的配置内容如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[[local|localrc]]
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD

enable_plugin neutron-vpnaas https://git.openstack.org/openstack/neutron-vpnaas
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service tempest

IPSEC_PACKAGE="libreswan"

LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2

SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
SWIFT_REPLICAS=1
SWIFT_DATA_DIR=$DEST/data

注意：本次测试第一次是使用默认IPSEC_PACKAGE=”strongwan”，最后发现创建vpn-service和ipsec-site-connection状态都是PENDING-CREATE，于是手动安装libreswan，将配置strongswan更改为librewan,具体更改配置步骤参照另一篇文档中使用软件包安装部署，当然你也可以跑devstack时候，在local.conf里面直接指定IPSEC_PACKAGE=”libreswan”，参照上面。

测试方案

两个不同的vm，各自创建VPN服务，只需要各自VPC对应的VPN-Service里的出口网关IP地址能通即可；可vpn-service里出口IP默认是Router Gateway IP，而只需要将两个vm的external网络设置为同一网段，这样就能达到目标。

具体步骤:

  1. 为两个vm创建对应的network和所在子网
  2. 创建router，并设置网关在同一段external网络，即”public”
  3. 各自network下创建vm，并创建ikepolicy，ipsecpolicy，和vpn-service
  4. 创建ipsec-site-connection，测试两个vm的连通性

网络拓扑:
CentOS测试拓扑

过程记录

  1. 创建vpn-network-1，和vpn-network-1，指定子网的CIDR为2.3.4.0/24

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    neutron net-create vpn-network-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new network:
    +---------------------------+--------------------------------------+
    | Field                     | Value                                |
    +---------------------------+--------------------------------------+
    | admin_state_up            | True                                 |
    | availability_zone_hints   |                                      |
    | availability_zones        |                                      |
    | created_at                | 2018-07-05T10:14:12Z                 |
    | description               |                                      |
    | id                        | e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e |
    | ipv4_address_scope        |                                      |
    | ipv6_address_scope        |                                      |
    | is_default                | False                                |
    | mtu                       | 1450                                 |
    | name                      | vpn-network-1                        |
    | port_security_enabled     | True                                 |
    | project_id                | 60ad4059478544e6a4d3d241fdbefa69     |
    | provider:network_type     | vxlan                                |
    | provider:physical_network |                                      |
    | provider:segmentation_id  | 15                                   |
    | revision_number           | 2                                    |
    | router:external           | False                                |
    | shared                    | False                                |
    | status                    | ACTIVE                               |
    | subnets                   |                                      |
    | tags                      |                                      |
    | tenant_id                 | 60ad4059478544e6a4d3d241fdbefa69     |
    | updated_at                | 2018-07-05T10:14:13Z                 |
    +---------------------------+--------------------------------------+
    neutron subnet-create --name vpn-subnet-1 vpn-network-1 2.3.4.0/24
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new subnet:
    +-------------------+------------------------------------------+
    | Field             | Value                                    |
    +-------------------+------------------------------------------+
    | allocation_pools  | {"start": "2.3.4.2", "end": "2.3.4.254"} |
    | cidr              | 2.3.4.0/24                               |
    | created_at        | 2018-07-05T10:14:51Z                     |
    | description       |                                          |
    | dns_nameservers   |                                          |
    | enable_dhcp       | True                                     |
    | gateway_ip        | 2.3.4.1                                  |
    | host_routes       |                                          |
    | id                | 79645d5f-59db-47b1-a996-b30d2e513140     |
    | ip_version        | 4                                        |
    | ipv6_address_mode |                                          |
    | ipv6_ra_mode      |                                          |
    | name              | vpn-subnet-1                             |
    | network_id        | e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e     |
    | project_id        | 60ad4059478544e6a4d3d241fdbefa69         |
    | revision_number   | 0                                        |
    | service_types     |                                          |
    | subnetpool_id     |                                          |
    | tags              |                                          |
    | tenant_id         | 60ad4059478544e6a4d3d241fdbefa69         |
    | updated_at        | 2018-07-05T10:14:51Z                     |
    +-------------------+------------------------------------------+

  2. 创建路由器vpn-router-1，为其添加子网vpn-subnet-1，设置其网关为”public”网络

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    [stack@vpnaas devstack]$ openstack network list
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | ID                                   | Name          | Subnets                                                                    |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | 0e34b977-8c21-4027-9029-1ff0d53c443c | private       | 00015b05-4e14-4535-864f-04a2991761aa, f30ddb58-d365-4154-88f7-baf0259a6363 |
    | cfc1f313-1a12-46d3-b9f0-8a0d0fc83994 | public        | eca99e88-279e-4a4d-9dcd-d2011bbcc58b, eefc51de-718a-4539-a167-e95cab039ae4 |
    | e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e | vpn-network-1 | 79645d5f-59db-47b1-a996-b30d2e513140                                       |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    [stack@vpnaas devstack]$ neutron router-create vpn-router-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new router:
    +-------------------------+--------------------------------------+
    | Field                   | Value                                |
    +-------------------------+--------------------------------------+
    | admin_state_up          | True                                 |
    | availability_zone_hints |                                      |
    | availability_zones      |                                      |
    | created_at              | 2018-07-05T10:16:23Z                 |
    | description             |                                      |
    | distributed             | False                                |
    | external_gateway_info   |                                      |
    | flavor_id               |                                      |
    | ha                      | False                                |
    | id                      | da70042a-aa92-4b81-a00d-630b8da0f3e2 |
    | name                    | vpn-router-1                         |
    | project_id              | 60ad4059478544e6a4d3d241fdbefa69     |
    | revision_number         | 1                                    |
    | routes                  |                                      |
    | status                  | ACTIVE                               |
    | tags                    |                                      |
    | tenant_id               | 60ad4059478544e6a4d3d241fdbefa69     |
    | updated_at              | 2018-07-05T10:16:23Z                 |
    +-------------------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron router-interface-add vpn-router-1 vpn-subnet-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Added interface 2c1b084d-f618-4cae-9a35-83718976fe40 to router vpn-router-1.
    [stack@vpnaas devstack]$ neutron router-gateway-set vpn-router-1 public
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Set gateway for router vpn-router-1

  3. 创建vm1,类似创建vpn-network-2及vm2

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    [stack@vpnaas devstack]$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e vm1
    +--------------------------------------+-----------------------------------------------------------------+
    | Property                             | Value                                                           |
    +--------------------------------------+-----------------------------------------------------------------+
    | OS-DCF:diskConfig                    | MANUAL                                                          |
    | OS-EXT-AZ:availability_zone          |                                                                 |
    | OS-EXT-SRV-ATTR:host                 | -                                                               |
    | OS-EXT-SRV-ATTR:hostname             | vm1                                                             |
    | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                               |
    | OS-EXT-SRV-ATTR:instance_name        |                                                                 |
    | OS-EXT-SRV-ATTR:kernel_id            |                                                                 |
    | OS-EXT-SRV-ATTR:launch_index         | 0                                                               |
    | OS-EXT-SRV-ATTR:ramdisk_id           |                                                                 |
    | OS-EXT-SRV-ATTR:reservation_id       | r-5ufc0lzr                                                      |
    | OS-EXT-SRV-ATTR:root_device_name     | -                                                               |
    | OS-EXT-SRV-ATTR:user_data            | -                                                               |
    | OS-EXT-STS:power_state               | 0                                                               |
    | OS-EXT-STS:task_state                | scheduling                                                      |
    | OS-EXT-STS:vm_state                  | building                                                        |
    | OS-SRV-USG:launched_at               | -                                                               |
    | OS-SRV-USG:terminated_at             | -                                                               |
    | accessIPv4                           |                                                                 |
    | accessIPv6                           |                                                                 |
    | adminPass                            | zWsm6KWAoBhc                                                    |
    | config_drive                         |                                                                 |
    | created                              | 2018-07-05T10:20:02Z                                            |
    | description                          | -                                                               |
    | flavor:disk                          | 1                                                               |
    | flavor:ephemeral                     | 0                                                               |
    | flavor:extra_specs                   | {}                                                              |
    | flavor:original_name                 | m1.tiny                                                         |
    | flavor:ram                           | 512                                                             |
    | flavor:swap                          | 0                                                               |
    | flavor:vcpus                         | 1                                                               |
    | hostId                               |                                                                 |
    | host_status                          |                                                                 |
    | id                                   | 5d88d5c3-ac9a-4330-bf96-6c953c215bf0                            |
    | image                                | cirros-0.3.5-x86_64-disk (5cd7bf05-9701-4e40-b6df-d2105457030c) |
    | key_name                             | -                                                               |
    | locked                               | False                                                           |
    | metadata                             | {}                                                              |
    | name                                 | vm1                                                             |
    | os-extended-volumes:volumes_attached | []                                                              |
    | progress                             | 0                                                               |
    | security_groups                      | default                                                         |
    | status                               | BUILD                                                           |
    | tags                                 | []                                                              |
    | tenant_id                            | 60ad4059478544e6a4d3d241fdbefa69                                |
    | updated                              | 2018-07-05T10:20:02Z                                            |
    | user_id                              | 08e7a814d45a4328a330c0bc4a41fa5d                                |
    +--------------------------------------+-----------------------------------------------------------------+
    [stack@vpnaas devstack]$ neutron net-create vpn-network-2
    [stack@vpnaas devstack]$ neutron subnet-create --name vpn-subnet-2 vpn-network-2 4.3.2.0/24
    [stack@vpnaas devstack]$ neutron router-create vpn-router-2
    [stack@vpnaas devstack]$ neutron router-interface-add vpn-router-2 vpn-subnet-2
    [stack@vpnaas devstack]$ neutron router-gateway-set vpn-router-2 public
    [stack@vpnaas devstack]$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=2a7d7d49-475f-4868-9d1f-c307bd05c8fe vm2
    [stack@vpnaas devstack]$ nova list
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    | ID                                   | Name | Status | Task State | Power State | Networks               |
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    | 5d88d5c3-ac9a-4330-bf96-6c953c215bf0 | vm1  | ACTIVE | -          | Running     | vpn-network-1=2.3.4.11 |
    | 7971cf68-1b0b-461b-a72c-0d0243928543 | vm2  | ACTIVE | -          | Running     | vpn-network-2=4.3.2.6  |
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    [stack@vpnaas devstack]$ openstack network list
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | ID                                   | Name          | Subnets                                                                    |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | 0e34b977-8c21-4027-9029-1ff0d53c443c | private       | 00015b05-4e14-4535-864f-04a2991761aa, f30ddb58-d365-4154-88f7-baf0259a6363 |
    | 2a7d7d49-475f-4868-9d1f-c307bd05c8fe | vpn-network-2 | 5035c74b-5d70-4d2b-b7cb-b1014a7c2e35                                       |
    | cfc1f313-1a12-46d3-b9f0-8a0d0fc83994 | public        | eca99e88-279e-4a4d-9dcd-d2011bbcc58b, eefc51de-718a-4539-a167-e95cab039ae4 |
    | e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e | vpn-network-1 | 79645d5f-59db-47b1-a996-b30d2e513140                                       |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    [stack@vpnaas devstack]$ openstack router list
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | ID                                   | Name         | Status | State | Distributed | HA    | Project                          |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | 6aaa82da-d904-468c-b643-220d96cf3841 | router1      | ACTIVE | UP    | False       | False | 3cf6626d7c3a4fc2b43deda4493a38ab |
    | 9ae51377-14b2-40c0-917e-c93a32ed6b76 | vpn-router-2 | ACTIVE | UP    | False       | False | 60ad4059478544e6a4d3d241fdbefa69 |
    | da70042a-aa92-4b81-a00d-630b8da0f3e2 | vpn-router-1 | ACTIVE | UP    | False       | False | 60ad4059478544e6a4d3d241fdbefa69 |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    [stack@vpnaas devstack]$ openstack subnet list
    +--------------------------------------+---------------------+--------------------------------------+---------------------+
    | ID                                   | Name                | Network                              | Subnet              |
    +--------------------------------------+---------------------+--------------------------------------+---------------------+
    | 00015b05-4e14-4535-864f-04a2991761aa | ipv6-private-subnet | 0e34b977-8c21-4027-9029-1ff0d53c443c | fd19:a538:3b0f::/64 |
    | 5035c74b-5d70-4d2b-b7cb-b1014a7c2e35 | vpn-subnet-2        | 2a7d7d49-475f-4868-9d1f-c307bd05c8fe | 4.3.2.0/24          |
    | 79645d5f-59db-47b1-a996-b30d2e513140 | vpn-subnet-1        | e6edeb26-b2b7-4c83-ad39-4fc2fe671e6e | 2.3.4.0/24          |
    | eca99e88-279e-4a4d-9dcd-d2011bbcc58b | ipv6-public-subnet  | cfc1f313-1a12-46d3-b9f0-8a0d0fc83994 | 2001:db8::/64       |
    | eefc51de-718a-4539-a167-e95cab039ae4 | public-subnet       | cfc1f313-1a12-46d3-b9f0-8a0d0fc83994 | 172.24.4.0/24       |
    | f30ddb58-d365-4154-88f7-baf0259a6363 | privateA            | 0e34b977-8c21-4027-9029-1ff0d53c443c | 10.0.0.0/26         |
    +--------------------------------------+---------------------+--------------------------------------+---------------------+

  4. 创建ikepolicy和ipsecpolicy和vpn-service

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    [stack@vpnaas devstack]$ neutron vpn-ikepolicy-create vpn-ikepolicy-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ikepolicy:
    +-------------------------+--------------------------------------+
    | Field                   | Value                                |
    +-------------------------+--------------------------------------+
    | auth_algorithm          | sha1                                 |
    | description             |                                      |
    | encryption_algorithm    | aes-128                              |
    | id                      | 1b6d0d3c-da17-4d61-961b-0b8585bfce21 |
    | ike_version             | v1                                   |
    | lifetime                | {"units": "seconds", "value": 3600}  |
    | name                    | vpn-ikepolicy-1                      |
    | pfs                     | group5                               |
    | phase1_negotiation_mode | main                                 |
    | project_id              | 60ad4059478544e6a4d3d241fdbefa69     |
    | tenant_id               | 60ad4059478544e6a4d3d241fdbefa69     |
    +-------------------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron vpn-ikepolicy-create vpn-ikepolicy-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ikepolicy:
    +-------------------------+--------------------------------------+
    | Field                   | Value                                |
    +-------------------------+--------------------------------------+
    | auth_algorithm          | sha1                                 |
    | description             |                                      |
    | encryption_algorithm    | aes-128                              |
    | id                      | 9932a063-1df6-4b99-968b-3320a45ff2e8 |
    | ike_version             | v1                                   |
    | lifetime                | {"units": "seconds", "value": 3600}  |
    | name                    | vpn-ikepolicy-2                      |
    | pfs                     | group5                               |
    | phase1_negotiation_mode | main                                 |
    | project_id              | 60ad4059478544e6a4d3d241fdbefa69     |
    | tenant_id               | 60ad4059478544e6a4d3d241fdbefa69     |
    +-------------------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron vpn-ipsecpolicy-create vpn-ipsecpolicy-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsecpolicy:
    +----------------------+--------------------------------------+
    | Field                | Value                                |
    +----------------------+--------------------------------------+
    | auth_algorithm       | sha1                                 |
    | description          |                                      |
    | encapsulation_mode   | tunnel                               |
    | encryption_algorithm | aes-128                              |
    | id                   | 73af30cb-e34b-4ca6-9dd8-1ea30b132284 |
    | lifetime             | {"units": "seconds", "value": 3600}  |
    | name                 | vpn-ipsecpolicy-1                    |
    | pfs                  | group5                               |
    | project_id           | 60ad4059478544e6a4d3d241fdbefa69     |
    | tenant_id            | 60ad4059478544e6a4d3d241fdbefa69     |
    | transform_protocol   | esp                                  |
    +----------------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron vpn-ipsecpolicy-create vpn-ipsecpolicy-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsecpolicy:
    +----------------------+--------------------------------------+
    | Field                | Value                                |
    +----------------------+--------------------------------------+
    | auth_algorithm       | sha1                                 |
    | description          |                                      |
    | encapsulation_mode   | tunnel                               |
    | encryption_algorithm | aes-128                              |
    | id                   | c40a7b02-2745-49c5-bfb4-435e4d98bd0a |
    | lifetime             | {"units": "seconds", "value": 3600}  |
    | name                 | vpn-ipsecpolicy-2                    |
    | pfs                  | group5                               |
    | project_id           | 60ad4059478544e6a4d3d241fdbefa69     |
    | tenant_id            | 60ad4059478544e6a4d3d241fdbefa69     |
    | transform_protocol   | esp                                  |
    +----------------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron vpn-service-create vpn-router-1 vpn-subnet-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new vpnservice:
    +----------------+--------------------------------------+
    | Field          | Value                                |
    +----------------+--------------------------------------+
    | admin_state_up | True                                 |
    | description    |                                      |
    | external_v4_ip | 172.24.4.14                          |
    | external_v6_ip | 2001:db8::b                          |
    | flavor_id      |                                      |
    | id             | f46567dc-6752-497e-a9bd-97bffcc40ba0 |
    | name           |                                      |
    | project_id     | 60ad4059478544e6a4d3d241fdbefa69     |
    | router_id      | da70042a-aa92-4b81-a00d-630b8da0f3e2 |
    | status         | PENDING_CREATE                       |
    | subnet_id      | 79645d5f-59db-47b1-a996-b30d2e513140 |
    | tenant_id      | 60ad4059478544e6a4d3d241fdbefa69     |
    +----------------+--------------------------------------+
    [stack@vpnaas devstack]$ neutron vpn-service-create vpn-router-2 vpn-subnet-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new vpnservice:
    +----------------+--------------------------------------+
    | Field          | Value                                |
    +----------------+--------------------------------------+
    | admin_state_up | True                                 |
    | description    |                                      |
    | external_v4_ip | 172.24.4.12                          |
    | external_v6_ip | 2001:db8::9                          |
    | flavor_id      |                                      |
    | id             | 056b7032-d26e-40a7-9e5a-0dd2cdd8e2b7 |
    | name           |                                      |
    | project_id     | 60ad4059478544e6a4d3d241fdbefa69     |
    | router_id      | 9ae51377-14b2-40c0-917e-c93a32ed6b76 |
    | status         | PENDING_CREATE                       |
    | subnet_id      | 5035c74b-5d70-4d2b-b7cb-b1014a7c2e35 |
    | tenant_id      | 60ad4059478544e6a4d3d241fdbefa69     |
    +----------------+--------------------------------------+

  5. 创建ipsec-site-connection

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    [stack@vpnaas devstack]$ neutron ipsec-site-connection-create --name connectionA --vpnservice-id f46567dc-6752-497e-a9bd-97bffcc40ba0 --ikepolicy-id 1b6d0d3c-da17-4d61-961b-0b8585bfce21 --ipsecpolicy-id 73af30cb-e34b-4ca6-9dd8-1ea30b132284 --peer-address 172.24.4.12 --peer-id 172.24.4.12 --peer-cidr 4.3.2.0/24 --psk lizenghui_key
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | ccab8f61-3998-4526-a0be-28dcda09871d               |
    | ikepolicy_id      | 1b6d0d3c-da17-4d61-961b-0b8585bfce21               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | 73af30cb-e34b-4ca6-9dd8-1ea30b132284               |
    | local_ep_group_id |                                                    |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | connectionA                                        |
    | peer_address      | 172.24.4.12                                        |
    | peer_cidrs        | 4.3.2.0/24                                         |
    | peer_ep_group_id  |                                                    |
    | peer_id           | 172.24.4.12                                        |
    | project_id        | 60ad4059478544e6a4d3d241fdbefa69                   |
    | psk               | lizenghui_key                                      |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | 60ad4059478544e6a4d3d241fdbefa69                   |
    | vpnservice_id     | f46567dc-6752-497e-a9bd-97bffcc40ba0               |
    +-------------------+----------------------------------------------------+
    neutron ipsec-site-connection-create --name connectionB --vpnservice-id 056b7032-d26e-40a7-9e5a-0dd2cdd8e2b7 --ikepolicy-id 9932a063-1df6-4b99-968b-3320a45ff2e8 --ipsecpolicy-id c40a7b02-2745-49c5-bfb4-435e4d98bd0a --peer-address 172.24.4.14 --peer-id 172.24.4.14 --peer-cidr 2.3.4.0/24 --psk lizenghui_key
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c               |
    | ikepolicy_id      | 9932a063-1df6-4b99-968b-3320a45ff2e8               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | c40a7b02-2745-49c5-bfb4-435e4d98bd0a               |
    | local_ep_group_id |                                                    |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | connectionB                                        |
    | peer_address      | 172.24.4.14                                        |
    | peer_cidrs        | 2.3.4.0/24                                         |
    | peer_ep_group_id  |                                                    |
    | peer_id           | 172.24.4.14                                        |
    | project_id        | 60ad4059478544e6a4d3d241fdbefa69                   |
    | psk               | lizenghui_key                                      |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | 60ad4059478544e6a4d3d241fdbefa69                   |
    | vpnservice_id     | 056b7032-d26e-40a7-9e5a-0dd2cdd8e2b7               |
    +-------------------+----------------------------------------------------+
    [stack@vpnaas devstack]$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name        | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c | connectionB | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.14  | psk       | ACTIVE |
    | ccab8f61-3998-4526-a0be-28dcda09871d | connectionA | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.12  | psk       | ACTIVE |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+

  6. 测试vm1和vm2的连通性，能够ping通，删掉site-connection后，无法ping通

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    [stack@vpnaas ~]$ openstack router list
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | ID                                   | Name         | Status | State | Distributed | HA    | Project                          |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | 6aaa82da-d904-468c-b643-220d96cf3841 | router1      | ACTIVE | UP    | False       | False | 3cf6626d7c3a4fc2b43deda4493a38ab |
    | 9ae51377-14b2-40c0-917e-c93a32ed6b76 | vpn-router-2 | ACTIVE | UP    | False       | False | 60ad4059478544e6a4d3d241fdbefa69 |
    | da70042a-aa92-4b81-a00d-630b8da0f3e2 | vpn-router-1 | ACTIVE | UP    | False       | False | 60ad4059478544e6a4d3d241fdbefa69 |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    [stack@vpnaas ~]$ nova list
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    | ID                                   | Name | Status | Task State | Power State | Networks               |
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    | 5d88d5c3-ac9a-4330-bf96-6c953c215bf0 | vm1  | ACTIVE | -          | Running     | vpn-network-1=2.3.4.11 |
    | 7971cf68-1b0b-461b-a72c-0d0243928543 | vm2  | ACTIVE | -          | Running     | vpn-network-2=4.3.2.6  |
    +--------------------------------------+------+--------+------------+-------------+------------------------+
    [stack@vpnaas ~]$ sudo ip netns exec qrouter-da70042a-aa92-4b81-a00d-630b8da0f3e2 ssh cirros@2.3.4.11
    The authenticity of host '2.3.4.11 (2.3.4.11)' can't be established.
    RSA key fingerprint is SHA256:quCmsfIs7PjSTxkwF+ZWTgVW6biLCUv2CovYtbMLkek.
    RSA key fingerprint is MD5:7c:c7:9e:54:0c:a2:1c:66:2a:18:37:d8:6d:d5:03:20.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '2.3.4.11' (RSA) to the list of known hosts.
    cirros@2.3.4.11's password:
    $ ping 4.3.2.6
    PING 4.3.2.6 (4.3.2.6): 56 data bytes
    64 bytes from 4.3.2.6: seq=0 ttl=62 time=11.136 ms
    64 bytes from 4.3.2.6: seq=1 ttl=62 time=2.416 ms
    64 bytes from 4.3.2.6: seq=2 ttl=62 time=1.563 ms
    64 bytes from 4.3.2.6: seq=3 ttl=62 time=1.402 ms
    64 bytes from 4.3.2.6: seq=4 ttl=62 time=1.457 ms
    64 bytes from 4.3.2.6: seq=5 ttl=62 time=1.745 ms
    64 bytes from 4.3.2.6: seq=6 ttl=62 time=1.462 ms
    64 bytes from 4.3.2.6: seq=7 ttl=62 time=1.726 ms
    ^C
    --- 4.3.2.6 ping statistics ---
    8 packets transmitted, 8 packets received, 0% packet loss
    round-trip min/avg/max = 1.402/2.863/11.136 ms
    $ exit
    Connection to 2.3.4.11 closed.
    [stack@vpnaas ~]$ sudo ip netns exec qrouter-9ae51377-14b2-40c0-917e-c93a32ed6b76 ssh cirros@4.3.2.6
    The authenticity of host '4.3.2.6 (4.3.2.6)' can't be established.
    RSA key fingerprint is SHA256:iJlXmthJVWjemtbNfCTqWjO7ePihSXrEpD1+ocCNCXI.
    RSA key fingerprint is MD5:d8:3b:2e:70:98:41:22:de:04:f3:02:69:08:50:99:06.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '4.3.2.6' (RSA) to the list of known hosts.
    cirros@4.3.2.6's password:
    $ ping 2.3.4.11
    PING 2.3.4.11 (2.3.4.11): 56 data bytes
    64 bytes from 2.3.4.11: seq=0 ttl=62 time=5.454 ms
    64 bytes from 2.3.4.11: seq=1 ttl=62 time=1.406 ms
    64 bytes from 2.3.4.11: seq=2 ttl=62 time=1.584 ms
    64 bytes from 2.3.4.11: seq=3 ttl=62 time=1.577 ms
    64 bytes from 2.3.4.11: seq=4 ttl=62 time=1.412 ms
    64 bytes from 2.3.4.11: seq=5 ttl=62 time=1.335 ms
    64 bytes from 2.3.4.11: seq=6 ttl=62 time=2.119 ms
    ^C
    --- 2.3.4.11 ping statistics ---
    7 packets transmitted, 7 packets received, 0% packet loss
    round-trip min/avg/max = 1.335/2.126/5.454 ms
    $ exit
    Connection to 4.3.2.6 closed.
    [stack@vpnaas ~]$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name        | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c | connectionB | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.14  | psk       | ACTIVE |
    | ccab8f61-3998-4526-a0be-28dcda09871d | connectionA | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.12  | psk       | ACTIVE |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    [stack@vpnaas ~]$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name        | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c | connectionB | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.14  | psk       | ACTIVE |
    | ccab8f61-3998-4526-a0be-28dcda09871d | connectionA | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.12  | psk       | ACTIVE |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    [stack@vpnaas ~]$ neutron ipsec-site-connection-delete b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Deleted ipsec_site_connection(s): b37e51cc-9cdd-45b0-98d8-c13a5fd2f41c
    [stack@vpnaas ~]$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name        | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    | ccab8f61-3998-4526-a0be-28dcda09871d | connectionA | 60ad4059478544e6a4d3d241fdbefa69 | 172.24.4.12  | psk       | ACTIVE |
    +--------------------------------------+-------------+----------------------------------+--------------+-----------+--------+
    [stack@vpnaas ~]$ sudo ip netns exec qrouter-da70042a-aa92-4b81-a00d-630b8da0f3e2 ssh cirros@2.3.4.11
    cirros@2.3.4.11's password:
    $ ping 4.3.2.6
    PING 4.3.2.6 (4.3.2.6): 56 data bytes
    ^C
    --- 4.3.2.6 ping statistics ---
    5 packets transmitted, 0 packets received, 100% packet loss
    $ exit
    Connection to 2.3.4.11 closed.
    [stack@vpnaas ~]$ sudo ip netns exec qrouter-9ae51377-14b2-40c0-917e-c93a32ed6b76 ssh cirros@4.3.2.6
    cirros@4.3.2.6's password:
    $ ping 2.3.4.11
    PING 2.3.4.11 (2.3.4.11): 56 data bytes
    ^C
    --- 2.3.4.11 ping statistics ---
    4 packets transmitted, 0 packets received, 100% packet loss
    $ exit
    Connection to 4.3.2.6 closed.
分析与总结
  • CentOS不支持ipsec_duiver类型为strongswan，推荐Fedora/RHEL/CentOS系统使用libreswan。
  • Endpoint group测试可参照另一篇文档 Endpoint group功能测试。
  • 如果已经在CentOS上装了strongswan，下载libreswan软件包，修改配置文件配置一下，将strongswan改成libreswan，然后更新neutron数据库，重启neutron相关服务即可。配置过程参照使用软件包安装部署。
  • 建立ipsec-site-connection连接时双方psk值必须一致匹配。

VPN test for Ubuntu

测试环境

云主机环境：Ubuntu16.04
Devstack版本：master
Openstack版本：master(Rocky，Queens)
IPsec_driver类型：strongswan

1
2
3
4
5
6
7
8
9
10
11
12
stack@vpn-test2:~$ neutron service-provider-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------+-------------+---------+
| service_type  | name        | default |
+---------------+-------------+---------+
| L3_ROUTER_NAT | single_node | False   |
| L3_ROUTER_NAT | ha          | False   |
| L3_ROUTER_NAT | dvrha       | False   |
| VPN           | strongswan  | True    |
| L3_ROUTER_NAT | dvr         | False   |
| VPN           | strongswan  | True    |
+---------------+-------------+---------+

环境配置

Devstack目录下local.conf的配置内容如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[[local|localrc]]

ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD

enable_plugin neutron-vpnaas https://git.openstack.org/openstack/neutron-vpnaas
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service tempest

IPSEC_PACKAGE="strongswan"

LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=2

SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
SWIFT_REPLICAS=1

测试方案

环境是ALL-IN-ONE单节点Openstack，两个不同的vm，各自创建VPN服务，只需要各自VPC对应的VPN-Service里的出口网关IP地址能通即可；可vpn-service里出口IP默认是Router Gateway IP，而只需要将两个vm的external网络设置为同一网段，这样就能达到目标。

具体步骤:

  1. 为两个vm创建对应的network和所在子网
  2. 创建router，并设置网关在同一段external网络，即”public”
  3. 各自network下创建vm，并创建ikepolicy，ipsecpolicy，和vpn-service
  4. 创建ipsec-site-connection，测试两个vm的连通性

网络拓扑:
ubuntu测试拓扑

过程记录

  1. 创建network和相应子网

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    stack@vpn-test2:~/devstack$ neutron net-create vpn-network-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new network:
    +---------------------------+--------------------------------------+
    | Field                     | Value                                |
    +---------------------------+--------------------------------------+
    | admin_state_up            | True                                 |
    | availability_zone_hints   |                                      |
    | availability_zones        |                                      |
    | created_at                | 2018-07-04T09:02:12Z                 |
    | description               |                                      |
    | id                        | 8f71bd0e-d26e-405b-8e48-68148fef8ed1 |
    | ipv4_address_scope        |                                      |
    | ipv6_address_scope        |                                      |
    | is_default                | False                                |
    | mtu                       | 1450                                 |
    | name                      | vpn-network-1                        |
    | port_security_enabled     | True                                 |
    | project_id                | da717c389f494c3e8e6d32f46142d610     |
    | provider:network_type     | vxlan                                |
    | provider:physical_network |                                      |
    | provider:segmentation_id  | 35                                   |
    | revision_number           | 1                                    |
    | router:external           | False                                |
    | shared                    | False                                |
    | status                    | ACTIVE                               |
    | subnets                   |                                      |
    | tags                      |                                      |
    | tenant_id                 | da717c389f494c3e8e6d32f46142d610     |
    | updated_at                | 2018-07-04T09:02:12Z                 |
    +---------------------------+--------------------------------------+
    stack@vpn-test2:~/devstack$ neutron subnet-create --name vpn-subnet-1 vpn-network-1 10.1.0.0/24 --gateway 10.1.0.1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new subnet:
    +-------------------+--------------------------------------------+
    | Field             | Value                                      |
    +-------------------+--------------------------------------------+
    | allocation_pools  | {"start": "10.1.0.2", "end": "10.1.0.254"} |
    | cidr              | 10.1.0.0/24                                |
    | created_at        | 2018-07-04T09:03:32Z                       |
    | description       |                                            |
    | dns_nameservers   |                                            |
    | enable_dhcp       | True                                       |
    | gateway_ip        | 10.1.0.1                                   |
    | host_routes       |                                            |
    | id                | 840b87e2-c78c-45cb-92e7-2272ef6e0821       |
    | ip_version        | 4                                          |
    | ipv6_address_mode |                                            |
    | ipv6_ra_mode      |                                            |
    | name              | vpn-subnet-1                               |
    | network_id        | 8f71bd0e-d26e-405b-8e48-68148fef8ed1       |
    | project_id        | da717c389f494c3e8e6d32f46142d610           |
    | revision_number   | 0                                          |
    | service_types     |                                            |
    | subnetpool_id     |                                            |
    | tags              |                                            |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610           |
    | updated_at        | 2018-07-04T09:03:32Z                       |
    +-------------------+--------------------------------------------+
    stack@vpn-test2:~/devstack$ neutron net-create vpn-network-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new network:
    +---------------------------+--------------------------------------+
    | Field                     | Value                                |
    +---------------------------+--------------------------------------+
    | admin_state_up            | True                                 |
    | availability_zone_hints   |                                      |
    | availability_zones        |                                      |
    | created_at                | 2018-07-04T09:03:55Z                 |
    | description               |                                      |
    | id                        | 137d45d3-820f-409c-aac7-12ef9be66cb2 |
    | ipv4_address_scope        |                                      |
    | ipv6_address_scope        |                                      |
    | is_default                | False                                |
    | mtu                       | 1450                                 |
    | name                      | vpn-network-2                        |
    | port_security_enabled     | True                                 |
    | project_id                | da717c389f494c3e8e6d32f46142d610     |
    | provider:network_type     | vxlan                                |
    | provider:physical_network |                                      |
    | provider:segmentation_id  | 87                                   |
    | revision_number           | 1                                    |
    | router:external           | False                                |
    | shared                    | False                                |
    | status                    | ACTIVE                               |
    | subnets                   |                                      |
    | tags                      |                                      |
    | tenant_id                 | da717c389f494c3e8e6d32f46142d610     |
    | updated_at                | 2018-07-04T09:03:55Z                 |
    +---------------------------+--------------------------------------+
    stack@vpn-test2:~/devstack$ neutron subnet-create --name vpn-subnet-2 vpn-network-2 10.2.0.0/24 --gateway 10.2.0.1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new subnet:
    +-------------------+--------------------------------------------+
    | Field             | Value                                      |
    +-------------------+--------------------------------------------+
    | allocation_pools  | {"start": "10.2.0.2", "end": "10.2.0.254"} |
    | cidr              | 10.2.0.0/24                                |
    | created_at        | 2018-07-04T09:04:39Z                       |
    | description       |                                            |
    | dns_nameservers   |                                            |
    | enable_dhcp       | True                                       |
    | gateway_ip        | 10.2.0.1                                   |
    | host_routes       |                                            |
    | id                | 4a0b78a7-3d04-4b0e-8a43-350a482c4564       |
    | ip_version        | 4                                          |
    | ipv6_address_mode |                                            |
    | ipv6_ra_mode      |                                            |
    | name              | vpn-subnet-2                               |
    | network_id        | 137d45d3-820f-409c-aac7-12ef9be66cb2       |
    | project_id        | da717c389f494c3e8e6d32f46142d610           |
    | revision_number   | 0                                          |
    | service_types     |                                            |
    | subnetpool_id     |                                            |
    | tags              |                                            |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610           |
    | updated_at        | 2018-07-04T09:04:39Z                       |
    +-------------------+--------------------------------------------+

  2. 创建router，设置网关

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    stack@vpn-test2:~/devstack$ neutron router-create vpn-router-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new router:
    +-------------------------+--------------------------------------+
    | Field                   | Value                                |
    +-------------------------+--------------------------------------+
    | admin_state_up          | True                                 |
    | availability_zone_hints |                                      |
    | availability_zones      |                                      |
    | created_at              | 2018-07-04T09:10:49Z                 |
    | description             |                                      |
    | distributed             | False                                |
    | external_gateway_info   |                                      |
    | flavor_id               |                                      |
    | ha                      | False                                |
    | id                      | f0906c65-e9b2-447c-ad25-73fecb7fd093 |
    | name                    | vpn-router-1                         |
    | project_id              | da717c389f494c3e8e6d32f46142d610     |
    | revision_number         | 0                                    |
    | routes                  |                                      |
    | status                  | ACTIVE                               |
    | tags                    |                                      |
    | tenant_id               | da717c389f494c3e8e6d32f46142d610     |
    | updated_at              | 2018-07-04T09:10:49Z                 |
    +-------------------------+--------------------------------------+
    stack@vpn-test2:~/devstack$ neutron router-interface-add vpn-router-1 vpn-subnet-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Added interface 0fbd2cfa-9e25-4ba2-a5ff-c5949d2bdd70 to router vpn-router-1.
    stack@vpn-test2:~/devstack$ neutron router-gateway-set vpn-router-1 public
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Set gateway for router vpn-router-1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
stack@vpn-test2:~/devstack$ neutron router-create vpn-router-2
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Created a new router:
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| admin_state_up          | True                                 |
| availability_zone_hints |                                      |
| availability_zones      |                                      |
| created_at              | 2018-07-04T09:13:56Z                 |
| description             |                                      |
| distributed             | False                                |
| external_gateway_info   |                                      |
| flavor_id               |                                      |
| ha                      | False                                |
| id                      | faf5fb10-8662-4862-bcb4-b3cee3c603be |
| name                    | vpn-router-2                         |
| project_id              | da717c389f494c3e8e6d32f46142d610     |
| revision_number         | 0                                    |
| routes                  |                                      |
| status                  | ACTIVE                               |
| tags                    |                                      |
| tenant_id               | da717c389f494c3e8e6d32f46142d610     |
| updated_at              | 2018-07-04T09:13:56Z                 |
+-------------------------+--------------------------------------+
stack@vpn-test2:~/devstack$ neutron router-interface-add vpn-router-2 vpn-subnet-2
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Added interface b0d7d178-00d1-44b2-87f9-322a1f5d9a6a to router vpn-router-2.
stack@vpn-test2:~/devstack$ neutron router-gateway-set vpn-router-2 public
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Set gateway for router vpn-router-2
stack@vpn-test2:~/devstack$

  1. 创建2个虚拟机实例

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    stack@vpn-test2:~/devstack$ openstack network list
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | ID                                   | Name          | Subnets                                                                    |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | 137d45d3-820f-409c-aac7-12ef9be66cb2 | vpn-network-2 | 4a0b78a7-3d04-4b0e-8a43-350a482c4564                                       |
    | 4df92675-1843-4b4b-9436-4932856ed8fe | public        | 4d801dd9-da36-4832-ad10-0be3229fea9a, 4fafa6f9-2f43-475d-95b0-5392dc3164fb |
    | 8f71bd0e-d26e-405b-8e48-68148fef8ed1 | vpn-network-1 | 840b87e2-c78c-45cb-92e7-2272ef6e0821                                       |
    | e95e8eac-c523-4066-a6fd-9ba03f7a9b20 | private       | 20516105-08ab-407a-8c8f-a16f3d2f60c0, bab7c9d8-2065-4dfc-9ff7-1973ebb4b604 |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    stack@vpn-test2:~/devstack$ openstack image list
    +--------------------------------------+--------------------------+--------+
    | ID                                   | Name                     | Status |
    +--------------------------------------+--------------------------+--------+
    | 695cf39a-6e94-46ed-9332-158da52361fc | cirros-0.3.5-x86_64-disk | active |
    +--------------------------------------+--------------------------+--------+
    stack@vpn-test2:~/devstack$ nova boot --flavor 1 --image 695cf39a-6e94-46ed-9332-158da52361fc --nic net-id=8f71bd0e-d26e-405b-8e48-68148fef8ed1 vpn-vm-1
    +--------------------------------------+-----------------------------------------------------------------+
    | Property                             | Value                                                           |
    +--------------------------------------+-----------------------------------------------------------------+
    | OS-DCF:diskConfig                    | MANUAL                                                          |
    | OS-EXT-AZ:availability_zone          |                                                                 |
    | OS-EXT-SRV-ATTR:host                 | -                                                               |
    | OS-EXT-SRV-ATTR:hostname             | vpn-vm-1                                                        |
    | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                               |
    | OS-EXT-SRV-ATTR:instance_name        |                                                                 |
    | OS-EXT-SRV-ATTR:kernel_id            |                                                                 |
    | OS-EXT-SRV-ATTR:launch_index         | 0                                                               |
    | OS-EXT-SRV-ATTR:ramdisk_id           |                                                                 |
    | OS-EXT-SRV-ATTR:reservation_id       | r-6hmzgn8b                                                      |
    | OS-EXT-SRV-ATTR:root_device_name     | -                                                               |
    | OS-EXT-SRV-ATTR:user_data            | -                                                               |
    | OS-EXT-STS:power_state               | 0                                                               |
    | OS-EXT-STS:task_state                | scheduling                                                      |
    | OS-EXT-STS:vm_state                  | building                                                        |
    | OS-SRV-USG:launched_at               | -                                                               |
    | OS-SRV-USG:terminated_at             | -                                                               |
    | accessIPv4                           |                                                                 |
    | accessIPv6                           |                                                                 |
    | adminPass                            | C3eFwVpuG7Su                                                    |
    | config_drive                         |                                                                 |
    | created                              | 2018-07-04T09:22:12Z                                            |
    | description                          | -                                                               |
    | flavor:disk                          | 1                                                               |
    | flavor:ephemeral                     | 0                                                               |
    | flavor:extra_specs                   | {}                                                              |
    | flavor:original_name                 | m1.tiny                                                         |
    | flavor:ram                           | 512                                                             |
    | flavor:swap                          | 0                                                               |
    | flavor:vcpus                         | 1                                                               |
    | hostId                               |                                                                 |
    | host_status                          |                                                                 |
    | id                                   | 70b959d7-3bc6-485a-b51c-07711b67757e                            |
    | image                                | cirros-0.3.5-x86_64-disk (695cf39a-6e94-46ed-9332-158da52361fc) |
    | key_name                             | -                                                               |
    | locked                               | False                                                           |
    | metadata                             | {}                                                              |
    | name                                 | vpn-vm-1                                                        |
    | os-extended-volumes:volumes_attached | []                                                              |
    | progress                             | 0                                                               |
    | security_groups                      | default                                                         |
    | status                               | BUILD                                                           |
    | tags                                 | []                                                              |
    | tenant_id                            | da717c389f494c3e8e6d32f46142d610                                |
    | updated                              | 2018-07-04T09:22:12Z                                            |
    | user_id                              | bde32ad388334927b25a5c2ff259c78d                                |
    +--------------------------------------+-----------------------------------------------------------------+
    stack@vpn-test2:~/devstack$ nova boot --flavor 1 --image 695cf39a-6e94-46ed-9332-158da52361fc --nic net-id=137d45d3-820f-409c-aac7-12ef9be66cb2 vpn-vm-2
    +--------------------------------------+-----------------------------------------------------------------+
    | Property                             | Value                                                           |
    +--------------------------------------+-----------------------------------------------------------------+
    | OS-DCF:diskConfig                    | MANUAL                                                          |
    | OS-EXT-AZ:availability_zone          |                                                                 |
    | OS-EXT-SRV-ATTR:host                 | -                                                               |
    | OS-EXT-SRV-ATTR:hostname             | vpn-vm-2                                                        |
    | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                               |
    | OS-EXT-SRV-ATTR:instance_name        |                                                                 |
    | OS-EXT-SRV-ATTR:kernel_id            |                                                                 |
    | OS-EXT-SRV-ATTR:launch_index         | 0                                                               |
    | OS-EXT-SRV-ATTR:ramdisk_id           |                                                                 |
    | OS-EXT-SRV-ATTR:reservation_id       | r-4geqbiv7                                                      |
    | OS-EXT-SRV-ATTR:root_device_name     | -                                                               |
    | OS-EXT-SRV-ATTR:user_data            | -                                                               |
    | OS-EXT-STS:power_state               | 0                                                               |
    | OS-EXT-STS:task_state                | scheduling                                                      |
    | OS-EXT-STS:vm_state                  | building                                                        |
    | OS-SRV-USG:launched_at               | -                                                               |
    | OS-SRV-USG:terminated_at             | -                                                               |
    | accessIPv4                           |                                                                 |
    | accessIPv6                           |                                                                 |
    | adminPass                            | BtUSoUTa375w                                                    |
    | config_drive                         |                                                                 |
    | created                              | 2018-07-04T09:24:04Z                                            |
    | description                          | -                                                               |
    | flavor:disk                          | 1                                                               |
    | flavor:ephemeral                     | 0                                                               |
    | flavor:extra_specs                   | {}                                                              |
    | flavor:original_name                 | m1.tiny                                                         |
    | flavor:ram                           | 512                                                             |
    | flavor:swap                          | 0                                                               |
    | flavor:vcpus                         | 1                                                               |
    | hostId                               |                                                                 |
    | host_status                          |                                                                 |
    | id                                   | e8e150dd-4a98-40df-ab35-28b3a791c1be                            |
    | image                                | cirros-0.3.5-x86_64-disk (695cf39a-6e94-46ed-9332-158da52361fc) |
    | key_name                             | -                                                               |
    | locked                               | False                                                           |
    | metadata                             | {}                                                              |
    | name                                 | vpn-vm-2                                                        |
    | os-extended-volumes:volumes_attached | []                                                              |
    | progress                             | 0                                                               |
    | security_groups                      | default                                                         |
    | status                               | BUILD                                                           |
    | tags                                 | []                                                              |
    | tenant_id                            | da717c389f494c3e8e6d32f46142d610                                |
    | updated                              | 2018-07-04T09:24:04Z                                            |
    | user_id                              | bde32ad388334927b25a5c2ff259c78d                                |
    +--------------------------------------+-----------------------------------------------------------------+

  2. 创建ikepolicy和ipsecpolicy

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    stack@vpn-test2:~/devstack$ neutron vpn-ikepolicy-create ikepolicy
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ikepolicy:
    +-------------------------+--------------------------------------+
    | Field                   | Value                                |
    +-------------------------+--------------------------------------+
    | auth_algorithm          | sha1                                 |
    | description             |                                      |
    | encryption_algorithm    | aes-128                              |
    | id                      | 3e5ae910-757f-46d2-b2c2-cb75df0abc39 |
    | ike_version             | v1                                   |
    | lifetime                | {"units": "seconds", "value": 3600}  |
    | name                    | ikepolicy                            |
    | pfs                     | group5                               |
    | phase1_negotiation_mode | main                                 |
    | project_id              | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id               | da717c389f494c3e8e6d32f46142d610     |
    +-------------------------+--------------------------------------+
    stack@vpn-test2:~/devstack$ neutron vpn-ipsecpolicy-create ipsecpolicy
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsecpolicy:
    +----------------------+--------------------------------------+
    | Field                | Value                                |
    +----------------------+--------------------------------------+
    | auth_algorithm       | sha1                                 |
    | description          |                                      |
    | encapsulation_mode   | tunnel                               |
    | encryption_algorithm | aes-128                              |
    | id                   | f7c16fe2-a0cb-4494-bcf3-a430d5699c4c |
    | lifetime             | {"units": "seconds", "value": 3600}  |
    | name                 | ipsecpolicy                          |
    | pfs                  | group5                               |
    | project_id           | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id            | da717c389f494c3e8e6d32f46142d610     |
    | transform_protocol   | esp                                  |
    +----------------------+--------------------------------------+

  3. 创建vpn-sercvice，其将用来后面建立site-to-site的连接

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    stack@vpn-test2:~/devstack$ neutron vpn-service-create --name VPNA --description "VPN service A" vpn-router-1 vpn-subnet-1
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new vpnservice:
    +----------------+--------------------------------------+
    | Field          | Value                                |
    +----------------+--------------------------------------+
    | admin_state_up | True                                 |
    | description    | VPN service A                        |
    | external_v4_ip | 172.24.4.18                          |
    | external_v6_ip | 2001:db8::b                          |
    | flavor_id      |                                      |
    | id             | 6725564d-a056-4c62-8ea8-3dff47a8e563 |
    | name           | VPNA                                 |
    | project_id     | da717c389f494c3e8e6d32f46142d610     |
    | router_id      | f0906c65-e9b2-447c-ad25-73fecb7fd093 |
    | status         | PENDING_CREATE                       |
    | subnet_id      | 840b87e2-c78c-45cb-92e7-2272ef6e0821 |
    | tenant_id      | da717c389f494c3e8e6d32f46142d610     |
    +----------------+--------------------------------------+
    stack@vpn-test2:~/devstack$ neutron vpn-service-create --name VPNB --description "VPN service B" vpn-router-2 vpn-subnet-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new vpnservice:
    +----------------+--------------------------------------+
    | Field          | Value                                |
    +----------------+--------------------------------------+
    | admin_state_up | True                                 |
    | description    | VPN service B                        |
    | external_v4_ip | 172.24.4.19                          |
    | external_v6_ip | 2001:db8::5                          |
    | flavor_id      |                                      |
    | id             | 4f7e2da2-92eb-4e30-aa7e-94ec66928d69 |
    | name           | VPNB                                 |
    | project_id     | da717c389f494c3e8e6d32f46142d610     |
    | router_id      | faf5fb10-8662-4862-bcb4-b3cee3c603be |
    | status         | PENDING_CREATE                       |
    | subnet_id      | 4a0b78a7-3d04-4b0e-8a43-350a482c4564 |
    | tenant_id      | da717c389f494c3e8e6d32f46142d610     |
    +----------------+--------------------------------------+

  4. 创建ipsec-site-connection

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    stack@vpn-test2:~/devstack$ neutron vpn-service-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+------+----------------------------------+--------------------------------------+----------------+
    | id                                   | name | tenant_id                        | router_id                            | status         |
    +--------------------------------------+------+----------------------------------+--------------------------------------+----------------+
    | 4f7e2da2-92eb-4e30-aa7e-94ec66928d69 | VPNB | da717c389f494c3e8e6d32f46142d610 | faf5fb10-8662-4862-bcb4-b3cee3c603be | PENDING_CREATE |
    | 6725564d-a056-4c62-8ea8-3dff47a8e563 | VPNA | da717c389f494c3e8e6d32f46142d610 | f0906c65-e9b2-447c-ad25-73fecb7fd093 | PENDING_CREATE |
    +--------------------------------------+------+----------------------------------+--------------------------------------+----------------+
    stack@vpn-test2:~/devstack$ neutron vpn-ikepolicy-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-----------+----------------------------------+----------------+----------------------+-------------+--------+
    | id                                   | name      | tenant_id                        | auth_algorithm | encryption_algorithm | ike_version | pfs    |
    +--------------------------------------+-----------+----------------------------------+----------------+----------------------+-------------+--------+
    | 3e5ae910-757f-46d2-b2c2-cb75df0abc39 | ikepolicy | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | v1          | group5 |
    +--------------------------------------+-----------+----------------------------------+----------------+----------------------+-------------+--------+
    stack@vpn-test2:~/devstack$ neutron vpn-ipsecpolicy-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+----------------+----------------------+--------+
    | id                                   | name        | tenant_id                        | auth_algorithm | encryption_algorithm | pfs    |
    +--------------------------------------+-------------+----------------------------------+----------------+----------------------+--------+
    | f7c16fe2-a0cb-4494-bcf3-a430d5699c4c | ipsecpolicy | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | group5 |
    +--------------------------------------+-------------+----------------------------------+----------------+----------------------+--------+
    stack@vpn-test2:~/devstack$ neutron ipsec-site-connection-create --name VPNConnectionA --vpnservice-id 6725564d-a056-4c62-8ea8-3dff47a8e563 --ikepolicy-id 3e5ae910-757f-46d2-b2c2-cb75df0abc39 --ipsecpolicy-id f7c16fe2-a0cb-4494-bcf3-a430d5699c4c --peer-address 172.24.4.19 --peer-id 172.24.4.19 --peer-cidr 10.2.0.0/24 --psk lizenghui
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c               |
    | ikepolicy_id      | 3e5ae910-757f-46d2-b2c2-cb75df0abc39               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | f7c16fe2-a0cb-4494-bcf3-a430d5699c4c               |
    | local_ep_group_id |                                                    |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | VPNConnectionA                                     |
    | peer_address      | 172.24.4.19                                        |
    | peer_cidrs        | 10.2.0.0/24                                        |
    | peer_ep_group_id  |                                                    |
    | peer_id           | 172.24.4.19                                        |
    | project_id        | da717c389f494c3e8e6d32f46142d610                   |
    | psk               | lizenghui                                          |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610                   |
    | vpnservice_id     | 6725564d-a056-4c62-8ea8-3dff47a8e563               |
    +-------------------+----------------------------------------------------+
    stack@vpn-test2:~/devstack$ neutron ipsec-site-connection-create --name VPNConnectionB --vpnservice-id 4f7e2da2-92eb-4e30-aa7e-94ec66928d69 --ikepolicy-id 3e5ae910-757f-46d2-b2c2-cb75df0abc39 --ipsecpolicy-id f7c16fe2-a0cb-4494-bcf3-a430d5699c4c --peer-address 172.24.4.18 --peer-id 172.24.4.18 --peer-cidr 10.1.0.0/24 --psk lizenghui
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | 033a5fe4-7505-474c-bb83-4f4f78b63e7a               |
    | ikepolicy_id      | 3e5ae910-757f-46d2-b2c2-cb75df0abc39               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | f7c16fe2-a0cb-4494-bcf3-a430d5699c4c               |
    | local_ep_group_id |                                                    |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | VPNConnectionB                                     |
    | peer_address      | 172.24.4.18                                        |
    | peer_cidrs        | 10.1.0.0/24                                        |
    | peer_ep_group_id  |                                                    |
    | peer_id           | 172.24.4.18                                        |
    | project_id        | da717c389f494c3e8e6d32f46142d610                   |
    | psk               | lizenghui                                          |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610                   |
    | vpnservice_id     | 4f7e2da2-92eb-4e30-aa7e-94ec66928d69               |
    +-------------------+----------------------------------------------------+

  5. 查看连接状态，如果立即查看status为PENDING_CREATE，则等待30~50秒后再次查看，即可看到状态变为ACTIVE。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    stack@vpn-test2:~/devstack$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+----------------+
    | id                                   | name           | tenant_id                        | peer_address | auth_mode | status         |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+----------------+
    | 033a5fe4-7505-474c-bb83-4f4f78b63e7a | VPNConnectionB | da717c389f494c3e8e6d32f46142d610 | 172.24.4.18  | psk       | PENDING_CREATE |
    | ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c | VPNConnectionA | da717c389f494c3e8e6d32f46142d610 | 172.24.4.19  | psk       | PENDING_CREATE |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+----------------+
    stack@vpn-test2:~/devstack$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name           | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+
    | 033a5fe4-7505-474c-bb83-4f4f78b63e7a | VPNConnectionB | da717c389f494c3e8e6d32f46142d610 | 172.24.4.18  | psk       | ACTIVE |
    | ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c | VPNConnectionA | da717c389f494c3e8e6d32f46142d610 | 172.24.4.19  | psk       | ACTIVE |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+

  6. 从vpn-router-2所在的namespace，看看是否能够ping通vm-1，同样查看在vpn-router-1所在namespace是否可以ping通vm-2,发现可以相互ping通。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    stack@vpn-test2:~/devstack$ nova list
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | ID                                   | Name     | Status | Task State | Power State | Networks                |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | 70b959d7-3bc6-485a-b51c-07711b67757e | vpn-vm-1 | ACTIVE | -          | Running     | vpn-network-1=10.1.0.12 |
    | e8e150dd-4a98-40df-ab35-28b3a791c1be | vpn-vm-2 | ACTIVE | -          | Running     | vpn-network-2=10.2.0.3  |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    stack@vpn-test2:~/devstack$ openstack router list
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | ID                                   | Name         | Status | State | Distributed | HA    | Project                          |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | bdcb18cf-4b83-47df-b0d5-b424eaeda1a6 | router1      | ACTIVE | UP    | False       | False | 9885c5369b824e81b01d0e5444f379f7 |
    | f0906c65-e9b2-447c-ad25-73fecb7fd093 | vpn-router-1 | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    | faf5fb10-8662-4862-bcb4-b3cee3c603be | vpn-router-2 | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host
           valid_lft forever preferred_lft forever
    17: qr-b0d7d178-00: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1
        link/ether fa:16:3e:a5:29:f9 brd ff:ff:ff:ff:ff:ff
        inet 10.2.0.1/24 brd 10.2.0.255 scope global qr-b0d7d178-00
           valid_lft forever preferred_lft forever
        inet6 fe80::f816:3eff:fea5:29f9/64 scope link
           valid_lft forever preferred_lft forever
    18: qg-16b32471-b8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1
        link/ether fa:16:3e:bd:3e:f1 brd ff:ff:ff:ff:ff:ff
        inet 172.24.4.19/24 brd 172.24.4.255 scope global qg-16b32471-b8
           valid_lft forever preferred_lft forever
        inet6 2001:db8::5/64 scope global
           valid_lft forever preferred_lft forever
        inet6 fe80::f816:3eff:febd:3ef1/64 scope link
           valid_lft forever preferred_lft forever
    stack@vpn-test2:~/devstack$ sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be  ping 10.1.0.12
    PING 10.1.0.12 (10.1.0.12) 56(84) bytes of data.
    64 bytes from 10.1.0.12: icmp_seq=1 ttl=63 time=3.20 ms
    64 bytes from 10.1.0.12: icmp_seq=2 ttl=63 time=0.812 ms
    64 bytes from 10.1.0.12: icmp_seq=3 ttl=63 time=0.648 ms
    64 bytes from 10.1.0.12: icmp_seq=4 ttl=63 time=0.722 ms
    64 bytes from 10.1.0.12: icmp_seq=5 ttl=63 time=0.801 ms
    64 bytes from 10.1.0.12: icmp_seq=6 ttl=63 time=0.657 ms
    64 bytes from 10.1.0.12: icmp_seq=7 ttl=63 time=0.760 ms
    64 bytes from 10.1.0.12: icmp_seq=8 ttl=63 time=0.633 ms
    64 bytes from 10.1.0.12: icmp_seq=9 ttl=63 time=0.612 ms
    64 bytes from 10.1.0.12: icmp_seq=10 ttl=63 time=0.635 ms
    64 bytes from 10.1.0.12: icmp_seq=11 ttl=63 time=0.697 ms
    stack@vpn-test2:~$ sudo ip netns exec qrouter-f0906c65-e9b2-447c-ad25-73fecb7fd093 ping 10.2.0.3
    PING 10.2.0.3 (10.2.0.3) 56(84) bytes of data.
    64 bytes from 10.2.0.3: icmp_seq=1 ttl=63 time=5.14 ms
    64 bytes from 10.2.0.3: icmp_seq=2 ttl=63 time=0.875 ms
    64 bytes from 10.2.0.3: icmp_seq=3 ttl=63 time=0.973 ms
    64 bytes from 10.2.0.3: icmp_seq=4 ttl=63 time=0.819 ms
    64 bytes from 10.2.0.3: icmp_seq=5 ttl=63 time=0.630 ms
    64 bytes from 10.2.0.3: icmp_seq=6 ttl=63 time=0.755 ms
    64 bytes from 10.2.0.3: icmp_seq=7 ttl=63 time=0.643 ms
    64 bytes from 10.2.0.3: icmp_seq=8 ttl=63 time=0.563 ms
    64 bytes from 10.2.0.3: icmp_seq=9 ttl=63 time=0.623 ms
    64 bytes from 10.2.0.3: icmp_seq=10 ttl=63 time=0.745 ms

  7. 能够Ping通，说明vpn连接已经建立，在qr，qg口抓包，确认ipsec流量，可以看到ESP加密包，下面是在路由器vpn-router-2上的qg口上的数据包

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    stack@vpn-test2:~/devstack$ sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be tcpdump -i qg-16b32471-b8 -v
    tcpdump: listening on qg-16b32471-b8, link-type EN10MB (Ethernet), capture size 262144 bytes
    19:48:00.826393 IP (tos 0x0, ttl 64, id 59316, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x120), length 132
    19:48:00.827128 IP (tos 0x0, ttl 64, id 18957, offset 0, flags [DF], proto UDP (17), length 70)
        172.24.4.19.59831 > host-192-168-1-4.openstacklocal.domain: 65426+ PTR? 18.4.24.172.in-addr.arpa. (42)
    19:48:00.828650 IP (tos 0x0, ttl 63, id 26517, offset 0, flags [DF], proto UDP (17), length 70)
        host-192-168-1-4.openstacklocal.domain > 172.24.4.19.59831: 65426 NXDomain 0/0/0 (42)
    19:48:00.828854 IP (tos 0x0, ttl 64, id 18958, offset 0, flags [DF], proto UDP (17), length 70)
        172.24.4.19.44962 > host-192-168-1-4.openstacklocal.domain: 9844+ PTR? 19.4.24.172.in-addr.arpa. (42)
    19:48:00.828902 IP (tos 0x0, ttl 64, id 27632, offset 0, flags [none], proto ESP (50), length 152)
        172.24.4.18 > 172.24.4.19: ESP(spi=0xc2eca09d,seq=0x120), length 132
    19:48:00.828902 IP (tos 0x0, ttl 63, id 9967, offset 0, flags [none], proto ICMP (1), length 84)
        10.1.0.12 > 10.2.0.1: ICMP echo reply, id 8568, seq 1, length 64
    19:48:00.829140 IP (tos 0x0, ttl 63, id 26518, offset 0, flags [DF], proto UDP (17), length 70)
        host-192-168-1-4.openstacklocal.domain > 172.24.4.19.44962: 9844 NXDomain 0/0/0 (42)
    19:48:00.829341 IP (tos 0x0, ttl 64, id 18959, offset 0, flags [DF], proto UDP (17), length 70)
        172.24.4.19.59801 > host-192-168-1-4.openstacklocal.domain: 17454+ PTR? 4.1.168.192.in-addr.arpa. (42)
    19:48:00.829841 IP (tos 0x0, ttl 64, id 18960, offset 0, flags [DF], proto UDP (17), length 67)
        172.24.4.19.48978 > host-192-168-1-4.openstacklocal.domain: 40542+ PTR? 1.0.2.10.in-addr.arpa. (39)
    19:48:00.830112 IP (tos 0x0, ttl 63, id 26520, offset 0, flags [DF], proto UDP (17), length 67)
        host-192-168-1-4.openstacklocal.domain > 172.24.4.19.48978: 40542 NXDomain 0/0/0 (39)
    19:48:00.830320 IP (tos 0x0, ttl 64, id 18961, offset 0, flags [DF], proto UDP (17), length 68)
        172.24.4.19.51925 > host-192-168-1-4.openstacklocal.domain: 56871+ PTR? 12.0.1.10.in-addr.arpa. (40)
    19:48:00.857281 IP (tos 0x0, ttl 63, id 26542, offset 0, flags [DF], proto UDP (17), length 103)
        host-192-168-1-4.openstacklocal.domain > 172.24.4.19.51925: 56871 NXDomain* 0/1/0 (75)
    19:48:01.828091 IP (tos 0x0, ttl 64, id 59451, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x121), length 132
    19:48:01.828958 IP (tos 0x0, ttl 64, id 27729, offset 0, flags [none], proto ESP (50), length 152)
        172.24.4.18 > 172.24.4.19: ESP(spi=0xc2eca09d,seq=0x121), length 132
    19:48:01.828958 IP (tos 0x0, ttl 63, id 10013, offset 0, flags [none], proto ICMP (1), length 84)
        10.1.0.12 > 10.2.0.1: ICMP echo reply, id 8568, seq 2, length 64
    19:48:02.829143 IP (tos 0x0, ttl 64, id 59462, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x122), length 132
    19:48:02.829728 IP (tos 0x0, ttl 64, id 27978, offset 0, flags [none], proto ESP (50), length 152)
        172.24.4.18 > 172.24.4.19: ESP(spi=0xc2eca09d,seq=0x122), length 132
    19:48:02.829728 IP (tos 0x0, ttl 63, id 10207, offset 0, flags [none], proto ICMP (1), length 84)
        10.1.0.12 > 10.2.0.1: ICMP echo reply, id 8568, seq 3, length 64
    19:48:03.829327 IP (tos 0x0, ttl 64, id 59603, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x123), length 132
    19:48:03.830344 IP (tos 0x0, ttl 64, id 28220, offset 0, flags [none], proto ESP (50), length 152)
        172.24.4.18 > 172.24.4.19: ESP(spi=0xc2eca09d,seq=0x123), length 132
    19:48:03.830344 IP (tos 0x0, ttl 63, id 10261, offset 0, flags [none], proto ICMP (1), length 84)
        10.1.0.12 > 10.2.0.1: ICMP echo reply, id 8568, seq 4, length 64
    19:48:04.830575 IP (tos 0x0, ttl 64, id 59761, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x124), length 132
    19:48:04.831213 IP (tos 0x0, ttl 64, id 28233, offset 0, flags [none], proto ESP (50), length 152)
        172.24.4.18 > 172.24.4.19: ESP(spi=0xc2eca09d,seq=0x124), length 132
    19:48:04.831213 IP (tos 0x0, ttl 63, id 10436, offset 0, flags [none], proto ICMP (1), length 84)
        10.1.0.12 > 10.2.0.1: ICMP echo reply, id 8568, seq 5, length 64
    19:48:05.829216 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.24.4.10 tell 172.24.4.19, length 28
    19:48:05.829479 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.24.4.19 tell 172.24.4.10, length 28
    19:48:05.829492 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.24.4.19 is-at fa:16:3e:bd:3e:f1 (oui Unknown), length 28
    19:48:05.829495 IP (tos 0x0, ttl 64, id 19157, offset 0, flags [DF], proto UDP (17), length 70)
        172.24.4.19.36595 > host-192-168-1-4.openstacklocal.domain: 39747+ PTR? 10.4.24.172.in-addr.arpa. (42)
    19:48:05.829570 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.24.4.10 is-at 26:8b:2c:c4:4f:48 (oui Unknown), length 28
    19:48:05.829618 IP (tos 0x0, ttl 64, id 60003, offset 0, flags [DF], proto ESP (50), length 152)
        172.24.4.19 > 172.24.4.18: ESP(spi=0xc3330682,seq=0x125), length 132

  8. 查看两个路由器的路由表信息

    1
    2
    3
    4
    5
    6
    7
    8
    stack@vpn-test2:~$ sudo ip netns exec qrouter-f0906c65-e9b2-447c-ad25-73fecb7fd093 ip route list
    default via 172.24.4.10 dev qg-472d97a3-80
    10.1.0.0/24 dev qr-0fbd2cfa-9e  proto kernel  scope link  src 10.1.0.1
    172.24.4.0/24 dev qg-472d97a3-80  proto kernel  scope link  src 172.24.4.18
    stack@vpn-test2:~$ sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be ip route list
    default via 172.24.4.10 dev qg-16b32471-b8
    10.2.0.0/24 dev qr-b0d7d178-00  proto kernel  scope link  src 10.2.0.1
    172.24.4.0/24 dev qg-16b32471-b8  proto kernel  scope link  src 172.24.4.19

  9. 删掉vpn服务连接实例后，看看路由表变动情况以及还能否ping通，发现不能ping通了

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    stack@vpn-test2:~$ neutron vpn-service-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+------+----------------------------------+--------------------------------------+--------+
    | id                                   | name | tenant_id                        | router_id                            | status |
    +--------------------------------------+------+----------------------------------+--------------------------------------+--------+
    | 4f7e2da2-92eb-4e30-aa7e-94ec66928d69 | VPNB | da717c389f494c3e8e6d32f46142d610 | faf5fb10-8662-4862-bcb4-b3cee3c603be | ACTIVE |
    | 6725564d-a056-4c62-8ea8-3dff47a8e563 | VPNA | da717c389f494c3e8e6d32f46142d610 | f0906c65-e9b2-447c-ad25-73fecb7fd093 | ACTIVE |
    +--------------------------------------+------+----------------------------------+--------------------------------------+--------+
    stack@vpn-test2:~$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name           | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+
    | 033a5fe4-7505-474c-bb83-4f4f78b63e7a | VPNConnectionB | da717c389f494c3e8e6d32f46142d610 | 172.24.4.18  | psk       | ACTIVE |
    | ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c | VPNConnectionA | da717c389f494c3e8e6d32f46142d610 | 172.24.4.19  | psk       | ACTIVE |
    +--------------------------------------+----------------+----------------------------------+--------------+-----------+--------+
    stack@vpn-test2:~$ neutron ipsec-site-connection-delete ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Deleted ipsec_site_connection(s): ce29fb78-6b5c-4e2c-8fa8-6c7a8662a14c
    stack@vpn-test2:~$ sudo ip netns exec qrouter-f0906c65-e9b2-447c-ad25-73fecb7fd093 ping 10.2.0.3
    PING 10.2.0.3 (10.2.0.3) 56(84) bytes of data.
    ^C
    --- 10.2.0.3 ping statistics ---
    376 packets transmitted, 0 received, 100% packet loss, time 375175ms
    stack@vpn-test2:~$ sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be  ping 10.1.0.12
    PING 10.1.0.12 (10.1.0.12) 56(84) bytes of data.
    ^C
    --- 10.1.0.12 ping statistics ---
    15 packets transmitted, 0 received, 100% packet loss, time 14077ms
    stack@vpn-test2:~$ sudo ip netns exec qrouter-faf5fb10-8662-4862-bcb4-b3cee3c603be ip route list
    default via 172.24.4.10 dev qg-16b32471-b8
    10.2.0.0/24 dev qr-b0d7d178-00  proto kernel  scope link  src 10.2.0.1
    172.24.4.0/24 dev qg-16b32471-b8  proto kernel  scope link  src 172.24.4.19
    stack@vpn-test2:~$ sudo ip netns exec qrouter-f0906c65-e9b2-447c-ad25-73fecb7fd093 ip route list
    default via 172.24.4.10 dev qg-472d97a3-80
    10.1.0.0/24 dev qr-0fbd2cfa-9e  proto kernel  scope link  src 10.1.0.1
    172.24.4.0/24 dev qg-472d97a3-80  proto kernel  scope link  src 172.24.4.18

至此，vpnaas的基本功能验证完成。

*! Endpoint group功能测试
测试方案

环境还是ALL-IN-ONE单节点Openstack，四不同的vm（vm1~vm2），vm1,vm2属于network-1下的不同子网，vm3，vm4属于network-2下面的不同子网。各自连接的路由器上创建VPN服务，分别创建endpoint-group，需要将两个router的出口网关设为同一网段。

具体步骤：

  1. 为四个vm创建对应的network和所在子网
  2. 创建router，并设置出口网关在同一段external网络，即”public”
  3. 各自network下创建vm，固定v4-ip，分别属于不同子网下面，并创建ikepolicy，ipsecpolicy，和vpn-service和endpoint group
  4. 创建和删除ipsec-site-connection后，测试四个vm之间的连通性

网络拓扑:
endpoint_group测试拓扑

测试步骤

  1. 创建用于测试endpoint group的network和子网

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    stack@vpn-test2:~$ neutron net-create eg-network-1
    stack@vpn-test2:~$ neutron net-create eg-network-2
    stack@vpn-test2:~$ neutron subnet-create --name eg-subnet-1 eg-network-1 10.3.0.0/24
    stack@vpn-test2:~$ neutron subnet-create --name eg-subnet-2 eg-network-1 10.4.0.0/24
    stack@vpn-test2:~$ neutron subnet-create --name eg-subnet-3 eg-network-2 10.5.0.0/24
    stack@vpn-test2:~$ neutron subnet-create --name eg-subnet-4 eg-network-2 10.6.0.0/24
    stack@vpn-test2:~$ openstack network list
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | ID                                   | Name          | Subnets                                                                    |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    | 137d45d3-820f-409c-aac7-12ef9be66cb2 | vpn-network-2 | 4a0b78a7-3d04-4b0e-8a43-350a482c4564                                       |
    | 274a1424-ccce-47c6-ad32-a092c51574a6 | eg-network-2  | 2a6799f1-2436-43f0-8dae-3b838bbb5167, 792a2075-fcf0-4622-a509-620f708bc316 |
    | 307b1ddf-36bc-4ace-a055-0bfe97f0da9d | eg-network-1  | 111c98b0-effc-4760-a270-f4003ca98ad2, 9584d7e3-8a93-49fb-8f8f-73d44c94d19d |
    | 4df92675-1843-4b4b-9436-4932856ed8fe | public        | 4d801dd9-da36-4832-ad10-0be3229fea9a, 4fafa6f9-2f43-475d-95b0-5392dc3164fb |
    | 8f71bd0e-d26e-405b-8e48-68148fef8ed1 | vpn-network-1 | 840b87e2-c78c-45cb-92e7-2272ef6e0821                                       |
    | e95e8eac-c523-4066-a6fd-9ba03f7a9b20 | private       | 20516105-08ab-407a-8c8f-a16f3d2f60c0, bab7c9d8-2065-4dfc-9ff7-1973ebb4b604 |
    +--------------------------------------+---------------+----------------------------------------------------------------------------+
    stack@vpn-test2:~$ openstack subnet list
    +--------------------------------------+---------------------+--------------------------------------+---------------------+
    | ID                                   | Name                | Network                              | Subnet              |
    +--------------------------------------+---------------------+--------------------------------------+---------------------+
    | 111c98b0-effc-4760-a270-f4003ca98ad2 | eg-subnet-1         | 307b1ddf-36bc-4ace-a055-0bfe97f0da9d | 10.3.0.0/24         |
    | 20516105-08ab-407a-8c8f-a16f3d2f60c0 | ipv6-private-subnet | e95e8eac-c523-4066-a6fd-9ba03f7a9b20 | fd60:9aee:5b2f::/64 |
    | 2a6799f1-2436-43f0-8dae-3b838bbb5167 | eg-subnet-3         | 274a1424-ccce-47c6-ad32-a092c51574a6 | 10.5.0.0/24         |
    | 4a0b78a7-3d04-4b0e-8a43-350a482c4564 | vpn-subnet-2        | 137d45d3-820f-409c-aac7-12ef9be66cb2 | 10.2.0.0/24         |
    | 4d801dd9-da36-4832-ad10-0be3229fea9a | ipv6-public-subnet  | 4df92675-1843-4b4b-9436-4932856ed8fe | 2001:db8::/64       |
    | 4fafa6f9-2f43-475d-95b0-5392dc3164fb | public-subnet       | 4df92675-1843-4b4b-9436-4932856ed8fe | 172.24.4.0/24       |
    | 792a2075-fcf0-4622-a509-620f708bc316 | eg-subnet-4         | 274a1424-ccce-47c6-ad32-a092c51574a6 | 10.6.0.0/24         |
    | 840b87e2-c78c-45cb-92e7-2272ef6e0821 | vpn-subnet-1        | 8f71bd0e-d26e-405b-8e48-68148fef8ed1 | 10.1.0.0/24         |
    | 9584d7e3-8a93-49fb-8f8f-73d44c94d19d | eg-subnet-2         | 307b1ddf-36bc-4ace-a055-0bfe97f0da9d | 10.4.0.0/24         |
    | bab7c9d8-2065-4dfc-9ff7-1973ebb4b604 | privateA            | e95e8eac-c523-4066-a6fd-9ba03f7a9b20 | 10.0.0.0/26         |
    +--------------------------------------+---------------------+--------------------------------------+---------------------+

  2. 创建两个eg-router，并连接对应子网，设置对外网关

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    stack@vpn-test2:~$ neutron router-create eg-router-1
    stack@vpn-test2:~$ neutron router-create eg-router-2
    stack@vpn-test2:~$ neutron router-interface-add eg-router-1 eg-subnet-1
    stack@vpn-test2:~$ neutron router-interface-add eg-router-1 eg-subnet-2
    stack@vpn-test2:~$ neutron router-interface-add eg-router-2 eg-subnet-3
    stack@vpn-test2:~$ neutron router-interface-add eg-router-2 eg-subnet-4
    stack@vpn-test2:~$ neutron router-gateway-set eg-router-1 public
    Set gateway for router eg-router-1
    stack@vpn-test2:~$ neutron router-gateway-set eg-router-2 public
    Set gateway for router eg-router-2

  3. 创建虚机eg-vm-1~eg-vm-4

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    stack@vpn-test2:~$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=307b1ddf-36bc-4ace-a055-0bfe97f0da9d,v4-fixed-ip=10.3.0.3 eg-vm-1
    stack@vpn-test2:~$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=307b1ddf-36bc-4ace-a055-0bfe97f0da9d,v4-fixed-ip=10.4.0.4 eg-vm-2
    stack@vpn-test2:~$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=274a1424-ccce-47c6-ad32-a092c51574a6,v4-fixed-ip=10.5.0.5 eg-vm-3
    stack@vpn-test2:~$ nova boot --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=274a1424-ccce-47c6-ad32-a092c51574a6,v4-fixed-ip=10.6.0.6 eg-vm-4
    stack@vpn-test2:~$ nova list
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | ID                                   | Name     | Status | Task State | Power State | Networks                |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | c6c3a388-fe16-424a-8a68-0899b7578977 | eg-vm-1  | ACTIVE | -          | Running     | eg-network-1=10.3.0.3   |
    | 2dd3e32c-49f5-4db9-9271-a880eb7c6806 | eg-vm-2  | ACTIVE | -          | Running     | eg-network-1=10.4.0.4   |
    | f5040c29-2904-4318-879f-05ce51940c37 | eg-vm-3  | ACTIVE | -          | Running     | eg-network-2=10.5.0.5   |
    | 384c7696-a673-4039-b2aa-363199327c92 | eg-vm-4  | ACTIVE | -          | Running     | eg-network-2=10.6.0.6   |
    | 70b959d7-3bc6-485a-b51c-07711b67757e | vpn-vm-1 | ACTIVE | -          | Running     | vpn-network-1=10.1.0.12 |
    | e8e150dd-4a98-40df-ab35-28b3a791c1be | vpn-vm-2 | ACTIVE | -          | Running     | vpn-network-2=10.2.0.3  |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+

  4. 创建用于测试endpoint group的ikepolicy，ipsecpolicy

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    stack@vpn-test2:~$ neutron vpn-ikepolicy-create eg-ikepolicy
    stack@vpn-test2:~$ neutron vpn-ipsecpolicy-create eg-ipsecpolicy
    stack@vpn-test2:~$ neutron vpn-ikepolicy-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+--------------+----------------------------------+----------------+----------------------+-------------+--------+
    | id                                   | name         | tenant_id                        | auth_algorithm | encryption_algorithm | ike_version | pfs    |
    +--------------------------------------+--------------+----------------------------------+----------------+----------------------+-------------+--------+
    | 74796b9d-3695-4759-a0bc-bd0c6bdb8c9f | ikepolicy    | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | v1          | group5 |
    | 85c27997-ab43-4782-815b-b4de285ae74a | eg-ikepolicy | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | v1          | group5 |
    +--------------------------------------+--------------+----------------------------------+----------------+----------------------+-------------+--------+
    stack@vpn-test2:~$ neutron vpn-ipsecpolicy-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+----------------+----------------------------------+----------------+----------------------+--------+
    | id                                   | name           | tenant_id                        | auth_algorithm | encryption_algorithm | pfs    |
    +--------------------------------------+----------------+----------------------------------+----------------+----------------------+--------+
    | 35305802-8e52-4983-adab-946a58e44c91 | eg-ipsecpolicy | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | group5 |
    | f7c16fe2-a0cb-4494-bcf3-a430d5699c4c | ipsecpolicy    | da717c389f494c3e8e6d32f46142d610 | sha1           | aes-128              | group5 |
    +--------------------------------------+----------------+----------------------------------+----------------+----------------------+--------+

  5. 创建vpn-service和endpoint-group。
    注意：此时创建vpn-service时，不指定子网，只指定路由器

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    neutron vpn-service-create --name eg-VPNA eg-router-1
    stack@vpn-test2:~$ neutron vpn-endpoint-group-create --name eg-locals --type subnet --value eg-subnet-1 --value eg-subnet-2
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new endpoint_group:
    +-------------+--------------------------------------+
    | Field       | Value                                |
    +-------------+--------------------------------------+
    | description |                                      |
    | endpoints   | 111c98b0-effc-4760-a270-f4003ca98ad2 |
    |             | 9584d7e3-8a93-49fb-8f8f-73d44c94d19d |
    | id          | 9827c31e-4255-41de-91e7-eeefa73db311 |
    | name        | eg-locals                            |
    | project_id  | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id   | da717c389f494c3e8e6d32f46142d610     |
    | type        | subnet                               |
    +-------------+--------------------------------------+
    stack@vpn-test2:~$ neutron vpn-endpoint-group-create --name eg-peers --type cidr --value 10.5.0.0/24 --value 10.6.0.0/24
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new endpoint_group:
    +-------------+--------------------------------------+
    | Field       | Value                                |
    +-------------+--------------------------------------+
    | description |                                      |
    | endpoints   | 10.5.0.0/24                          |
    |             | 10.6.0.0/24                          |
    | id          | df0e8f64-5e03-49c8-ae5a-1ec9f62cf563 |
    | name        | eg-peers                             |
    | project_id  | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id   | da717c389f494c3e8e6d32f46142d610     |
    | type        | cidr                                 |
    +-------------+--------------------------------------+
    stack@vpn-test2:~$ neutron vpn-service-create --name eg-VPNB eg-router-2
    stack@vpn-test2:~$ neutron vpn-endpoint-group-create --name eg-locals-2 --type subnet --value eg-subnet-3 --value eg-subnet-4
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new endpoint_group:
    +-------------+--------------------------------------+
    | Field       | Value                                |
    +-------------+--------------------------------------+
    | description |                                      |
    | endpoints   | 2a6799f1-2436-43f0-8dae-3b838bbb5167 |
    |             | 792a2075-fcf0-4622-a509-620f708bc316 |
    | id          | 0fd07dbc-bf75-4b55-88a1-56cc072e437d |
    | name        | eg-locals-2                          |
    | project_id  | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id   | da717c389f494c3e8e6d32f46142d610     |
    | type        | subnet                               |
    +-------------+--------------------------------------+
    stack@vpn-test2:~$ neutron vpn-endpoint-group-create --name eg-peers-2 --type cidr --value 10.3.0.0/24 --value 10.4.0.0/24
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new endpoint_group:
    +-------------+--------------------------------------+
    | Field       | Value                                |
    +-------------+--------------------------------------+
    | description |                                      |
    | endpoints   | 10.3.0.0/24                          |
    |             | 10.4.0.0/24                          |
    | id          | b6448c42-a274-4475-b335-4987cddfdba5 |
    | name        | eg-peers-2                           |
    | project_id  | da717c389f494c3e8e6d32f46142d610     |
    | tenant_id   | da717c389f494c3e8e6d32f46142d610     |
    | type        | cidr                                 |
    +-------------+--------------------------------------+
    stack@vpn-test2:~$ neutron vpn-endpoint-group-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------+----------------------------------+--------+------------------------------------------------------------------------------------+
    | id                                   | name        | tenant_id                        | type   | endpoints                                                                          |
    +--------------------------------------+-------------+----------------------------------+--------+------------------------------------------------------------------------------------+
    | 0fd07dbc-bf75-4b55-88a1-56cc072e437d | eg-locals-2 | da717c389f494c3e8e6d32f46142d610 | subnet | [u'2a6799f1-2436-43f0-8dae-3b838bbb5167', u'792a2075-fcf0-4622-a509-620f708bc316'] |
    | 9827c31e-4255-41de-91e7-eeefa73db311 | eg-locals   | da717c389f494c3e8e6d32f46142d610 | subnet | [u'111c98b0-effc-4760-a270-f4003ca98ad2', u'9584d7e3-8a93-49fb-8f8f-73d44c94d19d'] |
    | b6448c42-a274-4475-b335-4987cddfdba5 | eg-peers-2  | da717c389f494c3e8e6d32f46142d610 | cidr   | [u'10.3.0.0/24', u'10.4.0.0/24']                                                   |
    | df0e8f64-5e03-49c8-ae5a-1ec9f62cf563 | eg-peers    | da717c389f494c3e8e6d32f46142d610 | cidr   | [u'10.5.0.0/24', u'10.6.0.0/24']                                                   |
    +--------------------------------------+-------------+----------------------------------+--------+------------------------------------------------------------------------------------+

  6. 创建ipsec-site-connection，等待30到50秒后查看vpn-service和ipsec-site-connection的状态，状态为ACTIVE，说明功能正常。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    stack@vpn-test2:~$ neutron ipsec-site-connection-create --name eg-vpnconnection1 --vpnservice-id e64a8cac-3b3e-46c0-9154-69b42b162d36 --ikepolicy-id 85c27997-ab43-4782-815b-b4de285ae74a --ipsecpolicy-id 35305802-8e52-4983-adab-946a58e44c91 --peer-address 172.24.4.17 --peer-id 172.24.4.17 --local-ep-group eg-locals --peer-ep-group eg-peers --psk lizenghui
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | 34fd710c-c989-453c-ab30-43e35300ec76               |
    | ikepolicy_id      | 85c27997-ab43-4782-815b-b4de285ae74a               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | 35305802-8e52-4983-adab-946a58e44c91               |
    | local_ep_group_id | 9827c31e-4255-41de-91e7-eeefa73db311               |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | eg-vpnconnection1                                  |
    | peer_address      | 172.24.4.17                                        |
    | peer_cidrs        |                                                    |
    | peer_ep_group_id  | df0e8f64-5e03-49c8-ae5a-1ec9f62cf563               |
    | peer_id           | 172.24.4.17                                        |
    | project_id        | da717c389f494c3e8e6d32f46142d610                   |
    | psk               | lizenghui                                          |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610                   |
    | vpnservice_id     | e64a8cac-3b3e-46c0-9154-69b42b162d36               |
    +-------------------+----------------------------------------------------+
    stack@vpn-test2:~$ neutron ipsec-site-connection-create --name eg-vpnconnection2 --vpnservice-id f883f3f5-66be-414a-b90f-32f84aa77655 --ikepolicy-id 85c27997-ab43-4782-815b-b4de285ae74a --ipsecpolicy-id 35305802-8e52-4983-adab-946a58e44c91 --peer-address 172.24.4.15 --peer-id 172.24.4.15 --local-ep-group eg-locals-2 --peer-ep-group eg-peers-2 --psk lizenghui
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Created a new ipsec_site_connection:
    +-------------------+----------------------------------------------------+
    | Field             | Value                                              |
    +-------------------+----------------------------------------------------+
    | admin_state_up    | True                                               |
    | auth_mode         | psk                                                |
    | description       |                                                    |
    | dpd               | {"action": "hold", "interval": 30, "timeout": 120} |
    | id                | 429e0919-6e80-457e-a8d7-6f28f2d0df77               |
    | ikepolicy_id      | 85c27997-ab43-4782-815b-b4de285ae74a               |
    | initiator         | bi-directional                                     |
    | ipsecpolicy_id    | 35305802-8e52-4983-adab-946a58e44c91               |
    | local_ep_group_id | 0fd07dbc-bf75-4b55-88a1-56cc072e437d               |
    | local_id          |                                                    |
    | mtu               | 1500                                               |
    | name              | eg-vpnconnection2                                  |
    | peer_address      | 172.24.4.15                                        |
    | peer_cidrs        |                                                    |
    | peer_ep_group_id  | b6448c42-a274-4475-b335-4987cddfdba5               |
    | peer_id           | 172.24.4.15                                        |
    | project_id        | da717c389f494c3e8e6d32f46142d610                   |
    | psk               | lizenghui                                          |
    | route_mode        | static                                             |
    | status            | PENDING_CREATE                                     |
    | tenant_id         | da717c389f494c3e8e6d32f46142d610                   |
    | vpnservice_id     | f883f3f5-66be-414a-b90f-32f84aa77655               |
    +-------------------+----------------------------------------------------+
    stack@vpn-test2:~$ neutron vpn-service-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+---------+----------------------------------+--------------------------------------+----------------+
    | id                                   | name    | tenant_id                        | router_id                            | status         |
    +--------------------------------------+---------+----------------------------------+--------------------------------------+----------------+
    | 4f7e2da2-92eb-4e30-aa7e-94ec66928d69 | VPNB    | da717c389f494c3e8e6d32f46142d610 | faf5fb10-8662-4862-bcb4-b3cee3c603be | ACTIVE         |
    | aff9287d-22dd-43d9-b707-afd6257ac70e | VPNA    | da717c389f494c3e8e6d32f46142d610 | f0906c65-e9b2-447c-ad25-73fecb7fd093 | PENDING_CREATE |
    | e64a8cac-3b3e-46c0-9154-69b42b162d36 | eg-VPNA | da717c389f494c3e8e6d32f46142d610 | 704fcac7-57d3-4fca-a21e-1b838a440f2d | ACTIVE         |
    | f883f3f5-66be-414a-b90f-32f84aa77655 | eg-VPNB | da717c389f494c3e8e6d32f46142d610 | d8a62641-a84f-4d3d-a48f-ead48d85e701 | ACTIVE         |
    +--------------------------------------+---------+----------------------------------+--------------------------------------+----------------+
    stack@vpn-test2:~$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name              | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | 34fd710c-c989-453c-ab30-43e35300ec76 | eg-vpnconnection1 | da717c389f494c3e8e6d32f46142d610 | 172.24.4.17  | psk       | ACTIVE |
    | 429e0919-6e80-457e-a8d7-6f28f2d0df77 | eg-vpnconnection2 | da717c389f494c3e8e6d32f46142d610 | 172.24.4.15  | psk       | ACTIVE |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+

  7. 登陆到创建的vm，验证vm1，vm2，vm3，vm4之间的连通性，发现它们之间是互相通的

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    stack@vpn-test2:~$ nova list
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | ID                                   | Name     | Status | Task State | Power State | Networks                |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    | c6c3a388-fe16-424a-8a68-0899b7578977 | eg-vm-1  | ACTIVE | -          | Running     | eg-network-1=10.3.0.3   |
    | 2dd3e32c-49f5-4db9-9271-a880eb7c6806 | eg-vm-2  | ACTIVE | -          | Running     | eg-network-1=10.4.0.4   |
    | f5040c29-2904-4318-879f-05ce51940c37 | eg-vm-3  | ACTIVE | -          | Running     | eg-network-2=10.5.0.5   |
    | 384c7696-a673-4039-b2aa-363199327c92 | eg-vm-4  | ACTIVE | -          | Running     | eg-network-2=10.6.0.6   |
    | 70b959d7-3bc6-485a-b51c-07711b67757e | vpn-vm-1 | ACTIVE | -          | Running     | vpn-network-1=10.1.0.12 |
    | e8e150dd-4a98-40df-ab35-28b3a791c1be | vpn-vm-2 | ACTIVE | -          | Running     | vpn-network-2=10.2.0.3  |
    +--------------------------------------+----------+--------+------------+-------------+-------------------------+
    stack@vpn-test2:~$ openstack router list
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | ID                                   | Name         | Status | State | Distributed | HA    | Project                          |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    | 704fcac7-57d3-4fca-a21e-1b838a440f2d | eg-router-1  | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    | bdcb18cf-4b83-47df-b0d5-b424eaeda1a6 | router1      | ACTIVE | UP    | False       | False | 9885c5369b824e81b01d0e5444f379f7 |
    | d8a62641-a84f-4d3d-a48f-ead48d85e701 | eg-router-2  | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    | f0906c65-e9b2-447c-ad25-73fecb7fd093 | vpn-router-1 | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    | faf5fb10-8662-4862-bcb4-b3cee3c603be | vpn-router-2 | ACTIVE | UP    | False       | False | da717c389f494c3e8e6d32f46142d610 |
    +--------------------------------------+--------------+--------+-------+-------------+-------+----------------------------------+
    stack@vpn-test2:~$ sudo ip netns exec qrouter-704fcac7-57d3-4fca-a21e-1b838a440f2d ssh cirros@10.3.0.3
    cirros@10.3.0.3's password:
    Permission denied, please try again.
    cirros@10.3.0.3's password:
    $ ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
        inet6 ::1/128 scope host
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
        link/ether fa:16:3e:9a:10:1c brd ff:ff:ff:ff:ff:ff
        inet 10.3.0.3/24 brd 10.3.0.255 scope global eth0
        inet6 fe80::f816:3eff:fe9a:101c/64 scope link
           valid_lft forever preferred_lft forever
    $ ping 10.5.0.5
    PING 10.5.0.5 (10.5.0.5): 56 data bytes
    64 bytes from 10.5.0.5: seq=0 ttl=62 time=5.781 ms
    64 bytes from 10.5.0.5: seq=1 ttl=62 time=1.803 ms
    64 bytes from 10.5.0.5: seq=2 ttl=62 time=1.809 ms
    64 bytes from 10.5.0.5: seq=3 ttl=62 time=1.707 ms
    64 bytes from 10.5.0.5: seq=4 ttl=62 time=1.300 ms
    64 bytes from 10.5.0.5: seq=5 ttl=62 time=1.342 ms
    64 bytes from 10.5.0.5: seq=6 ttl=62 time=1.263 ms
    64 bytes from 10.5.0.5: seq=7 ttl=62 time=1.409 ms
    64 bytes from 10.5.0.5: seq=8 ttl=62 time=1.356 ms
    ^C
    --- 10.5.0.5 ping statistics ---
    9 packets transmitted, 9 packets received, 0% packet loss
    round-trip min/avg/max = 1.263/1.974/5.781 ms
    $ ping 10.6.0.6
    PING 10.6.0.6 (10.6.0.6): 56 data bytes
    64 bytes from 10.6.0.6: seq=0 ttl=62 time=6.258 ms
    64 bytes from 10.6.0.6: seq=1 ttl=62 time=2.096 ms
    64 bytes from 10.6.0.6: seq=2 ttl=62 time=1.644 ms
    64 bytes from 10.6.0.6: seq=3 ttl=62 time=1.441 ms
    64 bytes from 10.6.0.6: seq=4 ttl=62 time=1.516 ms
    64 bytes from 10.6.0.6: seq=5 ttl=62 time=1.546 ms
    64 bytes from 10.6.0.6: seq=6 ttl=62 time=2.268 ms
    64 bytes from 10.6.0.6: seq=7 ttl=62 time=2.222 ms
    ^C
    --- 10.6.0.6 ping statistics ---
    8 packets transmitted, 8 packets received, 0% packet loss
    round-trip min/avg/max = 1.441/2.373/6.258 ms
    $ ping 10.4.0.4
    PING 10.4.0.4 (10.4.0.4): 56 data bytes
    64 bytes from 10.4.0.4: seq=0 ttl=63 time=10.127 ms
    64 bytes from 10.4.0.4: seq=1 ttl=63 time=1.999 ms
    64 bytes from 10.4.0.4: seq=2 ttl=63 time=1.438 ms
    64 bytes from 10.4.0.4: seq=3 ttl=63 time=1.125 ms
    64 bytes from 10.4.0.4: seq=4 ttl=63 time=1.217 ms
    64 bytes from 10.4.0.4: seq=5 ttl=63 time=1.245 ms
    64 bytes from 10.4.0.4: seq=6 ttl=63 time=1.202 ms
    64 bytes from 10.4.0.4: seq=7 ttl=63 time=1.120 ms
    ^C
    --- 10.4.0.4 ping statistics ---
    8 packets transmitted, 8 packets received, 0% packet loss
    round-trip min/avg/max = 1.120/2.434/10.127 ms

  8. 删除ipsec-site-connection连接，另一个ipsec-site-connection连接down掉。再测试连通性，发现vm1与eg-network-2下面的vm3、vm4已无法ping通，而与vm2能ping通。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    stack@vpn-test2:~$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name              | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | 34fd710c-c989-453c-ab30-43e35300ec76 | eg-vpnconnection1 | da717c389f494c3e8e6d32f46142d610 | 172.24.4.17  | psk       | ACTIVE |
    | 429e0919-6e80-457e-a8d7-6f28f2d0df77 | eg-vpnconnection2 | da717c389f494c3e8e6d32f46142d610 | 172.24.4.15  | psk       | ACTIVE |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    stack@vpn-test2:~$ neutron ipsec-site-connection-delete 34fd710c-c989-453c-ab30-43e35300ec76
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    Deleted ipsec_site_connection(s): 34fd710c-c989-453c-ab30-43e35300ec76
    stack@vpn-test2:~$ neutron ipsec-site-connection-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | id                                   | name              | tenant_id                        | peer_address | auth_mode | status |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    | 429e0919-6e80-457e-a8d7-6f28f2d0df77 | eg-vpnconnection2 | da717c389f494c3e8e6d32f46142d610 | 172.24.4.15  | psk       | DOWN   |
    +--------------------------------------+-------------------+----------------------------------+--------------+-----------+--------+
    stack@vpn-test2:~$ sudo ip netns exec qrouter-704fcac7-57d3-4fca-a21e-1b838a440f2d ssh cirros@10.3.0.3
    cirros@10.3.0.3's password:
    $ ping 10.5.0.5
    PING 10.5.0.5 (10.5.0.5): 56 data bytes
    ^C
    --- 10.5.0.5 ping statistics ---
    10 packets transmitted, 0 packets received, 100% packet loss
    $ ping 10.6.0.6
    PING 10.6.0.6 (10.6.0.6): 56 data bytes
    ^C
    --- 10.6.0.6 ping statistics ---
    6 packets transmitted, 0 packets received, 100% packet loss
    $ ping 10.4.0.4
    PING 10.4.0.4 (10.4.0.4): 56 data bytes
    64 bytes from 10.4.0.4: seq=0 ttl=63 time=4.499 ms
    64 bytes from 10.4.0.4: seq=1 ttl=63 time=1.958 ms
    64 bytes from 10.4.0.4: seq=2 ttl=63 time=1.169 ms
    64 bytes from 10.4.0.4: seq=3 ttl=63 time=1.189 ms
    64 bytes from 10.4.0.4: seq=4 ttl=63 time=1.290 ms
    64 bytes from 10.4.0.4: seq=5 ttl=63 time=1.231 ms
    ^C
    --- 10.4.0.4 ping statistics ---
    6 packets transmitted, 6 packets received, 0% packet loss
    round-trip min/avg/max = 1.169/1.889/4.499 ms
结果与分析

  • 查看agent，没有单独的neutron-vpn-agent，说明现在L3 agent起了vpn agent的作用

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    stack@vpn-test2:~$ neutron agent-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +--------------------------------------+--------------------+-----------+-------------------+-------+----------------+---------------------------+
    | id                                   | agent_type         | host      | availability_zone | alive | admin_state_up | binary                    |
    +--------------------------------------+--------------------+-----------+-------------------+-------+----------------+---------------------------+
    | 4aaf04b5-a843-4af1-9c75-c5169d9c24a9 | Metadata agent     | vpn-test2 |                   | :-)   | True           | neutron-metadata-agent    |
    | 79de8e88-b5e2-458d-9fd1-f94b19b91524 | DHCP agent         | vpn-test2 | nova              | :-)   | True           | neutron-dhcp-agent        |
    | be88d7e9-44d9-41b3-8da2-29943df76fa6 | Open vSwitch agent | vpn-test2 |                   | :-)   | True           | neutron-openvswitch-agent |
    | dc1ee8b1-2a0c-4da7-b096-c54f7b0e071a | L3 agent           | vpn-test2 | nova              | :-)   | True           | neutron-l3-agent          |
    +--------------------------------------+--------------------+-----------+-------------------+-------+----------------+---------------------------+

  • 查看ipsec driver，dvr和ha都未启用，支持VPN功能实现的ipsec_driver为strongswan。关于HA router对VPNaaS功能的影响还有待进一步测试。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    stack@vpn-test2:~$ neutron service-provider-list
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    +---------------+-------------+---------+
    | service_type  | name        | default |
    +---------------+-------------+---------+
    | L3_ROUTER_NAT | single_node | False   |
    | L3_ROUTER_NAT | ha          | False   |
    | L3_ROUTER_NAT | dvrha       | False   |
    | VPN           | strongswan  | True    |
    | L3_ROUTER_NAT | dvr         | False   |
    | VPN           | strongswan  | True    |
    +---------------+-------------+---------+

  • 当前版本支持传统的指定子网CIDR的vpn连接方式，也支持建立endpoint group的多个本地子网的方式，并且后者是现在和以后推荐使用的。

  • Ubuntu系统支持strongswan类型的ipsec_driver，CentOS系统不支持。

  • 当ipsec-site-connection一旦建立起来后，其连接所有用到的资源（ikepolicy，ipsecpolicy，subnet，endpoint group等）都不能被删除或更新。

    1
    2
    3
    4
    stack@vpn-test2:~$ neutron vpn-ikepolicy-delete 85c27997-ab43-4782-815b-b4de285ae74a
    neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
    IKEPolicy 85c27997-ab43-4782-815b-b4de285ae74a is in use by existing IPsecSiteConnection and can't be updated or deleted
    Neutron server returns request_ids: ['req-afe0c4a0-0775-452a-a4dd-535274456373']

  • 建立endpoint group时指定type时，后面的value必须一致，要么都为subnet，要么都为cidr。

  • 建立ipsec-site-connection连接时双方psk值必须一致匹配。