OpenStackネットワーク入門 Midokura Japan Takaaki Suzuki
OpenStackネットワーク入門
Midokura JapanTakaaki Suzuki
Agenda
- History of Neutron- Nova Network- Quantum- Neutron- Packet Flow- Neutron and SDN
History of Neutron...
History of Neutron...
● In 2010, OpenStack debut with its monolithic networking model → Nova Network
2010
Nova
Network
History of Neutron...
● Nova Networkの機能o L2o DHCP
2010
Nova
Network
History of Neutron...
● In 2011, pluggable networking model was proposed to the community → Quantum
Nova
NetworkQuantum
OVS MidoNet…...OVS plugin and Vendor plugins
2010 2011
History of Neutron...● Quantum
o L3/L2o DHCPo FloatingIPo Secutiry Group
2011
Quantum
History of Neutron...
● In 2013, OpenStack Foundation received a notice that Quantum was a trademark and they had to change the name → Neutron
Nova
NetworkQuantum
OVS MidoNet…...
OVS plugin and Vendor plugins
Neutron
OVS MidoNet…...
20112010 2013
History of Neutron...
● Neutron (Juno)o L3/L2o DHCPo FloatingIPo Secutiry Group
Neutron
o LBaaS
o FWaaS
o VPNaaS
o DVR(分散ルーティング)
o L3HA
Server
eth0
LinuxBridge
VM01
192.168.10.2/24
Nova Network (Flat DHCP Manager)
L2 Switch Vlan1
nova-network
nova-compute
VM02
192.168.10.3/24
dnsmasq
Server
eth0
LinuxBridge
VM01
192.168.10.2/24
Nova Network (VLAN Manager)
L2 Switch
Trunk
nova-network
nova-compute
VM02
192.168.10.3/24
dnsmasqLinuxBridge
dnsmasq
VM01
172.16.0.2/24
VM02
172.16.0.2324
VLAN100 VLAN200
VLAN100
VLAN200
Server
iptables PRE-POST Routing
(DNAT,SNAT)
eth0
VM01
192.168.10.2/24
L2 Switch
nova-network
nova-compute
VM02
192.168.10.3/24
Vlan1
Nova Network (Routing NAT)
DNAT 119.15.120.4 : 172.16.0.2
SNAT any: 119.15.120.3
eth1
LinuxBridge
dnsmasq
Server
iptables PRE-POST Routing
(DNAT,SNAT)
Server
eth0
VM01
192.168.10.2/24
L2 Switch
nova-compute
VM02
192.168.10.3/24
VLAN1
Nova Network (Routing NAT)
DNAT 119.15.120.4 : 172.16.0.2
SNAT any: 119.15.120.3
eth1
LinuxBridge
dnsmasq
eth0
nova-network
GW: 192.168.10.1
Nova Network
● Linux Bridgeやiptablesを使う.
● ネットワークを分ける為に
VLANを利用している.
● L3ルーティングは外部機器におまかせ.
● 枯れた技術を好む人もいる.
Compute
ovs-agent
Quantum
Controller
Quantum-server
Compute
ovs-agent
Compute
quantum
openvswitch
plugin-agent
Network
quantum-l3-agent
quantum-dchp-agent
quantum-metadata-agent
quantum-openvswitch-plugin-
agent
OpenvSwitch
Plugin
Quantum Server (API)
Controller
Quantum-server
Quantum Server (API)● Plugin形式で様々なPluginを1つ選べる
Controller
Quantum-server
OpenvSwitch
Plugin
MidoNet
PluginLinuxBridge
Plugin
Quantum L2 agent
Compute
quantum
openvswitch
plugin-agent
Quantum L2 agent
● VM用の仮想L2設定自動的に設定してくれる.
Compute
quantum
openvswitch
plugin-agent
VM
VM
VM
VM
VM
VM
Quantum L3/DHCP/Metadata agent
Network
quantum-l3-agent
quantum-dchp-agent
quantum-metadata-agent
quantum-openvswitch-plugin-
agent
Quantum L3/DHCP/Metadata agent
● DHCP、インターネット外部接続
● VM用メタデータの提供
(hostname, SSH鍵等)
Network
quantum-l3-agent
quantum-dchp-agent
quantum-metadata-agent
quantum-openvswitch-plugin-
agent
Compute
ovs-agent
Quantum
Controller
Quantum-server
Compute
ovs-agent
Compute
quantum
openvswitch
plugin-agent
Network
quantum-l3-agent
quantum-dchp-agent
quantum-metadata-agent
quantum-openvswitch-plugin-
agent
OpenvSwitch
Plugin
Quantum
・Agentを使って様々な機能を提供する.・Plugin形式により様々な技術を利用できる.・NetoworkノードはL3機能(Routing, NAT)
ComputeノードではL2機能が利用できる.
Quantum -> Neutron
Compute
ovs-agent
Neutron
Controller
neutron-server
Compute
ovs-agent
Compute
neutron
openvswitch
agent
Network
neutron-l3-agent
neutron-dchp-agent
neutron-metadata-agent
neutron-openvswitch-agent
OpenvSwitch
Plugin
Compute
ovs-agent
Neutron (Juno)
Controller
neutron-server
Compute
ovs-agent
Compute
neutron
openvswitch
agent
Network
neutron-l3-agent (FWaaS)
neutron-dchp-agent
neutron-metadata-agent
neutron-openvswitch-agent
neutron-lbaas-agent
neutron-vpn-agent ML2
Plugin
Neutron Multi Layer 2 Plugin
Controller
neutron-server
ML2
Plugin
Neutron Multi Layer 2 Plugin
Controller
neutron-server
ML2
Plugin
Type Driver
GRE
VXLAN
FLAT
Mechanism
Driver
OpenvSwitch
Cisco
Arista
Neutron Multi Layer 2 Plugin
Controller
neutron-server
ML2
Plugin
Type Driver
GRE
VXLAN
FLAT
Mechanism
Driver
OpenvSwitch
Cisco
Arista
Neutron Multi Layer 2 Plugin
Controller
neutron-server
ML2
Plugin
Type Driver
GRE
VXLAN
FLAT
Mechanism
Driver
OpenvSwitch
LinuxBridge
Cisco
Compute
neutron
openvswitch
agent
Compute
neutron
cisco
agent
Neutron Multi Layer 2 Plugin
Controller
neutron-server
ML2
Plugin
Type Driver
GRE
VXLAN
FLAT
Mechanism
Driver
OpenvSwitch
LinuxBridge
Cisco
Compute
neutron
openvswitch
agent
Compute
neutron
cisco
agent
VM
VM
VM
VM
VM
VM
VM
VM
様々なネットワーク技術を複数組合せて利用できる.
Network
neutron-l3-agent (FWaaS)
neutron-dchp-agent
neutron-metadata-agent
neutron-openvswitch-agent
neutron-lbaas-agent
neutron-vpn-agent
Neutron (Juno)
Neutronでユーザーができること
Neutronでユーザーができること
Neutronでユーザーができること
1. 仮想ネットワーク(L2)の作成
① ① ①
Neutronでユーザーができること
2. 仮想L3ルータの作成
neutron router-create
① ① ①
② ②
3. ネットワークとルータの接続(router-interface-add)
Neutronでユーザーができること
① ① ①
② ②
③ ③ ③
Neutronでユーザーができること
4. ルータと外部ネットワークの接続(router-gateway-set)
① ① ①
② ②
③ ③ ③
④ ④
Neutronでユーザーができること
5. FloatingIP(グローバルIP DNAT/SNAT)をVMに付与できる.① ① ①
② ②
③ ③ ③
④ ④
⑤⑤ ⑤ ⑤
Neutronでユーザーができること
6. VMのポートに対してSecurityGroupを作成しSSHやHTTPの通信を許可等を適用することができる.
① ① ①
② ②
③ ③ ③
④ ④
⑤⑤ ⑤ ⑤⑥ ⑥ ⑥ ⑥ ⑥ ⑥
Neutronでユーザーができること
① ① ①
② ②
③ ③ ③
④ ④
⑤⑤ ⑤ ⑤⑥ ⑥ ⑥ ⑥ ⑥ ⑥
*外部ネットワーク作成は管理者(admin権限)のみ
Neutron環境のパケットフロー
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM VM VM
eth0
VM VM
qbr43d***qbr5db***qbr5db*** qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron
openvswitch
agent
eth0VXLAN
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM VM VM
eth0
VM VM
qbr43d***qbr5db***qbr5db*** qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron
openvswitch
agent
eth0VXLAN
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM VM VM
eth0
VM VM
qbr43d***qbr5db***qbr5db*** qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron
openvswitch
agent
eth0VXLAN
tap interface
veth interface
LinuxBridge
ovs bridge
ovs bridge
vxlan-port
patch
tap interface
LinuxBridge
veth interface
ovs bridge
ovs bridge
patch
vxlan-port
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
br-int
VXLAN
DHCP
namespace
br-ex
router
namespace
Networknode
eth1eth0
neutron-l3-agent
neutron-openvswitch-agent
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
br-int
VXLAN
DHCP
namespace
br-ex
router
namespace
Networknode
eth1eth0
neutron-l3-agnet
neutron-openvswitch-agent
Network Namespace
Network Namespace
Network Namespace
ip netnsip netns exec qrouter-*** ip linkip netns exec qrouter-*** netstat -nr
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
eth1
neutron-l3-agnet
neutron-openvswitch-agent
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
eth1
GigabitEthenet0/0
119.15.120.129neutron-l3-agnet
neutron-openvswitch-agent
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
router namespaceルーティングテーブル192.168.73.0 255.255.255.0 qr-4894a511-cf CONNECTED
192.168.83.0 255.255.255.0 qr-5ad834a4-df CONNECTED
eth1
neutron-l3-agnet
neutron-openvswitch-agent
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
router namespaceルーティングテーブル192.168.73.0 255.255.255.0 qr-4894a511-cf CONNECTED
192.168.83.0 255.255.255.0 qr-5ad834a4-df CONNECTED
119.15.120.130 255.255.255.128 qg-ad4adf-1e CONNECTED
0.0.0.0 0.0.0.0 119.15.120.129 qg-ad4adf-1e STATIC
eth1
GigabitEthenet0/0
119.15.120.129
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
qg-ad4adf22-1e: 119.120.130
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
eth1
neutron-l3-agnet
neutron-openvswitch-agent
router
namespace
router
namespace
router
namespace
ルーターnamespaceがどんどん作成される
Neutron環境のパケットフロー
br-tun
br-int
br-ex
router
namespace
Networknode
eth1
neutron-l3-agnet
neutron-openvswitch-agent
router
namespace
router
namespace
router
namespace
router
namespacerouter
namespace
router
namespace router
namespace
router
namespacerouter
namespace
router
namespace
router
namespace router
namespacerouter
namespacerouter
namespace router
namespacerouter
namespace
router
namespace
router
namespacerouter
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespace
router
namespacerouter
namespace
router
namespace
ユーザーからルーターを定義すればするほどネットワークノードがボトルネックに...
Distributed Virtual Router
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agenteth0VXLAN
router
namespace
router
namespace
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agenteth0VXLAN
router
namespace
router
namespace
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agenteth0VXLAN
router
namespace
eth1 eth1
router
namespace
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agenteth0VXLAN
router
namespace
router
namespace
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
router namespaceインターフェースqr-4894a511-cf: 192.168.73.1
qr-5ad834a4-df: 192.168.83.1
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agent
eth0VXLAN
router
namespace
FloatingIP
namespace
br-ex
eth1 eth1
router
namespace
br-ex
FloatingIP
namespace
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agent
eth0VXLAN
router
namespace
FloatingIP
namespace
br-ex
eth1 eth1
router
namespace
br-ex
FloatingIP
namespace
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***qbr43d***
VM VM
br-tun
br-int
br-tun
br-int
Compute
neutron-openvswitch-agent
neutron-l3-agent
eth0VXLAN
router
namespace
FloatingIP
namespace
br-ex
eth1 eth1
router
namespace
br-ex
FloatingIP
namespace
Neutron環境のパケットフロー (DVR)
Compute
neutron-openvswitch-agent
neutron-l3-agent
VMVM
eth0
qbr5db***
br-tun
router
namespace
VXLAN
br-tunbr-ex
SNAT
namespace
Networknode
eth1
neutron-l3-agnet
neutron-openvswitch-agent
eth0
eth1
br-ex
FloatingIP
namespacebr-int
br-int
L3HA
L3HA OVS Network Node
● Active/Standby deployment with VRRP● Up to two Network Nodes
Network Node
(Active)
Network Node
(Standby)
L3 agentL3 agent
Ext Switch Ext Switch
VRRP
DHCP and MetaData
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
br-int
VXLAN
DHCP
namespace
Networknode eth2eth0
neutron-l3-agnet
neutron-openvswitch-agent
neutron-dhcp-agent
neutron-metadata-agent
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
VXLAN
DHCP
namespace
Networknode eth2eth0
neutron-l3-agnet
neutron-openvswitch-agent
neutron-dhcp-agent
neutron-metadata-agent
dnsmasq
br-int
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
VXLAN
DHCP
namespace
Networknode eth2eth0
neutron-l3-agnet
neutron-openvswitch-agent
neutron-dhcp-agent
neutron-metadata-agent
dnsmasq
br-int
metadataproxy
metadataagent
Novaapi
Networknode
Neutron環境のパケットフロー
Compute
neutron
openvswitch
agent
VMVM
eth0
VM VM
qbr43d***qbr5db***
br-tun
br-int
br-tun
br-int
VXLAN
br-ex
router
namespace
eth0
neutron-l3-agent
neutron-openvswitch-agent
DHCP
namespace metadataproxy
metadataagent
Novaapi
eth2
NeutronとSDN製品
OpenStack Kilo (Neutron)
● 4/30リリース
Brocade ML2 driver for MLX and ICX switches
Brocade L3 routing plugin for MLX switch
Brocade Vyatta vRouter L3 Plugin
Brocade Vyatta vRouter Firewall Driver
Brocade Vyatta vRouter VPN Driver
Cisco Nexus
http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/data_sheet_c78-727737.html
Contrail
http://www.slideshare.net/natiueno/contrail-overview-open-stack-days-tokyofeb2015
IBM SDN-VE
http://www-06.ibm.com/systems/jp/networking/software/sdnve/
MidoNet
http://midonet.org/midonet-tv.php
MidoNet and Cumulus
Janさんお願いします!