- 1.OpenStack + KVM = .com VPS OpenStack + KVM = .com VPS OSC
2012 Kyoto: 2012/08/03OpenStack User MTG#07: 2012/08/28 OSC 2012
Tokyo/Fall: 2012/09/08
http://www.slideshare.net/chroum/2012-osc-kyoto-openstack-vps-kvm
11294
2. OpenStack + KVM = .com VPS .com VPS VPS CentOS 6.x +
griddynamics.net OpenStack Diablo VPS OpenStack OSS DevOps
GlusterFS 3.3betanoVNCdisk resizeFreeBSD virtio 21294 3. OpenStack
+ KVM = .com VPS GMO (1) .com VPS VZ(2)
GMOVirtuozzo(OpenVZ)OSswapBest Effort EL6, KVM .com VPS KVMLinux
KVM KVMVPS webLB(Brocade ADX) (3) .com Windows Desktop
Fusion-IOHyper-V, RDP GMO GMO VPS GMO Public (Xen, KVM) KVMPlesk
API31294 4. OpenStack + KVM = .com VPS .com VPS KVM ( native KVM)
ISO uploadOSLive CD boot http/https/ssh virtIO ON/OFF GMO( )
VIF(spoongIP) REST API OpenStack Diablo()(1)41294 5. OpenStack +
KVM = .com VPS .com VPS KVM web (.Net + IIS) Windows Server DB
(mysql) (Linux KVM) websocket-VNC(noVNC) ssh/console sftp rabbitmq
dhcp ldap(dhcp, ssh) nova/keystone DB(mysql) nova Manage/API ISO
upload storage (GlusterFS, NFS) Linux KVM (nova Compute) CentOS
6.xnova Compute, collectd, kvm51294 6. OpenStack + KVM = .com VPS
.com VPS KVM InternetL3 SW L3 SWAPI / Manage web (.Net) API /
Manage mysqldb API/ DBAPI / Manage API/nova Managenova Computenova
Computenova Computenova Computenova Computenova Compute glancenova
Computenova Computenova Computenova Compute sftpconsoleconsole
consolesftpconsoledhcpvnc RAID 10 local diskRAID 10 local disk
mqAPI / Manage mqldapmysqlmqdbnova DB Storagekeystone DBStorage ISO
storage (NFS)61294 7. OpenStack + KVM = .com VPS .com VPS KVM .com
VPS VZ(Virtuozzo) IN/OUTFirewall >> Port80 port, 22 port
(php...) >> >> Linux bridge mail gateway(SPAM) >>
>> VPS KVM FirewallLinux bridge(ebtables) 71294 8. OpenStack
+ KVM = .com VPS .com VPS KVM Flat DHCP + VLAN bridge
(FirewallLinux bridge) GMONet() L3 SW(dhcp relay) L2 SW Rack L2 SW
Linux bonding(act.-stan.) Linux VLAN Linux bridge VM VIF(tap -
virtio NIC) 81294 9. OpenStack + KVM = .com VPS .com VPS KVM
InternetGMO core backboneL3 SWL3 SW L3 SWL2 SWL3 SWdhcp
relaydhcpeth0 eth1consolenova Compute dhcp bond0
bridge.VLAN1libvirt network lterconsole ldap bridge.VLAN2ebtalbes
bridge.VLAN3tc htb (iproute) LDAPVMMACcollectd ADDRESSIPeth0VM eth0
eth0eth0 VMVM VM91294 10. OpenStack + KVM = .com VPS OpenStack
griddynamics.net Diablo CentOS6 CentOS 6.xpythonpython 2.6back port
libvirtCentOS http://yum.griddynamics.net/yum/diablo-centos/ =>
() src.rpm github.spec stable CentOS 6.3OSlibvirt diablo-centos
(EPEL(6) OpenStack) 101294 11. OpenStack + KVM = .com VPS OpenStack
griddynamics.net Diablo CentOS6 w VPS API (# ) => (CentOS 6.x)
111294 12. OpenStack + KVM = .com VPS OpenStack griddynamics.net
Diablo CentOS6 ) griddynamics dashboard > glanceVMVM 121294 13.
OpenStack + KVM = .com VPS OSVPS... ) VMTerminate : ... (virsh
destroy )...>> ACPIshutdownw w python-nova-*.rpm
/usr/lib/python2.6/site-packages/nova/compute/
/usr/lib/python2.6/site-packages/nova/virt/ ...131294 14. OpenStack
+ KVM = .com VPS shutoff(1)libvirt (nova/virt/driver.pyw) (destroy
call) (Essexreboot(SOFT)) Terminate:OFF + VPSvmACPI shutdowncall
shutoff diff -uNr
../usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py
./virt/libvirt/connection.py ---
../usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py
2012-07-31 12:01:28.890335240 +0900 +++
./virt/libvirt/connection.py2012-07-31 10:15:55.186268025 +0900 @@
-1901,3 +1953,63 @@ def set_host_enabled(self, host, enabled):
"""Sets the specified hosts ability to accept new instances."""
pass + + + def shutoff(self, instance, network_info, flag):# GMO
+instance_name = instance[name] + +try: + virt_dom =
self._lookup_by_name(instance_name) +except exception.NotFound: +
virt_dom = None141294 15. OpenStack + KVM = .com VPS shutoff(2) + +
# If the instance is already terminated, were still happy + #
Otherwise, destroy it + if virt_dom is not None: + try:
+virt_dom.shutdown() + except libvirt.libvirtError as e: +is_okay =
False +errcode = e.get_error_code() +if errcode ==
libvirt.VIR_ERR_OPERATION_INVALID: +# If the instance if already
shut off, we get this: +# Code=55 Error=Requested operation is not
valid: +# domain is not running +(state, _max_mem, _mem, _cpus, _t)
= virt_dom.info() +if state == power_state.SHUTOFF: +is_okay = True
+ +if not is_okay: +LOG.warning(_("Error from libvirt during
shutdown of " + "%(instance_name)s. Code=%(errcode)s " +
"Error=%(e)s") % + locals()) +raise + + 151294 16. OpenStack + KVM
= .com VPS shutoff(3) + """Called at an interval until the VM is
gone.""" + timeout_count = range(FLAGS.shutoff_retry_count) + while
timeout_count:# Y.Kawada +try: +state =
self.get_info(instance_name)[state] +except exception.NotFound:
+msg = _("During shutoff, %s disappeared.") % instance_name
+LOG.info(msg) +break +if state == power_state.SHUTOFF: +msg =
_("Instance %s shutoff successfully. stat:%s") % (instance_name,
state) +LOG.info(msg) +break + + timeout_count.pop() +
utils.my_logger(waiting... shutoff pw_state: %s % state) + if
len(timeout_count) == 0: + utils.my_logger("Error shutoff: %s time
over, try to destroy." % instance_name) + +""" try to destroy."""
+msg = _("try destroy. :%s") % (instance_name) +LOG.info(msg)
+self.destroy(instance, network_info, cleanup=False) +break +
time.sleep(1) + + return True161294 17. OpenStack + KVM = .com VPS
) dashboard : vnc(noVNC) >> noVNCEssex(python) >> noVNC
(javascript) noVNC (html5 Canvas + WebSocketvnc(rfb))
OpenStackauthhttp://www.moongift.jp/2011/02/20110219-2/https://github.com/kanaka/noVNChttp://novnc.com/
python-nova-*.rpm /usr/lib/python2.6/site-packages/nova/vnc/ ...
Essex (EssexnoVNCxvp) 171294 18. OpenStack + KVM = .com VPS noVNC
... 181294 19. OpenStack + KVM = .com VPS OSVPS... API Diablo(),
Essex()() computevmlimit Diablolimithost(limit) vcpu, memory
(Essex) try: except: VIF(VMattach) VMVIFIDvm 191294 20. OpenStack +
KVM = .com VPS try:except: )API except catch diff -uNr
../usr/lib/python2.6/site-packages/nova/vnc/auth.py ./vnc/auth.py
--- ../usr/lib/python2.6/site-packages/nova/vnc/auth.py 2012-07-31
12:01:28.900514682 +0900 +++ ./vnc/auth.py2012-07-31
10:15:55.190267491 +0900 @@ -52,7 +52,12 @@if not token:referrer =
req.environ.get(HTTP_REFERER) -auth_params =
urlparse.parse_qs(urlparse.urlparse(referrer).query) + try: +
auth_params = urlparse.parse_qs(urlparse.urlparse(referrer).query)
+ except: + LOG.audit(_("Unauthorized Access: (%s)"), req.environ)
+ return webob.exc.HTTPForbidden(detail=Unauthorized) +if token in
auth_params: token = auth_params[token][0] 201294 21. OpenStack +
KVM = .com VPS VIFVPS) vm( iu1 + - + ID) iu1-000001c8ID(16)vneth0,
eth1 ... >> sufx -0 , -1 ...vn000001c8-0 (tap) >> eth0
(VM)(collectd)(tc)ebtables 211294 22. OpenStack + KVM = .com VPS
VIF ) diff -uNr
../usr/lib/python2.6/site-packages/nova/virt/libvirt/vif.py
./virt/libvirt/vif.py ---
../usr/lib/python2.6/site-packages/nova/virt/libvirt/vif.py
2012-07-31 12:01:28.890335240 +0900 +++
./virt/libvirt/vif.py2012-07-31 10:15:55.186268025 +0900 @@ -38,7
+38,7 @@ class LibvirtBridgeDriver(VIFDriver): """VIF driver for
Linux bridge.""" - def _get_configurations(self, network, mapping):
+ def _get_configurations(self, network, mapping, instance): """Get
a dictionary of VIF configurations for bridge type.""" # Assume
that the gateway also acts as the dhcp server. gateway6 =
mapping.get(gateway6) @@ -59,6 +59,7 @@extra_params = "n" result =
{ + name: instance[name].split(-)[1], # Y.Kawada id: mac_id,
bridge_name: network[bridge], mac_address: mapping[mac], @@ -90,7
+91,7 @@network[bridge],network[bridge_interface]) - return
self._get_configurations(network, mapping) +return
self._get_configurations(network, mapping, instance) def
unplug(self, instance, network, mapping): """No manual unplugging
required."""221294 23. OpenStack + KVM = .com VPS OpenStack: , , ,
OFF >> shutoff call >> shutoff > start API
call(libvirt reboot kvm)231294 24. OpenStack + KVM = .com VPS
Internet.com VPS : API L3 SWL3 SWAPI / Manage web (.Net)API /
Manage mysqldb API/ DBAPI / Manage API/nova Managea Computeova
Compute nova Computeglance nova Compute nova Computesftp
consoleconsole console sftp consoledhcpvncRAID 10 local diskmq API
/ Manage mq ldap mysqlmqdbnova DB 241294Storage keystone DB 25.
OpenStack + KVM = .com VPS OpenStack: html5 vnc (1) OpenStack
dashboardvncnoVNC >> Top >>ajaxAPIw251294 26. OpenStack
+ KVM = .com VPS html5 vnc (2) noVNC websocketCanvas web browser
(IE8 Chrome Frame)261294 27. / DBOpenStack + KVM = .com VPS API /
Manage API/nova Managhtml5 vnc (3)g
sftpconsoleconsoleconsolesftpconsoledhcp vnck mqAPI / Manage
mqldapmysqlmqdb nova DBekeystone DBgee271294 28. OpenStack + KVM =
.com VPS html5 vnc (3): nginxwebsocketreverse proxy + dnsnginx
websocket reverse proxy patch(nginx
1.3)https://github.com/yaoweibin/nginx_tcp_proxy_modulePort
6080vncproxyPort 80/443[n-gohko@vnc1001 ~]$ ps axu | grep nova |
grep -v greproot 15509 0.4 2.0 451048 81164 ? S Jul11 152:13
/usr/bin/python /usr/bin/nova-vncproxy
--flagfile=/etc/nova/nova.conf
--logfile=/var/log/nova/nova-vncproxy.log
--pidfile=/var/run/nova/nova-vncproxy.pid[n-gohko@vnc1001 ~]$ ps
axu | grep nginx | grep -v greproot 1303 0.0 0.0 93188 1492 ? Ss
Jun26 0:00 nginx: master process /usr/sbin/nginx -c
/etc/nginx/nginx.confnginx 1304 0.1 0.0 94100 3472 ? S Jun26 54:44
nginx: worker process 281294 29. OpenStack + KVM = .com VPS html5
vnc (3): nginx websocket reverse proxytcp {proxy_read_timeout
6000;proxy_send_timeout 6000;upstream websockets { server
172.20.113.31:6080;#check interval=3000 rise=2 fall=5
timeout=1000;check_http_send "GET / HTTP/1.0rnrn";}server { #listen
6081; server_name _; listen 443;access_log
/var/log/nginx/access.ssl.log;ssl on;ssl_certificate
/etc/nginx/ssl/ast.myvps.jp/ast.myvps.jp.pem;ssl_certificate_key
/etc/nginx/ssl/ast.myvps.jp/ast.myvps.jp.key;ssl_protocolsSSLv3
TLSv1;ssl_ciphers HIGH:!ADH:!MD5;tcp_nodelay on;proxy_pass
websockets;}} 291294 30. OpenStack + KVM = .com VPS :
collectdcollectd-virtEPEL6collectd-virtcollectd-rrdtool(rrd)
libvirt nova-computevmrrdrrd301294 31. OpenStack + KVM = .com VPS :
collectd(rrdmysqlcsv)[root@cnode-a1001 novaclient]# rpm -qa | grep
collectdcollectd-4.10.3-1.el6.x86_64collectd-web-4.10.3-1.el6.x86_64collectd-rrdtool-4.10.3-1.el6.x86_64collectd-virt-4.10.3-1.el6.x86_64311294
32. OpenStack + KVM = .com VPS html5 serial console: anyterm +
conservernova-console(ajax-term)anyterm(OpenStack
Diabloajax-term)http://anyterm.org/anytermweb (anyterm -->
console)(OpenStack Essexajax-term)Pointpop-upURL [Enter `^Ec? for
help]iu1-000007ff 321294 33. OpenStack + KVM = .com VPS html5
serial console: anyterm token
https://console1001.myvps.jp/anyterm/proxy/11048/
anyterm.html?token=bb268280-8e4e-41c1-894f-
ac325079530f&title=iu1-000007ff nova-consolehttpd apache
reverse proxy (Port 80/443)[root@console1001 ~]# ps -aef | grep
httpd | grep -v greproot 1523 1 0 Jul09 ? 00:00:38
/usr/sbin/httpdapache 3385 1523 0 Jul29 ?00:00:04
/usr/sbin/httpdapache 3386 1523 0 Jul29 ?00:00:04
/usr/sbin/httpdapache 3387 1523 0 Jul29 ?00:00:04
/usr/sbin/httpdapache 23084 1523 0 Jul31 ?00:00:03
/usr/sbin/httpdapache 26971 1523 0 Aug01 ? 00:00:02
/usr/sbin/httpd[root@console1001 ~]# netstat -antp | grep
pythontcp0 0 172.20.113.36:33841 172.20.113.54:3306 ESTABLISHED
2712/pythontcp0 0 172.20.113.36:44081 172.20.113.32:5672
ESTABLISHED 2712/pythontcp0 0 172.20.113.36:44084
172.20.113.32:5672 ESTABLISHED 2712/python 331294 34. OpenStack +
KVM = .com VPS ssh serial console: ldap ssh + conserverssh serial
consolessh loginshellconsolesshvmanytermssh serial console [Enter
`^Ec? for help]conservervirsh console 341294 35. OpenStack + KVM =
.com VPS ISO upload: ldap ssh(sftp) chroot + NFSldap sshconsolesftp
upload (openssh 5.4p1patch)NFS storagechroot(NFS chroot)>>
proftpdmod_sftpquota 351294 36. OpenStack + KVM = .com VPS ISO
upload: ldap ssh(sftp) chroot + NFSNFS serverGlusterFS () GlusterFS
3.3beta ()brick...GlusterFS quotaquota>> quota
checkGlusterFSlockquotaISOGlusterFS...(GlusterFS 3.3 ) 361294 37.
OpenStack + KVM = .com VPS OSS: OS Nagios 3.3.1 (EPEL6)Hardware IBM
Director + HP SIM (Freenot OSS)HPg7OSagent()HPgen8age: cobbler
(kickstart network install) (EPEL6)cobblerhostdhcp, dns,
Nagioscobblertrigger(IBM Director, HP SIM) 371294 38. OpenStack +
KVM = .com VPS cobbler triggernagios [n-gohko@v2cob1001 libvirt]$
find /var/lib/cobbler/triggers/ /var/lib/cobbler/triggers/
/var/lib/cobbler/triggers/sync /var/lib/cobbler/triggers/sync/post
/var/lib/cobbler/triggers/sync/post/201-post-chmod-644-tftpboot-kick.sh
/var/lib/cobbler/triggers/sync/post/nagios-add-hosts.sh
/var/lib/cobbler/triggers/sync/post/301-post-chmod-644-tftpboot-kick.sh
/var/lib/cobbler/triggers/sync/pre /var/lib/cobbler/triggers/delete
/var/lib/cobbler/triggers/delete/file
/var/lib/cobbler/triggers/delete/file/post
/var/lib/cobbler/triggers/delete/file/pre
/var/lib/cobbler/triggers/delete/distro
/var/lib/cobbler/triggers/delete/distro/post
/var/lib/cobbler/triggers/delete/distro/pre
/var/lib/cobbler/triggers/delete/mgmtclass
/var/lib/cobbler/triggers/delete/mgmtclass/post
/var/lib/cobbler/triggers/delete/mgmtclass/pre
/var/lib/cobbler/triggers/delete/package
/var/lib/cobbler/triggers/delete/package/post
/var/lib/cobbler/triggers/delete/package/pre
/var/lib/cobbler/triggers/delete/system
/var/lib/cobbler/triggers/delete/system/post
/var/lib/cobbler/triggers/delete/system/pre
/var/lib/cobbler/triggers/delete/profile ...381294 39. OpenStack +
KVM = .com VPS DevOps: 1) GlusterFS 3.3betagluser.org>>
>> 391294 40. OpenStack + KVM = .com VPS DevOps: 2) virt
resizedisk image 20GB + disk>> GlanceOS(CentOS 6.2)>>
GMO >> LVMdisk(10%) imageresizefsckOS OS >> image
401294 41. OpenStack + KVM = .com VPS DevOps: 3) vncGoogle chrome
dev 20.xnoVNC(((())))noVNCupdatechrome 20.xnoVNC(html5) 411294 42.
OpenStack + KVM = .com VPS DEMO: DNSname base DNSsub domain NS
Internet sub domain NS MacBook-Air:$ dig ns d.myvps.jpwww
;d.myvps.jp.IN NS MacBook-Air:$ dig www.d.myvps.jp @d-ns01.myvps.jp
;; ANSWER SECTION:;; QUESTION SECTION: d.myvps.jp.5 INNS
d-ns01.myvps.jp.;www.d.myvps.jp.IN A d.myvps.jp.5 INNS
d-ns02.myvps.jp. ;; ANSWER SECTION: www.d.myvps.jp. 5 INA
157.7.129.212 ;; ADDITIONAL SECTION: d-ns01.myvps.jp. 86400 IN A
157.7.129.212 MacBook-Air:$ dig www.d.myvps.jp @d-ns02.myvps.jp
d-ns02.myvps.jp. 86400 IN A 157.7.129.213 ;; QUESTION SECTION:
;www.d.myvps.jp.IN ADNS;; ANSWER SECTION: www.d.myvps.jp. 5 INA
157.7.129.213 DNS + DNS +web serverweb server L7(reverse proxy)
L7(reverse proxy) d-ns01. d-ns02.w01.w02. 157.7.129.212
157.7.129.213157.7.129.214 157.7.129.215 reverse proxyw01, w02
421294 43. OpenStack + KVM = .com VPS (1)200 > compute
nodeDiabloOpenStack DiabloVPSOpenStack Hypervisor
Driver(Essex)VPSsimple(m(_;))m )IP(Floting IP)IPv6>> >>
vipIPNIC 431294 44. OpenStack + KVM = .com VPS (2)
http://b.hatena.ne.jp/articles/201206/9110 441294 45. OpenStack +
KVM = .com VPS (2) vnc 451294 46. OpenStack + KVM = .com VPS
(3)qemu-img convertCentOS 6.3, Essex? 2nd Unit(OpenStackversion
upHosting)CentOS 6.x + EPEL Essex()API ==> VPS Stack ()(LB)
461294 47. OpenStack + KVM = .com VPS Special Thanks!GMO Internet
Inc., System Div., SI Team and Cloud Team Yamada san 471294