KVM Virtualization Cookbook · 2018-11-02 · administering large-scale, highly available Linux ... Installing and configuring the OpenStack Neutron networking service Getting ...

TitlePageKVMVirtualizationCookbookLearnhowtoeffectivelyuseKVMinproductionKonstantinIvanov

BIRMINGHAM-MUMBAI

Copyright

KVMVirtualizationCookbook

Copyright©2017PacktPublishing

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:June2017

Productionreference:1140617

PublishedbyPacktPublishingLtd.LiveryPlace35LiveryStreetBirminghamB32PB,UK.

ISBN978-1-78829-467-6

www.packtpub.com

http://www.packtpub.com

Credits

Author

KonstantinIvanov

CopyEditor

DiptiMankame

Reviewer

JayPayne

ProjectCoordinator

JudieJose

AcquisitionEditor

RahulNair

Proofreader

SafisEditing

ContentDevelopmentEditor

DevikaBattike

Indexer

RekhaNair

TechnicalEditor

PrachiSawant

Graphics

KirkD'Penha

ProductionCoordinator

AparnaBhagat

AbouttheAuthorKonstantinIvanovisaLinuxsystemsengineer,anopensourcedeveloper,andatechnologybloggerwhohasbeendesigning,configuring,deploying,andadministeringlarge-scale,highlyavailableLinuxenvironmentsformorethan15years.

Hisinterestsincludelargedistributedsystemsandtaskautomation,alongwithsolvingtechnicalchallengesinvolvingmultipletechnologystacks.

KonstantinreceivedtwomasterofscienceincomputersciencedegreesfromuniversitiesinBulgariaandtheUnitedStates,specializinginsystemandnetworksecurityandsoftwareengineering.

Inhissparetime,heloveswritingtechnologyblogsandspendingtimewithhistwoboys.

Tomyamazingwife,Deepa,mytrustycompanionthroughthemanylongdaysofwriting.

AbouttheReviewerJayPaynehasbeenadatabaseadministrator5atRackspaceforover10years,workingonthedesign,development,implementation,andoperationofstoragesystems.

Previously,Jayworkedonbillingandsupportsystemsforhostingcompanies.Forthelast20years,hehasprimarilyfocusedonthedatalifecycle,fromdatabasearchitecture,administration,operations,andreportingtodisasterrecoveryandcompliance.Hehasdomainexperienceinhosting,finance,billing,andcustomer-supportindustries.

www.PacktPub.comForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatservice@packtpub.comformoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www.packtpub.com/mapt

Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksandvideocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvanceyourcareer.

http://www.PacktPub.com



https://www.packtpub.com/mapt

Whysubscribe?

FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

CustomerFeedbackThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpusimprove,pleaseleaveusanhonestreviewonthisbook'sAmazonpageathttps://www.amazon.com/dp/178829467X.

Ifyou'dliketojoinourteamofregularreviewers,youcane-mailusatcustomerreviews@packtpub.com.WeawardourregularreviewerswithfreeeBooksandvideosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!

https://www.amazon.com/dp/178829467X

TableofContents

PrefaceWhatthisbookcoversWhatyouneedforthisbookWhothisbookisforSections

GettingreadyHowtodoit…Howitworks…There'smore…Seealso

ConventionsReaderfeedbackCustomersupport

DownloadingtheexamplecodeDownloadingthecolorimagesofthisbookErrataPiracyQuestions

1. GettingStartedwithQEMUandKVMIntroductionInstallingandconfiguringQEMU

GettingreadyHowtodoit...Howitworks...

Managingdiskimageswithqemu-imgGettingreadyHowtodoit...Howitworks...There'smore...

PreparingimagesforOSinstallationwithqemu-nbdGettingreadyHowtodoit...Howitworks...

InstallingacustomOSontheimagewithdebootstrapGettingready

Howtodoit...Howitworks...

ResizinganimageGettingreadyHowtodoit...Howitworks...

Usingpre-existingimagesGettingreadyHowtodoit...Howitworks...There'smore...Seealso

Runningvirtualmachineswithqemu-system-*GettingreadyHowtodoit...Howitworks...

StartingtheQEMUVMwithKVMsupportGettingreadyHowtodoit...Howitworks...There'smore...

ConnectingtoarunninginstancewithVNCGettingreadyHowtodoit...Howitworks...

2. UsinglibvirttoManageKVMIntroductionInstallingandconfiguringlibvirt


DefiningKVMinstancesGettingreadyHowtodoit...Howitworks...There'smore...

Starting,stopping,andremovingKVMinstancesGettingready


InspectingandeditingKVMconfigsGettingreadyHowtodoit...Howitworks...

BuildingnewKVMinstanceswithvirt-installandusingtheconsoleGettingreadyHowtodoit...Howitworks...

ManagingCPUandmemoryresourcesinKVMGettingreadyHowtodoit...Howitworks...There'smore...

AttachingblockdevicestovirtualmachinesGettingreadyHowtodoit...Howitworks...

SharingdirectoriesbetweenarunningVMandthehostOSGettingreadyHowtodoit...Howitworks...There'smore...

AutostartingKVMinstancesGettingreadyHowtodoit...Howitworks...

WorkingwithstoragepoolsGettingreadyHowtodoit...Howitworks...There'smore...

ManagingvolumesGettingreadyHowtodoit...Howitworks...

ManagingsecretsGettingready


3. KVMNetworkingwithlibvirtIntroductionTheLinuxbridge

GettingreadyHowtodoit...Howitworks...There'smore...

TheOpenvSwitchGettingreadyHowtodoit...Howitworks...There'smore...

ConfiguringNATforwardingnetworkGettingreadyHowtodoit...Howitworks...

ConfiguringbridgednetworkGettingreadyHowtodoit...Howitworks...

ConfiguringPCIpassthroughnetworkGettingreadyHowtodoit...Howitworks...

ManipulatingnetworkinterfacesGettingreadyHowtodoit...Howitworks...

4. MigratingKVMInstancesIntroductionManualofflinemigrationusinganiSCSIstoragepool


ManualofflinemigrationusingGlusterFSsharedvolumes


OnlinemigrationusingthevirshcommandwithsharedstorageGettingreadyHowtodoit...Howitworks...There'smore...

OfflinemigrationusingthevirshcommandandlocalimageGettingreadyHowtodoit...Howitworks...

OnlinemigrationusingthevirshcommandandlocalimageGettingreadyHowtodoit...Howitworks...

5. MonitoringandBackupofKVMVirtualMachinesIntroductionResourceusagecollectionwithlibvirt


MonitoringKVMinstanceswithSensuGettingreadyHowtodoit...Howitworks...There'smore...

SimpleKVMbackupswithtarandrsyncGettingreadyHowtodoit...Howitworks...

CreatingsnapshotsGettingreadyHowtodoit...Howitworks...

ListingsnapshotsGettingready


InspectingsnapshotsGettingreadyHowtodoit...Howitworks...

EditingsnapshotsGettingreadyHowtodoit...Howitworks...

RevertingsnapshotsGettingreadyHowtodoit...Howitworks...

DeletingsnapshotsGettingreadyHowtodoit...Howitworks...

6. DeployingKVMInstanceswithOpenStackIntroductionPreparingthehostfortheOpenStackdeployment


InstallingandconfiguringtheOpenStackKeystoneidentityserviceGettingreadyHowtodoit...Howitworks...

InstallingandconfiguringtheOpenStackGlanceimageserviceGettingreadyHowtodoit...Howitworks...

InstallingandconfiguringtheOpenStackNovacomputeserviceGettingreadyHowtodoit...Howitworks...

InstallingandconfiguringtheOpenStackNeutronnetworkingserviceGettingreadyHowtodoit...

Howitworks...BuildingandinspectingKVMinstanceswithOpenStack


StoppingKVMinstanceswithOpenStackGettingreadyHowtodoit...Howitworks...

TerminatingKVMinstanceswithOpenStackGettingreadyHowtodoit...Howitworks...

7. UsingPythontoBuildandManageKVMInstancesIntroductionInstallingandusingthePythonlibvirtlibrary


DefiningKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...

Starting,stopping,anddeletingKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...

InspectingKVMinstanceswithPythonGettingreadyHowtodoit...Howitworks...There'smore...

BuildingasimpleRESTAPIserverwithlibvirtandbottleGettingreadyHowtodoit...Howitworks...

There'smore...8. KernelTuningforKVMPerformance

IntroductionTuningthekernelforlowI/Olatency


MemorytuningforKVMguestsGettingreadyHowtodoit...Howitworks...

CPUperformanceoptionsGettingreadyHowtodoit...Howitworks...

NUMAtuningwithlibvirtGettingreadyHowtodoit...Howitworks...Thereismore...

TuningthekernelfornetworkperformanceGettingreadyHowtodoit...Howitworks...

PrefaceThefoundationofmostmodernclouddeploymentsissomesortofvirtualizationtechnology,suchasKernel-basedVirtualMachine(KVM).KVMhasbeenpartofthemainstreamLinuxkernelsinceversion2.6.20,releasedinFebruary2007,andsincethenhasenjoyedwidesystemadoption.

Virtualizationingeneralnotonlyprovidesawaytofullyutilizeserverresources,butalsoallowsgreatermultitenancy,alongwithrunningvariousworkloadsonthesamesystem.

TheOpenStackcloudoperatingsystemusesKVMasitsdefaultcomputedriver,providingacentralizedwayofmanagingthelifecycleofvirtualmachines:frombuilding,resizing,andmigratingtopausingandterminating.

ThisbookisaboutKVMandhowtobuildandmanagevirtualmachinesinthemostefficientway.UnlikecontainerizationsolutionssuchasDocker,KVMisdesignedtorunanentireoperatingsystemratherthanasingleprocess.Althoughcontainerizationhasitsadvantages,fullvirtualizationprovidesextrasecuritybyhavingthehypervisorlayerbetweentheguestOSandthehostandaddedstabilitybyrunningdifferentguestkernels(kernelpanicoftheguestinstancewillnotbringtheentirehostdown)orentirelydifferentoperatingsystems.

Thisbooktakesaratherdirectandpragmaticstep-by-stepapproach--youwilllearnhowtocreatecustomguestimages,installandconfigureQEMUandlibvirt,resizeandmigrateinstances,deploymonitoring,andprovisionguestsusingOpenStackandPython.

WhatthisbookcoversChapter1,GettingStartedwithQEMUandKVM,providesrecipesforinstallingandconfiguringQEMU,creatingandmanagingdiskimages,andrunningvirtualmachineswiththeqemu-systemutility.

Chapter2,UsinglibvirttoManageKVM,coverseverythingthatisneededtoinstall,configure,andrunKVMinstancesusinglibvirt.Youwilllearnwhatpackagesandtoolsarerequired,alongwithdifferentwaysofconfiguringvirtualmachinesusingXMLdefinitionfiles.Bytheendofthischapter,youwillhaveaLinuxsystemwithrunningKVMinstances.

Chapter3,KVMNetworkingwithlibvirt,willpresentrecipesforworkingwiththeLinuxBridgeandOpenvSwitchandwilldemonstratehowtoconnectKVMinstancesusingNAT,bridged,andPCIpass-throughnetworking.

Chapter4,MigratingKVMInstances,willshowexamplesonhowtoperformofflineandonlinemigrationofrunningKVMvirtualmachines.

Chapter5,MonitoringandBackupofKVMVirtualMachines,willpresentexamplesonhowtodeploycompletemonitoringsystemswithSensuandUchiwaanddemonstratehowtocreatesnapshotstouseasbackups.

Chapter6,DeployingKVMInstanceswithOpenStack,demonstrateshowtoprovisionKVMinstanceswithOpenStack.ItbeginsbyintroducingthevariouscomponentsthatmakeOpenStackandhowtousetheLXCNovadrivertoautomaticallyprovisionvirtualmachines.

Chapter7,UsingPythontoBuildandManageKVMInstances,willpresentrecipesforbuilding,starting,andmanagingthelifecycleofKVMinstancesusingthePythonlibvirtlibrary.WewillalsoseeexamplesonhowtobuildasimpleRESTfulAPItoworkwithKVM.

Chapter8,KernelTuningforKVMPerformance,showsrecipesfortuningthehostOSforbetterI/O,CPU,memory,andnetworkutilization.ThepresentedexamplescanalsobeusedinsidetheKVMinstances,dependingontheir

workload.

WhatyouneedforthisbookAbeginner-levelknowledgeofLinuxandthecommandlineisrequiredtofollowalongandruntherecipes.SomePythonexperienceisrequiredtofullyunderstandandbeabletoruntheexamplesinChapter7,UsingPythontoBuildandManageKVMInstances.

MostrecipesinthisbookhavebeentestedonbaremetalserverswithprocessorssupportingvirtualizationandthelatestversionofUbuntuLinux.

WhothisbookisforThisbookisforanyonewhoiscuriousaboutvirtualizationwithKVM--fromLinuxadministratorswhoarelookingforin-depthunderstandingofhowKVMcanbedeployedandmanagedinlargescaleproductionenvironmentstosoftwaredevelopersthatneedaquickandeasywaytoprototypecodeinisolatedguests.ADevOpsengineerismostlikelythebestjobtitleforthosewhowanttoreadthebookfromcovertocoverandtryallexamples.

SectionsInthisbook,youwillfindseveralheadingsthatappearfrequently(Gettingready,Howtodoit,Howitworks,There'smore,andSeealso).

Togiveclearinstructionsonhowtocompletearecipe,weusethesesectionsasfollows:

GettingreadyThissectiontellsyouwhattoexpectintherecipe,anddescribeshowtosetupanysoftwareoranypreliminarysettingsrequiredfortherecipe.

Howtodoit…Thissectioncontainsthestepsrequiredtofollowtherecipe.

Howitworks…Thissectionusuallyconsistsofadetailedexplanationofwhathappenedintheprevioussection.

There'smore…Thissectionconsistsofadditionalinformationabouttherecipeinordertomakethereadermoreknowledgeableabouttherecipe.

SeealsoThissectionprovideshelpfullinkstootherusefulinformationfortherecipe.

ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.

Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:"Managingdiskimageswithqemu-img."

Ablockofcodeissetasfollows:

importlibvirt

frombottleimportrun,request,get,post,HTTPResponse

deflibvirtConnect():

try:

conn=libvirt.open('qemu:///system')

exceptlibvirt.libvirtError:

conn=None

returnconn

Anycommand-lineinputoroutputiswrittenasfollows:

root@kvm:~#apt-getupdate

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:"Thememory_checkfortheKVMinstanceisnowshowingintheUchiwadashboard."

Warningsorimportantnotesappearinaboxlikethis.

Tipsandtricksappearlikethis.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook-whatyoulikedordisliked.Readerfeedbackisimportantforusasithelpsusdeveloptitlesthatyouwillreallygetthemostoutof.

Tosendusgeneralfeedback,[email protected],andmentionthebook'stitleinthesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideatwww.packtpub.com/authors.

http://www.packtpub.com/authors

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforthisbookfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

Youcandownloadthecodefilesbyfollowingthesesteps:

1. Loginorregistertoourwebsiteusingyoure-mailaddressandpassword.2. HoverthemousepointerontheSUPPORTtabatthetop.3. ClickonCodeDownloads&Errata.4. EnterthenameofthebookintheSearchbox.5. Selectthebookforwhichyou'relookingtodownloadthecodefiles.6. Choosefromthedrop-downmenuwhereyoupurchasedthisbookfrom.7. ClickonCodeDownload.

YoucanalsodownloadthecodefilesbyclickingontheCodeFilesbuttononthebook'swebpageatthePacktPublishingwebsite.Thispagecanbeaccessedbyenteringthebook'snameintheSearchbox.PleasenotethatyouneedtobeloggedintoyourPacktaccount.

Oncethefileisdownloaded,pleasemakesurethatyouunziporextractthefolderusingthelatestversionof:

WinRAR/7-ZipforWindowsZipeg/iZip/UnRarXforMac7-Zip/PeaZipforLinux

ThecodebundleforthebookisalsohostedonGitHubathttps://github.com/PacktPublishing/KVM-Virtualization-Cookbook.Wealsohaveothercodebundlesfromourrichcatalogofbooksandvideosavailableathttps://github.com/PacktPublishing/.Checkthemout!

http://www.packtpub.com

http://www.packtpub.com/support

https://github.com/PacktPublishing/KVM-Virtualization-Cookbook

https://github.com/PacktPublishing/

DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttps://www.packtpub.com/sites/default/files/downloads/KVMVirtualizationCookbook_ColorImages.pdf.

https://www.packtpub.com/sites/default/files/downloads/KVMVirtualizationCookbook_ColorImages.pdf.

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks-maybeamistakeinthetextorthecode-wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.

Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.

http://www.packtpub.com/submit-errata

https://www.packtpub.com/books/content/support

PiracyPiracyofcopyrightedmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.IfyoucomeacrossanyillegalcopiesofourworksinanyformontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusatcopyright@packtpub.comwithalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthorsandourabilitytobringyouvaluablecontent.

QuestionsIfyouhaveaproblemwithanyaspectofthisbook,[email protected],andwewilldoourbesttoaddresstheproblem.

GettingStartedwithQEMUandKVMInthischapter,wewillcoverthefollowingtopics:

InstallingandconfiguringQEMUManagingdiskimageswithqemu-imgPreparingimagesforOSinstallationwithqemu-nbdInstallingacustomOSontheimagewithdebootstrapResizinganimageUsingpre-existingimagesRunningvirtualmachineswithqemu-system-*StartingtheQEMUVMwithKVMsupportConnectingtoarunninginstancewithVNC

IntroductionQuickEmulator(QEMU)isthemaincomponentoftheQEMU/KVMvirtualizationtechnologysuit.Itprovideshardwarevirtualizationandprocessoremulation.QEMUrunsinuserspaceand,withouttheneedforkernel,driverscanstillprovidefastsystememulation.QEMUsupportstwooperatingmodes:

Fullsystememulation,whereQEMUemulatesanentirecomputersystem,includingtheCPUtypeandperipheralsUsermodeemulation,whereQEMUcanrunaprocessthathasbeencompiledonadifferentCPUarchitecturenatively

Inthisbook,wearegoingtofocusonfullsystememulationwiththehardwareaccelerationsupportprovidedbytheKernel-basedVirtualMachine(KVM)hypervisor.

Inthischapter,wewillstartbyinstallingQEMUonLinux,thenexplorevariousexamplesofbuilding,managing,andusingdiskimagesforthevirtualinstances.Wewillthenhaveanin-depthlookatrunningQEMUinfullsystememulationmode,usingtheprovidedbinaries.WewillseeexamplesofusingtheKVMkernelmoduletoacceleratetheQEMUprocesses.Finally,wearegoingtoendthechapterwithdetailsonhowtoconnecttothevirtualmachineswestartedearlier,usingVNCclients.

InstallingandconfiguringQEMUInthisrecipe,wewilllookatinstallingQEMUonasingleserverwiththeprovideddistributionpackages.Forproductionenvironments,werecommendusingprecompiled,packagedversionsofQEMUforeasierandmoreconsistentdeployments.However,wearegoingtoseeanexampleofhowtocompileQEMUfromsource,incaseyouneedacertainversionthatyoumightwanttopackagelater.

GettingreadyDependingonyourLinuxdistribution,thepackagenameandinstallationcommandswilldiffer.Youcanuseyoursystem'spackagemanager,suchasapt,dnf,oryumtosearchforanypackagescontainingtheQEMUstringandgetfamiliarwithwhatisavailableforyourparticularLinuxvariant.ThesourcecodecanbedownloadedfromtheofficialQEMUprojectwebsiteathttp://www.qemu-project.org/download/#source.

http://www.qemu-project.org/download/#source

Howtodoit...PerformthefollowingstepstoinstallQEMUfrompackagesonUbuntu/DebianandRHEL/CentOSdistributions:

1. OnUbuntu/Debiandistributions,updateyourpackagesindex:


2. Installthepackage:

root@kvm:~#apt-getinstall-yqemu

3. OnCentOS/RHELdistributionsexecute:

root@kvm:~#yuminstallqemu-kvm

Toinstallfromsource,executethefollowing:

1. Downloadthearchivefirst:

root@kvm:~#cd/usr/src&&wget

http://download.qemu-project.org/qemu-2.8.0.tar.xz

2. Extractthefilesfromthearchive:

root@kvm:/usr/src#tarxvJfqemu-2.8.0.tar.xz&&cdqemu-2.8.0

3. Configureandcompilethesourcecode:

root@kvm:/usr/src/qemu-2.8.0#./configure

root@kvm:/usr/src/qemu-2.8.0#make&&makeinstall

Howitworks...InstallingQEMUisquitetrivial,aswejustsaw.Let'shavealookatwhattheQEMUmetapackageinstalledonUbuntulookslike:

root@kvm:~#dpkg--list|grepqemu

iiipxe-qemu1.0.0+git-20150424.a25a16d-1ubuntu1allPXEbootfirmware-ROM

imagesforqemu

iiqemu1:2.5+dfsg-5ubuntu10.8amd64fastprocessoremulator

iiqemu-block-extra:amd641:2.5+dfsg-5ubuntu10.8amd64extrablockbackend

modulesforqemu-systemandqemu-utils

iiqemu-slof20151103+dfsg-1ubuntu1allSlimlineOpenFirmware--QEMUPowerPC

version

iiqemu-system1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulationbinaries

iiqemu-system-arm1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(arm)

iiqemu-system-common1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(commonfiles)

iiqemu-system-mips1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(mips)

iiqemu-system-misc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(miscelaneous)

iiqemu-system-ppc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(ppc)

iiqemu-system-sparc1:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(sparc)

iiqemu-system-x861:2.5+dfsg-5ubuntu10.8amd64QEMUfullsystememulation

binaries(x86)

iiqemu-user1:2.5+dfsg-5ubuntu10.8amd64QEMUusermodeemulationbinaries

iiqemu-user-binfmt1:2.5+dfsg-5ubuntu10.8amd64QEMUusermodebinfmt

registrationforqemu-user

iiqemu-utils1:2.5+dfsg-5ubuntu10.8amd64QEMUutilities

root@kvm:~#

Fromtheprecedingoutput,wecanseethattherearefewpackagesinvolved.Ifyouareinterested,youcanreadtheindividualdescriptiontogetmorefamiliarwithwhateachpackageprovides.

It'sworthmentioningthatallbinariesprovidedfromtheearlier-mentionedpackagesstartwiththeprefixQEMU.Youcanusetabcompletiontoseethelistofavailableexecutables:

root@kvm:~#qemu-

qemu-aarch64qemu-ioqemu-mips64elqemu-ppc64qemu-sparc32plusqemu-system-lm32

qemu-system-mipselqemu-system-sh4qemu-system-xtensa

qemu-alphaqemu-m68kqemu-mipselqemu-ppc64abi32qemu-sparc64qemu-system-m68k

qemu-system-moxieqemu-system-sh4ebqemu-system-xtensaeb

qemu-armqemu-make-debian-rootqemu-mipsn32qemu-ppc64leqemu-system-aarch64

qemu-system-microblazeqemu-system-or32qemu-system-sparcqemu-tilegx

qemu-armebqemu-microblazeqemu-mipsn32elqemu-s390xqemu-system-alphaqemu-

system-microblazeelqemu-system-ppcqemu-system-sparc64qemu-unicore32

qemu-crisqemu-microblazeelqemu-nbdqemu-sh4qemu-system-armqemu-system-mips

qemu-system-ppc64qemu-system-tricoreqemu-x86_64

qemu-i386qemu-mipsqemu-or32qemu-sh4ebqemu-system-crisqemu-system-mips64

qemu-system-ppc64leqemu-system-unicore32

qemu-imgqemu-mips64qemu-ppcqemu-sparcqemu-system-i386qemu-system-mips64el

qemu-system-ppcembqemu-system-x86_64

root@kvm:~#

Wecanseethatthere'sasingleexecutableforeachCPUarchitecturetypethatcanbeemulated.

Managingdiskimageswithqemu-imgTorunvirtualmachines,QEMUneedsimagestostorethefilesystemoftheguestOS.Theimageitselfisatypeoffile,anditrepresentstheguestfilesystemresidingonavirtualdisk.QEMUsupportsvariousimagesandprovidestoolstocreateandmanagethem.Inthisrecipe,wearegoingtobuildablankdiskimagewiththeqemu-imgutility.

GettingreadyTousethisrecipe,weneedtohavetheqemu-imgutilityinstalled.Ifyoufollowedthestepsinthefirstrecipe,youshouldhavethatcovered.TocheckwhatimagetypesaresupportedonyourLinuxdistribution,runthefollowingcommand:

root@kvm:~#qemu-img-h|grepSupported

Supportedformats:bochsvvfatrbdvpcparallelstftpftpftpsrawhttpsqcow

dmghttpqcow2quorumnull-aiocloopvdiiscsinull-covhdxblkverifyfilevmdk

host_cdromblkdebughost_devicesheepdogqednbd

root@kvm:~#

Fromtheprecedingoutput,wecanseethattherearemanysupportedimagesonthetestsystemthatweareusing.MakesurethatyourQEMUversionsupportstherawimagetype,asit'sthedefaultandthatiswhatwearegoingtouseinthisrecipe.Oneofthemostcommonlyusedimagetypeisqcow2,whichsupportscopyonwrite,compression,encryption,andsnapshotting.Wearegoingtoleveragethatinlaterrecipes.

PleasenotethateventhoughQEMUsupportsmultipleformats,thatdoesnotnecessarilymeanthatyoucanrunvirtualmachinesonthem.However,qemu-imgcanbeusedtoconvertdifferentimagestorawandqcow2formats.Forbestperformance,useraworqcow2imageformats.

Howtodoit...Performthefollowingstepstocreateablankrawimageofaspecifiedsizeandtoverifythatthefilewascreatedonthehost:

1. Createarawimagenameddebian.imgwithsizeof10GB:

root@kvm:~#qemu-imgcreate-frawdebian.img10G

Formatting'debian.img',fmt=rawsize=10737418240

root@kvm:~#

2. Checkthatthefilewascreated:

root@kvm:~#ls-lahdebian.img

-rw-r--r--1rootroot10GFeb1016:58debian.img

root@kvm:~#

3. Examinethefiletype:

root@kvm:~#file-sdebian.img

debian.img:data

root@kvm:~#

4. Obtainmoreinformationabouttheimage:

root@kvm:~#qemu-imginfodebian.img

image:debian.img

fileformat:raw

virtualsize:10G(10737418240bytes)

disksize:0

root@kvm:~#

Howitworks...Theqemu-imgutilityallowsustocreate,convert,andmodifyguestimages.

Instep1,weusedthe-fflagspecifyingtheimageformat;inthiscase,raw,thenameoftheimagetobecreatedandthesizeingigabytes.

Instep4,weusedtheinfosubcommandtogatheradditionalinformationabouttheexistingimage.Notehowthedisksizeisshowingascurrentlybeingzero.Thisisduetothefactthatthisisablankimage,notcontainingafilesystem.Wearegoingtocreateoneinthenextrecipe.

There'smore...Inthisrecipe,welistedthesupporteddiskimageformatsbyQEMU.Thefollowingisabriefdescriptionofthemostcommontypesthatyoumightencounter:

raw:Rawdiskimageformat.Thisisthedefaultformatandcanbeoneofthefastestfile-basedformats.Ifyouformatthisimagewithafilesystemthatsupportsholes,forexample,EXT3,thenonlysectorsthathavedatawillusespace.Themaindrawbackoftherawimagesisthelackoffeatures,makingthemidealfortestingandquickprototyping.qcow2:Aswementionedintheprevioussection,thisisoneofthemostfeature-richformats.ItsupportsVMsnapshots,compression,andencryptionforthepriceofslightlyreducedperformance.qcow:ThisisanolderQEMUimageformatthatsupportsbackingfiles,compactimagefiles,encryption,andcompression.dmg:ThisistheMacdiskimageformat.TheMacdiskimageprovidessecurepasswordprotectionandcompression,anditismostcommonlyusedtodistributesoftware,ratherthanrunningvirtualmachines.nbd:Thenetworkblockdevice,typicallyusedforaccessingremotestoragedevices.vdi:ThisdiskformatisusedbytheOracleVirtualBoxsoftwareandcanbeusedtorunvirtualmachinesonvariousCPUplatforms.vmdk:ThisistheVMwarediskimagetype,whereasinglevirtualharddiskcanspanmultiplefiles.vhdx:MicrosoftHyper-Vusesthisimageformat.Itprovideslargestoragecapacity,datacorruptionprotectionduringpowerfailuresandread/writeoptimizationforlargerdiskimages.

Inthisbook,wearegoingtousetherawandqcow2diskformats,astheyprovidethebestperformanceandtoolsetforrunningandmanipulatingthem.

PreparingimagesforOSinstallationwithqemu-nbdInthepreviousrecipe,wecreatedablankrawimage.Inthisrecipe,wearegoingtomakeapartitionandafilesystemonit,gettingtheimagereadyforfullguestOSinstallation.Whencreatingthepartitionandfilesystem,youshouldconsiderthetypeofloadthatthevirtualinstancewillcreate.IfyourapplicationsrunninginsideVMareIObound,youmightconsiderXFSfortheimagefilesystem.Forthisrecipe,wearegoingtouseEXT4,asmostLinuxdistributionssupportitoutofthebox.

GettingreadyForthisrecipe,wearegoingtousethefollowingtools:

qemu-nbd

sfdisk

Thenbdkernelmodulemkfs

MostLinuxdistributionsshouldalreadyhavethetoolsinstalled.Ifthat'snotthecase,consultyourdistribution'sdocumentationonhowtoinstallthem.

Howtodoit...Performthefollowingstepsoutlinedtopartitionandcreateafilesystemontheblankimage:

1. Loadthenbdkernelmodule:

root@kvm:~#modprobenbd

root@kvm:~#

2. Usingtheqemu-nbdtool,associatetheblankimagefiletothe/dev/nbd0blockdevice:

root@kvm:~#qemu-nbd--format=raw--connect=/dev/nbd0

debian.img

root@kvm:~#

3. Createtwopartitionsontheblockdevice.Onewillbeusedforswap,andtheotherastherootpartitionfortheguestOS:

root@kvm:~#sfdisk/dev/nbd0<<EOF

>,1024,82

>;

>EOF

Checkingthatno-oneisusingthisdiskrightnow...

OK

Disk/dev/nbd0:cannotgetgeometry

Disk/dev/nbd0:1305cylinders,255heads,63sectors/track

sfdisk:ERROR:sector0doesnothaveanmsdossignature

/dev/nbd0:unrecognizedpartitiontabletype

Oldsituation:

Nopartitionsfound

Newsituation:

Units=cylindersof8225280bytes,blocksof1024bytes,countingfrom

0

DeviceBootStartEnd#cyls#blocksIdSystem

/dev/nbd0p10+10231024-8225279+82Linuxswap/Solaris

/dev/nbd0p2102413042812257132+83Linux

/dev/nbd0p30-000Empty

/dev/nbd0p40-000Empty

Warning:noprimarypartitionismarkedbootable(active)

ThisdoesnotmatterforLILO,buttheDOSMBRwillnotbootthisdisk.

Successfullywrotethenewpartitiontable

Re-readingthepartitiontable...

IfyoucreatedorchangedaDOSpartition,/dev/foo7,say,thenuse

dd(1)

tozerothefirst512bytes:ddif=/dev/zeroof=/dev/foo7bs=512count=1

(Seefdisk(8).)

root@kvm:~#

4. Listtheavailableblockdevicesafterthepartitioning:

root@kvm:~#ls-la/dev/nbd0*

brw-rw----1rootdisk43,0Feb1018:24/dev/nbd0

brw-rw----1rootdisk43,1Feb1018:24/dev/nbd0p1


root@kvm:~#

5. Createtheswappartition:

root@kvm:~#mkswap/dev/nbd0p1

Settingupswapspaceversion1,size=508KiB(520192bytes)

nolabel,UUID=c246fe39-1bc5-4978-967c-806264771d69

root@kvm:~#

6. MaketheEXT4filesystemontherootpartition:

root@kvm:~#mkfs.ext4/dev/nbd0p2

mke2fs1.42.13(17-May-2015)

Discardingdeviceblocks:failed-Input/outputerror

Creatingfilesystemwith26209284kblocksand655360inodes

FilesystemUUID:2ffa23de-579a-45ad-abbc-2a179de67f11

Superblockbackupsstoredonblocks:

32768,98304,163840,229376,294912,819200,884736,1605632

Allocatinggrouptables:done

Writinginodetables:done

Creatingjournal(32768blocks):done

Writingsuperblocksandfilesystemaccountinginformation:done

root@kvm:~#

Howitworks...Wetakeadvantageofthefunctionalitythatthenbdkernelmoduleprovidesbyallowingustoassociatearawimagefiletoablockdeviceusingtheqemu-nbdutility.Togetmoreinformationaboutthekernelmodulerunthefollowingcode:

root@kvm:~#modinfonbd

filename:/lib/modules/4.4.0-62-generic/kernel/drivers/block/nbd.ko

license:GPL

description:NetworkBlockDevice

srcversion:C67096AF2AE3C738DBE0B7E

depends:

intree:Y

vermagic:4.4.0-62-genericSMPmod_unloadmodversions

parm:nbds_max:numberofnetworkblockdevicestoinitialize(default:16)(int)

parm:max_part:numberofpartitionsperdevice(default:0)(int)

root@kvm:~#

Wecanexaminetheblockdevicemetadatacreatedinstep2byrunningthefollowingcommand:

root@kvm:~#file-s/dev/nbd0

/dev/nbd0:x86bootsector

root@kvm:~#

Aftercreatingthetwonewpartitionsinstep3,thetypeoftheimagefilehaschanged.Let'sexamineitagain:

root@kvm:~#file-sdebian.img

debian.img:x86bootsector

root@kvm:~#

Wechosetousethesfdiskutilitytocreatethepartitions,butyoucanusethefdiskutilityinteractivelyinsteadifyouprefer.Theendresultwillbethesame.

Nowthatwehaveanimagefilethatcontainstwopartitionsandafilesystem,wecanproceedwithinstallingtheguestOSinthenextrecipe.

InstallingacustomOSontheimagewithdebootstrapInthisrecipe,wearegoingtousethedebootstraputilitytoinstallaDebiandistributionontherawimagewepreparedintheprevioustworecipes.ThedebootstrapcommandisusedtobootstrapabasicDebiansystemusingaspecificpublicmirror.Bytheendofthisrecipe,weshouldhaveanimagecontaininganentireLinuxdistribution,readyforQEMUexecution.

GettingreadyWearegoingtoneedthefollowinginordertocompletethisrecipe:

TheblockdevicescreatedinthepreviousrecipeThedebootstraputilityThechrootutility

Toensurethattheswapandrootblockdevicesarestillpresentonthesystem,runthefollowing:

root@kvm:~#ls-la/dev/nbd0*

brw-rw----1rootdisk43,0Feb1018:24/dev/nbd0



root@kvm:~#

Ifthat'snotthecase,pleaserefertothePreparingimagesforOSinstallationwithqemu-nbdrecipeonhowtoassociatetherawimagewiththe/deb/nbd0blockdevice.

Toinstallthedebootstraputility,ifnotalreadypresentonyoursystem,executethefollowingcode:

root@kvm:~#aptinstall-ydebootstrap

...

Settingupdebootstrap(1.0.78+nmu1ubuntu1.2)...

root@kvm:~#

Howtodoit...FollowthesestepsoutlinedtoinstallanewDebianLinuxdistributionontherawimage:

1. MounttherootpartitionfromtheNetworkBlockDevice(NBD)deviceandensurethatitwasmountedsuccessfully:

root@kvm:~#mount/dev/nbd0p2/mnt/

root@kvm:~#mount|grepmnt

/dev/nbd0p2on/mnttypeext4(rw)

root@kvm:~#

2. InstallthelateststableDebiandistributionontherootpartitionmountedon/mntfromthespecifiedpublicrepository:

root@kvm:~#debootstrap--arch=amd64--include="openssh-servervim"

stable/mnt/http://httpredir.debian.org/debian/

...

I:Basesysteminstalledsuccessfully.

root@kvm:~#

3. Ensuretherootfilesystemwascreated,bylistingallthefilesatthemountedlocation:

root@kvm:~#ls-lah/mnt/

total100Kdrwxr-xr-x22rootroot4.0KFeb1017:19.

drwxr-xr-x23rootroot4.0KFeb1015:29..

drwxr-xr-x2rootroot4.0KFeb1017:19bin

drwxr-xr-x2rootroot4.0KDec2817:42boot

drwxr-xr-x4rootroot4.0KFeb1017:18dev

drwxr-xr-x55rootroot4.0KFeb1017:19etc

drwxr-xr-x2rootroot4.0KDec2817:42home

drwxr-xr-x12rootroot4.0KFeb1017:19lib

drwxr-xr-x2rootroot4.0KFeb1017:18lib64

drwx------2rootroot16KFeb1017:06lost+found

drwxr-xr-x2rootroot4.0KFeb1017:18media

drwxr-xr-x2rootroot4.0KFeb1017:18mnt

drwxr-xr-x2rootroot4.0KFeb1017:18opt

drwxr-xr-x2rootroot4.0KDec2817:42proc

drwx------2rootroot4.0KFeb1017:18root

drwxr-xr-x4rootroot4.0KFeb1017:19run

drwxr-xr-x2rootroot4.0KFeb1017:19sbin

drwxr-xr-x2rootroot4.0KFeb1017:18srv

drwxr-xr-x2rootroot4.0KApr62015sys

drwxrwxrwt2rootroot4.0KFeb1017:18tmp

drwxr-xr-x10rootroot4.0KFeb1017:18usr

drwxr-xr-x11rootroot4.0KFeb1017:18var

root@kvm:~#

4. Bindandmountthedevicesdirectoryfromthehosttotheimagefilesystem:

root@kvm:~#mount--bind/dev//mnt/dev

root@kvm:~#

5. Ensurethatthenbddevicesarenowpresentinsidethemountlocation:

root@kvm:~#ls-la/mnt/dev/|grepnbd0

brw-rw----1rootdisk43,0Feb1018:24nbd0

brw-rw----1rootdisk43,1Feb1018:26nbd0p1

brw-rw----1rootdisk43,2Feb1018:26nbd0p2

root@kvm:~#

6. Changethedirectorynamespacetobetherootfilesystemoftheimageandensuretheoperationsucceeded:

root@kvm:~#chroot/mnt/

root@kvm:/#pwd

/

root@kvm:/#

7. Checkthedistributionversioninsidethechrootenvironment:

root@kvm:/#cat/etc/debian_version

8.7

root@kvm:/#

8. Mounttheprocandsysfsvirtualfilesystemsinsidethechrootedenvironment:

root@kvm:/#mount-tprocnone/proc

root@kvm:/#mount-tsysfsnone/sys

root@kvm:/#

9. Whilestillinsidethechrootedlocation,installtheDebiankernelmetapackageandthegrub2utilities:

root@kvm:/#apt-getinstall-y--force-yeslinux-image-amd64grub2

IfaskedtoselecttargetdeviceforGRUBtoinstallon,donotselectanyandjustcontinue.

10. InstallGRUBontherootdevice:

root@kvm:/#grub-install/dev/nbd0--force

Installingfori386-pcplatform.

grub-install:warning:thismsdos-stylepartitionlabelhasnopost-MBR

gap;embeddingwon'tbepossible.

grub-install:warning:Embeddingisnotpossible.GRUBcanonlybe

installedinthissetupbyusingblocklists.However,blocklistsare

UNRELIABLEandtheiruseisdiscouraged..

Installationfinished.Noerrorreported.

root@kvm:/#

11. UpdatetheGRUBconfigsandtheinitrdimage:

root@kvm:/#update-grub2

Generatinggrubconfigurationfile...

Foundlinuximage:/boot/vmlinuz-3.16.0-4-amd64

Foundinitrdimage:/boot/initrd.img-3.16.0-4-amd64

done

root@kvm:/#

12. Changetherootpasswordoftheguest:

root@kvm:/#passwd

EnternewUNIXpassword:

RetypenewUNIXpassword:

passwd:passwordupdatedsuccessfully

root@kvm:/#

13. AllowaccesstothepseudoTerminalinsidethenewguestOS:

root@kvm:/#echo"pts/0">>/etc/securetty

root@kvm:/#

14. Changethesystemdrunleveltothemulti-userlevel:

root@kvm:/#systemctlset-defaultmulti-user.target

Createdsymlinkfrom/etc/systemd/system/default.targetto

/lib/systemd/system/multi-user.target.

root@kvm:/#

15. Addtherootmountpointtothefstabfile,soitcanpersistreboots:

root@kvm:/#echo"/dev/sda2/ext4defaults,discard00">/etc/fstab

16. Unmountthefollowingfilesystemsaswearedoneusingthemfornow:

root@kvm:/#umount/proc//sys//dev/

17. Exitthechrootedenvironment:

root@kvm:/#exit

exit

root@kvm:~#

18. InstallGRUBontherootpartitionoftheblockdeviceassociatedwiththerawimage:

root@kvm:~#grub-install/dev/nbd0--root-directory=/mnt--

modules="biosdiskpart_msdos"--force

Installingfori386-pcplatform.

grub-install:warning:thismsdos-stylepartitionlabelhasnopost-MBR

gap;embeddingwon'tbepossible.

grub-install:warning:Embeddingisnotpossible.GRUBcanonlybe

installedinthissetupbyusingblocklists.However,blocklistsare

UNRELIABLEandtheiruseisdiscouraged..

Installationfinished.Noerrorreported.

root@kvm:~#

19. UpdatetheGRUBconfigurationfiletoreflectthecorrectblockdevicefortheguestimage:

root@kvm:~#sed-i's/nbd0p2/sda2/g'/mnt/boot/grub/grub.cfg

root@kvm:~#

20. Unmountthenbd0device:

root@kvm:~#umount/mnt

root@kvm:~#

21. Disassociatethenbd0devicefromtherawimage:

root@kvm:~#qemu-nbd--disconnect/dev/nbd0

/dev/nbd0disconnected

root@kvm:~#

Howitworks...Alothashappenedintheprevioussection,solet'sstepthroughthecommandsandtalkalittlebitmoreaboutwhatexactlywasperformedandwhy.

Instep1,wemountedtherootpartitionwecreatedearlieronthe/dev/nbd0p2deviceto/mnt,sowecanuseit.Oncemounted,instep2,weinstalledanentireDebiandistributiononthatdeviceusingthemount-pointasthetarget.

InordertoinstalltheGRUBbootloaderontherootpartitionoftheimage,webindandmountedthe/devdirectoryfromthehostfilesystemtotheimagefilesystemin/mntinstep4.

Theninstep6,weusedthechroottooltochangeourdirectorynamespacetobe/mnt,sowecanperformoperations,aswearedirectlyonthenewOS.

Instep8,wemountedtheprocandsysfsvirtualfilesystemsinsidetheimagebecausetheGRUBbootloadertoolexpectthem.

Instep9,weproceededtoinstallthekernelsourceandGRUBtoolsinpreparationofinstallingthebootloaderonthebootpartitionandinstep10weinstalledthebootloader.

Instep11,theGRUBconfigurationfilesweregeneratedandthebootramdiskimagewasupdated.

Insteps12,13,and14,wechangedtherootpasswordandensuredwegetaccesstothepseudoTerminal,sowecanlogintotheVMlaterandchangetherun-levelfromthedefaultgraphicalinterfacetothemultiuser.

SincethefstabfileisemptyrightafterinstallingtheDebianOSontheimage,wehavetoaddtherootmountpoint,ortheVMwillnotbeabletostart.Thiswasaccomplishedinstep15.

Insteps16and17,weperformedsomecleaningupbyunmountingthefilesystemswemountedearlierandexitedthechrootenvironment.

Backonthehostfilesysteminstep18,weinstalledGRUBonthenbd0devicebyspecifyingthemountedlocationoftheimage.

Instep19,weupdatedtheGRUBconfigdevicenametobesda2becausethisisthenamethatwillappearinsidethevirtualmachineoncewestartit.Thenbd0p2nameisonlypresentwhilewehavetheassociationbetweentherawimageandthenetworkblockdeviceonthehostOS.FromtheVMperspective,thesecondpartitioninsidetheimagewecreatedbyisnamedsda2bydefault.

Andfinally,insteps20and21,weperformedsomecleaningbyremovingthemountpointanddisassociatingtherawimagefromthenetworkblockdevicenbd0.

ResizinganimageInthisrecipe,wearegoingtoexaminehowtoresizeanexistingrawimage,thepartitionshostedonitandthefilesystemontopofthepartitions.Wearegoingtobeusingtherawimagethatwebuildinthepreviousrecipes,whichcontainsaswapandarootpartitionwithanEXT4filesystemformattedonit.

GettingreadyForthisrecipe,wearegoingtousethefollowingtools:

qemu-img

losetup

tune2fs

e2fsck

kpartx

fdisk

resize2fs

MostoftheutilitiesshouldalreadybeinstalledonUbuntuwiththeexceptionofkpartx.Toinstallit,runthefollowing:

root@kvm:~#aptinstallkpartx

Howtodoit...Thenextstepsdemonstratehowtoaddadditionalspacetotherawimagewecreatedearlier,extendtherootpartition,andresizethefilesystem.Bytheendofthisrecipe,theoriginalrawimagefilesystemsizeshouldhavechangedfrom10Gto20G.

1. Obtainthecurrentsizeoftheimage:


image:debian.img

fileformat:raw


disksize:848M

root@kvm:~#

2. Addadditional10GBtotheimage:

root@kvm:~#qemu-imgresize-frawdebian.img+10GB

Imageresized.

root@kvm:~#

Pleasenotethatnotallimagetypessupportresizing.Inordertoresizesuchanimage,youwillneedtoconvertittorawimagefirstusingtheqemu-imgconvertcommand.

3. Checkthenewsizeoftheimage:


image:debian.img

fileformat:raw


disksize:848M

root@kvm:~#

4. Printthenameofthefirstunusedloopdevice:

root@kvm:~#losetup-f

/dev/loop0

root@kvm:~#

5. Associatethefirstunusedloopdevicewiththerawimagefile:

root@kvm:~#losetup/dev/loop1debian.img

root@kvm:~#

6. Readthepartitioninformationfromtheassociatedloopdeviceandcreatethedevicemappings:

root@kvm:~#kpartx-av/dev/loop1

addmaploop1p1(252:0):01024linear7:12048


root@kvm:~#

7. Examinethenewdevicemaps,representingthepartitionsontherawimage:

root@kvm:~#ls-la/dev/mapper

total0

drwxr-xr-x2rootroot100Mar919:10.

drwxr-xr-x20rootroot4760Mar919:10..

crw-------1rootroot10,236Feb1023:25control

lrwxrwxrwx1rootroot7Mar919:10loop1p1

lrwxrwxrwx1rootroot7Mar919:10loop1p2

root@kvm:~#

8. Obtainsomeinformationfromtherootpartitionmapping:

root@kvm:~#tune2fs-l/dev/mapper/loop1p2

tune2fs1.42.13(17-May-2015)

Filesystemvolumename:<none>

Lastmountedon:/

FilesystemUUID:96a73752-489a-435c-8aa0-8c5d1aba3e5f

Filesystemmagicnumber:0xEF53

Filesystemrevision#:1(dynamic)

Filesystemfeatures:has_journalext_attrresize_inodedir_index

filetypeneeds_recoveryextentflex_bgsparse_super

large_filehuge_fileuninit_bgdir_nlinkextra_isizeFilesystem

flags:signed_directory_hash

Defaultmountoptions:user_xattracl

Filesystemstate:clean

Errorsbehavior:Continue

FilesystemOStype:Linux

Inodecount:655360

Blockcount:2620928

Reservedblockcount:131046

Freeblocks:2362078

Freeinodes:634148

Firstblock:0

Blocksize:4096

Fragmentsize:4096

ReservedGDTblocks:639

Blockspergroup:32768

Fragmentspergroup:32768

Inodespergroup:8192

Inodeblockspergroup:512

Flexblockgroupsize:16

Filesystemcreated:FriFeb1023:29:012017

Lastmounttime:ThuMar919:09:252017

Lastwritetime:ThuMar919:08:232017

Mountcount:12

Maximummountcount:-1

Lastchecked:FriFeb1023:29:012017

Checkinterval:0(<none>)

Lifetimewrites:1621MB

Reservedblocksuid:0(userroot)

Reservedblocksgid:0(grouproot)

Firstinode:11

Inodesize:256

Requiredextraisize:28

Desiredextraisize:28

Journalinode:8

Defaultdirectoryhash:half_md4

DirectoryHashSeed:f101cccc-944e-4773-8644-91ebf4bd4f2d

Journalbackup:inodeblocks

root@kvm:~#

9. Checkthefilesystemontherootpartitionofthemappeddevice:

root@kvm:~#e2fsck/dev/mapper/loop1p2

e2fsck1.42.13(17-May-2015)

/dev/mapper/loop1p2:recoveringjournalSettingfreeblockscountto

2362045(was2362078)/dev/mapper/loop1p2:clean,21212/655360files,

258883/2620928blocks

root@kvm:~#

10. Removethejournalfromtherootpartitiondevice:

root@kvm:~#tune2fs-O^has_journal/dev/mapper/loop1p2

tune2fs1.42.13(17-May-2015)

root@kvm:~#

11. Ensurethatthejournalinghasbeenremoved:

root@kvm:~#tune2fs-l/dev/mapper/loop1p2|grep"features"

Filesystemfeatures:ext_attrresize_inodedir_indexfiletypeextent

flex_bgsparse_superlarge_filehuge_fileuninit_bgdir_nlink

extra_isize

root@kvm:~#

12. Removethepartitionmappings:

root@kvm:~#kpartx-dv/dev/loop1

deldevmap:loop1p2

deldevmap:loop1p1

root@kvm:~#

13. Detachtheloopdevicefromtheimage:

root@kvm:~#losetup-d/dev/loop1

root@kvm:~#

14. Associatetherawimagewiththenetworkblockdevice:

root@kvm:~#qemu-nbd--format=raw--connect=/dev/nbd0debian.img

root@kvm:~#

15. Usingfdisk,listtheavailablepartitions,thendeletetherootpartition,recreateit,andwritethechanges:

root@kvm:~#fdisk/dev/nbd0

Command(mforhelp):p

Disk/dev/nbd0:21.5GB,21474836480bytes

255heads,63sectors/track,2610cylinders,total41943040sectors

Units=sectorsof1*512=512bytes

Sectorsize(logical/physical):512bytes/512bytes

I/Osize(minimum/optimal):512bytes/512bytes

Diskidentifier:0x00000000

DeviceBootStartEndBlocksIdSystem

/dev/nbd0p11164505598225279+82Linuxswap/Solaris

/dev/nbd0p216450560209648242257132+83Linux

Command(mforhelp):d

Partitionnumber(1-4):2

Command(mforhelp):n

Partitiontype:

pprimary(1primary,0extended,3free)

eextended

Select(defaultp):p

Partitionnumber(1-4,default2):2

Firstsector(16450560-41943039,default16450560):

Usingdefaultvalue16450560

Lastsector,+sectorsor+size{K,M,G}(16450560-41943039,default

41943039):

Usingdefaultvalue41943039

Command(mforhelp):w

Thepartitiontablehasbeenaltered!

Callingioctl()tore-readpartitiontable.

Syncingdisks.

root@kvm:~#

16. Associatethefirstunusedloopdevicewiththerawimagefile,likewedidinstep5:

root@kvm:~#losetup/dev/loop1debian.img

17. Readthepartitioninformationfromtheassociatedloopdeviceandcreatethedevicemappings:

root@kvm:~#kpartx-av/dev/loop1



root@kvm:~#

18. Afterthepartitioningiscomplete,performafilesystemcheck:

root@kvm:~#e2fsck-f/dev/mapper/loop1p2

e2fsck1.42.13(17-May-2015)

Pass1:Checkinginodes,blocks,andsizes

Pass2:Checkingdirectorystructure

Pass3:Checkingdirectoryconnectivity

Pass4:Checkingreferencecounts

Pass5:Checkinggroupsummaryinformation

/dev/mapper/loop1p2:21212/655360files(0.2%non-contiguous),

226115/2620928blocks

root@kvm:~#

19. Resizethefilesystemontherootpartitionofthemappeddevice:

root@kvm:~#resize2fs/dev/nbd0p2

resize2fs1.42.13(17-May-2015)

Resizingthefilesystemon/dev/mapper/loop1p2to5242368(4k)blocks.

Thefilesystemon/dev/mapper/loop1p2isnow5242368(4k)blockslong.

root@kvm:~#

20. Createthefilesystemjournalbecauseweremoveditearlier:

root@kvm:~#tune2fs-j/dev/mapper/loop1p2

tune2fs1.42.13(17-May-2015)

Creatingjournalinode:done

root@kvm:~#

21. Removethedevicemappings:

root@kvm:~#kpartx-dv/dev/loop1

deldevmap:loop1p2

deldevmap:loop1p1

root@kvm:~#losetup-d/dev/loop1

root@kvm:~#

Howitworks...ResizinganimageforVMcanbesomewhatinvolving,aswesawfromallthestepsintheprevioussection.ThingscangetcomplicatedwhentherearemultipleLinuxpartitionsinsidethesameimage,evenmoresoifwearenotusingLogicalVolumeManagement(LVM).Let'sstepthroughallthecommandsweranearlierandexplaininmoredetailswhyweranthemandwhattheydo.

Instep1,weconfirmedthecurrentsizeoftheimagebeing10GB.

Instep2,weadded10GBattheendoftheimageandconfirmthenewimagesizeinstep3.

Recallthattheimagewebuiltfromearlierrecipescontainstwopartitions,swapandroot.Weneedawaytomanipulatethemindividually.Particularly,wewouldliketoallocatetheextraspaceweaddedinstep2totherootpartition.Todothatweneedtoexposeitasablockdevicethatwecaneasilymanipulatewithstandarddiskandfilesystemutilities.Weaccomplishedthatusingthelosetupcommandinstep5,resultinginamappingbetweentheimageandanewblockdevicenamed/dev/loop1.Instep6,weexposedtheindividualpartitionsastwonewdevicemappings.The/dev/mapper/loop1p2istherootpartitionthatwewouldliketoappendtheunuseddiskspaceto.

Beforewecanresizethepartitionedontheloopdevice,weneedtochecktheintegrityofthefilesystemonit,andthisiswhatwedidinstep9.Becauseweareusingajournalingfilesystem,weneedtoremovethejournalpriortoresizing.Wedothatinstep10andmadesurethatthehas_journalattributeisnotshowingafterrunningthetune2fscommandinstep11.

Now,weneedtoworkdirectlyonthemainblockdeviceandnottheindividualpartitions.Weremovethemappingsinsteps12and13andassociatedanewblockdevicewiththeimagefileusingtheqemu-nbdcommandinstep14.Thenew/dev/nbd0blockdevicenowrepresentstheentirediskoftheguestVMandit'sadirectmappingtowhat'sinsidetherawimage.Wecanusethisblockdevicejustlikeanyotherregulardisk,mostimportantlywecanusetoolssuch

asfdisktoexamineandmanipulatethepartitionsresidingonit.

Instep15,weusethefdiskutilitytodeletetherootpartitionandrecreateit.Thisdoesnotdestroyanyfilesystemdata,butchangesthemetadata,allocatingtheextraspaceweaddedearlieraspartoftherootpartition.

Nowthattheblockdevicehasallthediskspaceallocatedtotherootpartition,weneedtoextendthefilesystemthatisontopofit.Wedothatbyfirstrecreatingtheindividualpartitionmappingslikewedidearlier,toexposetherootpartitiondirectlysothatwecanyetagainmanipulateit.Wedothatinsteps16and17.

Insteps18and19,wechecktheintegrityoftherootfilesystem,thenweresizeittothemaximumavailablediskspaceontherootpartitionthatitresides.

Finally,instep20,weremovethemappingsagain.Nowtheimage,therootpartitioninsidetheimage,andtheEXT4filesystemontopoftheLinuxpartitionhavebeenresizedto20GB.

YoucancheckthenewrootpartitionsizebystartinganewQEMUinstanceusingtheimage.Wearegoingtodojustthatinaseparaterecipeinthischapter.

Usingpre-existingimagesIntheInstallingacustomOSontheimagewithdebootstraprecipe,wesawhowtousethedebootstrapcommandtoinstallDebianonanimagewebuilt.MostLinuxvendorsprovidealreadybuiltimagesoftheirdistributionsforvariousarchitectures.InstallableimagesarealsoavailableformanuallyinstallingtheguestOS.Inthisrecipe,wearegoingtodemonstratehowtoobtainandexamineCentOSandDebianimagesthathavealreadybeenbuilt.Inalaterrecipe,wearegoingtoshowhowtostartQEMU/KVMinstancesusingthosesameimages.

GettingreadyForthisrecipe,wearegoingtoneedQEMUinstalledonthehostOS.ForinstructionsonhowtoinstallQEMU,pleaserefertotheInstallingandconfiguringQEMUrecipefromthischapter.Wearealsogoingtoneedthewgetutilitytodownloadtheimagesfromtheupstreampublicrepositories.

Howtodoit...ToobtainDebianWheezyimagesforusewithQEMUandKVM,performthefollowing:

1. Downloadtheimageusingwget:

root@kvm:~tmp#wget

https://people.debian.org/~aurel32/qemu/amd64/debian_wheezy_amd64_standard.qcow2

--2017-03-0922:07:20--2Resolvingpeople.debian.org

(people.debian.org)...2001:41c8:1000:21::21:30,5.153.231.30

Connectingtopeople.debian.org

(people.debian.org)|2001:41c8:1000:21::21:30|:443...connected.HTTP

requestsent,awaitingresponse...200OKLength:267064832(255M)

Savingto:‘debian_wheezy_amd64_standard.qcow2’

debian_wheezy_amd64_standard.qcow2100%.

[===================================>]254.69M35.8MB/sin8.3s2017-

03-0922:07:29(30.9MB/s)-‘debian_wheezy_amd64_standard.qcow2’saved

[267064832/267064832]

root@kvm:~#

2. Inspectthetypeoftheimage:

root@kvm:~#qemu-imginfodebian_wheezy_amd64_standard.qcow2

image:debian_wheezy_amd64_standard.qcow2

fileformat:qcow2


disksize:261M

cluster_size:65536

Formatspecificinformation:

compat:1.1

lazyrefcounts:false

refcountbits:16

corrupt:false

root@kvm:~#

TodownloadCentOSimagesrunthefollowingcommands:

1. Downloadtheimageusingwget:

root@kvm:/tmp#wget

https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-

GenericCloud.qcow2--2017-03-0922:11:34--

https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-

GenericCloud.qcow2Resolvingcloud.centos.org(cloud.centos.org)...

2604:4500::2a8a,136.243.75.209Connectingtocloud.centos.org

(cloud.centos.org)|2604:4500::2a8a|:443...connected.HTTPrequest

sent,awaitingresponse...200OKLength:1361182720(1.3G)Saving

to:‘CentOS-7-x86_64-GenericCloud.qcow2’

CentOS-7-x86_64-GenericCloud.qcow2100%

[=========================================>]1.27G22.3MB/sin54s

2017-03-0922:12:29(24.0MB/s)-‘CentOS-7-x86_64-GenericCloud.qcow2’

saved[1361182720/1361182720]

FINISHED--2017-03-0922:12:29--Totalwallclocktime:54s

Downloaded:1files,1.3Gin54s(24.0MB/s)

root@kvm:/tmp#

2. Inspectthetypeoftheimage:

root@kvm:~#qemu-imginfoCentOS-7-x86_64-GenericCloud.qcow2

image:CentOS-7-x86_64-GenericCloud.qcow2

fileformat:qcow2

virtualsize:8.0G(8589934592bytes)

disksize:1.3G

cluster_size:65536


compat:0.10

refcountbits:16

root@kvm:~#

Howitworks...TherearemanypublicrepositoriesontheInternetthatprovideimagesofvarioustypes,mostcommonlyqcow2forusewithQEMU/KVM.Intheprevioussection,weusedtheofficialCentOSrepositorytoobtaintheimageandananotheronecontainingprebuiltimagesforDebian.

Bothimagesareintheqcow2format,asweconfirmedinstep2.

There'smore...Sofar,we'veonlyseenhowtobuild,examine,manipulate,anddownloadimages.Inthenextrecipe,wearegoingtofocusonhowtoactuallyusetheimagestostartQEMU/KVMinstances.

SeealsoExaminethefollowinglinkstogetmoreinformationaboutwhatprebuiltimagesareavailableforthelisteddistributions:

OfficialUbuntuimages:https://uec-images.ubuntu.com/releases/OfficialCentOSimages:https://cloud.centos.org/centos/OfficialDebianimages:http://cdimage.debian.org/cdimage/openstack/OfficialFedoraimages:https://alt.fedoraproject.org/cloud/OfficialopenSUSEimages:http://download.opensuse.org/repositories/Cloud:/Images:/

https://uec-images.ubuntu.com/releases/

https://cloud.centos.org/centos/

http://cdimage.debian.org/cdimage/openstack/

https://alt.fedoraproject.org/cloud/

http://download.opensuse.org/repositories/Cloud:/Images:/

Runningvirtualmachineswithqemu-system-*Inthisrecipe,wearegoingtodemonstratehowtostartvirtualmachineswithQEMU.QEMUprovidesbinariesthatcanemulatedifferentCPUarchitecturesusingeithercustomorprebuiltimagesfortheguestOS.

IfyoucompletedtheInstallingandconfiguringQEMUrecipe,youshouldhaveahostthatcontainsthefollowingbinaries:

root@kvm:~#ls-la/usr/bin/qemu-system-*

-rwxr-xr-x1rootroot8868848Jan2512:49/usr/bin/qemu-system-aarch64

-rwxr-xr-x1rootroot7020544Jan2512:49/usr/bin/qemu-system-alpha

-rwxr-xr-x1rootroot8700784Jan2512:49/usr/bin/qemu-system-arm

-rwxr-xr-x1rootroot3671488Jan2512:49/usr/bin/qemu-system-cris

-rwxr-xr-x1rootroot8363680Jan2512:49/usr/bin/qemu-system-i386

-rwxr-xr-x1rootroot3636640Jan2512:49/usr/bin/qemu-system-lm32

-rwxr-xr-x1rootroot6982528Jan2512:49/usr/bin/qemu-system-m68k

-rwxr-xr-x1rootroot3652224Jan2512:49/usr/bin/qemu-system-microblaze

-rwxr-xr-x1rootroot3652224Jan2512:49/usr/bin/qemu-system-microblazeel

-rwxr-xr-x1rootroot8132992Jan2512:49/usr/bin/qemu-system-mips

-rwxr-xr-x1rootroot8356672Jan2512:49/usr/bin/qemu-system-mips64

-rwxr-xr-x1rootroot8374336Jan2512:49/usr/bin/qemu-system-mips64el

-rwxr-xr-x1rootroot8128896Jan2512:49/usr/bin/qemu-system-mipsel

-rwxr-xr-x1rootroot3578592Jan2512:49/usr/bin/qemu-system-moxie

-rwxr-xr-x1rootroot3570848Jan2512:49/usr/bin/qemu-system-or32

-rwxr-xr-x1rootroot8701760Jan2512:49/usr/bin/qemu-system-ppc

-rwxr-xr-x1rootroot9048000Jan2512:49/usr/bin/qemu-system-ppc64

lrwxrwxrwx1rootroot17Jan2512:49/usr/bin/qemu-system-ppc64le->qemu-

system-ppc64

-rwxr-xr-x1rootroot8463680Jan2512:49/usr/bin/qemu-system-ppcemb

-rwxr-xr-x1rootroot6894528Jan2512:49/usr/bin/qemu-system-sh4

-rwxr-xr-x1rootroot6898624Jan2512:49/usr/bin/qemu-system-sh4eb

-rwxr-xr-x1rootroot4032000Jan2512:49/usr/bin/qemu-system-sparc

-rwxr-xr-x1rootroot7201696Jan2512:49/usr/bin/qemu-system-sparc64

-rwxr-xr-x1rootroot3704704Jan2512:49/usr/bin/qemu-system-tricore

-rwxr-xr-x1rootroot3554912Jan2512:49/usr/bin/qemu-system-unicore32

-rwxr-xr-x1rootroot8418656Jan2512:49/usr/bin/qemu-system-x86_64

-rwxr-xr-x1rootroot3653024Jan2512:49/usr/bin/qemu-system-xtensa

-rwxr-xr-x1rootroot3642752Jan2512:49/usr/bin/qemu-system-xtensaeb

root@kvm:~#

EachcommandcanstartaQEMU-emulatedinstanceforthespecificCPUarchitecture.Forthisrecipe,wearegoingtobeusingtheqemu-system-x86_64utility.

GettingreadyTocompletethisrecipe,youwillneedthefollowing:

TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeTheCentOSqcow2imagewedownloadedintheUsingpre-existingimagesrecipe

Let'shavealookatwhatCPUarchitecturesQEMUsupportsonthehostsystem:

root@kvm:~#qemu-system-x86_64--cpuhelp

x86qemu64QEMUVirtualCPUversion2.5+

x86phenomAMDPhenom(tm)9550Quad-CoreProcessor

x86core2duoIntel(R)Core(TM)[email protected]

x86kvm64CommonKVMprocessor

x86qemu32QEMUVirtualCPUversion2.5+

x86kvm32Common32-bitKVMprocessor

x86coreduoGenuineIntel(R)[email protected]

x86486

x86pentium

x86pentium2

x86pentium3

x86athlonQEMUVirtualCPUversion2.5+

x86n270Intel(R)Atom(TM)[email protected]

x86ConroeIntelCeleron_4x0(Conroe/MeromClassCore2)

x86PenrynIntelCore2DuoP9xxx(PenrynClassCore2)

x86NehalemIntelCorei79xx(NehalemClassCorei7)

x86WestmereWestmereE56xx/L56xx/X56xx(Nehalem-C)

x86SandyBridgeIntelXeonE312xx(SandyBridge)

x86IvyBridgeIntelXeonE3-12xxv2(IvyBridge)

x86Haswell-noTSXIntelCoreProcessor(Haswell,noTSX)

x86HaswellIntelCoreProcessor(Haswell)

x86Broadwell-noTSXIntelCoreProcessor(Broadwell,noTSX)

x86BroadwellIntelCoreProcessor(Broadwell)

x86Opteron_G1AMDOpteron240(Gen1ClassOpteron)

x86Opteron_G2AMDOpteron22xx(Gen2ClassOpteron)

x86Opteron_G3AMDOpteron23xx(Gen3ClassOpteron)

x86Opteron_G4AMDOpteron62xxclassCPU

x86Opteron_G5AMDOpteron63xxclassCPU

x86hostKVMprocessorwithallsupportedhostfeatures(onlyavailableinKVM

mode)

RecognizedCPUIDflags:

fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmovpatpse36pn

clflushdsacpimmxfxsrssesse2sshttmia64pbe

pni|sse3pclmulqdq|pclmuldqdtes64monitords_cplvmxsmxesttm2ssse3cidfma

cx16xtprpdcmpciddcasse4.1|sse4_1sse4.2|sse4_2x2apicmovbepopcnttsc-

deadlineaesxsaveosxsaveavxf16crdrandhypervisor

fsgsbasetsc_adjustbmi1hleavx2smepbmi2ermsinvpcidrtmmpxavx512frdseed

adxsmappcommitclflushoptclwbavx512pfavx512eravx512cd

syscallnx|xdmmxextfxsr_opt|ffxsrpdpe1gbrdtscplm|i643dnowext3dnow

lahf_lmcmp_legacysvmextapiccr8legacyabmsse4amisalignsse3dnowprefetch

osvwibsxopskinitwdtlwpfma4tcenodeid_msrtbmtopoextperfctr_core

perfctr_nb

invtsc

xstorexstore-enxcryptxcrypt-enace2ace2-enphephe-enpmmpmm-en

kvmclockkvm_nopiodelaykvm_mmukvmclockkvm_asyncpfkvm_steal_timekvm_pv_eoi

kvm_pv_unhaltkvmclock-stable-bit

nptlbrvsvm_locknrip_savetsc_scalevmcb_cleanflushbyasiddecodeassists

pause_filterpfthreshold

xsaveoptxsavecxgetbv1xsaves

arat

root@kvm:~#

Fromtheprecedingoutput,wecanseethelistofCPUsthatwecanpassasparameterstothe-cpuflaginordertoemulatethatCPUtypeinsideourvirtualmachine.

Howtodoit...Tostartanewvirtualmachineusingtheqemu-systemutility,performthefollowingsteps:

1. StartanewQEMUvirtualmachineusingthex86_64CPUarchitecture:

root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpu

Nehalem-m1024-driveformat=raw,index=2,file=debian.img-daemonize

root@kvm:~#

2. Ensurethattheinstanceisrunning:

root@kvm:~#pgrep-lfaqemu

3527qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m1024-

driveformat=raw,index=2,file=debian.img-daemonize

root@kvm:~#

3. TerminatetheDebianQEMUinstance:

root@kvm:~#pkillqemu

root@kvm:~#

4. StartanewQEMUinstanceusingtheprebuiltCentOSimage:

root@kvm:~#qemu-system-x86_64-vnc146.20.141.254:0-m1024-hda

CentOS-7-x86_64-GenericCloud.qcow2-daemonize

root@kvm:~#



3546qemu-system-x86_64-vnc146.20.141.254:0-m1024-hdaCentOS-7-

x86_64-GenericCloud.qcow2-daemonize

root@kvm:~#

6. TerminatetheCentOSQEMUinstance:


root@kvm:~#

MakesuretoreplacetheIPaddressofthe-vncparameterwiththeonefromyourhostmachine.

Howitworks...HowtostartavirtualmachinewithQEMU/KVMdependsgreatlyonthetypeofimageandhowthepartitionsarestructuredinsidethatimage.

Weusedtwodifferentimagetypeswithdifferentpartitioningschemestodemonstratethisconcept.

Instep1,weusedtheqemu-system-x86_64commandtoemulateax86_64CPUarchitecture,specificallywepassedthe-cpuNehalemflag,emulatingtheNehalemCPUmodel.WepassedtheIPaddressofourhostasaparametertothe-vncflag.ThisstartsaVNCserverintheVMsothatwecanlateruseaVNCclienttoconnecttotheQEMUinstance.Wespecifiedtheamountofmemorytobeallocatedtotheinstance,inthiscase,1GBwiththe-mflag.WeinstructedQEMUthatwearegoingtousearawimagewiththeformat=rawoptionandthenameandlocationoftheactualimagewiththefile=debian.imgparameter.

Recallthatthisrawimagecontainstwopartitionswiththesecondpartitioncontainingtherootfilesystemwherethebootloaderislocated.ThisisveryimportanttorememberbecauseweneedtospecifyfromwhatpartitionindextheguestOSshouldload.Wedothatwiththeindex=2flag.Finally,wepassthe-daemonizeparametertobackgroundtheQEMUprocess.

Instep4,westartedanotherQEMUinstance,thistimeusingtheqcow2CentOSimagewedownloadedearlier.Wedidnothavetospecifyfromwhatpartitionweneedtobootfromthisthistimebecausemostprebuiltimagesusethefirstpartition,oronlyhaveonepartition.Wealsousedthe-hdaflaginsteadofthe-driveparameter,justtodemonstratethatbothoptionscanbeusedwiththesameresult.The-hdaflagtellsQEMUthefirstdiskfortheinstanceshouldbeloadedfromthefilenamethatfollowsit.

StartingtheQEMUVMwithKVMsupportInthisrecipe,wearegoingtostartaQEMUvirtualmachinewithKVMacceleration.Kernel-basedVirtualMachine(KVM)isafullvirtualizationtechnologyforCPUarchitecturesthatsupportvirtualizationextensions.ForIntel-basedprocessors,thisistheIntelVT,andforAMDCPUS,itistheAMD-Vhardwareextension.ThemainpartsofKVMaretwoloadablekernelmodules,namedkvm.ko,whichprovidesthemainvirtualizationfunctionality,andasecondkernelmodulethatisprocessorspecific,kvm-intel.koandkvm-amd.koforbothmainCPUvendors.

QEMUistheuserspacecomponenttocreatevirtualmachines,whereKVMresidesinkernelspace.IfyoucompletedtheRunningvirtualmachineswithqemu-system-*recipe,youmightnotethatthedifferencebetweenrunningaKVMvirtualmachineandrunninganonacceleratedQEMUinstanceisjustasinglecommand-lineoption.

GettingreadyInordertostartaKVMinstance,youwillneedthefollowing:

TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeProcessorthatsupportsvirtualizationTheKVMkernelmodules

TocheckwhetheryourCPUsupportsvirtualization,runthefollowingcode:

root@kvm:~#cat/proc/cpuinfo|egrep"vmx|svm"|uniq

flags:fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmovpatpse36

clflushdtsacpimmxfxsrssesse2sshttmpbesyscallnxpdpe1gbrdtscplm

constant_tscarch_perfmonpebsbtsrep_goodnoplxtopologynonstop_tsc

aperfmperfeagerfpupnipclmulqdqdtes64monitords_cplvmxsmxesttm2ssse3

sdbgfmacx16xtprpdcmpciddcasse4_1sse4_2x2apicmovbepopcnt

tsc_deadline_timeraesxsaveavxf16crdrandlahf_lmabmepbtpr_shadowvnmi

flexpriorityeptvpidfsgsbasetsc_adjustbmi1avx2smepbmi2ermsinvpcidcqm

xsaveoptcqm_llccqm_occup_llcdthermaratplnpts

root@kvm:~#

Thepresenceofthevmx(forIntel)orsvm(forAMD)flagsindicatethatyourCPUsupportsthevirtualizationextensions.

Theflagsfromthecpuinfocommandoutputsimplymeanthatyourprocessorsupportsvirtualization;however,makesurethatthisfeatureisenabledintheBIOSofyoursystem;otherwise,theKVMinstancewillfailtostart.

TomanuallyloadtheKVMkernelmoduleandensurethatit'sbeenloaded,runthefollowingcode:

root@kvm:~#modprobekvm

root@kvm:~#lsmod|grepkvm

kvm4558430

root@kvm:~#

Howtodoit...TostartaKVMinstance,ensurethatit'srunningandfinallyterminateit,executethefollowing:

1. StartaQEMUinstancewithKVMsupport:

root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m

1024-driveformat=raw,index=2,file=debian.img-enable-kvm-daemonize

root@kvm:~#



4895qemu-system-x86_64-namedebian-vnc146.20.141.254:0-m1024-

driveformat=raw,index=2,file=debian.img-enable-kvm-daemonize

root@kvm:~#

3. Terminatetheinstance:


root@kvm:~#

Howitworks...TostartaQEMU/KVMvirtualmachine,allwehadtododifferentlyfromwhatweperformedintheInstallingandconfiguringQEMUrecipeispassthe-enable-kvmflagtotheqemu-system-x86_64command.

Instep1,wespecifiedanamefortheVMwiththe-nameflag,providedtheIPaddressofourphysicalhosttothe-vncflag,enablingVNCaccessforthevirtualinstance,allocated1GBofmemorywiththe-mflag,specifiedthepartitionwherethebootloaderislocatedwiththeindex=2parameter,theimageformat,andname,andfinallyweenabledKVMhardwareaccelerationwiththe-enable-kvmparameteranddeamonizedtheprocesswiththe-daemonizeflag.

Instep2,weensuredthattheinstanceisrunningandweterminateditinstep3.

There'smore...Asanalternativetodirectlyrunningtheqemu-system-*commands,onUbuntusystemsthere'stheqemu-kvmpackagethatprovidesthe/usr/bin/kvmbinary.Thisfileisawrappertotheqemu-system-x86_64command,anditpassesthe-enable-kvmparametertoitautomatically.

Toinstallthepackageandusethekvmcommandinstead,runthefollowing:

root@kvm:~#aptinstallqemu-kvm

...

root@kvm:~#kvm-namedebian-vnc146.20.141.254:0-cpuNehalem-m1024-drive

format=raw,index=2,file=debian.img-daemonize


25343qemu-system-x86_64-enable-kvm-namedebian-vnc146.20.141.254:0-cpu


root@kvm:~#

YoumighthavenotedthatstartingandstoppingQEMU/KVMinstancesissomewhatofamanualprocess,especiallyhavingtokilltheinstanceprocessinordertostopit.InChapter2,UsinglibvirttoManageKVM,wearegoingtowalkyouthroughasetofrecipesthatwillmakemanagingthelifecycleofKVMvirtualmachinesmucheasier,withtheuserspacetoolsthatthelibvirtpackageprovides.

ConnectingtoarunninginstancewithVNCInthisrecipe,wearegoingtoconnecttoarunningKVMinstanceusingaVNCclient.Onceconnected,wearegoingtologinandchecktheCPUtypeandavailablememoryoftheinstance.We'vealreadyseenhowtostartQEMU/KVMinstanceswithVNCsupportinthepreviousrecipes,butwearegoingtodoitagain,incaseyouarenotreadingthisbookfromcovertocover.

VirtualNetworkComputing(VNC)usestheRemoteFrameBuffer(RFB)protocoltoremotelycontrolanothersystem.Itrelaysthescreenfromtheremotecomputerbacktotheclient,allowingthefullkeyboardandmousecontrol.

TherearemanydifferentVNCclientandserverimplementations,butforthisrecipe,wearegoingtouseafreelyavailableversionnamedchickenoftheVNCformacOS.Youcandownloadtheclientfromhttps://sourceforge.net/projects/cotvnc/.

https://sourceforge.net/projects/chicken/

GettingreadyInordertocompletethisrecipe,youwillneedthefollowing:

TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipeThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipeAprocessorthatsupportsvirtualizationTheloadedKVMkernelmodulesThechickenoftheVNCclient,installed,asdescribedintheprevioussection

Howtodoit...1. StartanewKVM-acceleratedqemuinstance:

root@kvm:~#qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpu


root@kvm:~#



4987qemu-system-x86_64-namedebian-vnc146.20.141.254:0-cpuNehalem

-m1024-driveformat=raw,index=2,file=debian.img-daemonize

root@kvm:~#

3. StarttheVNCclientandconnecttotheVNCserverontheIPaddressanddisplayportyouspecifiedinstep1:

TheVNCloginscreen

4. Logintotheinstanceusingtherootuser,thenchecktheCPUtypeandavailablememoryasshownhere:

VNCsession

Howitworks...Instep1,westartedanewQEMUinstancewithKVMaccelerationandenabledaVNCserveronitwiththespecifiedIPaddressanddisplayport.WespecifiedtheamountofavailablememoryandtheCPUmodelname.

Instep4,weloggedintheinstanceusingtherootuserandthepasswordwecreatedwhenbuildingtheimage,thenobtainedtheCPUinformationbyrunningthelscpucommand.NotehowtheCPUmodelnamematcheswhatwespecifiedwiththe-cpuflagwhenwestartedthevirtualmachine.Next,wecheckedtheallocatedmemorywiththefreecommand,whichalsomatcheswhatwepreviouslyspecifiedwiththe-mparameter.

UsinglibvirttoManageKVMInthischapter,wewillcoverthefollowingtopics:

InstallingandconfiguringlibvirtDefiningKVMinstancesStarting,stopping,andremovingKVMinstancesInspectingandeditingKVMconfigsBuildingnewKVMinstanceswithvirt-installandusingtheconsoleManagingCPUandmemoryresourcesinKVMAttachingblockdevicestovirtualmachinesSharingdirectoriesbetweenarunningVMandthehostOSAutostartingKVMinstancesWorkingwithstoragepoolsManagingvolumesManagingsecrets

IntroductionInthepreviouschapter,wesawexamplesofprovisioningvirtualmachinesusingtheQEMUtoolsetandtheKVMkernelmodules.TheQEMUcommandsareconvenientforquicklystartingvirtualinstances;however,theydon'tprovideaneasywayofconfiguringandadministeringthelifecycleofthevirtualmachines.

Inthischapter,wearegoingtoworkwiththelibvirttoolset.Libivrtprovidesvarioususerspacecommandsandlanguagebindingsinordertobuild,configure,start,stop,migrate,terminate,anddootherfunctionstomanageyourvirtualmachines.Itprovidessupportfordifferentvirtualizationtechnologies,suchasQEMU/KVM,XEN,andcontainerswithLXC.

Wewillstartbyinstallingandconfiguringthelibvirttools,thenmoveontocreatingvirtualmachinesusingtheXMLconfigurationfilesthatlibvirtsupportsandexploremanyofthefunctionalitiesthatthetoolkitprovidesinordertomanagethelifecycleofKVMinstances.Alltherecipesinthischapteraregoingtobeinthecontextofbuildinghighlyavailable,multitenantenvironments.

InstallingandconfiguringlibvirtInthisrecipe,wearegoingtoinstalllibvirtfrompackagesprovidedbytheLinuxdistributionofchoiceandseewhatconfigurationfilesandoptionsareavailableforconfiguringit.Aswithanyotherproduction-readytools,werecommendusingpackagesforyourproductionenvironmentforeaseandconsistencyofdeployment;however,compilingthelatestversionfromthesourceisalsoanoptionifthepackagesfromyourLinuxvendorareolder.

GettingreadyDependingonyourLinuxdistribution,thepackagenameandinstallationcommandswilldiffer.Youcanuseyoursystem'spackagemanager,suchasapt,dnf,oryumtosearchforanypackagescontainingthelibvirtstringandgetfamiliarwithwhatisavailableforyourparticularLinuxvariant.Thesourcecodecanbedownloadedfromtheofficiallibvirtprojectwebsiteathttp://www.qemu-project.org/download/#source.

https://libvirt.org/downloads.html

Howtodoit...Toinstalllibvirtfrompackagesandsourcefollowthefollowingsteps:

1. OnUbuntu,installthepackagebyrunning:

root@kvm:~#aptupdate&&aptinstalllibvirt-bin

root@kvm:~#

2. Ensurethatthelibvirtdaemonisrunningbyexecuting:

root@kvm:~#pgrep-lfalibvirtd

36667/usr/sbin/libvirtd

root@kvm:~#

3. Examinethedefaultconfiguration:

root@kvm:~#cat/etc/libvirt/libvirtd.conf|grep-vi"#"

|sed'/^$/d'

unix_sock_group="libvirtd"

unix_sock_ro_perms="0777"

unix_sock_rw_perms="0770"

auth_unix_ro="none"

auth_unix_rw="none"

root@kvm:~#

4. DisablethesecuritydriverinQEMUbyeditingtheqemuconfigurationfileasfollows:

root@kvm:~#vim/etc/libvirt/qemu.conf

...

security_driver="none"

...

root@kvm:~#

5. Restartthelibvirtdaemon:

root@kvm:~#/etc/init.d/libvirt-binrestart

libvirt-binstop/waiting

libvirt-binstart/running,process1158

root@kvm:~#

DependingonyourLinuxdistribution,thenameofthelibvirtservicemaybedifferent.OnRHEL/CentOS,thenameoftheserviceislibvirtd;torestartit,runservicelibvirtdrestart.

6. Examineallconfigurationfilesinthelibvirtdirectory:

root@kvm:~#ls-la/etc/libvirt/

total76



drwxr-xr-x2rootroot4096Feb52016hooks

-rw-r--r--1rootroot518Feb52016libvirt.conf

-rw-r--r--1rootroot13527Feb52016libvirtd.conf

-rw-r--r--1rootroot1176Feb52016lxc.conf

drwxr-xr-x2rootroot4096Mar2123:16nwfilter

drwxr-xr-x3rootroot4096Mar2123:57qemu

-rw-------1rootroot16953Mar2123:18qemu.conf

-rw-r--r--1rootroot2170Feb52016qemu-lockd.conf

-rw-r--r--1rootroot2213Feb52016virtlockd.conf

-rw-r--r--1rootroot1217Feb52016virt-login-shell.conf

root@kvm:~#

Howitworks...Instep1,weinstalledthepackageonUbuntu.Thepostinstallscriptstartedthelibvirtdaemonafterthepackagewassuccessfullyinstalled.Weverifiedthatinstep2.

Instep3,weexaminedthemainconfigurationfilefortheservice-sidedaemon-libvirtd.TheprocessrunsonthehostOSandmanagestasksforthevirtualmachines,suchasconfiguration,lifecyclemanagement,migration,storage,andnetworking,aswearegoingtoseelaterinthischapter.TheuserspacetoolsprovidedbythepackageweinstalledcommunicatewiththedaemonbysendingrequestsonalocalUnixdomainsocket.Thedefaultoptionswesawinstep3aresufficientfortherecipesinthischapter,buttheconfigurationfileisratherlarge.Weencourageyoutogothroughitandgetfamiliarwiththerestoftheavailableconfigurationoptions.Thefileisverywelldocumented.

Instep4,wedisabledthesecuritydriverforQEMU.BydefaultonRHEL/CentOSsystems,QEMUisconfiguredtouseSELinux.UbuntudistributionsuseAppArmor.Forsimplicity,wedisablethatfunctionalityinthisstep;however,inproduction,youshouldtakeadvantageoftheextrasecuritythatamandatoryaccesscontrolsystemsuchasSELinuxprovides.

Anychangetothelibvirtconfigurationfilerequiresarestart.Werestartthelibvirtserviceinstep5.

Therearefewimportantconfigurationfilesthatweneedtobefamiliar,whicharelistedinstep6:

libvirt.confistheclient-sideconfigurationfileforthevirshcommandthatwearegoingtouseinthisrecipe.WecanspecifyURIaliasesinit.Thedefaultsshouldbesufficient.libvirtd.confistheserver-sideconfigurationfile,aswesawinstep3.Itprovidesvarioussecurityoptions,requestlimits,andloggingcontrols.Forthepurposeofthisbook,thedefaultsaresufficient.qemu.confisthemainconfigurationfilefortheQEMUdriverthatlibvirtuses.WecanconfigureoptionssuchastheVNCserveraddress,the

securitydriverthatwesawinstep4andtheuserandgroupfortheQEMUprocess.OncewecreateaQEMU/KVMvirtualmachine,the/etc/libvirt/qemu/directorywillcontaintheXMLconfigurationdefinitionforthatinstance,aswearegoingtoseeinthefollowingrecipes.Finally,the/etc/libvirt/qemu/networks/directorycontainsconfigurationfilesforthenetworking.Wearegoingtoexplorethoseinmoredetaillaterinthischapter.

DefiningKVMinstancesInthisrecipe,wearegoingtodefineavirtualinstancebycreatingasimpleXMLconfigurationfilethatlibvirtcanusetobuildthevirtualmachine.WearegoingtodescribesomeoftheXMLschemablocksandlookatexamplesofhowtogeneratetheXMLdefinitionfileusingthevirt-installcommandratherthanwritingitmanually.

GettingreadyForthisrecipe,wearegoingtoneedthefollowing:

TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVM.ThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipefromthepreviouschapter.

YoucanuseyourownvirtualmachineimageordownloadonefromtheInternet,asweshowedintheUsingpre-existingimagesrecipeinChapter1,GettingStartedwithQEMUandKVM.

Howtodoit...TodefineanewKVMvirtualmachine,runthecommandsoutlinedhere:

1. ListallvirtualmachinesonthehostOS:

root@kvm:~#virshlist--all

IdNameState

----------------------------------------------------

root@kvm:~#

2. CreatethefollowingXMLdefinitionfile:

root@kvm:~#catkvm1.xml

<domaintype='kvm'id='1'>

<name>kvm1</name>

<memoryunit='KiB'>1048576</memory>

<vcpuplacement='static'>1</vcpu>

<os>

<typearch='x86_64'machine='pc-i440fx-trusty'>hvm</type>

<bootdev='hd'/>

</os>

<on_poweroff>destroy</on_poweroff>

<on_reboot>restart</on_reboot>

<on_crash>restart</on_crash>

<devices>

<emulator>/usr/bin/qemu-system-x86_64</emulator>

<disktype='file'device='disk'>

<drivername='qemu'type='raw'/>

<sourcefile='/tmp/debian.img'/>

<targetdev='hda'bus='ide'/>

<aliasname='ide0-0-0'/>

<addresstype='drive'controller='0'bus='0'target='0'unit='0'/>

</disk>

<interfacetype='network'>

<sourcenetwork='default'/>

<targetdev='vnet0'/>

<modeltype='rtl8139'/>

<aliasname='net0'/>

<addresstype='pci'domain='0x0000'bus='0x00'slot='0x03'

function='0x0'/>

</interface>

<graphicstype='vnc'port='5900'autoport='yes'

listen='146.20.141.158'>

<listentype='address'address='146.20.141.158'/>

</graphics>

</devices>

<seclabeltype='none'/>

</domain>

root@kvm:~#

3. Definethevirtualmachine:

root@kvm:~#virshdefinekvm1.xml

Domainkvm1definedfromkvm1.xml

root@kvm:~#

4. Listallinstancesinallstates:


IdNameState

----------------------------------------------------

-kvm1shutoff

root@kvm:~#

Howitworks...Instep1,weusedthevirshcommandandsuppliedallargumenttolistallactiveandinactiveinstances.Asexpected,westartedwithnoinstancesdefined.

Instep2,wecreatedadefinitionfileforanewKVMinstance.WeusedasmallsubsectionoftheavailableXMLschemaattributestosetthefollowingoptions:

TherootelementoftheXMLfileisrequiredforallvirtualmachinedefinitionsandisnameddomain.Ithastwoattributes--typeandid.Wespecifiedkvmasthetypeandanidas1becausethisisourfirstKVMvirtualmachine.Allotherattributesaredefinedunderthedomainrootelement.Wespecifiedanamefortheinstancewiththenameattribute.ThememoryattributedefinestheavailablememorytotheVM,inourcase,1GB.ThevcpuelementdefinesthemaximumnumberofvirtualCPUsallocatedfortheguestOS.Wespecified1,andweusedtheoptionalattributeplacementthatindicatestheCPUplacementmode;inthisexample,static.StaticplacementindicatesthatthevirtualinstancewillbepinnedtoalltheavailablephysicalCPUs.TheOSelementdefinesthearchitectureoftheVMwiththeuseofthetypeelement.Thehvmoptionindicatesthatwearegoingtousefullvirtualization,whichisgoingtobeKVM,asspecifiedinthedomaintypeattributewesawearlier.WespecifythebootdevicetheVMwillstartfromwiththe<bootdev>element.Thenextthreeelementsspecifytheactiontobetakenwhentheguestrequestsapoweroff,rebootoritcrashes.Inourexample,theVMwillbedestroyedwhentheguestOSispoweredoffandrestartedwhentheguestrebootsorcrashes.ThelargestsectionoftheXMLdefinitionisthedevicessection,whereweusevariousXMLelementstodescribedevicesprovidedtotheguestOS.Theemulatorelementspecifiesthepathtotheemulatorbinary.WearegoingtousethesameQEMUemulatorbinaryqemu-system-x86_64weusedinChapter1,GettingStartedwithQEMUandKVM.Inthelastfewsectionsofthedevicesattribute,wedefinethetypeofvirtualdiskweareusing,inthis

example,therawimagewebuiltinthepreviouschapter.Inasimilarfashion,wedescribetheVNCserverthattheguestshouldstartandthenetworkinterfaceinsidetheguestOS.

Withtheconfigfileinplace,wedefinedtheinstanceinstep3,usingtheimagewecreatedearlierin/tmp.

Onceanewinstancehasbeendefined,itdoesnotautomaticallystartbydefault.Wecanseethatthestatusofthenewinstanceisshutoffinstep4.

ForinformationonalloftheavailableXMLelementsandtheirattributes,pleaserefertotheofficialdocumentationathttp://libvirt.org/formatdomain.html.

http://libvirt.org/formatdomain.html

There'smore...ConfiguringavirtualmachinebywritingtheXMLfile,canbequitetediousanderror-prone.AneasierwayofcreatingtheVMfromanexistingimage,orfromaninstallationmedia(whichcanbephysical,virtual,oranetworklocation),isusingthevirt-installtool.LetsseeanexampleofcreatingthesameKVMinstanceusingthattool.

1. Westartbyinstallingthepackage:

root@kvm:~#aptinstallvirtinst

...

root@kvm:~#

2. Next,wedefineandstartthenewinstancebyinvokingthevirt-installcommand(ifaninstancewiththesamenamealreadyexist,you'llneedtodestroyandundefineitfirst):

root@kvm:~#virt-install--namekvm1--ram1024--disk

path=/tmp/debian.img,format=raw--graphicsvnc,listen=146.20.141.158--

noautoconsole--hvm--import

Startinginstall...

Creatingdomain...|0B00:00

Domaincreationcompleted.Youcanrestartyourdomainbyrunning:

virsh--connectqemu:///systemstartkvm1

root@kvm:~#

3. ThenewVMhasnowbeendefinedandstarted.Toconfirm,execute:


IdNameState

----------------------------------------------------

10kvm1running

root@kvm:~#

4. Wecanseethevirtualmachinedefinitionfilethatwasautomaticallygeneratedbyrunningthefollowingcode:

root@kvm:~#cat/etc/libvirt/qemu/kvm1.xml



<domaintype='kvm'>

<name>kvm1</name>

<uuid>c3892cbf-812a-2448-7ad2-098ea8381066</uuid>


<currentMemoryunit='KiB'>1048576</currentMemory>


<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>

<clockoffset='utc'/>




<devices>







</disk>

<controllertype='usb'index='0'>


function='0x2'/>

</controller>

<controllertype='pci'index='0'model='pci-root'/>

<controllertype='ide'index='0'>


function='0x1'/>

</controller>


<macaddress='52:54:00:59:e3:4e'/>




function='0x0'/>

</interface>

<serialtype='pty'>

<targetport='0'/>

</serial>

<consoletype='pty'>

<targettype='serial'port='0'/>

</console>

<inputtype='mouse'bus='ps2'/>

<inputtype='keyboard'bus='ps2'/>

<graphicstype='vnc'port='-1'autoport='yes'

listen='146.20.141.158'>


</graphics>

<video>

<modeltype='cirrus'vram='9216'heads='1'/>


function='0x0'/>

</video>

<memballoonmodel='virtio'>


function='0x0'/>

</memballoon>

</devices>

</domain>

root@kvm:~#

Starting,stopping,andremovingKVMinstancesInthepreviousrecipe,wesawhowtodefinenewKVMvirtualmachinebyeithermanuallywritingtheXMLdefinitionfileorusingthevirt-installtooltodefinetheinstanceforus.

IfyoudefineanewinstancefromanXMLfile,bydefaulttheinstancewillnotstartautomatically.Inthisrecipe,wewillseehowtostartaninstancethatwaspreviouslyconfigured.


TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVM.ThecustomrawDebianimagewebuiltintheInstallingacustomOSontheimagewithdebootstraprecipefromthepreviouschapter.ThevirshtoolprovidedbycompletingtheInstallingandconfiguringlibvirtrecipe.ThedefinedinstancefromtheDefiningKVMinstancesrecipeinashutoffstate.

Howtodoit...Thefollowingstepsoutlinetheprocessoflisting,starting,andstoppingKVMinstancesusingthevirshcommand:

1. Listallinstancesinallstates:


IdNameState

----------------------------------------------------

-kvm1shutoff

root@kvm:~#

2. Startthenewlydefinedinstanceandverifyitsstatus:

root@kvm:~#virshstartkvm1

Domainkvm1started

root@kvm:~#


IdNameState

----------------------------------------------------

1kvm1running

root@kvm:~#

3. Examinetherunningprocessforthevirtualmachine:


1686/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-i440fx-

trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp

1,sockets=1,cores=1,threads=1-uuida9dfd1a1-7dd1-098e-a926-db9526785a9e

-no-user-config-nodefaults-chardev

socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait

-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-

shutdown-bootstrict=on-devicepiix3-usb-

uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive

file=/tmp/debian.img,if=none,id=drive-ide0-0-0,format=raw-deviceide-

hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1-netdev

tap,fd=24,id=hostnet0-device

rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:ce:dd:f2,bus=pci.0,addr=0x3

-chardevpty,id=charserial0-deviceisa-

serial,chardev=charserial0,id=serial0-vnc146.20.141.158:0-device

cirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-

pci,id=balloon0,bus=pci.0,addr=0x4

root@kvm:~#

4. TerminatetheVMandensureitsstatuschangedfromrunningtoshutoff:

root@kvm:~#virshdestroykvm1

Domainkvm1destroyed


IdNameState

----------------------------------------------------

-kvm1shutoff

root@kvm:~#

5. Removetheinstancedefinition:

root@kvm:~#virshundefinekvm1

Domainkvm1hasbeenundefined


IdNameState

----------------------------------------------------

root@kvm:~#

Howitworks...Instep1,welistalldefinedinstances,regardlessoftheirstate.Fromtheoutput,wecanseethatwecurrentlyhaveoneinstancethatwedefinedintheearlierrecipe.

Instep2,westartedthevirtualmachineandensureditsstatushadchangedtorunning.

IfyoucompletedtheRunningVirtualMachineswithqemu-system-*recipefromChapter1,GettingStartedwithQEMUandKVM,youmightnotethattheXMLdefinitionforthisVMisverysimilartoallthecommand-lineoptionsweusedtostarttheQEMUinstance.Wecanseethesimilaritiesofhowthenewinstancewasstartedinstep3.ThemaindifferenceisthelargernumberofparametersthatlibvirtpassedtotheQEMUexecutable.

Finally,insteps4and5,westoppedtheVMandremoveditsdefinitionfile.TherawimageweusedfortheVMisstillavailablehoweverandcanbeusedagain.

InspectingandeditingKVMconfigsInthisrecipe,wearegoingtousethevirshtooltoinspectandedittheconfigurationforanexistingvirtualmachine.Aswesawearlier,oncewedefineandstartaKVMinstance,libvirtcreatestheXMLdefinitionfileinthe/etc/libvirt/qemu/directory.Wecandumptheguestconfigurationtodisk,forinspection,ortobackitup.Withthevirshcommandwecanalsoperformupdatestotheconfigurationinplace,aswewillseelaterinthisrecipe.


TheQEMUbinaries,providedafterfollowingtheInstallingandconfiguringQEMUrecipefromChapter1,GettingStartedwithQEMUandKVMThecustomrawDebianimagewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromthepreviouschapter,oranyothervirtualmachineimage,ineitherraworqcow2formatThevirshtoolprovidedbycompletingtheInstallingandconfiguringlibvirtrecipeArunninglibvirtKVMinstance

Howtodoit...ThefollowingstepsoutlinetheprocessofinspectingandeditingtheXMLdefinitionofaKVMinstance:

1. EnsurethatyouhavearunningKVMinstancewithlibvirt,ifnot,followthestepsinthepreviousrecipe:

root@kvm:~#virshlist

IdNameState

----------------------------------------------------

11kvm1running

root@kvm:~#

2. Dumptheinstanceconfigurationfiletostandardoutput(stdout).Formoreinformationonstdoutrefertofolllowinglink:

https://en.wikipedia.org/wiki/Standard_streams

root@kvm:~#virshdumpxmlkvm1


<name>kvm1</name>

<uuid>9eb9a2e9-abb2-54c5-5cb3-dc86728e70fc</uuid>




<resource>

<partition>/machine</partition>

</resource>

<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>






https://en.wikipedia.org/wiki/Standard_streams



</disk>


<aliasname='usb0'/>


function='0x2'/>

</controller>

<controllertype='pci'index='0'model='pci-root'>

<aliasname='pci.0'/>

</controller>


<aliasname='ide0'/>


function='0x1'/>

</controller>


<macaddress='52:54:00:d1:70:df'/>




<aliasname='net0'/>


function='0x0'/>

</interface>

<serialtype='pty'>

<sourcepath='/dev/pts/0'/>

<targetport='0'/>

<aliasname='serial0'/>

</serial>

<consoletype='pty'tty='/dev/pts/0'>




</console>



<graphicstype='vnc'port='5900'autoport='yes'

listen='146.20.141.158'>


</graphics>

<video>


<aliasname='video0'/>


function='0x0'/>

</video>


<aliasname='balloon0'/>


function='0x0'/>

</memballoon>

</devices>


</domain>

root@kvm:~#

3. Savetheconfigurationtoanewfile,asfollows:

root@kvm:~#virshdumpxmlkvm1>kvm1.xml

root@kvm:~#headkvm1.xml


<name>kvm1</name>

<uuid>9eb9a2e9-abb2-54c5-5cb3-dc86728e70fc</uuid>




<resource>


</resource>

<os>

root@kvm:~#

4. EdittheconfigurationinplaceandchangetheavailablememoryfortheVM:

root@kvm:~#virsheditkvm1

Domainkvm1XMLconfigurationedited.

root@kvm:~#

Howitworks...Libvirtprovidestwomainwaystomanipulatetheconfigurationdefinitionsofthevirtualinstances.Wecaneitherdumptheconfigfromanexistinginstance,aswedidinsteps2and3,oredittheXMLdefinitioninplace,aswedidinstep4.

SavingthecurrentconfigurationtoafileisaconvenientwaytobackuptheVMdefinition.ItalsoprovidesawayofdefininganewinstancebyeditingthesavedfileandjustchangingthenameandIDofthevirtualmachine.WecanthenusethatfiletostartanewVMonthesame,oradifferenthost,assumingthatthefilesystemorimageisalsoavailable.Wearegoingtoseeexamplesofmigratingandbackingupvirtualmachineswithlibvirtinlaterrecipes.

Whenmakingchangesinplace,asshowninstep4,thedefaultsystem$EDITORwillbeused.Onceintheeditingmode,notethattheXMLfilecontainsinformationaboutthecurrentstateofthevirtualinstance.The<uuid>and<currentMemory>attributesaresuchexamples.IfyouwouldliketochangetheavailablememoryfortheVM,afterupdatingthe<memory>attribute,youmightneedtodeletethe<currentMemory>stanza.Ifthereareanyissueswiththeedit,libvirtwillcomplainwithanerrormessageandpresentthefollowingoptions:


error:XMLerror:currentmemory'1048576k'exceedsmaximum'524288k'

Failed.Tryagain?[y,n,f,?]:n

Domainkvm1XMLconfigurationnotchanged.

root@kvm:~#

Alsokeepinmindthat,ifyouwouldliketocreateanewinstancefromthedumpofanexistingone,youwillneedtochangethe<name>anddeletethe<uuid>attributes,asthelatterwillbeautogeneratedoncethenewinstancehasbeendefined.

BuildingnewKVMinstanceswithvirt-installandusingtheconsoleIntheConnectingtotherunninginstancewithVNCrecipefromChapter1,GettingStartedwithQEMUandKVM,youlearnedhowtoconnecttoaQEMU/KVMvirtualmachinethatwasrunningaVNCserver.Thisisagreatwaytoconnecttoaninstancethatisbeinginstalledorintheprocessofbootinginordertointeractwithit.

Sofar,we'veusedthecustomrawimagethatwecreatedearlier,whichcontainsaninstallationofDebian.RecallfromChapter1,GettingStartedwithQEMUandKVM,thatweusedthedebootstrapcommandtoinstalltheOSinsidetheimagefile.Inthisrecipe,wearegoingtousethevirt-installtooltoinstallanewLinuxdistribution,usingtheprovidedupstreamInternetrepository,asthesourceoftheinstallationandthenusethevirshcommandtoattachtotherunninginstance,usingtheconsole.


ThevirshcommandThevirt-installcommandInternetconnectivityinordertodownloadtheinstallationfiles

Howtodoit...TobuildanewKVMinstanceandconnecttoitusingtheconsole,performthefollowingsteps:

1. InstallanewKVMvirtualmachineusingtheofficialDebianrepository:

root@kvm:~#virt-install--namekvm1--ram1024--extra-args="text

console=tty0utf8console=ttyS0,115200"--graphics

vnc,listen=146.20.141.158--hvm--

location=http://ftp.us.debian.org/debian/dists/stable/main/installer-

amd64/--diskpath=/tmp/kvm1.img,size=8

RetrievingfileMANIFEST...|3.3kB00:00...

Retrievingfilelinux...|6.0MB00:00...

Retrievingfileinitrd.gz...|29MB00:00...

Creatingstoragefilekvm1.img|8.0GB00:00

WARNINGUnabletoconnecttographicalconsole:virt-viewernot

installed.Pleaseinstallthe'virt-viewer'package.

Domaininstallationstillinprogress.Youcanreconnectto

theconsoletocompletetheinstallationprocess.

root@kvm:~#

2. Attachtotheconsoletocompletetheinstallationbyrunningthefollowingcode:

root@kvm:~#virshconsolekvm1

Connectedtodomainkvm1

Escapecharacteris^]

3. Onceconnectedtotheconsole,youshouldbepresentedwithascreensimilartotheonehere:

Theconsoleoutputonceconnectedwiththevirshconsolecommand

4. Completetheinstallationbyfollowingthetextmenuprompts.5. StartthenewlyprovisionedVM:


Domainkvm1started

root@kvm:~#

6. UsingyourfavoriteVNCclient,connecttotheinstance,loginwiththeusernameandpasswordyoucreatedduringtheinstallationprocessinstep3andenabletheserialconsoleaccessbyrunningthefollowingcommand:

root@debian:~#[email protected]

root@debian:~#[email protected]

root@debian:~#

7. ClosetheVNCsessionandconnecttothevirtualinstancefromthehostOS,usingvirsh:



Escapecharacteris^]

DebianGNU/Linux8debianttyS0

debianlogin:root

Password:

Lastlogin:WedMar2216:38:10CDT2017ontty1

Linuxdebian3.16.0-4-amd64#1SMPDebian3.16.39-1+deb8u2(2017-03-07)

x86_64

TheprogramsincludedwiththeDebianGNU/Linuxsystemarefree

software;

theexactdistributiontermsforeachprogramaredescribedinthe

individualfilesin/usr/share/doc/*/copyright.

DebianGNU/LinuxcomeswithABSOLUTELYNOWARRANTY,totheextent

permittedbyapplicablelaw.

root@debian:~#free-m

totalusedfreesharedbufferscached

Mem:1000989024943

-/+buffers/cache:44956

Swap:3820382

root@debian:~#

8. DisconnectfromtheconsoleusingtheCtrl+]keycombination.9. Examinetheimagefilecreatedaftertheinstallation:

root@kvm:~#qemu-imginfo/tmp/kvm1.img

image:/tmp/kvm1.img

fileformat:raw


disksize:1.9G

root@kvm:~#

Ifyouarenotusingsystemd-basedinitsystemonthedistributionfortheKVMmachine,inordertoallowaccesstotheserialconsoleoftheinstance,youwillneedtoeditthe/etc/securettyorthe/etc/inittabfiles.

Howitworks...Alothappenedinthisrecipe,soletsgothroughallthestepsinmoredetail.

Instep1,westartedtheinstallationprocessforanewKVMinstanceusingthevirt-installutility.Wespecifiedtheserialconsoletobeenabledduringtheinstallationprocesswiththe--extra-argsparameter.Wealsousedthe--locationflagtotelllibvirtthelocationoftheinstallationfilesforthelatestDebiandistribution.WethenspecifiedthelocationandsizeoftheimagefilethatwillcontaintheguestOSfilesystem.Sincethisfiledidnotexist,virt-installcreateditasarawimage,asshowninstep9.

Withconsoleaccessenabledfortheinstallation,wewereabletoconnecttotheconsoleinstep2andcompletetheinstallationprocessinsteps3and4.

Aftertheinstallationcompleted,theconsolesessionwasterminatedandthenewKVMinstancereadytobestarted.Westartedtheinstanceinstep5.

Inordertoenableconsoleaccessontheserialport,wefirstconnectedtotherunningVMusingaVNCclientandinstructsystemdtostartedtheconsoleserviceinstep6.

Withconsoleaccessenabled,wewereabletoconnecttotheserialconsoleusingthevirshtoolinstep7.

Withallthiscompleted,wenowhavetwowaysofconnectingtoarunningKVMinstanceusingeitherVNCortheconsole.

Inthelaterrecipe,wewillenablenetworkingintheguestOSandprovideathirdwaytoconnectusingSSH.

ManagingCPUandmemoryresourcesinKVMChangingtheamountofallocatedmemoryorthenumberofCPUscanbedoneeitherbyeditingtheXMLdefinitionfortheVMorusingthelibvirttoolset.Inthisrecipe,wearegoingtolookatexamplesofchangingboththememoryandtheCPUcountforaKVMinstance.


ArunningKVMinstancewith1GBofmemory,1CPUallocated,andconsoleaccessThelibvirtpackageAguestOSwithatleast4GBofavailablememoryandminimumof4CPUs

Howtodoit...ToinspectandupdatethememoryandCPUresourcesassignedtoavirtualmachinefollowtheprocessoutlinedhere:

1. Getmemorystatisticsfortherunninginstance:

root@kvm:~#virshdommemstatkvm1

actual1048576

swap_in0

rss333644

root@kvm:~#

2. UpdatetheavailablememoryfortheVMto2GB:

root@kvm:~#virshsetmemkvm1--size1049000

root@kvm:~#

3. Stoptherunninginstance:


Domainkvm1destroyed

root@kvm:~#

4. Setthemaximumusablememoryto2GB:

root@kvm:~#virshsetmaxmemkvm1--size2097152

root@kvm:~#

5. Starttheinstance:


Domainkvm1started

root@kvm:~#

6. Checkthecurrentallocatedmemory:

root@kvm:~#virshdommemstatkvm1

actual2097152

swap_in0

rss214408

root@kvm:~#

7. ConnecttotheKVMinstanceandcheckthememoryintheguestOS:



Escapecharacteris^]


debianlogin:root

Password:

...

root@debian:~#free-m

totalusedfreesharedbufferscached

Mem:20109319175840

-/+buffers/cache:431966

Swap:3820382

root@debian:~#

root@kvm:~#

8. CheckthememorysettingsintheinstanceXMLdefinition:

root@kvm:~#virshdumpxmlkvm1|grepmemory


root@kvm:~#

9. GetinformationabouttheguestCPUs:

root@kvm:~#virshvcpuinfokvm1

VCPU:0

CPU:29

State:running

CPUtime:9.7s

CPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

root@kvm:~#

10. ListthenumberofvirtualCPUsusedbytheguestOS:

root@kvm:~#virshvcpucountkvm1

maximumconfig1

maximumlive1

currentconfig1

currentlive1

root@kvm:~#

11. ChangethenumberofallocatedCPUsto4fortheVM:


...


...


root@kvm:~#

12. EnsurethattheCPUcountupdatetookeffect:

root@kvm:~#virshvcpucountkvm1

maximumconfig4

maximumlive4

currentconfig4

currentlive4

root@kvm:~#virshdumpxmlkvm1|grep-icpu


root@kvm:~#

Howitworks...Instep1,wegatheredsomememorystatisticsfortherunningKVMinstance.Fromtheoutput,wecanseethattheVMisconfiguredwith1GBofmemoryindicatedbytheactualparameter,andit'scurrentlyusing333644KBofmemory.

Instep2,weupdatedtheavailablememoryto2GBandthenproceededtoupdatethemaximummemorythatcanbeallocatedtotheinstanceinstep4.Inordertoperformthatoperation,theinstancehadtobestoppedfirst,asshowninstep3.

Insteps6,7,and8,wemadesurethattheupdatestookplacebyfirstinvokingthedommemstatsubcommand,thenconnectedtotheVMsconsoleandfinallycheckedthecurrentconfigurationbydumpingtheinstancedefinition.

ThevirshcommandprovidesfewsubcommandstoinspecttheCPUstateforarunningVM.Insteps9and10,welistedtheallocatedvirtualCPUsforthekvm1instance,inthiscase,justoneandthecurrentstate,load,andaffinity.

Finally,insteps11and12,weupdatetheXMLdefinitionoftheinstance,allocatingfourCPUsandlistedthenewcount.

There'smore...Inthisrecipe,weusedthevirshcommandwithvarioussubcommandsinoneliners.Thisisparticularlyusefulifweneedtorunthecommandsfromascript.Thevirshcommandalsoprovidesaninteractiveterminal,whichsavessometyping,andprovidescontextualhelp.Tostartthevirtualization-interactiveterminal,runthefollowingcode:

root@kvm:~#virsh

Welcometovirsh,thevirtualizationinteractiveterminal.

Type:'help'forhelpwithcommands

'quit'toquit

virsh#

Typinghelpwilllistallavailablesubcommandswithashortdescription.Toobtainmoreinformationforaparticularsubcommandtype:

virsh#helpvcpucount

NAME

vcpucount-domainvcpucounts

SYNOPSIS

vcpucount<domain>[--maximum][--active][--live][--config][--current][--

guest]

DESCRIPTION

ReturnsthenumberofvirtualCPUsusedbythedomain.

OPTIONS

[--domain]<string>domainname,idoruuid

--maximumgetmaximumcountofvcpus

--activegetnumberofcurrentlyactivevcpus

--livegetvaluefromrunningdomain

--configgetvaluetobeusedonnextboot

--currentgetvalueaccordingtocurrentdomainstate

--guestretrievevcpucountfromtheguestinsteadofthehypervisor

virsh#

Allthestepsweperformedinthisrecipecanbedoneintheinteractiveterminal.

AttachingblockdevicestovirtualmachinesInthisrecipe,wearegoingtoexamineafewdifferentwaysofaddingnewblockdevicestoaKVMinstance.Thenewblockdevicecanthenbepartitioned,formatted,andusedasaregularblockdeviceinsidetheguestOS.Wecanadddiskstoliverunninginstances,orwecanattachthempersistentlybycreatingXMLdefinitionsfortheindividualblockdevicesoffline.FromthehostOS,wecanpresentanytypeofblockdevicefiletotheguest,includingiSCSItargets,LVMlogicalvolumes,orimagefiles.

GettingreadyForthisrecipe,wewillneed:

ArunningKVMinstancewithconsoleaccessTheddutility

Howtodoit...ToattachanewblockdevicetoaKVMguest,runthefollowing:

1. Createanew1GBimagefile:

root@kvm:~#ddif=/dev/zeroof=/tmp/new_disk.imgbs=1Mcount=1024

1024+0recordsin

1024+0recordsout

1073741824bytes(1.1GB)copied,0.670831s,1.6GB/s

root@kvm:~#

2. AttachthefileasanewdisktotheKVMinstance:

root@kvm:~#virshattach-diskkvm1/tmp/new_disk.imgvda--live

Diskattachedsuccessfully

root@kvm:~#

3. ConnecttotheKVMinstanceviatheconsole:



Escapecharacteris^]


debianlogin:root

Password:

...

root@debian:~#

4. Printthekernelringbufferandcheckforthenewblockdevice:

root@debian:~#dmesg|grepvda

[3664.134978]sd2:0:2:0:[vda]2097152512-bytelogicalblocks:(1.07

GB/1.00GiB)

[3664.135248]sd2:0:2:0:[vda]WriteProtectisoff

[3664.135251]sd2:0:2:0:[vda]ModeSense:63000008

[3664.135340]sd2:0:2:0:[vda]Writecache:enabled,readcache:

enabled,doesn'tsupportDPOorFUA

[3664.138254]vda:unknownpartitiontable

[3664.139008]sd2:0:2:0:[vda]AttachedSCSIdisk

root@debian:~#

5. Examinethenewblockdevice:

root@debian:~#fdisk-l/dev/vda

Disk/dev/vda:1GiB,1073741824bytes,2097152sectors

Units:sectorsof1*512=512bytes



root@debian:~#

6. DumptheinstanceconfigurationfromthehostOS:



...

<devices>




<sourcefile='/tmp/kvm1.img'/>




</disk>



<sourcefile='/tmp/new_disk.img'/>

<targetdev='vda'bus='scsi'/>

<aliasname='scsi0-0-2'/>


</disk>

</devices>

</domain>

root@kvm:~#

7. Getinformationaboutthenewdisk:

root@kvm:~#virshdomblkstatkvm1vda

vdard_req119

vdard_bytes487424

vdawr_req0

vdawr_bytes0

vdaflush_operations0

vdard_total_times29149092

vdawr_total_times0

vdaflush_total_times0

root@kvm:~#

8. Detachthedisk:

root@kvm:~#virshdetach-diskkvm1vda--live

Diskdetachedsuccessfully

root@kvm:~#

9. Copyorcreateanewrawimage:

root@kvm:~#cp/tmp/new_disk.img/tmp/other_disk.img

root@kvm:~#

10. Writethefollowingconfigfile:

root@kvm:~#catother_disk.xml


<drivername='qemu'type='raw'cache='none'/>

<sourcefile='/tmp/other_disk.img'/>

<targetdev='vdb'/>

</disk>

root@kvm:~#

11. Attachthenewdevice:

root@kvm:~#virshattach-devicekvm1--liveother_disk.xml

Deviceattachedsuccessfully

root@kvm:~#

12. Detachtheblockdevice:

root@kvm:~#virshdetach-devicekvm1other_disk.xml--live

Devicedetachedsuccessfully

root@kvm:~#

Howitworks...AttachingmorediskstorunningKVMinstancescanbequiteuseful,especiallywhenusingLVMinsidetheguestOS,asthisallowsforextendingthelogicalvolumes,thusaddingmorediskspaceonthego.Libvirtprovidestwodifferentmethodsforthisaswesawinthestepsoutlinedearlier.Wecanusethevirshattach-diskcommandbypassingthelocationoftheimagefileandthenameofthenewblockdevicefortheguestVMaswesawinstep2.

Instep1,wecreatedanewrawimageusingtheddcommand;however,wecouldhaveusedtheqemu-imgtoolaswesawintheManagingDiskimageswithqemu-imgandddrecipefromChapter1,GettingStartedwithQEMUandKVM.

Afterattachingthenewdiskinstep2,insteps3,4,and5,weconnectedtoVMandverifiedthatanewblockdeviceisindeedpresent.ThisisalsoreflectedintheXMLdefinitionoftheinstanceinstep6.

TomakethenewdeviceavailableafteraVMrestartandpersisttheXMLdefinitionchanges,passthe--persistoptiontothevirshattach-diskcommand.

Instep7,wedisplaysomeinformationaboutthenewdisk.Thisdataisquiteusefulinordertomonitortheread/writerequestsfortheblockdevice,withouthavingtoattachtothevirtualinstance.

Instep8,wedetachedthediskfromtherunningKVMinstance.Ifyoudumptheinstancedefinitionatthispoint,youwillnotetheabsenceoftheextradisk.

Analternativewayofattachingablockdeviceisshowninstep10.WefirstcreateanewXMLfilewiththedefinitionoftheblockdeviceweareattaching.Notehowsimilarthedefinitionistotheoutputinstep6.

Instep11,wedetachthenewdeviceyetagain.NotethatwehavetospecifythesamedeviceXMLdefinitionfileinordertodothat.

OncethediskisvisibleinsidetheguestOS,wecanuseitasaregularblockdevice,wecanpartitionit,createafilesystem,andmountit.

device,wecanpartitionit,createafilesystem,andmountit.

SharingdirectoriesbetweenarunningVMandthehostOSInthepreviousrecipe,wesawtwoexamplesonhowtoattachdiskstoarunningKVMinstance.Inthisrecipe,wearegoingtoshareadirectoryfromthehostOSandmakeitavailableinthevirtualmachine.Wecanonlyperformthisactiononastoppedinstancehowever.Ifyou'vebeenfollowingalong,youshouldalreadyhavealibvirtKVMinstancethatyoucanuse.

GettingreadyTheprerequisitesforthisrecipeareasfollows:

StoppedlibvirtKVMinstancewithconsoleaccessAguestOSwiththe9pandvirtiokernelmodules(availableonmostLinuxdistributionsbydefault)

Howtodoit...ToshareadirectoryfromthehostOStotheKVMguest,executethefollowing:

1. CreateanewdirectoryonthehostOSandaddafiletoit:

root@kvm:~#mkdir/tmp/shared

root@kvm:~#touch/tmp/shared/file

root@kvm:~#

2. AddthefollowingdefinitiontothestoppedKVMinstance:


...

<devices>

...

<filesystemtype='mount'accessmode='passthrough'>

<sourcedir='/tmp/shared'/>

<targetdir='tmp_shared'/>

</filesystem>

...

</devices>

...


root@kvm:~#

3. StarttheVM:


Domainkvm1started

root@kvm:~#

4. Connecttotheconsoleasfollows:



Escapecharacteris^]


debianlogin:root

Password:

...

root@debian:~#

5. Ensurethatthe9pandthevirtiokernelmodulesareloaded:

root@debian:~#lsmod|grep9p

9pnet_virtio170060

9pnet6163219pnet_virtio

virtio_ring175133virtio_pci,virtio_balloon,9pnet_virtio

virtio130583virtio_pci,virtio_balloon,9pnet_virtio

root@debian:~#

6. Mounttheshareddirectoryto/mnt:

root@debian:~#mount-t9p-otrans=virtiotmp_shared/mnt

root@debian:~#

7. Listthenewmount:

root@debian:~#mount|greptmp_shared

tmp_sharedon/mnttype9p(rw,relatime,sync,dirsync,trans=virtio)

root@debian:~#

8. EnsurethatthesharedfileisvisibleinthehostOS:

root@debian:~#ls-la/mnt/

total8



-rw-r--r--1rootroot0Mar2311:25file

root@debian:~#

Howitworks...Let'sgetthroughthestepsandseewhatwasaccomplishedinmoredetailsintheprevioussection.

Instep1,wecreateadirectoryandafilethatwewanttosharewiththeguestOS.Then,onthestoppedKVMinstance,weaddedthenew<filesystem>definitioninstep2.Weusedthemounttypebecausewearemountingadirectoryandspecifiedtheaccessmode,whichspecifiesthesecuritymodeforaccessingthesharedresource.Therearethreeaccessmodes:

passthrough:Thisisthedefaultmode,whichaccessestheshareddirectoryusingthepermissionsoftheuserinsidetheguestOSmapped:Inthismode,theshareddirectoryanditsfilesareaccessedusingthepermissionsoftheQEMUuser,inheritedfromthehostsquash:Thismodeissimilartothepassthroughmode;however,thefailuresofprivilegedoperationssuchaschmodareignored

Withthenewdefinitioninplace,westarttheVMinstep3andconnecttoitinstep4.

OntheDebianvirtualmachinewehavebeenusing,therequiredkernelmodulehasbeenloadedwhentheVMstarted.IfthisisnotthecaseforyourVM,loadthemodulesbyrunning:

root@debian:~#modprobe9pvirtio

root@debian:~#

Themainactionhappensinstep6,wherewemounttheshareddirectoryandensurethatithasbeensuccessfullymountedandthefilepresentinthesubsequentsteps.

There'smore...Inthischapter,wehavebeenstartingKVMvirtualmachinesusingthevirshcommand,providedbythelibvirttoolsetandlibraries.Ifyouchecktheprocesstreeafterstartingaguest,youcanseethatvirshcommandactuallycallsthe/usr/bin/qemu-system-x86_64binary.IfyourecallfromtheRunningvirtualmachineswithqemu-system-*recipeinChapter1,GettingStartedwithQEMUandKVM;thisisexactlywhatweusedtostartQEMU/KVMvirtualmachines.

NotetheprocessthatthelibvirtdaemonstartedwhenweranstartedtheKVMinstanceinthisrecipe:


6233/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-i440fx-


2,sockets=2,cores=1,threads=1-uuid6ad84d8a-229d-d1f6-ecfc-d29a25fcfa03-no-

user-config-nodefaults-chardev

socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait-mon

chardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-shutdown-boot

strict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-device

lsi,id=scsi0,bus=pci.0,addr=0x5-drivefile=/tmp/kvm1.img,if=none,id=drive-ide0-

0-0,format=raw-deviceide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-

0,bootindex=1-fsdevlocal,security_model=passthrough,id=fsdev-

fs0,path=/tmp/shared-devicevirtio-9p-pci,id=fs0,fsdev=fsdev-

fs0,mount_tag=tmp_shared,bus=pci.0,addr=0x6-netdevtap,fd=25,id=hostnet0-

devicertl8139,netdev=hostnet0,id=net0,mac=52:54:00:c5:c8:9d,bus=pci.0,addr=0x3

-chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-

vnc146.20.141.158:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-device

virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

root@kvm:~#

Insteadofusinglibvirt,wecanstartanewguestOSwiththesameshareddirectoryweuseinthisrecipe,byjustrunningthefollowing,justmakesuretostopthelibvirtinstancewestartedearlierfirst:

root@kvm:~#qemu-system-x86_64-namedebian-fsdev

local,id=tmp,path=/tmp/shared,security_model=passthrough-devicevirtio-9p-

pci,fsdev=tmp,mount_tag=tmp_shared-enable-kvm-usbdevicetablet-vnc

146.20.141.158:0-m1024-driveformat=raw,file=/tmp/kvm1.img-daemonize

root@kvm:~#

YoushouldbeabletouseyourVNCclienttoconnecttotheguestandperformthesamestepstomounttheshareddirectory,aswedidearlier.

AutostartingKVMinstancesOnceaKVMinstancehasbeendefinedandstarted,itwillrununtilthehostOSisup.OncethehostOSrestarts,instancesbuildwithlibvirtwillnotautomaticallystartoncethehostisupandthelibvirtdaemonisrunning.Inthisrecipe,wearegoingtochangethisbehaviorandensurevirtualinstancestartwhenthelibvirtdaemonstarts.

GettingreadyForthisrecipe,wearegoingtoneedasingleKVMinstancebuildwithlibvirt.

Howtodoit...ToconfigureaKVMguesttoautomaticallystartafteraserver,orlibvirtdrestart,runthefollowing:

1. EnabletheVMautostart:

root@kvm:~#virshautostartkvm1

Domainkvm1markedasautostarted

root@kvm:~#

2. Obtaininformationfortheinstance:

root@kvm:~#virshdominfokvm1

Id:31

Name:kvm1

UUID:6ad84d8a-229d-d1f6-ecfc-d29a25fcfa03

OSType:hvm

State:running

CPU(s):2

CPUtime:10.9s

Maxmemory:2097152KiB

Usedmemory:1048576KiB

Persistent:yes

Autostart:enable

Managedsave:no

Securitymodel:none

SecurityDOI:0

root@kvm:~#

3. Stoptherunninginstanceandensurethatitisintheshutoffstate:


Domainkvm1destroyed


IdNameState

----------------------------------------------------

-kvm1shutoff

root@kvm:~#

4. Stopthelibvirtdaemonandensurethatitisnotrunning:

root@kvm:~#/etc/init.d/libvirt-binstop


root@kvm:~#pgrep-lfalibvirtd

root@kvm:~#

5. Startbackthelibvirtdaemon:

root@kvm:~#/etc/init.d/libvirt-binstart


root@kvm:~#

6. Listallrunninginstances:


IdNameState

----------------------------------------------------

2kvm1running

root@kvm:~#

7. Disabletheautostartoption:

root@kvm:~#virshautostartkvm1--disable

Domainkvm1unmarkedasautostarted

root@kvm:~#

8. Verifythechange:

root@kvm:~#virshdominfokvm1|grep-iautostart

Autostart:disable

root@kvm:~#

Howitworks...Inthissimplerecipe,weenabledtheautostartfeatureofalibvirtcontrolledKVMinstance.

Instep1,weenabledautostartandverifiedthatithasbeenenabledinstep2.

Next,tosimulateaserverrestart,wefirststoptherunninginstanceinstep3andthelibvirtdaemoninstep4.

Instep5,westartedthelibvirtdaemonbackandobservethatitstartedthevirtualmachineaswell,asseeninstep6.

Finally,insteps7and8,wedisabletheautostartfeatureandensurethatitindeedhasbeendisabled.

WorkingwithstoragepoolsLibvirtprovidesacentralizedwayofmanaginginstancevolumes(beingimagefilesordirectories)bydefiningstoragepools.Astoragepoolisacollectionofvolumesthatthencanbeassignedtovirtualmachinesandusedtohosttheirfilesystemsoraddedasadditionalblockdevices.ThemainbenefitsofusingstoragepoolsistheabilityforlibvirttopresentandmanagethegivenstoragetypetoVMsinacentralizedway.

Asofthiswriting,thefollowingstoragepoolbackendsareavailable:

DirectorybackendLocalfilesystembackendNetworkfilesystembackendLogicalbackendDiskbackendiSCSIbackendSCSIbackendMultipathbackendRADOSblockdevicebackendSheepdogbackendGlusterbackendZFSbackendVirtuozzostoragebackend

Inthisrecipe,wearegoingtocreateadirectory-backedstoragepool,moveanexistingimagetoit,andthenprovisionanewKVMinstanceusingthestoragepoolandvolume.


TheDebianrawimagewecreatedintheBuildingnewKVMinstanceswithvirt-installandusingtheconsolerecipeThelibvirtpackage

Howtodoit...Thefollowingstepsdemonstratehowtocreateanewstoragepool,inspectit,andassignittoavirtualmachine:

1. CopytherawDebianimagefilewecreatedintheBuildingnewKVMinstanceswithvirt-installandusingtheconsolerecipeearlierinthischapter:

root@kvm:~#cp/tmp/kvm1.img/var/lib/libvirt/images/

root@kvm:~#

2. Createthefollowingstoragepooldefinition:

root@kvm:~#catfile_storage_pool.xml

<pooltype="dir">

<name>file_virtimages</name>

<target>

<path>/var/lib/libvirt/images</path>

</target>

</pool>

root@kvm:~#

3. Definethenewstoragepool:

root@kvm:~#virshpool-definefile_storage_pool.xml

Poolfile_virtimagesdefinedfromfile_storage_pool.xml

root@kvm:~#

4. Listallstoragepools:

root@kvm:~#virshpool-list--all

NameStateAutostart

-------------------------------------------

file_virtimagesinactiveno

root@kvm:~#

5. Startthenewstoragepoolandensurethatit'sactive:

root@kvm:~#virshpool-startfile_virtimages

Poolfile_virtimagesstarted


NameStateAutostart

-------------------------------------------

file_virtimagesactiveno

root@kvm:~#

6. Enabletheautostartfeatureonthestoragepool:

root@kvm:~#virshpool-autostartfile_virtimages

Poolfile_virtimagesmarkedasautostarted


NameStateAutostart

-------------------------------------------

file_virtimagesactiveyes

root@kvm:~#

7. Obtainmoreinformationaboutthestoragepool:

root@kvm:~#virshpool-infofile_virtimages

Name:file_virtimages

UUID:d51d500b-8885-4c26-8000-2ae46ffe9018

State:running

Persistent:yes

Autostart:yes

Capacity:219.87GiB

Allocation:7.99GiB

Available:211.88GiB

root@kvm:~#

8. Listallvolumesthatareapartofthestoragepool:

root@kvm:~#virshvol-listfile_virtimages

NamePath

----------------------------------------------------

kvm1.img/var/lib/libvirt/images/kvm1.img

root@kvm:~#

9. Obtaininformationonthevolume:

root@kvm:~#virshvol-info/var/lib/libvirt/images/kvm1.img

Name:kvm1.img

Type:file

Capacity:8.00GiB

Allocation:1.87GiB

root@kvm:~#

10. StartnewKVMinstanceusingthestoragepoolandvolume,thenensurethatit'srunning:

root@kvm:~#virt-install--namekvm1--ram1024--graphics

vnc,listen=146.20.141.158--hvm--diskvol=file_virtimages/kvm1.img--

import

Startinginstall...




root@kvm:~#


IdNameState

----------------------------------------------------

3kvm1running

root@kvm:~#

Howitworks...WestartthisrecipewithanimageofaDebianOSthatweinstalledearlierinthebook;however,youcanuseanempty,raw,orqcow2image,addittothestoragepool,andinstallthevirtualmachineOSonitwithalmostnochangestotherecipestepsifyoudon'thavethatimagealready.

Instep1,wecopytheVMimagetothedefaultlibvirtstoragepoollocationin/var/lib/libvirt/images/,butyoucancreateyourowndirectory,thelocationdoesnotmatteraslongasit'sdefinedinthestoragepoolconfigurationfile.Wedothatinstep2.

Instep3,wedefinethenewstoragepool,byspecifyinganame,targetdirectory,andthetypeofthepool,inthiscase,adirectorybackendpool.Wethenproceedtolistthenewpoolinstep4.Notethat,oncedefined,westillneedtostartit,justlikedefininganewKVMinstancefromanXMLfile.Bydefault,theautostartoptionisnotenabledonanewstoragepool.

Instep5,westartthestoragepoolandensurethatit'sactive.Wethenproceedtoenabletheautostartfeaturesothatthevolumescanbeusedincasethehostserverrestartsinstep6.

Althoughnotmandatory,wecheckthemetadataprovidedforthestoragepoolanditsvolumesinstep7.Notethattheallocationfieldshowshowmuchspaceisusedbythevolumesinthepool.Wecurrentlyhaveasinglerawimagewiththatexactsize.

Instep8,welistallvolumesthatareapartofthenewstoragepoolandobtainfurtherinformationaboutthesinglevolumeinstep9.

Finallyinstep10,westartanewKVMinstanceusingthestoragepoolandvolumebypassingthestoragepoolandvolumenamestothevoldisktype.

There'smore...Let'slookataslightlymorecomplicatedexampleofusingstoragepoolsbydefininganiSCSI-backedpool.

CreatinganiSCSItargetandloggingitontheinitiatorserverisbeyondthescopeofthisrecipe,soweassumethatyouhaveaniSCSItargetreadytobeusedfromaremoteserver.Thenewstoragepooldefinitionisasfollows:

<pooltype='iscsi'>

<name>iscsi_virtimages</name>

<source>

<hostname='iscsi-target.linux-admins.net'/>

<devicepath='iqn.2004-04.ubuntu:ubuntu16:iscsi.libvirtkvm'/>

</source>

<target>

<path>/dev/disk/by-path</path>

</target>

</pool>

Thefileisverysimilartothedirectory-backedstoragepool,themaindifferencearethefollowingattributes:

The<host>attributespecifiesthehostnameoftheiSCSItargetserverthatisexportingtheiSCSILUNThe<device>specifiesthenameoftheiSCSILUNwearegoingtologinOnceanewiSCSIblockdevicehasbeenloggedin,itwillappearinthelocationspecifiedin<path>,onmostLinuxdistributionsinthe/dev/disk/by-pathdirectory

Wedefineandstartthenewstoragepoolthesamewaywedidinsteps3and5earlierintherecipe.Oncethestoragepoolisactive,libvirtwilllogtheremoteiSCSItargetLUNs.WecanlisttheavailableiSCSIvolumesasusual:

root@kvm:~#virshvol-listiscsi_virtimages

NamePath

-----------------------------------------

10.0.0.1/dev/disk/by-path/ip-10.184.226.106:3260-iscsi-iqn.2004-

04.ubuntu:ubuntu16:iscsi.libvirtkvm-lun-1

root@kvm:~#

TostartanewinstallationprocessusingtheiSCSIvolumeasthetargetforthe

guestOSfilesystem,runthefollowingcode:

root@kvm:~#virt-install--namekvm1--ram1024--extra-args="textconsole=tty0

utf8console=ttyS0,115200"--graphicsvnc,listen=146.20.141.158--hvm--

location=http://ftp.us.debian.org/debian/dists/stable/main/installer-amd64/--

diskvol=iscsi_virtimages/10.0.0.1

Startinginstall...

...

root@kvm:~#

FormoreinformationabouttheXMLdefinitionoftheotherbackendtypes,pleaserefertohttps://libvirt.org/storage.html.

https://libvirt.org/storage.html

ManagingvolumesInthepreviousrecipe,wesawhowtocreatenewstoragepools,addavolumetoit,andcreateanewKVMinstanceusingthatvolume.Inthisrecipe,wearegoingtofocusonmanipulatingvolumesthatareapartofanexistingstoragepool.Strictlyspeaking,wearenotrequiredtousestoragepoolsandvolumesinordertobuildVMs.Wecanuseothertoolstomanageandmanipulatethevirtualinstanceimages,suchastheqemu-imgutility.Usingvolumesisjustaconvenienceforhavingacentralizedstoragerepositoryofvariousbackendtypes.

GettingreadyThemainrequirementofthisrecipeistohaveanexistingstoragepoolwiththedirectorybackend.Ifyouskippedthepreviousrecipe,nowisthattimetocreateanewone,aswe'llbeusingittomanipulatevolumes.

Howtodoit...Tocreate,inspectandassignvolumestoaninstance,runthefollowing:

1. Listtheavailablestoragepools:


NameStateAutostart

-------------------------------------------

file_virtimagesactiveyes

root@kvm:~#

2. Listtheavailablevolumes,thatareapartofthestoragepool:


NamePath

--------------------------------------------------------------------


root@kvm:~#

3. Createanewvolumewiththespecifiedsize:

root@kvm:~#virshvol-create-asfile_virtimagesnew_volume.img9G

Volnew_volume.imgcreated

root@kvm:~#

4. Listthevolumesonthefilesystem:

root@kvm:~#ls-lah/var/lib/libvirt/images/

total11G

drwx--x--x2rootroot4.0KMar2320:38.

drwxr-xr-x8rootroot4.0KMar2123:16..

-rwxr-xr-x1libvirt-qemukvm8.0GMar2320:23kvm1.img

-rw-------1rootroot9.0GMar2320:38new_volume.img

root@kvm:~#

5. Obtaininformationaboutthenewvolume:

root@kvm:~#qemu-imginfo/var/lib/libvirt/images/new_volume.img

image:/var/lib/libvirt/images/new_volume.img

fileformat:raw


disksize:9.0G

root@kvm:~#

6. Usethevirshcommandtogetevenmoreinformation:

root@kvm:~#virshvol-infonew_volume.img--poolfile_virtimages

Name:new_volume.img

Type:file

Capacity:9.00GiB

Allocation:9.00GiB

root@kvm:~#

7. Dumpthevolumeconfiguration:

root@kvm:~#virshvol-dumpxmlnew_volume.img--poolfile_virtimages

<volumetype='file'>

<name>new_volume.img</name>

<key>/var/lib/libvirt/images/new_volume.img</key>

<source>

</source>

<capacityunit='bytes'>9663676416</capacity>

<allocationunit='bytes'>9663680512</allocation>

<target>

<path>/var/lib/libvirt/images/new_volume.img</path>

<formattype='raw'/>

<permissions>

<mode>0600</mode>

<owner>0</owner>

<group>0</group>

</permissions>

<timestamps>

<atime>1490301514.446004048</atime>

<mtime>1490301483.698003615</mtime>

<ctime>1490301483.702003615</ctime>

</timestamps>

</target>

</volume>

root@kvm:~#

8. Resizethevolumeanddisplaythenewsize:

root@kvm:~#virshvol-resizenew_volume.img10G--poolfile_virtimages

Sizeofvolume'new_volume.img'successfullychangedto10G

root@kvm:~#virshvol-infonew_volume.img--poolfile_virtimages

Name:new_volume.img

Type:file

Capacity:10.00GiB

Allocation:9.00GiB

root@kvm:~#

9. Deletethevolumeandlistallavailablevolumesinthestoragepool:

root@kvm:~#virshvol-deletenew_volume.img--poolfile_virtimages

Volnew_volume.imgdeleted


NamePath

------------------------------------------------------------------


root@kvm:~#

10. Clonetheexistingvolume:

root@kvm:~#virshvol-clonekvm1.imgkvm2.img--poolfile_virtimages

Volkvm2.imgclonedfromkvm1.img


NamePath

--------------------------------------------------------------------



root@kvm:~#

Howitworks...Westartthisrecipewiththefile_virtimagesstoragepoolwecreatedinthepreviousrecipe.Welistallstoragepoolsinstep1toconfirmthat.Instep2,weseethatourstoragepoolcontainsasinglevolume.Nosurpriseshereaswecreatedthatinthelastrecipeinthischapter.

Instep3,wecreateanewvolume,byspecifyingitsname,size,andthestoragepoolwewantittobeapartof.Sincethisisadirectory-backedstoragepool,wecanseethevolumeasarawimagefileinstep4.

Insteps5and6,wecollectmoreinformationaboutthenewvolume.Wecanseethatitisaraw,thereforebydefaultasparseimage.Sparseimagesdon'tallocateallofthediskspaceandgrowasmoredataisbeingwrittentoit.

Instep7,wedumpthedefinitionofthevolume.Wecanusethattodefineanewvolumelateronwiththevirshvol-createcommand.

Libvirtprovidesaconvenientwaytoresizeexistingimages.Thisiswhatwedoinstep8--weresizetheimageto10GB.Wecannowseethattheallocationsizeissmallerthanthecapacity;thisisbecausetheimageisraw.

Finally,instep9,wedeletetheimage,thoughwecouldhaveusedittoinstallanewvirtualmachine,asshownintheWorkingwithstoragepoolsrecipe.

Inthelaststep,weusetheexistingDebianimageandcreatedaclonevolumefromit.StartingavirtualmachineusingtheclonedvolumewillresultinanidenticalKVMinstance,astheoneweclonedthevolumefrom.ThiscombinedwithadumpoftheinstancedefinitionisagreatwaytobackupyourKVMinstances,aslongasyoustorethevolumeimagefileandtheXMLdefinitionfiletoaremotelocation.WearegoingtoexplorebackingupKVMinstancesinlaterrecipes.

ManagingsecretsLibvirtprovidesanAPItocreate,store,andusesecrets.Secretsareobjectsthatcontainsensitiveinformationsuchaspasswords,thatcanbeassociatedwithdifferentvolumebackendtypes.RecallfromtheWorkingwithstoragepoolsrecipe,whichwecreatedaniSCSIpoolandvolumefromaremoteiSCSItargetanduseditastheimageforaKVMguest.Inproductionenvironments,moreoftenthannotiSCSItargetsarepresentedwithCHAPauthentication.Inthisrecipe,wearegoingtocreateasecrettobeusedwithaniSCSIvolume.


AstoragepoolwithaniSCSI-backedvolumeThelibvirtpackage

Howtodoit...Todefineandlistsecretswithlibvirt,performthestepsoutlinedhere:

1. Listallavailablesecrets:

root@kvm:~#virshsecret-list

UUIDUsage

-------------------------------------------------------------------

root@kvm:~#

2. Createthefollowingsecretsdefinition:

root@kvm:~#catvolume_secret.xml

<secretephemeral='no'>

<description>PassphrasefortheiSCSIiscsi-target.linux-admins.net

targetserver</description>

<usagetype='iscsi'>

<target>iscsi_secret</target>

</usage>

</secret>

root@kvm:~#

3. Createthesecretandensurethatithasbeensuccessfullycreated:

root@kvm:~#virshsecret-definevolume_secret.xml

Secret7ad1c208-c2c5-4723-8dc5-e2f4f576101acreated

root@kvm:~#virshsecret-list

UUIDUsage

-----------------------------------------------------------------

7ad1c208-c2c5-4723-8dc5-e2f4f576101aiscsiiscsi_secret

root@kvm:~#

4. Setavalueforthesecret:

root@kvm:~#virshsecret-set-value7ad1c208-c2c5-4723-8dc5-e2f4f576101a

$(echo"some_password"|base64)

Secretvalueset

root@kvm:~#

5. CreateanewiSCSIpooldefinitionfile:

root@kvm:~#catiscsi.xml

<pooltype='iscsi'>

<name>iscsi_virtimages</name>

<source>

<hostname='iscsi-target.linux-admins.net'/>

<devicepath='iqn.2004-04.ubuntu:ubuntu16:iscsi.libvirtkvm'/>

<authtype='chap'username='iscsi_user'>

<secretusage='iscsi_secret'/>

</auth>

</source>

<target>


</target>

</pool>

root@kvm:~#

Howitworks...Instep1,welistallavailablesecretsthatlibvirtknowsabout.Sincewehaven'tcreatedany,thelistisempty.

Instep2,wecreatetheXMLdefinitionofthesecret.TheXMLelementsthatweusetodefinethesecretare:

The<secret>rootelement,withanoptionalephemeralattribute,tellinglibvirtthatthepasswordshouldonlybestoredinmemory,ifsettoyes.The<description>attributecontaininganarbitrarydescription.The<usage>elementspecifieswhatthesecretsisgoingtobeusedforanditstype.Inthisexample,thetypeattributeissettoiSCSI.Theotheravailabletypesarevolume,ceph,andtls.Thetypeattributeismandatory.The<target>elementthatspecifiesanarbitrarynameistobeusedintheiSCSIpooldefinition.

Withtheconfigurationfileinplace,wecreatethesecretinstep3.Iftheoperationissuccessful,libvirtreturnsanUUIDthatidentifiesthesecret.

Instep4,wesetavalueforthesecret,bybase64encodingthesome_passwordstring,whichisthepasswordfortheiSCSItargetwewouldliketouse,asastoragepoolvolume.

Andfinallyinstep5,weaddthe<auth>attributeunderthe<source>sectionoftheiSCSIpooldefinition.NotethatthesecretwewouldliketheiSCSIvolumetouseisspecifiedinthe<secretusage='iscsi_secret'/>attribute.Libvirtcannowusetheiscsi_secretnametolocatetheactualpasswordthatithasstored.

KVMNetworkingwithlibvirtInthischapter,wearegoingtocoverthefollowingtopics:

TheLinuxbridgeTheOpenvSwitchConfiguringNATforwardingnetworkConfiguringbridgednetworkConfiguringPCIpassthroughnetworkManipulatingnetworkinterfaces

IntroductionWithlibvirt,wecandefinedifferentnetworktypesforourKVMguests,usingthealreadyfamiliarXMLdefinitionsyntaxandthevirshandvirt-installuserspacetools.Inthischapter,wearegoingtodeploythreedifferentnetworktypes,explorethenetworkXMLformat,andseeexamplesonhowtodefineandmanipulatevirtualinterfacesfortheKVMinstances.

TobeabletoconnectthevirtualmachinestothehostOSortoeachother,wearegoingtousetheLinuxbridgeandtheOpenvSwitch(OVS)daemons,userspacetools,andkernelmodules.BothsoftwarebridgingtechnologiesaregreatatcreatingSoftware-definedNetworking(SDN)ofvariouscomplexity,inaconsistentandeasy-to-manipulatemanner.TheLinuxbridgeandOVSbothactasabridge/switchthatthevirtualinterfacesoftheKVMguestscanconnectto.

Withallthisinmind,let'sstartbylearningmoreaboutthesoftwarebridgesinLinux.

TheLinuxbridgeTheLinuxbridgeisasoftwarelayer2devicethatprovidessomeofthefunctionalityofaphysicalbridgedevice.ItcanforwardframesbetweenKVMguests,thehostOS,andvirtualmachinesrunningonotherservers,ornetworks.TheLinuxbridgeconsistsoftwocomponents--auserspaceadministrationtoolthatwearegoingtouseinthisrecipeandakernelmodulethatperformsalltheworkofconnectingmultipleEthernetsegmentstogether.Eachsoftwarebridgewecreatecanhaveanumberofportsattachedtoit,wherenetworktrafficisforwardedtoandfrom.WhencreatingKVMinstances,wecanattachthevirtualinterfacesthatareassociatedwiththemtothebridge,whichissimilartoplugginganetworkcablefromaphysicalserver'sNICtoabridge/switchdevice.Beingalayer2device,theLinuxbridgeworkswithMACaddressesandmaintainsakernelstructuretokeeptrackofportsandassociatedMACaddressesintheformofaContentAddressableMemory(CAM)table.

Inthisrecipe,wearegoingtocreateanewLinuxbridgeandusethebrctlutilitytomanipulateit.


RecentLinuxkernelwithenabled802.1dEthernetbridgingoptions

Tocheckwhetheryourkerneliscompiledwiththosefeaturesorexposedaskernelmodules,runthefollowingcommand:

root@kvm:~#cat/boot/config-`uname-r`|grep-ibridg

#PC-cardbridges

CONFIG_BRIDGE_NETFILTER=y

CONFIG_NF_TABLES_BRIDGE=m

CONFIG_BRIDGE_NF_EBTABLES=m

CONFIG_BRIDGE_EBT_BROUTE=m

CONFIG_BRIDGE_EBT_T_FILTER=m

CONFIG_BRIDGE_EBT_T_NAT=m

CONFIG_BRIDGE_EBT_802_3=m

CONFIG_BRIDGE_EBT_AMONG=m

CONFIG_BRIDGE_EBT_ARP=m

CONFIG_BRIDGE_EBT_IP=m

CONFIG_BRIDGE_EBT_IP6=m

CONFIG_BRIDGE_EBT_LIMIT=m

CONFIG_BRIDGE_EBT_MARK=m

CONFIG_BRIDGE_EBT_PKTTYPE=m

CONFIG_BRIDGE_EBT_STP=m

CONFIG_BRIDGE_EBT_VLAN=m

CONFIG_BRIDGE_EBT_ARPREPLY=m

CONFIG_BRIDGE_EBT_DNAT=m

CONFIG_BRIDGE_EBT_MARK_T=m

CONFIG_BRIDGE_EBT_REDIRECT=m

CONFIG_BRIDGE_EBT_SNAT=m

CONFIG_BRIDGE_EBT_LOG=m

#CONFIG_BRIDGE_EBT_ULOGisnotset

CONFIG_BRIDGE_EBT_NFLOG=m

CONFIG_BRIDGE=m

CONFIG_BRIDGE_IGMP_SNOOPING=y

CONFIG_BRIDGE_VLAN_FILTERING=y

CONFIG_SSB_B43_PCI_BRIDGE=y

CONFIG_DVB_DDBRIDGE=m

CONFIG_EDAC_SBRIDGE=m

#VMEBridgeDrivers

root@kvm:~#

Thebridgekernelmodule

Toverifythatthemoduleisloadedandtoobtainmoreinformationaboutitsversionandfeatures,executethefollowingcommand:

root@kvm:~#lsmod|grepbridge

bridge1109250

stp129762garp,bridge

llc145523stp,garp,bridge

root@kvm:~#

root@kvm:~#modinfobridge

filename:/lib/modules/3.13.0-107-generic/kernel/net/bridge/bridge.ko

alias:rtnl-link-bridge

version:2.3

license:GPL

srcversion:49D4B615F0B11CA696D8623

depends:stp,llc

intree:Y

vermagic:3.13.0-107-genericSMPmod_unloadmodversions

signer:Magrathea:Glaciersigningkey

sig_key:E1:07:B2:8D:F0:77:39:2F:D6:2D:FD:D7:92:BF:3B:1D:BD:57:0C:D8

sig_hashalgo:sha512

root@kvm:~#

Thebridge-utilspackagethatprovidesthetooltocreateandmanipulatetheLinuxbridgeTheabilitytocreatenewKVMguestsusinglibvirtortheQEMUutilitiesoranexistingKVMinstancefromthepreviouschapters

Howtodoit...Tocreate,list,andmanipulateanewLinuxbridge,followthesesteps:

1. InstalltheLinuxbridgepackage,ifitisnotalreadypresent:

root@kvm:~#aptinstallbridge-utils

2. BuildanewKVMinstanceusingtherawimagefromtheInstallingacustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVM,ifyouarenotreadingthisbookcovertocover:

root@kvm:~#virt-install--namekvm1--ram1024--disk

path=/tmp/debian.img,format=raw--graphicsvnc,listen=146.20.141.158--

noautoconsole--hvm--import

Startinginstall...




root@kvm:~#

3. Listalltheavailablebridgedevices:

root@kvm:~#brctlshow

bridgenamebridgeidSTPenabledinterfaces

virbr08000.fe5400559bd6yesvnet0

root@kvm:~#

4. Bringthevirtualbridgedown,deleteit,andensurethatit'sbeendeleted:

root@kvm:~#ifconfigvirbr0down

root@kvm:~#brctldelbrvirbr0



root@kvm:~#

5. Createanewbridgeandbringitup:

root@kvm:~#brctladdbrvirbr0



virbr08000.000000000000no

root@kvm:~#ifconfigvirbr0up

root@kvm:~#

6. AssignanIPaddresstobridge:

root@kvm:~#ipaddradd192.168.122.1devvirbr0

root@kvm:~#ipaddrshowvirbr0

39:virbr0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscnoqueue

stateUNKNOWNgroupdefault

link/ether32:7d:3f:80:d7:c6brdff:ff:ff:ff:ff:ff

inet192.168.122.1/32scopeglobalvirbr0

valid_lftforeverpreferred_lftforever

inet6fe80::307d:3fff:fe80:d7c6/64scopelink


root@kvm:~#

7. ListthevirtualinterfacesonthehostOS:

root@kvm:~#ipas|grepvnet

38:vnet0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast

stateUNKNOWNgroupdefaultqlen500

root@kvm:~#

8. Addthevirtualinterfacevnet0tothebridge:

root@kvm:~#brctladdifvirbr0vnet0

root@kvm:~#brctlshowvirbr0


virbr08000.fe5400559bd6novnet0

root@kvm:~#

9. EnabletheSpanningTreeProtocol(STP)onbridgeandobtainmoreinformation:

root@kvm:~#brctlstpvirbr0on

root@kvm:~#brctlshowstpvirbr0

virbr0

bridgeid8000.fe5400559bd6

designatedroot8000.fe5400559bd6

rootport0pathcost0

maxage20.00bridgemaxage20.00

hellotime2.00bridgehellotime2.00

forwarddelay15.00bridgeforwarddelay15.00

ageingtime300.00

hellotimer0.26tcntimer0.00

topologychangetimer0.00gctimer90.89

flags

vnet0(1)

portid8001stateforwarding

designatedroot8000.fe5400559bd6pathcost100

designatedbridge8000.fe5400559bd6messageagetimer0.00

designatedport8001forwarddelaytimer0.00

designatedcost0holdtimer0.00

flags

root@kvm:~#

10. FrominsidetheKVMinstance,bringtheinterfaceup,requestanIPaddress,andtestconnectivitytothehostOS:

root@debian:~#ifconfigeth0up

root@debian:~#dhclienteth0

root@debian:~#ipaseth0

2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast

stateUPgroupdefaultqlen1000

link/ether52:54:00:55:9b:d6brdff:ff:ff:ff:ff:ff

inet192.168.122.92/24brd192.168.122.255scopeglobaleth0


inet6fe80::5054:ff:fe55:9bd6/64scopelink


root@debian:~#

root@debian:~#ping192.168.122.1-c3

PING192.168.122.1(192.168.122.1)56(84)bytesofdata.

64bytesfrom192.168.122.1:icmp_seq=1ttl=64time=0.276ms



---192.168.122.1pingstatistics---

3packetstransmitted,3received,0%packetloss,time1999ms

rttmin/avg/max/mdev=0.226/0.253/0.276/0.027ms

root@debian:~#

Howitworks...Whenwefirstinstalledandstartedthelibvirtdaemon,afewthingshappenedautomatically:

AnewLinuxbridgewascreatedwiththenameandIPaddressdefinedinthe/etc/libvirt/qemu/networks/default.xmlconfigurationfileThednsmasqservicewasstartedwithaconfigurationspecifiedinthe/var/lib/libvirt/dnsmasq/default.conffile

Let'sexaminethedefaultlibvirtbridgeconfiguration:

root@kvm:~#cat/etc/libvirt/qemu/networks/default.xml

<network>

<name>default</name>

<bridgename="virbr0"/>

<forward/>

<ipaddress="192.168.122.1"netmask="255.255.255.0">

<dhcp>

<rangestart="192.168.122.2"end="192.168.122.254"/>

</dhcp>

</ip>

</network>

root@kvm:~#

Thisisthedefaultnetworkthatlibvirtcreatedforus,specifyingthebridgename,IPaddress,andtheIPrangeusedbytheDHCPserverthatwasstarted.Wearegoingtotalkaboutlibvirtnetworkinginmuchmoredetaillaterinthischapter;however,we'veshownitheretohelpyouunderstandwherealltheIPaddressesandthebridgenamecamefrom.

WecanseethataDHCPserverisrunningonthehostOSanditsconfigurationfilebyrunningthefollowingcommand:

root@kvm:~#pgrep-lfadnsmasq

38983/usr/sbin/dnsmasq--conf-file=/var/lib/libvirt/dnsmasq/default.conf

root@kvm:~#cat/var/lib/libvirt/dnsmasq/default.conf

##WARNING:THISISANAUTO-GENERATEDFILE.CHANGESTOITARELIKELYTOBE

##OVERWRITTENANDLOST.Changestothisconfigurationshouldbemadeusing:

##virshnet-editdefault

##orotherapplicationusingthelibvirtAPI.

##

##dnsmasqconffilecreatedbylibvirt

strict-order

user=libvirt-dnsmasq

pid-file=/var/run/libvirt/network/default.pid

except-interface=lo

bind-dynamic

interface=virbr0

dhcp-range=192.168.122.2,192.168.122.254

dhcp-no-override

dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases

dhcp-lease-max=253

dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile

addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

root@kvm:~#

Fromtheconfigurationfileearlier,notehowtheIPaddressrangefortheDHCPserviceandthenameofthevirtualbridgematchwhatisconfiguredinthedefaultlibvirtnetworkfilethatwejustsaw.

Withallthisinmind,let'sstepthroughalltheactionsweperformedearlier:

Instep1,weinstalledtheuserspacetoolbrctlthatweusetocreate,configure,andinspecttheLinuxbridgeconfigurationintheLinuxkernel.

Instep2,weprovisionedanewKVMinstanceusingacustomrawimagecontainingtheguestOS.Thisstepisnotrequiredifyoucompletedtherecipesinthepreviouschapters.

Instep3,weinvokedthebridgeutilitytolistallavailablebridgedevices.Fromtheoutput,wecanobservethatcurrentlythere'sonebridge,namedvirbr0,whichlibvirtcreatedautomatically.Notethatundertheinterfacescolumn,wecanseethevnet0interface.ThisisthevirtualNICthatwasexposedtothehostOS,whenwestartedtheKVMinstance.Thismeansthatthevirtualmachineisconnectedtothehostbridge.

Instep4,wefirstbringthebridgedowninordertodeleteit,thenweusethebrctlcommandagaintoremovethebridgeandensurethatit'snotpresentonthehostOS.

Instep5,werecreatedthebridgeandbroughtitbackup.Wedothistodemonstratethestepsrequiredtocreateanewbridge.

Instep6,wereassignedthesameIPaddresstothebridgeandlistedit.

Insteps7and8,welistallvirtualinterfacesonthehostOS.Becauseweonly

haveoneKVMguestcurrentlyrunningontheserver,weonlyseeonevirtualinterface,thatis,vnet0.Wethenproceedtoadd/connectthevirtualNICtothebridge.

Instep9,weenabledtheSTPonthebridge.STPisalayer2protocolthathelpspreventnetworkloopsifwehaveredundantnetworkpaths.Thisisespeciallyusefulinlarger,morecomplexnetworktopologies,wheremultiplebridgesareconnectedtogether.

Finally,instep10,weconnecttotheKVMguestusingtheconsole,listitsinterfaceconfiguration,andensurethatwecanpingthebridgeonthehostOS.Inordertodothat,weneedtobringthenetworkinterfaceinsidetheguestupwithifconfigeth0up,thenobtainanIPaddresswiththedhclienteth0commandfromthednsmasqserverrunningonthehost.

There'smore...TherearefewmoreusefulcommandswecanuseontheLinuxbridge.

WealreadyknowthatabridgeforwardsframesbasedontheMACaddressescontainedtherein.ToexaminethetableofMACaddressesthebridgeknowsabout,runthefollowingcommand:

root@kvm:~#brctlshowmacsvirbr0

portnomacaddrislocal?ageingtimer

152:54:00:55:9b:d6no268.02

1fe:54:00:55:9b:d6yes0.00

root@kvm:~#

Fromtheprecedingoutput,wecanseethatthebridgehasrecordedtwoMACaddressesonitsonlyport.Thefirstrecordisanonlocaladdress,anditbelongstothenetworkinterfaceinsidetheKVMinstances.WecanconfirmthatbyconnectingtotheKVMguestasfollows:



Escapecharacteris^]

root@debian:~#ipaseth0

2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_faststateUP

groupdefaultqlen1000




root@debian:~#

ThesecondMACaddressistheaddressofthebridgeitselfandtheMACaddressofthevirtualinterface,belongingtotheKVMvirtualmachine,exposedtothehostOS.Toconfirmthis,runthefollowingcommand:

root@kvm:~#ifconfig|grep"fe:54:00:55:9b:d6"

virbr0Linkencap:EthernetHWaddrfe:54:00:55:9b:d6

vnet0Linkencap:EthernetHWaddrfe:54:00:55:9b:d6

root@kvm:~#

Whenthebridgeseesaframeononeofitsports,itrecordsthetimethenafterasetamountoftimenotseeingthesameMACaddressagain,itwillremovetherecordfromtheitsCAMtable.Wecansetthetimelimitinsecondsbeforethe

bridgewillexpiretheMACaddressentrybyexecutingthefollowingcommand:

root@kvm:~#brctlsetageingvirbr0600

root@kvm:~#

Thebrctlcommandiswelldocumented;tolistallavailablesubcommands,runitwithoutanyparameters:

root@kvm:~#brctl

Usage:brctl[commands]

commands:

addbr<bridge>addbridge

delbr<bridge>deletebridge

addif<bridge><device>addinterfacetobridge

delif<bridge><device>deleteinterfacefrombridge

hairpin<bridge><port>{on|off}turnhairpinon/off

setageing<bridge><time>setageingtime

setbridgeprio<bridge><prio>setbridgepriority

setfd<bridge><time>setbridgeforwarddelay

sethello<bridge><time>sethellotime

setmaxage<bridge><time>setmaxmessageage

setpathcost<bridge><port><cost>setpathcost

setportprio<bridge><port><prio>setportpriority

show[<bridge>]showalistofbridges

showmacs<bridge>showalistofmacaddrs

showstp<bridge>showbridgestpinfo

stp<bridge>{on|off}turnstpon/off

root@kvm:~#

MostLinuxdistributionspackagethebrctlutilityandthisiswhatweusedinthisrecipe.However,tousethelatestversion,orifapackageisnotavailableforyourdistribution,wecanbuildtheutilityfromsourcebycloningtheprojectwithgit,thenconfigureandcompile:

root@kvm:~#cd/usr/src/

root@kvm:/usr/src#apt-getupdate&&apt-getinstallbuild-essentialautomake

pkg-configgit

root@kvm:/usr/src#gitclone

git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git

Cloninginto'bridge-utils'...

remote:Countingobjects:654,done.

remote:Total654(delta0),reused0(delta0)

Receivingobjects:100%(654/654),131.72KiB|198.00KiB/s,done.

Resolvingdeltas:100%(425/425),done.

Checkingconnectivity...done.

root@kvm:/usr/src#cdbridge-utils/

root@kvm:/usr/src/bridge-utils#autoconf

root@kvm:/usr/src/bridge-utils#./configure&&make&&makeinstall

root@kvm:/usr/src/bridge-utils#brctl--version

bridge-utils,1.5

root@kvm:/usr/src/bridge-utils#

Fromtheprecedingoutput,wecanseethatwefirstclonedthegitrepositoryfor

thebridge-utilsprojectandthencompiledthesourcecode.

OnaRedHat/CentOShost,theprocessissimilar:

[root@centos~]#cd/usr/src/

[root@centossrc]#

[root@centossrc]#yumgroupinstall"Developmenttools"

[root@centossrc]#gitclone

git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git

Cloninginto'bridge-utils'...

remote:Countingobjects:654,done.

remote:Total654(delta0),reused0(delta0)

Receivingobjects:100%(654/654),131.72KiB|198.00KiB/s,done.

Resolvingdeltas:100%(425/425),done.

Checkingconnectivity...done.

[root@centossrc]#cdbridge-utils

[root@centosbridge-utils]#autoconf

[root@centosbridge-utils]#./configure&&make&&makeinstall

[root@centosbridge-utils]#brctl--version

bridge-utils,1.5

[root@centosbridge-utils]#

TheOpenvSwitchOVSisanothersoftwarebridging/switchingdevicethatcanbeusedtocreatevariousvirtualnetworktopologiesandconnectKVMinstancestoit.OVScanbeusedinsteadoftheLinuxbridge,anditprovidesanextensivefeatureset,includingpolicyrouting,AccessControlLists(ACLs),QualityofService(QoS)policing,trafficmonitoring,flowmanagement,VLANtagging,GREtunneling,andmuchmore.

Inthisrecipe,wearegoingtoinstall,configure,andusetheOVSbridgetoconnectaKVMinstancetothehostOS,inasimilarwaytowhatwedidinthepreviousrecipewiththeLinuxbridge.

GettingreadyInorderforthisrecipetowork,weneedtoensurethefollowing:

TheLinuxbridgeisdeleted,ifpresent,andOVSisinstalledWehaveatleastoneKVMinstancerunning

Howtodoit...TocreateanewOVSbridgeandattachthevirtualinterfaceofaKVMguest,followthesesteps:

1. RemovetheexistingLinuxbridge,ifany:




root@kvm:~#ifconfigvirbr0down

root@kvm:~#brctldelbrvirbr0



root@kvm:~#

OnsomeLinuxdistributions,ithelpstounloadthekernelmodulefortheLinuxbridgebeforeusingOVS.Todothis,executeroot@kvm:/usr/src#modprobe-rbridge.

2. InstalltheOVSpackageonUbuntu:

root@kvm:~#apt-getinstallopenvswitch-switch

...

Settingupopenvswitch-common(2.0.2-0ubuntu0.14.04.3)...

Settingupopenvswitch-switch(2.0.2-0ubuntu0.14.04.3)...

openvswitch-switchstart/running

...

root@kvm:~#

3. EnsurethattheOVSprocessesarerunning:

root@kvm:~#pgrep-lfaswitch

22255ovsdb-server/etc/openvswitch/conf.db-vconsole:emer-vsyslog:err

-vfile:info--remote=punix:/var/run/openvswitch/db.sock--private-

key=db:Open_vSwitch,SSL,private_key--

certificate=db:Open_vSwitch,SSL,certificate--bootstrap-ca-

cert=db:Open_vSwitch,SSL,ca_cert--no-chdir--log-

file=/var/log/openvswitch/ovsdb-server.log--

pidfile=/var/run/openvswitch/ovsdb-server.pid--detach--monitor

22264ovs-vswitchd:monitoringpid22265(healthy)

22265ovs-vswitchdunix:/var/run/openvswitch/db.sock-vconsole:emer-

vsyslog:err-vfile:info--mlockall--no-chdir--log-

file=/var/log/openvswitch/ovs-vswitchd.log--

pidfile=/var/run/openvswitch/ovs-vswitchd.pid--detach--monitor

root@kvm:~#

4. EnsurethattheOVSkernelmodulehasbeenloaded:

root@kvm:~#lsmod|grepswitch

openvswitch709890

gre137961openvswitch

vxlan376111openvswitch

libcrc32c126441openvswitch

root@kvm:~#

5. ListtheavailableOVSswitches:

root@kvm:~#ovs-vsctlshow

e5164e3e-7897-4717-b766-eae1918077b0

ovs_version:"2.0.2"

root@kvm:~#

6. CreateanewOVSswitch:

root@kvm:~#ovs-vsctladd-brvirbr1


e5164e3e-7897-4717-b766-eae1918077b0

Bridge"virbr1"

Port"virbr1"

Interface"virbr1"

type:internal

ovs_version:"2.0.2"

root@kvm:~#

7. AddtheinterfaceoftherunningKVMinstancetotheOVSswitch:

root@kvm:~#ovs-vsctladd-portvirbr1vnet0


e5164e3e-7897-4717-b766-eae1918077b0

Bridge"virbr1"

Port"virbr1"

Interface"virbr1"

type:internal

Port"vnet0"

Interface"vnet0"

ovs_version:"2.0.2"

root@kvm:~#

8. ConfigureanIPaddressontheOVSswitch:

root@kvm:~#ipaddradd192.168.122.1/24devvirbr1

root@kvm:~#ipaddrshowvirbr1

41:virbr1:<BROADCAST,UP,LOWER_UP>mtu1500qdiscnoqueuestateUNKNOWN

groupdefault

link/etherb2:52:e0:73:89:4ebrdff:ff:ff:ff:ff:ff

inet192.168.122.1/24scopeglobalvirbr1


inet6fe80::b0a8:c2ff:fed4:bb3f/64scopelink


root@kvm:~#

9. ConfigureanIPaddressinsidetheKVMguestandensureconnectivitytothehostOS(iftheimagedoesnothaveconsoleaccessconfigure,connecttoitusingVNC):



Escapecharacteris^]

root@debian:~#ifconfigeth0up&&ipaddradd192.168.122.210/24dev

eth0

root@debian:~#ipaddrshoweth0

2:eth0:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscpfifo_fast

stateUPgroupdefaultqlen1000


inet192.168.122.210/24scopeglobaleth0




root@debian:~#ping192.168.122.1





^C




root@debian:~#

Howitworks...Inordertosimplifyoursetupandavoidconflicts,it'sprudenttofirstremovetheLinuxbridgebeforecreatinganewOVSswitch.Wedeletethebridgeinstep1andoptionallyunloadedthekernelmodule.

Instep2,weinstalltheOVSpackagethatalsostartsthemainOVSdaemonovs-vswitchdresponsibleforcreatingandmodifyingthebridges/switchesonthehostOS.

Instep4,wemakesurethattheOVSkernelmodulehasbeenloaded,andwelistallavailableOVSswitchesonthehostinstep5.

Insteps6and7,wecreateanewOVSswitchandaddtheKVMvirtualinterfacetotheswitch.

Theovsdb-serverprocessthatwasalsostartedafterinstallingthepackage,asseenfromtheoutputinstep3,isadatabaseenginethatusesJSONRemoteProcedureCalls(RPC)tocommunicatewiththemainOVSdaemon.Theovsdbserverprocessstoresinformation,suchastheswitchnetworkflows,ports,andQoStonamejustfew.Youcanquerythedatabasebyrunningthefollowingcommand:

root@kvm:~#ovsdb-clientlist-dbs

Open_vSwitch

root@kvm:~#ovsdb-clientlist-tables

Table

-------------------------

Port

Manager

Bridge

Interface

SSL

IPFIX

Open_vSwitch

Queue

NetFlow

Mirror

QoS

Controller

Flow_Table

sFlow

Flow_Sample_Collector_Set

root@kvm:~#ovsdb-clientdumpOpen_vSwitch

...

Porttable

_uuidbond_downdelaybond_fake_ifacebond_modebond_updelayexternal_ids

fake_bridgeinterfaceslacpmacnameother_configqosstatisticsstatustag

trunksvlan_mode

----------------------------------------------------------------------------

---------------------------------------------------------------------------

------------------------------------------------------------

9b4b743d-66b2-4779-9dd8-404b3aa55e180false[]0{}false[e7ed4e2b-a73c-46c7-

adeb-a203be56587c][][]"virbr1"{}[]{}{}[][][]

f2a033aa-9072-4be3-808e-6e0fce67ce7b0false[]0{}false[86a10eed-698f-4ccc-

b3b7-dd20c13e3ee3][][]"vnet0"{}[]{}{}[][][]

...

root@kvm:~#

Notefromtheprecedingoutputthatthenewswitchvirbr1andportvnet0arenowdisplaying,whenqueryingtheOVSdatabase.

Insteps8and9,weassignIPaddressestotheOVSswitchandtheKVMguestandensurethatwecanreachthehostbridgefrominsidethevirtualmachine.

There'smore...OVSisarathercomplexsoftwareswitch;inthisrecipe,weonlyscratchedthesurface.Inthenextfewrecipes,wecanuseboththeLinuxbridgeandOVS,withminorconfigurationchangesinlibvirt,whichwearegoingtopointoutaswego.

ToremovetheKVMvirtualinterfacefromtheOVSswitch,executethefollowingcommand:

root@kvm:~#ovs-vsctldel-portvirbr1vnet0

root@kvm:~#

TocompletelydeletetheOVSswitch,runthefollowingcommand:

root@kvm:~#ovs-vsctldel-brvirbr1&&ovs-vsctlshow

e5164e3e-7897-4717-b766-eae1918077b0

ovs_version:"2.0.2"

root@kvm:~#

FormoreinformationabouttheOVS,pleasevisittheprojectswebsite,http://openvswitch.org/.

http://openvswitch.org/

ConfiguringNATforwardingnetworkWhenthelibvirtdaemonstarts,itcreatesadefaultnetworkdefinedinthe/etc/libvirt/qemu/networks/default.xmlconfigurationfile.WhenanewKVMguestisbuildwithoutspecifyinganynetworkingoptions,itwillusethedefaultnetworktocommunicatewiththehostOSandotherguestsandnetworks.ThedefaultlibvirtnetworkisusingtheNetworkAddressTranslation(NAT)method.NATprovidesamappingfromoneIPaddressspacetoanother,bymodifyingtheIPaddressintheheaderoftheIPdatagrampacket.ThisisespeciallyusefulwhenthehostOSprovidesoneIPaddressallowingmultipleguestsonthesamehosttousethataddresstoestablishoutboundconnections.ThevirtualmachinesIPaddressesareessentiallytranslatedtoappearasthehostmachine'sIPaddress.

ThedefaultNATforwardingnetworkdefinesandsetsupaLinuxbridge,forthegueststoconnectto.Inthisrecipe,wearegoingtoexplorethedefaultNATnetworkandlearnabouttheXMLattributesusedtodefineit.Then,wearegoingtocreateanewNATnetworkandconnectourKVMguesttoit.


ALinuxhostwithlibvirtinstalledandthedaemonrunning.Theiptablesandiproute2packagesinstalledonthehostOS.Ifyouinstalledlibvirtfromapackage,chancesarethatiptablesandiproute2havebeeninstalled,asdependenciesofthelibvirtpackage.Ifyou'vebuiltlibvirtfromsource,youmightneedtoinstallthemmanually.ArunningKVMinstances.

Howtodoit...ToconfigureanewNATnetworkandconnectaKVMinstancetoit,runthefollowing:

1. Listallavailablenetworks:

root@kvm:~#virshnet-list--all

NameStateAutostartPersistent

----------------------------------------------------------

defaultactiveyesyes

root@kvm:~#

2. Dumptheconfigurationofthedefaultnetwork:

root@kvm:~#virshnet-dumpxmldefault

<networkconnections='1'>


<uuid>2ab5d22c-5928-4304-920e-bc43b8731bcf</uuid>

<forwardmode='nat'>

<nat>

<portstart='1024'end='65535'/>

</nat>

</forward>

<bridgename='virbr0'stp='on'delay='0'/>

<ipaddress='192.168.122.1'netmask='255.255.255.0'>

<dhcp>

<rangestart='192.168.122.2'end='192.168.122.254'/>

</dhcp>

</ip>

</network>

root@kvm:~#

3. ComparethatwiththeXMLdefinitionfileforthedefaultnetwork:

root@kvm:~#cat/etc/libvirt/qemu/networks/default.xml

<network>



<forward/>


<dhcp>

<rangestart="192.168.122.2"end="192.168.122.254"/>

</dhcp>

</ip>

</network>

root@kvm:~#

4. Listallrunninginstancesonthehost:


IdNameState

----------------------------------------------------

3kvm1running

root@kvm:~#

5. EnsurethattheKVMinstancesareconnectedtothedefaultLinuxbridge:




root@kvm:~#

6. CreateanewNATnetworkdefinition:

root@kvm:~#catnat_net.xml

<network>

<name>nat_net</name>


<forward/>


<dhcp>

<rangestart="10.10.10.2"end="10.10.10.254"/>

</dhcp>

</ip>

</network>

root@kvm:~#

7. Definethenewnetwork:

root@kvm:~#virshnet-definenat_net.xml

Networknat_netdefinedfromnat_net.xml

root@kvm:~#virshnet-list--all


----------------------------------------------------------

defaultactiveyesyes

nat_netinactivenoyes

root@kvm:~#

8. Startthenewnetworkandenableautostarting:

root@kvm:~#virshnet-startnat_net

Networknat_netstarted

root@kvm:~#virshnet-autostartnat_net

Networknat_netmarkedasautostarted

root@kvm:~#virshnet-list


----------------------------------------------------------

defaultactiveyesyes

nat_netactiveyesyes

root@kvm:~#

9. Obtainmoreinformationaboutthenewnetwork:

root@kvm:~#virshnet-infonat_net

Name:nat_net

UUID:fba2ca2b-8ca7-4dbb-beee-14799ee04bc3

Active:yes

Persistent:yes

Autostart:yes

Bridge:virbr1

root@kvm:~#

10. EdittheXMLdefinitionofthekvm1instanceandchangethenameofthesourcenetwork:


...


...

<sourcenetwork='nat_net'/>

...

</interface>

...


root@kvm:~#

11. RestarttheKVMguest:


Domainkvm1destroyed


Domainkvm1started

root@kvm:~#

12. Listallsoftwarebridgesonthehost:



virbr08000.000000000000yes

virbr18000.525400ba8e2cyesvirbr1-nic

vnet0

root@kvm:~#

13. ConnecttotheKVMinstancesandchecktheIPaddressoftheeth0interfaceandensureconnectivitytothehostbridge(iftheimageisnotconfiguredforconsoleaccess,useaVNCclientinstead):



Escapecharacteris^]


debianlogin:root

Password:

...

root@debian:~#ipaseth0|grepinet

inet10.10.10.92/24brd10.10.10.255scopeglobaleth0


root@debian:~#ifconfigeth0up&&dhclienteth0

root@debian:~#ping10.10.10.1-c3








root@debian:~#

14. OnthehostOS,examinewhichDHCPservicesarerunning:

root@kvm:~#pgrep-lfadnsmasq

38983/usr/sbin/dnsmasq--conf-

file=/var/lib/libvirt/dnsmasq/default.conf

40098/usr/sbin/dnsmasq--conf-

file=/var/lib/libvirt/dnsmasq/nat_net.conf

root@kvm:~#

15. ChecktheIPofthenewbridgeinterface:

root@kvm:~#ipasvirbr1

43:virbr1:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdiscnoqueue

stateUPgroupdefault

link/ether52:54:00:ba:8e:2cbrdff:ff:ff:ff:ff:ff

inet10.10.10.1/24brd10.10.10.255scopeglobalvirbr1


root@kvm:~#

16. ListtheiptablesrulesfortheNATtable:

root@kvm:~#iptables-L-n-tnat

ChainPREROUTING(policyACCEPT)

targetprotoptsourcedestination

ChainINPUT(policyACCEPT)


ChainOUTPUT(policyACCEPT)


ChainPOSTROUTING(policyACCEPT)


RETURNall--10.10.10.0/24224.0.0.0/24

RETURNall--10.10.10.0/24255.255.255.255

MASQUERADEtcp--10.10.10.0/24!10.10.10.0/24masqports:1024-65535

MASQUERADEudp--10.10.10.0/24!10.10.10.0/24masqports:1024-65535

MASQUERADEall--10.10.10.0/24!10.10.10.0/24

RETURNall--192.168.122.0/24224.0.0.0/24

RETURNall--192.168.122.0/24255.255.255.255

MASQUERADEtcp--192.168.122.0/24!192.168.122.0/24masqports:1024-

65535

MASQUERADEudp--192.168.122.0/24!192.168.122.0/24masqports:1024-

65535

MASQUERADEall--192.168.122.0/24!192.168.122.0/24

RETURNall--192.168.122.0/24224.0.0.0/24

RETURNall--192.168.122.0/24255.255.255.255

MASQUERADEtcp--192.168.122.0/24!192.168.122.0/24masqports:1024-

65535

MASQUERADEudp--192.168.122.0/24!192.168.122.0/24masqports:1024-

65535

MASQUERADEall--192.168.122.0/24!192.168.122.0/24

root@kvm:~#

Howitworks...WestartbylistingallavailablenetworksonthehostOSinstep1.Aswecanseefromtheoutputofthevirshcommand,there'sonlyonedefaultnetworkrunning.

Instep2,weexaminetheconfigurationofthedefaultnetwork.TheXMLdefinitionusesthefollowingattributes:

The<network>attributeistherootelement,instructinglibvirtthatwearedefininganetwork.The<name>elementspecifiesthenameofthenetworkandneedstobeunique.The<uuid>attributeprovidesagloballyuniqueidentifierforthevirtualnetworkandifomitted,itwillbeautogenerated.The<forward>elementanditsmodeattributedefinethenetworkasbeingconnectedtothehostnetworkstack,usingNAT.Ifthiselementismissing,libvirtwillcreateanisolatednetwork.The<nat>subelementfurtherdefinesthe<port>rangethatwillbeusedwhilethehostispreformingNAT.The<bridge>elementspecifiesthebridgetobecreated,itsname,andSTPoptions.The<ip>attributedefinestheIPrangefortheDHCPservertoassignaddressestotheguestVMs.

Instep3,welookattheconfigfileforthedefaultnetworkonthis.Notethatsomeoftheattributesaremissing.Libvirtautogeneratescertainattributesandassignsdefaultvalueswhereappropriate.

Instep4and5,wemakesurethatwehavearunninginstanceconnectedtothedefaultLinuxbridge.

Instep6,wecreateanewnetworkdefinitionusingthedefaultnetworkasatemplate.WechangethenameofthenetworkanddefinenewIPrange.

Withthenewnetworkdefinitionfileready,insteps7and8,wedefinethenewnetwork,startit,andmakesurethatitwillautomaticallystartwhenthelibvirt

daemonstarts,inthecaseofaserverreboot.

Afterobtainingmoreinformationaboutthenewlycreatednetworkinstep9,weproceedtoedittheXMLdefinitionoftheKVMguestinstep10.TomaketheVMpartofthenewnetwork,allweneedtodoisupdatethe<sourcenetwork>element.

AfterrestartingtheKVMguestinstep11,weproceedtolistallavailablesoftwarebridgesonthehostOSinstep12.Notethatwenowhavetwobridges,withthenewbridgehavingtheVMsvirtualinterfacevnet0connectedtoit.

WethenconnecttotherunningKVMguestandensurethatitseth0networkinterfacehasobtainedanIPaddressfromtheDHCPserverrunningonthehostandthattheIPispartoftheaddressrangeweconfiguredearlier.Wealsoensuredconnectivitytothehostbridgeusingthepingcommand.

BackonthehostOS,insteps14and15,wecheckwhatDHCPservicesarerunning.Note,fromtheoutputofthepgrepcommand,thatwenowhavetwodnsmasqprocessesrunning:oneforeachdefinednetwork.

TheNATforwardingisachievedbysettingiptablesrulesaswecanseeinstep18.EachtimewedefineandstartanewNATnetwork,libvirtcreatestherequiredrulesiniptables.Fromtheoutputinstep18,wecanobservethepresenceoftwosetsofNATrules,oneforeachrunningNATnetwork.

ConfiguringbridgednetworkWithfullbridging,wecanconnecttheKVMguestsdirectlytothehostnetwork,withoutusingNAT.However,thissetuprequiresanIPaddress,whichispartofthehostsubnet,foreachvirtualmachine.IfyoucannotallocatethatmanyIPaddresses,considerusingtheNATnetworksetup,asdescribedintheConfiguringNATforwardingnetworkrecipegivenbefore.Inthisnetworkingmode,thevirtualmachinesstillusethehostOSbridgeforconnectivity;however,thebridgeenslavesthephysicalinterfacethatisgoingtobeusedfortheguests.


AserverwithatleasttwophysicalinterfacesTheabilitytoprovisionandstartKVMinstanceswithlibvirtArunningKVMinstance

Howtodoit...Todefineanewbridgednetworkandattachaguesttoit,followthesteps:

1. Takedowntheinterfacewearegoingtobridge:

root@kvm:~#ifdowneth1

root@kvm:~#

2. Editthenetworkconfigurationfileonthehostandreplacetheeth1blockwiththefollowing,ifyourhostOSisDebian/Ubuntu:

root@kvm:~#vim/etc/network/interfaces

...

autovirbr2

ifacevirbr2inetstatic

address192.168.1.2

netmask255.255.255.0

network192.168.1.0

broadcast192.168.1.255

gateway192.168.1.1

bridge_portseth1

bridge_stpon

bridge_maxwait0

...

root@kvm:~#

3. IfusingRedHat/CentOSdistributions,editthefollowingtwofilesinstead:

root@kvm:~#cat/etc/sysconfig/ifcfg-eth1

DEVICE=eth1

NAME=eth1

NM_CONTROLLED=yes

ONBOOT=yes

TYPE=Ethernet

BRIDGE=virbr2

root@kvm:~#cat/etc/sysconfig/ifcfg-bridge_net

DEVICE=virbr2

NAME=virbr2

NM_CONTROLLED=yes

ONBOOT=yes

TYPE=Bridge

STP=on

IPADDR=192.168.1.2

NETMASK=255.255.255.0

GATEWAY=192.168.1.1

root@kvm:~#

4. Startthenewinterfaceup:

root@kvm:~#ifupvirbr2

root@kvm:~#

5. DisablesendingpacketstoiptablesthatoriginatefromtheguestVMs:

root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-iptables=0

net.bridge.bridge-nf-call-iptables=0

root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-iptables=0

net.bridge.bridge-nf-call-iptables=0

root@kvm:~#sysctl-wnet.bridge.bridge-nf-call-arptables=0

net.bridge.bridge-nf-call-arptables=0

root@kvm:~#

6. Listallbridgesonthehost:

root@kvm:~##brctlshow


virbr08000.000000000000yes

virbr28000.000a0ac60210yeseth1

root@kvm:~#

7. EdittheXMLdefinitionfortheKVMinstance:


...

<interfacetype='bridge'>

<sourcebridge='virbr2'/>

</interface>

...


root@kvm:~#

8. RestarttheKVMinstance:


Domainkvm1destroyed


Domainkvm1started

root@kvm:~#

Howitworks...Tosetupbridgenetworking,insteps1and2,wefirstbringthephysicalinterface(eth1inthisexample)down,inordertoenslaveit(makeitapartofthenewbridgewearegoingtocreate).Wethencreatethenetworkconfiguration,specifyingthenewbridgeandthephysicalinterfacethatisgoingtobeapartofthatbridge.Thisineffectmapsthesubnetthatisconfiguredonthephysicalinterfacetothebridge.Ifyourserverhasonlyonenetworkinterface,youcanstillenslaveit.However,youwillneedanadditionalwayofconnectingtotheserverbecauseonceyoubringyourmaininterfacedown,youwilllooseconnectivityandtroubleshootingmightbeimpossibleoveranSSHconnection.

Oncethenewbridgehasbeenconfigured,westartitinstep3.

Instep4,weinstructthekernelnottoapplyiptablerulestoanytrafficoriginatingfromthevirtualguestsconnectedtotheLinuxbridgebecausewearenotusinganyNATrules.

Withthenewinterfaceup,wecannowseethebridgeandtheenslavedphysicalinterfaceattachedtoit,instep5.

Instep6,weedittheXMLdefinitionofthekvm1instance,wherewespecifythetypeofnetworkwewouldliketouse;forthisrecipe,it'sthebridgenetwork.IfyourecallfromtheConfiguringNATforwardingnetworkrecipe,weusedthenetworktypeinsteadofbridgeandwespecifiedalibvirtnetworkname,insteadofthebridgename.

Finally,afterrestartingtheKVMinstanceinstep7,theguestOSshouldnowbeabletoreachotherinstancesthatareapartofthesamesubnetwithoutusingNAT.

ConfiguringPCIpassthroughnetworkTheKVMhypervisorsupportsdirectlyattachingPCIdevicesfromthehostOStothevirtualmachines.WecanusethisfeaturetoattachanetworkinterfacedirectlytotheguestOS,withouttheneedforusingNATorsoftwarebridges.Inthisrecipe,wearegoingtoattachaNetworkInterfaceCard(NIC)thatsupportsSR-IOVSingleRootI/OVirtualization(SR-IOV)fromthehosttotheKVMguest.SR-IOVisaspecificationthatallowsaPeripheralComponentInterconnectExpress(PCIe)devicetoappearasmultipleseparatephysicaldevicesthatcanbesharedbetweenmanyvirtualmachinesonthesamehost,bypassingthehypervisorlayer,thusachievingnativenetworkspeeds.CloudproviderssuchasAmazonAWSexposethisfeatureforitsEC2computeinstancesthroughAPIcalls.

GettingreadyInordertocompletethisrecipe,wearegoingtoneedthefollowing:

AphysicalhostwithNICthatsupportsSR-IOVA802.1QbhcapableswitchwithconnectiontothephysicalserverCPUwitheithertheIntelVT-dorAMDIOMMUextensionsLinuxhostwithlibvirtinstalled,ready-to-provisionKVMinstances

Howtodoit...TosetupanewPCIpassthroughnetworkfollowthesteps:

1. EnumeratealldevicesonthehostOS:

root@kvm:~#virshnodedev-list--tree

computer

|

+-net_lo_00_00_00_00_00_00

+-net_ovs_system_0a_c6_62_34_19_b4

+-net_virbr1_nic_52_54_00_ba_8e_2c

+-net_vnet0_fe_54_00_55_9b_d6

...

|

+-pci_0000_00_03_0

||

|+-pci_0000_03_00_0

|||

||+-net_eth0_58_20_b1_00_b8_61

||

|+-pci_0000_03_00_1

||

|+-net_eth1_58_20_b1_00_b8_61

|

...

root@kvm:~#

2. ListallPCIEthernetadapters:

root@kvm:~#lspci|grepEthernet

03:00.0Ethernetcontroller:IntelCorporation82599ES10-Gigabit

SFI/SFP+NetworkConnection(rev01)

03:00.1Ethernetcontroller:IntelCorporation82599ES10-Gigabit

SFI/SFP+NetworkConnection(rev01)

root@kvm:~#

3. ObtainmoreinformationaboutNICthattheeth1deviceisusing:

root@kvm:~#virshnodedev-dumpxmlpci_0000_03_00_1

<device>

<name>pci_0000_03_00_1</name>

<path>/sys/devices/pci0000:00/0000:00:03.0/0000:03:00.1</path>

<parent>pci_0000_00_03_0</parent>

<driver>

<name>ixgbe</name>

</driver>

<capabilitytype='pci'>

<domain>0</domain>

<bus>3</bus>

<slot>0</slot>

<function>1</function>

<productid='0x10fb'>82599ES10-GigabitSFI/SFP+Network

Connection</product>

<vendorid='0x8086'>IntelCorporation</vendor>

</capability>

</device>

root@kvm:~#

4. Convertthedomain,bus,slot,andfunctionvaluestohexadecimal:

root@kvm:~#printf%x0

0


3


0


1

root@kvm:~#

5. Createanewlibvirtnetworkdefinitionfile:

root@kvm:~#catpassthrough_net.xml

<network>

<name>passthrough_net</name>

<forwardmode='hostdev'managed='yes'>

<pfdev='eth1'/>

</forward>

</network>

root@kvm:~#

6. Define,start,andenableautostartingonthenewlibvirtnetwork:

root@kvm:~#virshnet-definepassthrough_net.xml

Networkpassthrough_netdefinedfrompassthrough_net.xml

root@kvm:~#virshnet-startpassthrough_net

Networkpassthrough_nettstarted

root@kvm:~#virshnet-autostartpassthrough_net

Networkpassthrough_netmarkedasautostarted

root@kvm:~#virshnet-list


----------------------------------------------------------

defaultactiveyesyes

passthrough_netactiveyesyes

root@kvm:~#

7. EdittheXMLdefinitionfortheKVMguest:


...

<devices>

...

<interfacetype='hostdev'managed='yes'>

<source>


function='0x0'/>

</source>

<virtualporttype='802.1Qbh'/>

</interface>


<sourcenetwork='passthrough_net'>

</interface>

...

</devices>

...


root@kvm:~#



Domainkvm1destroyed


Domainkvm1started

root@kvm:~#

9. ListtheVirtualFunctions(VFs)providedbySR-IOVNIC:

root@kvm:~#virshnet-dumpxmlpassthrough_net

<networkconnections='1'>

<name>passthrough_net</name>

<uuid>a4233231-d353-a112-3422-3451ac78623a</uuid>

<forwardmode='hostdev'managed='yes'>

<pfdev='eth1'/>


function='0x1'/>


function='0x3'/>


function='0x5'/>


function='0x7'/>


function='0x1'/>


function='0x3'/>


function='0x5'/>

</forward>

</network>

root@kvm:~#

Howitworks...InordertodirectlyattachPCINICfromthehostOStotheguestVM,wefirstneedtogathersomehardwareinformationaboutthedevice,suchasdomain,bus,slot,andfunctionIDs.Instep1,wecollectinformationaboutallavailabledevicesonthehostserver.Weareinterestedinusingtheeth1networkinterfaceforthisexample;therefore,wenotedowntheuniquePCIidentificationfromtheoutput--pci_0000_03_00_1inthiscase.

ToconfirmthisisindeedNICwewouldliketoexposetotheguest,welistallPCIdevicesinstep2.Fromtheoutput,wecanseethatthePCIIDisthesame03:00.1.

UsingthePCIIDfromstep1,weproceedtocollectmoreinformationaboutNICinstep3.Notethat0000_03_00_1IDisbrokendownintodomainID,busID,slotID,andfunctionID,asshownbytheXMLattributes.WearegoingtousethoseIDsinstep7;however,weneedtoconvertthemtohexadecimalsfirst,whichwedoinstep4.

Insteps5and6,wedefineanewlibvirtnetworkforourguest,startthenetwork,andenableautostartingincasethehostserverrestarts.Ifyoucompletedtheotherrecipesinthischapter,youshouldbealreadyfamiliarwithmostoftheattributesintheXMLdefinitionfileforthenetworkwejustcreated.Thehostdevmodedefinedinthe<forward>attributeiswhatinstructslibvirtthatthenewnetworkisgoingtousePCIpassthrough.Themanaged=yesparameter,asspecifiedinthe<forward>attribute,tellslibvirttofirstdetachthePCIdevicefromthehostbeforepassingitontotheguestandreattachingitbacktothehostaftertheguestterminates.Finally,the<pf>subelementspecifiesthephysicalinterfacethatwillbevirtualizedandpresentedtotheguest.

FormoreinformationontheavailableXMLattributes,pleaserefertohttp://libvirt.org/formatdomain.html.

Instep7,weedittheXMLdefinitionoftheKVMinstance,specifyingthePCIIDsweobtainedinstep3anddefinedaninterfacethatwillusethenewPCIpassthroughnetworkwecreatedinsteps5and6.

http://libvirt.org/formatdomain.html

passthroughnetworkwecreatedinsteps5and6.

WerestarttheKVMinstanceinstep8andfinallyverifythatthephysicalPCINICdeviceisnowpartofthenewpassthroughnetworkwedefinedearlier.NotethepresenceofmultiplePCItypedevices.ThisisbecausethePCIpassthroughdeviceweareusingsupportsSR-IOV.AllKVMgueststhatwillusethisnetworkwillnowbeabletodirectlyusethehostNICbyassigningoneofthelistedvirtualPCIdevices.

ManipulatingnetworkinterfacesLibvirtprovidesahandywaytomanagenetworkinterfacesonthehostthroughthealreadyfamiliarXMLdefinitionsyntax.Wecanusethevirshcommandtodefine,provision,anddeleteLinuxbridgesandobtainmoreinformationaboutexistingnetworkinterfaces,asyou'vealreadyseeninthischapter.

Inthisrecipe,wearegoingtodefineanewLinuxbridge,createit,andfinallyremoveitusingvirsh.Ifyourecallfromearlierrecipes,wecanmanipulatetheLinuxbridgethroughutilitiessuchasbrctl.Withlibvirt,however,wehaveawaytocontrolthisprogrammaticallybywritingthedefinitionfileandusingtheAPIbindings,aswe'llseeinChapter7,UsingPythontoBuildandManageKVMInstances.


ThelibvirtpackageinstalledonthehostALinuxhostwiththebridgekernelmodule

Howtodoit...Tocreateanewbridgeinterfaceusinglibvirt,runthefollowingcommands:

1. Createanewbridgeinterfaceconfigurationfile:

root@kvm:~#cattest_bridge.xml

<interfacetype='bridge'name='test_bridge'>

<startmode="onboot"/>

<protocolfamily='ipv4'>

<ipaddress='192.168.1.100'prefix='24'/>

</protocol>

<bridge>

<interfacetype='ethernet'name='vnet0'>

<macaddress='fe:54:00:55:9b:d6'/>

</interface>

</bridge>

</interface>

root@kvm:~#

2. Definethenewinterface:

root@kvm:~#virshiface-definetest_bridge.xml

Interfacetest_bridgedefinedfromtest_bridge.xml

root@kvm:~#

3. Listallinterfaceslibvirtknowsabout:

root@kvm:~#virshiface-list--all

NameStateMACAddress

---------------------------------------------------

bond0active58:20:b1:00:b8:61

bond0.129activebc:76:4e:20:10:6b

bond0.229activebc:76:4e:20:17:7e

eth0active58:20:b1:00:b8:61

eth1active58:20:b1:00:b8:61

loactive00:00:00:00:00:00

test_bridgeinactive

root@kvm:~#

4. Startthenewbridgeinterface:

root@kvm:~#virshiface-starttest_bridge

Interfacetest_bridgestarted

root@kvm:~#virshiface-list--all|greptest_bridge

test_bridgeactive4a:1e:48:e1:e7:de

root@kvm:~#

5. Listallbridgedevicesonthehost:



test_bridge8000.000000000000no

virbr08000.000000000000yes

virbr18000.525400ba8e2cyesvirbr1-nic

vnet0

root@kvm:~#

6. Checktheactivenetworkconfigurationofthenewbridge:

root@kvm:~#ipastest_bridge

46:test_bridge:<BROADCAST,MULTICAST,UP,LOWER_UP>mtu1500qdisc

noqueuestateUNKNOWNgroupdefault

link/ether4a:1e:48:e1:e7:debrdff:ff:ff:ff:ff:ff

inet192.168.1.100/24brd192.168.1.255scopeglobaltest_bridge


inet6fe80::481e:48ff:fee1:e7de/64scopelink


root@kvm:~#

7. ObtaintheMACaddressofbridge:

root@kvm:~#virshiface-mactest_bridge

4a:1e:48:e1:e7:de

root@kvm:~#

8. ObtainthenameofthebridgebasedbyprovidingitsMACaddress:

root@kvm:~#virshiface-name4a:1e:48:e1:e7:de

test_bridge

root@kvm:~#

9. Destroytheinterface,asfollows:

root@kvm:~#virshiface-destroytest_bridge

Interfacetest_bridgedestroyed


test_bridgeinactive

root@kvm:~#virshiface-undefinetest_bridge

Interfacetest_bridgeundefined


root@kvm:~#

Howitworks...Instep1,wewritetheXMLdefinitionforthenewnetworkinterface.Wespecifybridgeasthetype,anIPaddressfortheinterface,andoptionallyaMACaddress.

Insteps2and3,wedefinethenewbridgeinterfaceandlistit.Defininganinterfacedoesnotautomaticallymakeitactive,soweactivateitinstep4.

Activatingthebridgecreatestheactualinterfaceonthehost,asshowninstep5.

Instep6,weconfirmthattheIPandMACaddressassignedtothebridgeareindeedwhatwespecifiedinstep1.

Insteps7and8,weobtainboththenameandMACaddressusingthevirshutilityandfinally,instep9,weremovethebridgeinterface.

MigratingKVMInstancesInthischapter,wearegoingtodemonstratethefollowinglibvirtKVMmigrationconcepts:

ManualofflinemigrationusinganiSCSIstoragepoolManualofflinemigrationusingGlusterFSsharedvolumesOnlinemigrationusingthevirshcommandwithsharedstorageOfflinemigrationusingthevirshcommandandlocalimageOnlinemigrationusingthevirshcommandandlocalimage

IntroductionMigratingKVMinstancesistheprocessofsendingthestateoftheguestvirtualmachine'smemory,CPU,andvirtualizeddevicesattachedtoit,toadifferentserver.MigratingKVMinstancesisasomewhatcomplicatedprocess,dependingonwhatbackendstoragetheVMisusing(thatis,directory,imagefile,iSCSIvolume,sharedstorage,orstoragepools),thenetworkinfrastructure,andthenumberofblockdevicesattachedtotheguest.Therearefollowingthetwotypesofmigrationsasfaraslibvirtisconcerned:

Offlinemigrationinvolvesdowntimefortheinstance.ItworksbyfirstsuspendingtheguestVM,thencopyinganimageoftheguestmemorytothedestinationhypervisor.TheKVMmachineisthenresumedonthetargethost.IfthefilesystemoftheVMisnotonasharedstorage,thenitneedstobemovedtothetargetserveraswell.Livemigrationworksbymovingtheinstanceinitscurrentstatewithnoperceiveddowntime,preservingthememoryandCPUregisterstates.

Broadlyspeaking,theofflinemigrationinvolvesthefollowing:

StoppingtheinstanceDumpingitsXMLdefinitiontoafileCopyingtheguestfilesystemimagetothedestinationserver(ifnotusingsharedstorage)Definingtheinstanceonthedestinationhostandstartingit

Incontrast,theonlinemigrationrequiressharedstorage,suchasNFSorGlusterFS,removingtheneedtotransfertheguestfilesystemtothetargetserver.Thespeedofthemigrationdependsonhowoftenthememoryofthesourceinstanceisbeingupdated/writtento,thesizeofthememory,andtheavailablenetworkbandwidthbetweenthesourceandtargethosts.

Livemigrationfollowsthisprocess:

TheoriginalVMcontinuestorunwhilethecontentofitsmemoryisbeingtransferredtothetargethost

Libvirtmonitorsforanychangesinthealreadytransferredmemorypages,andiftheyhavebeenupdated,itretransmitsthemOncethememorycontenthasbeentransferredtothedestinationhost,theoriginalinstanceissuspendedandthenewinstanceonthetargethostisresumed

Inthischapter,wearegoingtoperformofflineandlivemigrationsusingiSCSIandGlusterFSwiththehelpofstoragepools.

ManualofflinemigrationusinganiSCSIstoragepoolInthisrecipe,wearegoingtosetupaniSCSItarget,configureastoragepoolforit,andcreateanewKVMinstanceusingtheattachediSCSIblockdeviceasitsbackendvolume.Then,wearegoingtoperformamanualofflinemigrationoftheinstancetoanewhost.


Twoserverswithlibvirtandqemuinstalledandconfigured,namedkvm1andkvm2.ThetwohostsmustbeabletoconnecttoeachotherusingSSHkeysandshorthostname.AserverwithanavailableblockdevicethatwillbeexportedasaniSCSItargetandreachablefrombothlibvirtservers.Ifablockdeviceisnotavailable,pleaserefertotheThere'smore...sectioninthisrecipeforinstructionsonhowtocreateoneusingaregularfile.ThenameoftheiSCSItargetserverinthisrecipeisiscsi_target.ConnectivitytoaLinuxrepositorytoinstalltheguestOS.

Howtodoit...ToperformamanualofflinemigrationofaKVMguestusinganiSCSIstoragepool,followthesesteps:

1. OntheiSCSItargethost,installtheiscsitargetpackageandkernelmodulepackage:

root@iscsi_target:~#apt-getupdate&&apt-getinstalliscsitarget

iscsitarget-dkms

2. Enablethetargetfunctionality:

root@iscsi_target:~#sed-i

's/ISCSITARGET_ENABLE=false/ISCSITARGET_ENABLE=true/g'

/etc/default/iscsitarget

root@iscsi_target:~#cat/etc/default/iscsitarget

ISCSITARGET_ENABLE=true

ISCSITARGET_MAX_SLEEP=3

#ietdoptions

#Seeietd(8)fordetails

ISCSITARGET_OPTIONS=""

root@iscsi_target:~#

3. ConfiguretheblockdevicetoexportwithiSCSI:

root@iscsi_target:~#cat/etc/iet/ietd.conf

Targetiqn.2001-04.com.example:kvm

Lun0Path=/dev/loop1,Type=fileio

Aliaskvm_lun


Replacethe/dev/loop1devicewiththeblockdeviceyouareexportingwithiSCSI.

4. RestarttheiSCSItargetservice:

root@iscsi_target:~#/etc/init.d/iscsitargetrestart

*RemovingiSCSIenterprisetargetdevices:[OK]

*StoppingiSCSIenterprisetargetservice:[OK]

*RemovingiSCSIenterprisetargetmodules:[OK]

*StartingiSCSIenterprisetargetservice[OK]


5. Onbothlibvirthosts,installtheiSCSIinitiator:

root@kvm1/2:~#apt-getupdate&&apt-getinstallopen-iscsi

6. Onbothlibvirtservers,enabletheiSCSIinitiatorserviceandstartit:

root@kvm1/2:~#sed-i's/node.startup=manual/node.startup=

automatic/g'/etc/iscsi/iscsid.conf

root@kvm1/2:~#/etc/init.d/open-iscsirestart

7. Frombothlibvirtinitiatorhosts,listwhatiSCSIvolumesareavailablebyqueryingtheiSCSItargetserver:

root@kvm1/2:~#iscsiadm-mdiscovery-tsendtargets-piscsi_target

10.184.226.74:3260,1iqn.2001-04.com.example:kvm



root@kvm:~#

8. Ononeofthelibvirtservers,createanewiSCSIstoragepool:

root@kvm1:~#catiscsi_pool.xml

<pooltype="iscsi">

<name>iscsi_pool</name>

<source>

<hostname="iscsi_target.example.com"/>

<devicepath="iqn.2001-04.com.example:kvm"/>

</source>

<target>


</target>

</pool>

root@kvm1:~#virshpool-defineiscsi_pool.xml

Pooliscsi_pooldefinedfromiscsi_pool.xml

root@kvm1:~#virshpool-list--all

NameStateAutostart

-------------------------------------------

iscsi_poolinactiveno

root@kvm1:~#

MakesuretoreplacethehostnameoftheiSCSItargetserverwithwhatisappropriateforyourenvironment.BothahostnameandanIPaddresscanbeusedwhenspecifyingtheiSCSItargethost.

9. StartthenewiSCISpool:

root@kvm1:~#virshpool-startiscsi_pool

Pooliscsi_poolstarted

root@kvm1:~#virshpool-list--all

NameStateAutostart

-------------------------------------------

iscsi_poolactiveno

root@kvm1:~#

10. ListtheavailableiSCSIvolumesfromthepoolandobtainmoreinformationonit:

root@kvm1:~#virshvol-list--pooliscsi_pool

NamePath

------------------------------------------------------------------------

------

unit:0:0:0/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0

root@kvm1:~#virshvol-infounit:0:0:0--pooliscsi_pool

Name:unit:0:0:0

Type:block

Capacity:10.00GiB

Allocation:10.00GiB

root@kvm1:~#

11. ListtheiSCSIsessionandtheassociatedblockdevices:

root@kvm1:~#iscsiadm-msession

tcp:[5]10.184.226.74:3260,1iqn.2001-04.com.example:kvm

root@kvm1:~#ls-la/dev/disk/by-path/

total0

drwxr-xr-x2rootroot100Apr1216:24.


lrwxrwxrwx1rootroot9Apr1216:24ip-10.184.22.74:3260-iscsi-

iqn.2001-04.com.example:kvm-lun-0->../../sdf

root@kvm1:~#

12. ExaminethepartitionschemeoftheiSCSIblockdevice:

root@kvm1:~#fdisk-l/dev/disk/by-path/ip-10.184.22.74\:3260-iscsi-

iqn.2001-04.com.example\:kvm-lun-0

Disk/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0:10.7GB,10737418240bytes





Diskidentifier:0x00000000

Disk/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0doesn'tcontainavalidpartitiontable

root@kvm1:~#

13. InstallanewKVMguestusingtheiSCSIvolumeandpool:

root@kvm1:~#virt-install--nameiscsi_kvm--ram1024--extra-args="text

console=tty0utf8console=ttyS0,115200"--graphicsvnc,listen=0.0.0.0--

hvm--


amd64/--diskvol=iscsi_pool/unit:0:0:0

Startinginstall...

RetrievingfileMANIFEST...|3.3kB00:00...

Retrievingfilelinux...

...

root@kvm1:~#virshconsoleiscsi_kvm

...

Requestingsystemreboot

[305.315002]reboot:Restartingsystem

root@kvm1:~#

14. Refreshthepartitiontablelistandexaminethenewblockdevicesaftertheinstallation:

root@kvm1:~#partprobe


total0




iqn.2001-04.com.example:kvm-lun-0->../../sdf


iqn.2001-04.com.example:kvm-lun-0-part1->../../sdf1





root@kvm1:~#fdisk-l/dev/sdf

Disk/dev/sdf:10.7GB,10737418240bytes





Diskidentifier:0x37eb1540

DeviceBootStartEndBlocksIdSystem

/dev/sdf1*2048200130551000550483Linux

/dev/sdf220015102209694714771855Extended

/dev/sdf5200151042096947147718482Linuxswap/Solaris

root@kvm1:~#

15. StartthenewKVMguestandensurethatit'srunning,andthatyoucanconnecttoitusingaVNCclient:

root@kvm1:~#virshstartiscsi_kvm

Domainiscsi_kvmstarted

root@kvm1:~#virshlist--all

IdNameState

----------------------------------------------------

19iscsi_kvmrunning

root@kvm1:~#

16. Tomanuallymigratetheinstancetoanewhost,firststoptheVMandtheiSCSIpool:

root@kvm1:~#virshdestroyiscsi_kvm

Domainiscsi_kvmdestroyed

root@kvm1:~#virshpool-destroyiscsi_pool

Pooliscsi_pooldestroyed


iscsiadm:Noactivesessions.

root@kvm1:~#

17. DumptheXMLconfigurationoftheKVMinstancetoafileandexamineit:

root@kvm1:~#virshdumpxmliscsi_kvm>iscsi_kvm.xml

root@kvm1:~#catiscsi_kvm.xml

<domaintype='kvm'>

<name>iscsi_kvm</name>

<uuid>306e05ed-e398-ef33-d6e2-3708e90b89a6</uuid>




<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>


<disktype='block'device='disk'>


<sourcedev='/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0'/>



</disk>



function='0x2'/>

</controller>

<controllertype='pci'index='0'model='pci-root'/>



function='0x1'/>

</controller>


<macaddress='52:54:00:8b:b8:e3'/>




function='0x0'/>

</interface>

<serialtype='pty'>

<targetport='0'/>

</serial>

<consoletype='pty'>


</console>



<graphicstype='vnc'port='-1'autoport='yes'listen='0.0.0.0'>


</graphics>

<video>



function='0x0'/>

</video>



function='0x0'/>

</memballoon>

</devices>

</domain>

root@kvm1:~#

18. RemotelycreatetheiSCSIstoragepoolfromthekvm1hosttothekvm2host:

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-define

iscsi_pool.xml

Pooliscsi_pooldefinedfromiscsi_pool.xml

root@kvm1:~#

IfyouarenotusingkeysfortheSSHconnectionbetweenboththeKVMhosts,youwillbeaskedtoprovideapasswordbeforethelibvirtcommandcanproceed.WerecommendthatyouuseSSHkeysonthelibvirthostsyouaremigratingbetween.

19. RemotelystarttheiSCSIpoolonthekvm2serverandensurethatit'srunning:

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-start

iscsi_pool

Pooliscsi_poolstarted

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systempool-list--all

NameStateAutostart

-------------------------------------------

iscsi_poolactiveno

root@kvm1:~#

YoucanalsoSSHtothekvm2serverandperformallofthepoolandvolumeoperationslocally.Wedoitremotelytodemonstratetheconcept.

20. RemotelylisttheavailableiSCSIvolumesonthekvm2nodefromthesourcehost:

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemvol-list--pool

iscsi_pool

NamePath

--------------------------------------------------------------------

unit:0:0:0/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0

root@kvm1:~#

21. SSHtothesecondKVMserverandensurethattheiSCSIblockdevicesarenowavailableonthehostOS:


tcp:[3]10.184.226.74:3260,1iqn.2001-04.com.example:kvm


total0


drwxr-xr-x6rootroot120Apr1217:44..


iqn.2001-04.com.example:kvm-lun-0->../../sdc


iqn.2001-04.com.example:kvm-lun-0-part1->../../sdc1





root@kvm2:~#

22. CompletethemigrationbyremotelydefiningtheKVMinstanceandstartingitonthetargethost:

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemdefineiscsi_kvm.xml

Domainiscsi_kvmdefinedfromiscsi_kvm.xml

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemlist--all

IdNameState

----------------------------------------------------

-iscsi_kvmshutoff

root@kvm:~#virsh--connectqemu+ssh://kvm2/systemstartiscsi_kvm

Domainiscsi_kvmstarted

root@kvm:~#virsh--connectqemu+ssh://kvm2/systemlist--all

IdNameState

----------------------------------------------------

3iscsi_kvmrunning

root@kvm1:~#

Howitworks...Inthisrecipe,wedemonstratedhowtomanuallyperformanofflinemigrationofaKVMinstancefromonehosttoanother,usinganiSCSIpool.IntheOnlinemigrationusingthevirshcommandrecipelaterinthischapter,wearegoingtoperformalivemigrationusingthesameiSCSIpoolandinstancewecreatedinthisrecipe,usingthevirshcommand,thusavoidingdowntimefortheinstance.

Let'sstepthroughtheprocessandexploreinmoredetailhowthemanualofflinemigrationwasaccomplished.

WestartwiththeserverthatisgoingtobepresentingtheiSCSItargetbyfirstinstallingtherequirediSCSItargetserverpackagesinstep1.Instep2,weenabletheiSCSItargetfunctionality,enablingtheservertoexportblockdevicesviatheiSCSIprotocol.Instep3,wespecifyanidentified(iSCSIqualifiedname)iqn.2001-04.com.example:kvmfortheiSCSItargetdevicethattheinitiatorsaregoingtouse.Weareusingthe/dev/loop1blockdeviceforthisexample.TheiSCSI-qualifiednamehastheformatiqn.yyyy-mm.naming-authority:uniquenamewhere:

iqn:ThisistheiSCSI-qualifiednameidentifieryyyy-mm:Thisistheyearandmonthwhenthenamingauthoritywasestablishednaming-authority:ThisisusuallyreversesyntaxoftheInternetdomainnameofthenamingauthorityorthedomainnameoftheserverUniquename:Thisisanynameyouwouldliketouse

FormoreinformationaboutiSCSIandthenamingschemeituses,pleaserefertohttps://en.wikipedia.org/wiki/ISCSI.

Withthetargetdefinitioninplace,instep4,werestarttheiSCSIserviceontheserver.

Insteps5and6,weinstallandconfiguretheiSCSIinitiatorserviceonbothKVMnodes,andinstep7,werequestallavailableiSCSItargets.Insteps8and

https://en.wikipedia.org/wiki/ISCSI

9,wedefineandstartanewiSCSI-basedstoragepool.Thesyntaxofthestoragepooldefinitionshouldlookfamiliarifyou'vecompletedtheWorkingwithstoragepoolsrecipefromChapter2,UsinglibvirttoManageKVM.

AftercreatingtheiSCSIstoragepool,weproceededtolistthevolumespartofthatpoolinstep10.Notethatwhenwestartedthepool,itloggedtheiSCSItargetin,resultinginanewblockdevicepresentinthe/dev/disk/by-path/directory,aswecanfurtherseeinstep11.WecannowusethisblockdevicelocallytoinstallanewLinuxOS.Instep12,wecanseethattheiSCSIblockdevicepresentedtothehostOSdoesnotyetcontainanypartitions.

Withthenewblockdevicepresent,weproceedtobuildanewKVMinstanceinstep13,specifyingthestoragepoolandvolumeasthetargetfortheinstallation.AftertheguestOSinstallationcompletes,wecannowseethattherearemultiplepartitionsontheiSCSIblockdeviceinstep14.Wethenproceedtostartthenewguestinstep15.

NowthatwehavearunningKVMinstanceusinganiSCSIblockdevice,wecanproceedwiththeofflinemanualmigrationfromthekvm1hoststothekvm2hosts.

WestartthemigrationprocessbyfirststoppingtherunningKVMinstanceandtheassociatedstoragepoolinstep16.WethendumptheXMLconfigurationoftheKVMguesttoafileinstep17.Wearegoingtouseittodefinetheguestonthetargetserver.Wehaveafewoptionsforthis:wecancopythefileovertothetargetserveranddefinetheinstancethereorwecandothatremotelyfromtheoriginalhost.

Insteps18and19,wecreatetheiSCSIstoragepoolremotelyfromtheoriginalhosttothetargethost.Wecouldhaveloggedintothetargethostandperformedthesameoperationslocallyaswellwiththesameresult.Thepointhereisthatwecanusetheqemu+sshconnectionstringtoremotelyconnecttootherqemuinstancesoverSSH.Insteps20and21,weensurethatthesameiSCSIvolumehasbeensuccessfullyloggedintothetargethost.

Finally,instep22,wedefinetheinstanceonthetargethostusingtheXMLconfigurationwedumpedinstep17andthenstartit.BecauseweareusingthesameXMLdefinitionfileandthesameiSCSIblockdevicecontainingtheguestOSfilesystem,wenowhaveexactlythesameinstancecreatedonthenewserver,thuscompletingtheofflinemigration.

server,thuscompletingtheofflinemigration.

There'smore...IftheiSCSItargetserverdoesnothaveanyavailableblockdevicestoexport,wecancreateanewblockdeviceusingaregularfilebyfollowingthestepsoutlinedhere:

1. Createanewimagefileofagivensize:

root@iscsi_target:~#truncate--size10Gxvdb.img

root@iscsi_target:~#file-sxvdb.img

xvdb.img:data

root@kvmiscsi_target:~#qemu-imginfoxvdb.img

image:xvdb.img

fileformat:raw


disksize:0


2. Ensurethattheloopkernelmoduleiscompiledin(orloaditwithmodprobeloop)andfindthefirstavailableloopdevicetouse:

root@iscsi_target:~#grep'loop'/lib/modules/`uname-r`/modules.builtin

kernel/drivers/block/loop.ko

root@iscsi_target:~#losetup--find

/dev/loop0


3. Associatetherawfileimagewiththefirstavailableloopdevice:

root@iscsi_target:~#losetup/dev/loop0xvdb.img

root@iscsi_target:~#losetup--all

/dev/loop0:[10300]:263347(/root/xvdb.img)


Instep1,wecreateanewimagefileusingthetruncatecommand.

Instep2,welistthefirstavailableblockdevicetouseandinstep3,weassociateitwiththerawimagefilewecreatedinstep1.Theresultisanewblockdeviceavailableas/dev/loop0thatwecanusetoexportiniSCSI.

ManualofflinemigrationusingGlusterFSsharedvolumesIntheManualofflinemigrationusinganiSCSIstoragepoolrecipe,wecreatedaniSCSIstoragepoolanduseditwhileperformingmanualofflinemigration.Withstoragepools,wecandelegatetheoperationofasharedstoragetolibvirtratherthanmanuallyhavingtologin/logoutiSCSItargets,forexample.Thisisespeciallyusefulwhenweperformlivemigrationswiththevirshcommand,aswearegoingtoseeinthenextrecipe.Eventhoughtheuseofstoragepoolsisnotrequired,itsimplifiesandcentralizesthemanagementofbackendvolumes.

Inthisrecipe,wearegoingtousetheGlusterFSnetworkfilesystemtodemonstrateanalternativewayofmanuallymigratingaKVMinstance,thistimenotusingstoragepools.

GlusterFShasthefollowingtwocomponents:

Servercomponent:ThisrunstheGlusterFSdaemonandexportslocalblockdevicesnamedbricksasvolumesthatcanbemountedbytheclientcomponentClientcomponent:ThisconnectstotheGlusterFSclusteroverTCP/IPandcanmounttheexportedvolumes

Therearethefollowingthreetypesofvolumes:

Distributed:ThesearevolumesthatdistributefilesthroughouttheclusterReplicated:ThesearevolumesthatreplicatedataacrosstwoormorenodesinthestorageclusterStriped:Thesearestripefilesacrossmultiplestoragenodes

Forhighavailability,wearegoingtousetwoGFSnodesusingthereplicatedvolumes(twobrickscontainingthesamedata).

GettingreadyTocompletethisrecipe,wearegoingtousethefollowing:

TwoserversthatwillhosttheGlusterFSsharedfilesystem.TwohostsrunninglibvirtandqemuthatwillbeusedtomigratetheKVMguest.Allserversshouldbeabletocommunicatewitheachotherusinghostnames.BothservershostingthesharedvolumesshouldhaveoneblockdeviceavailableforuseasGlusterFSbricks.Ifablockdeviceisnotavailable,pleaserefertotheThere'smore...sectionoftheManualofflinemigrationusinganiSCSIstoragepoolrecipeinthischapteronhowtocreateoneusingaregularfile.ConnectivitytoaLinuxrepositorytoinstalltheguestOS.

Howtodoit...TomigrateaKVMguestusingasharedGlusterFSbackendstore,runthefollowing:

1. Onbothserversthatwillhostthesharedvolumes,installGlusterFS:

root@glusterfs1/2:~#apt-getupdate&&apt-getinstallglusterfs-server

2. FromoneoftheGlusterFSnodes,probetheotherinordertoformacluster:

root@glusterfs1:~#glusterpeerstatus

peerstatus:Nopeerspresent

root@glusterfs1:~#glusterpeerprobeglusterfs2

peerprobe:success

root@glusterfs1:~#

3. VerifythattheGlusterFSnodesareawareofeachother:

root@glusterfs1:~#glusterpeerstatus

NumberofPeers:1

Hostname:glusterfs2

Port:24007

Uuid:923d152d-df3b-4dfd-9def-18dbebf2b76a

State:PeerinCluster(Connected)

root@glusterfs1:~#

4. OnbothGlusterFShosts,createafilesystemontheblockdevicesthatwillbeusedasGlusterFSbricksandmountthem:

root@glusterfs1/2:~#mkfs.ext4/dev/loop5

...

Allocatinggrouptables:done

Writinginodetables:done

Creatingjournal(32768blocks):done

Writingsuperblocksandfilesystemaccountinginformation:done

root@glusterfs1/2:~#mount/dev/loop5/mnt/

root@glusterfs1/2:~#mkdir/mnt/bricks

root@glusterfs1/2:~#

Makesuretoreplacetheblockdevicenamewithwhatisappropriateonyoursystem.

5. FromoneoftheGlusterFSnodes,createthereplicatedstoragevolume,usingthebricksfrombothserversandthenlistit:

root@glusterfs1:~#glustervolumecreatekvm_gfsreplica2transporttcp

glusterfs1:/mnt/bricks/gfs1glusterfs2:/mnt/bricks/gfs2

volumecreate:kvm_gfs:success:pleasestartthevolumetoaccessdata

root@glusterfs1:~#glustervolumelist

kvm_gfs

root@glusterfs1:~#

6. FromoneoftheGlusterFShosts,startthenewvolumeandobtainmoreinformationonit:

root@glusterfs1:~#glustervolumestartkvm_gfs

volumestart:kvm_gfs:success

root@glusterfs1:~#glustervolumeinfo

VolumeName:kvm_gfs

Type:Replicate

VolumeID:69823a48-8b1b-469f-b06a-14ef6f33a6f5

Status:Started

NumberofBricks:1x2=2

Transport-type:tcp

Bricks:

Brick1:glusterfs1:/mnt/bricks/gfs1

Brick2:glusterfs2:/mnt/bricks/gfs2

root@glusterfs1:~#

7. Onbothlibvirtnodes,installtheGlusterFSclientandmounttheGlusterFSvolumethatwillbeusedtohosttheKVMimage:

root@kvm1/2:~#apt-getupdate&&apt-getinstallglusterfs-client

root@kvm1/2:~#mkdir/tmp/kvm_gfs

root@kvm1/2:~#mount-tglusterfsglusterfs1:/kvm_gfs/tmp/kvm_gfs

root@kvm1/2:~#

WhenmountingtheGlusterFSvolume,youcanspecifyeitheroneoftheclusternodes.Intheprecedingexample,wearemountingfromtheglusterfs1node.

8. Ononeofthelibvirtnodes,buildanewKVMinstance,usingthemountedGlusterFSvolume:

root@kvm1:~#virt-install--namekvm_gfs--ram1024--extra-args="text

console=tty0utf8console=ttyS0,115200"--graphicsvnc,listen=0.0.0.0--

hvm--


amd64/--disk/tmp/kvm_gfs/gluster_kvm.img,size=5

...

root@kvm1:~#

9. Ensurethatbothlibvirtnodescanseetheguestimage:

root@kvm1/2:~#ls-al/tmp/kvm_gfs/

total1820300


drwxrwxrwt6rootroot4096Apr1315:00..

-rwxr-xr-x1rootroot5368709120Apr1314:59gluster_kvm.img

root@kvm1/2:~#

10. TomanuallymigratetheKVMinstancefromonelibvirtnodetotheother,firststoptheinstanceanddumpitsXMLdefinition:

root@kvm1:~#virshdestroykvm_gfs

Domainkvm_gfsdestroyed

root@kvm1:~#virshdumpxmlkvm_gfs>kvm_gfs.xml

root@kvm1:~#

11. Fromthesourcelibvirtnode,definetheinstanceonthetargethost:

root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemdefinekvm_gfs.xml

Domainkvm_gfsdefinedfromkvm_gfs.xml


IdNameState

----------------------------------------------------

-kvm_gfsshutoff

root@kvm1:~#

12. StarttheKVMinstanceonthetargethosttocompletethemigration:

root@kvm2:~#virshstartkvm_gfs

Domainkvm_gfsstarted

root@kvm2:~#

WecanalsostarttheKVMinstanceonthedestinationhostfromthesourcehostusingtheqemu+sshconnectionasfollows:root@kvm1:~#virsh--connectqemu+ssh://kvm2/systemstartkvm_gfs

Howitworks...WebeginbyinstallingtheGlusterFSserver-sidepackageonbothserversinstep1.Then,instep2,weproceedtoformaclusterbysendingaprobefromthefirstGlusterFSnode.Iftheprobewassuccessful,wefurtherobtaininformationabouttheclusterinstep3.Instep4,wepreparetheblockdevicesonbothGlusterFSserversforusebycreatingafilesystemonthem,thenmountingthem.TheblockdeviceoncemountedwillcontainthebricksthatwillformavirtualreplicatedvolumeforGlusterFStoexport.

Instep5,wecreatethenewreplicatedvolumeononeofthenodes(thiswillaffecttheentireclusterandonlyneedstoberunfromoneGlusterFSnode).Wespecifythatthetypeisgoingtobereplicated,usingtheTCPprotocolandthelocationofthebrickswearegoingtouse.Oncethevolumeiscreated,westartitinstep6andgetmoreinformationaboutit.Notethatfromtheoutputofthevolumeinformation,wecanseethenumberofbricksinuseandtheirlocationinthecluster.

Instep7,weinstalltheGlusterFSclientcomponentonbothlibvirtserversandmounttheGFSvolume.BothKVMhostsnowsharethesamereplicatedstoragethatisphysicallyhostedontheGlusterFSnodes.WearegoingtousethatsharedstoragetohostthenewKVMimagefile.

Instep8,weproceedwiththeinstallationofanewKVMinstance,usingtheGlusterFSvolumethatwemountedinthepreviousstep.Oncetheinstallationiscomplete,weverifythatbothlibvirtserverscanseethenewKVMimage,instep9.

Westartthemanualmigrationinstep10byfirststoppingtherunningKVMinstance,thensavingitsconfigurationtothedisk.Instep11,weremotelydefinetheKVMguestusingtheXMLdumpandverifythatithasbeensuccessfullydefinedonthetargethost.Finally,westarttheKVMinstanceonthetargetserver,completingthemigration.

OnlinemigrationusingthevirshcommandwithsharedstorageThevirshcommandprovidesamigrateparameterthatwecanusetomigrateKVMinstancesbetweenhosts.Intheprevioustworecipes,wesawhowtomigrateinstancesmanuallywithdowntime.Inthisrecipe,wearegoingtoperformalivemigrationonaninstancethatuseseithertheiSCSIstoragepoolortheGlusterFSsharedvolumesthatweusedearlierinthischapter.

Ifyourecall,livemigrationonlyworkswhentheguestfilesystemresidesonsomesortofsharedmedia,suchasNFS,iSCSI,GlusterFS,orifwefirstcopytheimagefiletoallnodesandusethe--copy-storage-alloptionwithvirshmigrate,aswe'llseelaterinthischapter.

GettingreadyInordertocompletethisrecipe,wearegoingtoneedthefollowing:

Twolibvirthostswithasharedstoragebetweenthem.Ifyou'vecompletedtheearlierrecipesinthischapter,youcaneitherusetheiSCSIstoragepoolwecreatedandtheKVMinstancethatisusingitortheGFSsharedstoragewiththeKVMguest.Bothlibvirthostsshouldbeabletocommunicatewitheachotherusingshorthostnames.

Howtodoit...Toperformalivemigrationusingthesharedstorage,performtheoperationslistedhere:

1. EnsurethattheiSCSIKVMinstancewebuiltearlierisrunningonthesourcehost:


IdNameState

----------------------------------------------------

26iscsi_kvmrunning

root@kvm1:~#

2. Livemigratetheinstancetothesecondlibvirtserver(thetargetnodeshouldalreadyhavetheiSCSIpoolconfigured).Ifthisoperationerrorsout,pleaseconsulttheThere'smore...sectionofthisrecipefortroubleshootingtips:

root@kvm1:~#virshmigrate--liveiscsi_kvmqemu+ssh://kvm2/system

root@kvm1:~#

3. EnsurethattheKVMinstancehasbeenstoppedonthesourcehostandstartedonthetargetserver:


IdNameState

----------------------------------------------------

-iscsi_kvmshutoff


IdNameState

----------------------------------------------------

10iscsi_kvmrunning

root@kvm1:~#

4. Tomigratetheinstanceback,fromthekvm2node,runthefollowing:

root@kvm2:~#virshmigrate--liveiscsi_kvmqemu+ssh://kvm1/system


IdNameState

----------------------------------------------------


IdNameState

----------------------------------------------------

28iscsi_kvmrunning

root@kvm2:~#

Howitworks...WhenmigratingaKVMinstancethatisusingasharedstorage,suchastheiSCSIstoragepoolinthisexample,onceweinitiatethemigrationwiththemigrate--liveparameter,libvirttakescareofloggingouttheiSCSIsessionfromtheoriginalhostandloggingitintothetargetserver,thusmakingtheblockdevicecontainingtheguestfilesystempresentonthedestinationserverwithouttheneedtocopyallthedata.YoumighthavenotedthatthemigrationtookonlyafewsecondsbecausetheonlydatathatwasmigratedwasthememorypagesoftherunningVMonthesourcehost.

There'smore...DependingonyourLinuxdistributionandtheservertype(on-metaloracloudinstance)youarerunningthisrecipeon,youmightencounterafewcommonerrorswhentryingtomigratetheinstance.

Error:error:Unsafemigration:Migrationmayleadtodatacorruptionifdisksusecache!=none.Solution:EdittheXMLdefinitionoftheinstanceyouaretryingtomigrateandupdatethedriversectionoftheblockdevicetocontainthecache=noneattribute:

root@kvm1:~#virsheditiscsi_kvm

...

<devices>

...

<disktype='block'device='disk'>

<drivername='qemu'type='raw'cache='none'/>

<sourcedev='/dev/disk/by-path/ip-10.184.22.74:3260-iscsi-iqn.2001-

04.com.example:kvm-lun-0'/>



</disk>

...

</devices>

Error:error:Internalerror:Attempttomigrateguesttothesamehost02000100-0300-0400-0005-000600070008.Solution:Someservers,usuallyvirtualized,mayreturnthesamesystemUUID,whichcausesthemigrationtofail.Toseeifthisisthecase,runthefollowingonboththesourceandtargetmachines:

root@kvm1/2:~#virshsysinfo|grep-B5-A3uuid

<system>

<entryname='manufacturer'>FOXCONN</entry>

<entryname='product'>CL7100</entry>

<entryname='version'>PVT1-X05</entry>

<entryname='serial'>2M2542Z069</entry>

<entryname='uuid'>02000100-0300-0400-0005-000600070008</entry>

<entryname='sku'>NULL</entry>

<entryname='family'>IntelGrantleyEP</entry>

</system>

root@kvm1/2:~#

IftheUUIDisthesameonbothservers,editthelibvirtconfigurationfileandassignauniqueUUID,thenrestartlibvirt:

root@kvm2:~#vim/etc/libvirt/libvirtd.conf

...

host_uuid="02000100-0300-0400-0006-000600070008"

...

root@kvm2:~#/etc/init.d/libvirt-binrestart



root@kvm2:~#

Error:error:Unabletoresolveaddresskvm2.localdomainservice49152:Nameorserviceisnotknown.Solution:Thisindicatesthatlibvirtisunabletoresolvethehostnameoftheinstances.Makesurethatthehostnamedoesnotresolvetolocalhostandthatyoucanping,orSSHbetweenthesourceandtargetserversusingthehostnameinsteadoftheIPaddressoftheserver.Anexampleofaworkinghostfileforbothlibvirtnodesisasfollows:

root@kvm1:~#cat/etc/hosts

127.0.0.1localhost

10.184.226.106kvm1.example.comkvm1


root@kvm1:~#

root@kvm2:~#cat/etc/hosts

127.0.0.1localhost



root@kvm2:~#

Youcanfindmoreinformationabouttheoperationofaninstancebyexaminingthefollowinglogs:

root@kvm1:~#cat/var/log/libvirt/libvirtd.log

...

2017-04-1219:26:02.297+0000:33149:error:virCommandWait:2399:internal

error:Childprocess(/usr/bin/iscsiadm--modesession)unexpectedexitstatus

21

...

root@kvm1:~#cat/var/log/libvirt/qemu/iscsi_kvm.log

...

2017-04-1317:59:48.040+0000:startingup

LC_ALL=CPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin

QEMU_AUDIO_DRV=none/usr/bin/qemu-system-x86_64-nameiscsi_kvm-S-machinepc-

i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp

1,sockets=1,cores=1,threads=1-uuid306e05ed-e398-ef33-d6e2-3708e90b89a6-no-


socket,id=charmonitor,path=/var/lib/libvirt/qemu/iscsi_kvm.monitor,server,nowait

-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-shutdown-

bootstrict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive

file=/dev/disk/by-path/ip-10.184.226.74:3260-iscsi-iqn.2001-04.com.example:kvm-

lun-0,if=none,id=drive-ide0-0-0,format=raw,cache=none-deviceide-



rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:8b:b8:e3,bus=pci.0,addr=0x3-

chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-

vnc0.0.0.0:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-incomingtcp:

[::]:49153-devicevirtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

chardeviceredirectedto/dev/pts/0(labelcharserial0)

qemu:terminatingonsignal15frompid33148

2017-04-1318:34:49.684+0000:shuttingdown

...

root@kvm1:~#

OfflinemigrationusingthevirshcommandandlocalimagePerformingofflinemigrationwithvirshdoesnotrequireasharedstorage;however,weareresponsibleforprovidingtheguestfilesystemtothenewhost(bycopingtheimagefileandsoon).Theofflinemigrationtransferstheinstancedefinitionwithoutstartingtheguestonthedestinationhostandwithoutstoppingitonthesourcehost.Inthisrecipe,wearegoingtoperformanofflinemigrationusingthevirshcommandonarunningKVMguestusinganimagefileforitsfilesystem.

GettingreadyForthissimplerecipe,wearegoingtoneedthefollowing:

TwolibvirthostsandarunningKVMinstance.Ifoneisnotpresentonyourhost,youcaninstallandstartanewguestVMusingalocalimagefile:

root@kvm:~#virt-install--namekvm_no_sharedfs--ram1024--extra-

args="textconsole=tty0utf8console=ttyS0,115200"--graphics

vnc,listen=0.0.0.0--hvm--


amd64/--disk/tmp/kvm_no_sharedfs.img,size=5

Bothhostsshouldbeabletocommunicatewitheachotherusinghostnames.

Howtodoit...Toperformanofflinemigrationusingthevirshcommand,runthefollowing:

1. MakesurethatwehavearunningKVMinstance:


IdNameState

----------------------------------------------------

26kvm_no_sharedfsrunning

root@kvm1:~#

2. Migratetheinstanceusingtheofflinemode.Ifthisoperationerrorsout,pleaseconsulttheThere'smore...sectionoftheOnlinemigrationusingthevirshcommandrecipefortroubleshootingtips:

root@kvm1:~#virshmigrate--offline--persistentkvm_no_sharedfs

qemu+ssh://kvm2/system

root@kvm1:~#

3. Unlikethelivemigration,thesourceinstanceisstillrunning,andthedestinationinstanceisstopped:


IdNameState

----------------------------------------------------



IdNameState

----------------------------------------------------

-kvm_no_sharedfsshutoff

root@kvm1:~#

Howitworks...Offlinemigrationsarequitesimple;thevirshcommandtransfersthedefinitionfilefromthetargethosttothedestinationanddefinestheinstance.TheoriginalKVMguestisleftrunning.Inordertostartthemigratedinstance,itsimagefileneedstobetransferredtothedestinationfirstandbepresentontheexactsamelocationastheoneonthesourceserver.ThemaindifferencewhenperforminganofflinemigrationascomparedwithjustdumpingtheXMLfileanddefiningitonthedestinationhostisthatlibvirtmakesupdatestothedestinationXMLfile,suchasassigningnewUUIDs.

Intheearlier-mentionedexample,theonlytwonewflagsweretheofflineandpersistentflags.Thepriorspecifiesanofflinetypemigration,andthelatterleavesthedomainpersistentonthedestinationhost.

OnlinemigrationusingthevirshcommandandlocalimageInthisrecipe,wearegoingtolivemigratearunninginstance,withoutsharedstorage.


TwolibvirtserverswitharunningKVMinstanceusingalocalimagefile.WearegoingtousetheKVMguestwebuiltinthepreviousrecipe,Offlinemigrationusingthevirshcommandandlocalimage.Bothserversmustbeabletocommunicatewitheachotherusingtheirhostnames.

Howtodoit...Tomigrateaninstancewithoutsharedstorage,usethefollowingsteps:

1. EnsurethattheKVMguestisrunning:


IdNameState

----------------------------------------------------


root@kvm1:~#

2. Findthelocationoftheimagefile:

root@kvm1:~#virshdumpxmlkvm_no_sharedfs|grep"sourcefile"

<sourcefile='/tmp/kvm_no_sharedfs.img'/>

root@kvm1:~#

3. Transfertheimagefiletothedestinationhost:

root@kvm1:~#scp/tmp/kvm_no_sharedfs.imgkvm2:/tmp/

kvm_no_sharedfs.img100%5120MB243.8MB/s00:21

root@kvm1:~#

4. Migratetheinstanceandensurethatit'srunningonthedestinationhost:

root@kvm1:~#virshmigrate--live--persistent--verbose--copy-storage-

allkvm_no_sharedfsqemu+ssh://kvm2/system

Migration:[100%]


IdNameState

----------------------------------------------------

-kvm_no_sharedfsshutoff


IdNameState

----------------------------------------------------


root@kvm1:~#

5. Fromthedestinationhost,migratetheinstanceback,usingtheincrementalimagetransfer:

root@kvm2:~#virshmigrate--live--persistent--verbose--copy-storage-

inckvm_no_sharedfsqemu+ssh://kvm/system

Migration:[100%]

root@kvm2:~#

Howitworks...Afterweensurethatthesourceinstanceisinarunningstateinstep1,wetransfertheimagefiletothedestinationfile,intheexactsamelocationasthesourceinstep3.Withtheimagefileinplace,wecannowperformalivemigration,whichwedoinstep4andthenbackinstep5.

Thetwonewparameterswehaven'tusedsofarare--copy-storage-allandcopy-storage-inc.Thefirstoneinstructslibvirttotransfertheentireimagefiletothedestination,whereasthesecondperformsanincrementaltransfer,copyingonlythedatathathaschanged,reducingthetransfertime.

MonitoringandBackupofKVMVirtualMachinesInthischapter,wearegoingtocoverthefollowingtopics:

ResourceusagecollectionwithlibvirtMonitoringKVMinstanceswithSensuSimpleKVMbackupswithtarandrsyncCreatingsnapshotsListingsnapshotsInspectingsnapshotsEditingsnapshotsRevertingsnapshotsDeletingsnapshots

IntroductionItgoeswithoutsayingthatmonitoringandbackingupofproductionKVMinstancesisimportantinordertomeetuptimeService-LevelAgreements(SLAs)andtosatisfyhigh-availabilityandperformancerequirements.Monitoringandbackingupofvirtualmachinesisnotverydifferentfrommonitoringandbackingupofphysicalservers.Insomecases,it'sevenmoreconvenienttobackupasingleimagefileforVMorcreateasnapshot,ratherthanthefilesystemofanOSrunningonaphysicalserver.

Inthischapter,wearegoingtoseeexamplesonhowtogatherresourceusagemetricsforliveKVMinstancesandhowtomonitortheresourceusageandalertonpredefinedthresholdswithtoolssuchasSensu.Followingthis,wearegoingtofocusondifferentwaysofbackingupKVMguestsusingtoolssuchasrsync,andcreatingandmanagingsnapshotswiththehelpofthevirshcommand.

ResourceusagecollectionwithlibvirtThefirststepinmonitoringvirtualmachinesistogetfamiliarwiththetoolstocollectmetricsonthesubsystemswewouldliketolateralerton.Inthisrecipe,wearegoingtofocusonCPU,memory,andblockdeviceutilizationoftheKVMguests.WearealsogoingtolearnhowtousetheQEMUmonitoringsocketandtheQEMUguestagent.

Libvirtexposesasetofcallsthatthevirshcommandleveragestogatherresourceusageinformationonthespecifiedguest/domain.WearegoingtomonitorandalertontheinformationcollectedhereintheMonitoringKVMinstanceswithSensurecipelaterinthischapter.


AserverwithlibvirtinstalledandconfiguredArunningKVMinstance

Howtodoit...Tocollectvariousresourceusageinformationforarunninginstanceorthehypervisorhost,performthefollowing:

1. ObtaininformationonthehypervisorCPUutilization:

root@kvm:~#virshnodecpustats--percent

usage:0.0%

user:0.0%

system:0.0%

idle:100.0%

iowait:0.0%

root@kvm:~#

2. Collectinformationonthehypervisormemoryutilization:

root@kvm:~#virshnodememstats

total:131918328KiB

free:103633700KiB

buffers:195532KiB

cached:25874840KiB

root@kvm:~#

3. CheckthestateofaKVMinstance:

root@kvm:~#virshdomstatekvm1

running

root@kvm:~#

4. GetanumberofassignedvirtualCPUs(vCPU)foraKVMinstance:

root@kvm:~#virshvcpucount--currentkvm1--live

1

root@kvm:~#

5. CollectdetailedinformationaboutthevirtualCPUforaguest:


VCPU:0

CPU:29

State:running

CPUtime:118.8s


root@kvm:~#

6. GatherinformationaboutthevCPUtimersfortheguestVM:

root@kvm:~#virshcpu-stats--totalkvm1

Total:

cpu_time175.003045493seconds

user_time2.610000000seconds

system_time7.510000000seconds

root@kvm:~#

7. CollectgeneralinformationabouttheVM:

root@kvm:~#virshdominfokvm1

Id:30

Name:kvm1

UUID:bd167199-c1c4-de7e-4996-43a7f197e565

OSType:hvm

State:running

CPU(s):1

CPUtime:175.6s

Maxmemory:1048576KiB

Usedmemory:1048576KiB

Persistent:yes

Autostart:disable

Managedsave:no

Securitymodel:none

SecurityDOI:0

root@kvm:~#

8. CollectthememoryutilizationfortheVM:

root@kvm:~#virshdommemstat--livekvm1

actual1048576

swap_in0

rss252684

root@kvm:~#

9. GetinformationabouttheblockdevicesassociatedwiththeKVMinstance:

root@kvm:~#virshdomblklistkvm1

TargetSource

------------------------------------------------

hda/var/lib/libvirt/images/kvm1.img

root@kvm:~#

10. ObtainsizeinformationontheblockdevicefortheVM:

root@kvm:~#virshdomblkinfo--devicehdakvm1

Capacity:8589934592

Allocation:2012381184

Physical:2012381184

root@kvm:~#

11. GetanyblockdeviceerrorsfortheKVMinstance:

root@kvm:~#virshdomblkerrorkvm1

Noerrorsfound

root@kvm:~#

12. PrintblockdevicestatisticforaKVMguest:

root@kvm:~#virshdomblkstat--devicehda--humankvm1

Device:hda

numberofreadoperations:42053

numberofbytesread:106145280

numberofwriteoperations:10648

numberofbyteswritten:96768000

numberofflushoperations:4044

totaldurationofreads(ns):833974071

totaldurationofwrites(ns):1180545967

totaldurationofflushes(ns):3458623200

root@kvm:~#

Howitworks...Inthisrecipe,wecollectvarioushypervisorandguestresourceutilizationinformationfromthehostOS.Inlaterrecipes,wewillusethatinformationinamonitoringsystemtoalertandtriggeractionsbasedonsetcriteriaandthresholds.Let'sgothroughthestepsweperformedearlierinmoredetail.

WestartbycollectinginformationabouttheCPUutilizationonthehypervisor/hostOSinstep1.Wethenproceedtogatherthememoryutilizationonthephysicalhostinstep2.NotethatwecanalsouseotherLinuxcommandstodothisinsteadofvirsh,butithelpsdemonstratetheconcept.

MonitoringthestateofaKVMinstanceisimportantincasetheVMterminatesunexpectedlyordoesnotstartautomaticallyafteraserverreboot.Instep3,weobtainthecurrentstateofthevirtualmachine.

Insteps4,5,and6,wecollectinformationaboutthevirtualCPUoftheguest.WecanseethenumberofassignedCPUsalongwithotherusefulinformation,suchasthetimetheCPUspendsrunningkernelanduserspacecode.

Instep7,wecollectmoregeneralinformationaboutthevirtualmachine;ofnotableinterestisthetotalandusedamountofmemorythatwecansetalertingthresholdson.

Instep8,wegetinformationaboutthememoryutilizationoftheKVMinstance.Wecanseethetotal,swap,andresidentmemoryusageintheoutput.

Instep9,welisttheblockdevicesattachedtothevirtualmachine,andweusethatinstep10togetinformationaboutitscapacity.Ifthereareanyerrorsassociatedwiththeblockdevice,theoutputofthecommandinstep11willshowthat,whichwecanusetotriggermonitoringalerts.

MonitoringtheperformanceofablockdeviceattachedtoaKVMinstancecanbedoneusingtheoutputofthecommandinstep12.

There'smore...Whenwecreateavirtualmachinewithlibvirt,theQEMUprocessthatisstartedexposesamonitoringsocketthatwecanconnecttoandcollectinformationabouttheguest.

Let'sseehowthatlooksfortheKVMinstancewe'vebeenusing:

1. Gettheprocessinformationfortheguestinstance:

root@kvm:~#pgrep-lfakvm1

32332/usr/bin/qemu-system-x86_64-namekvm1-S-machinepc-

i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-smp

1,sockets=1,cores=1,threads=1-uuidbd167199-c1c4-de7e-4996-43a7f197e565

-no-user-config-nodefaults-chardev

socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitor,server,nowait

-monchardev=charmonitor,id=monitor,mode=control-rtcbase=utc-no-

shutdown-bootstrict=on-devicepiix3-usb-


file=/var/lib/libvirt/images/kvm1.img,if=none,id=drive-ide0-0-

0,format=raw-deviceide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-

0,id=ide0-0-0,bootindex=1-netdevtap,fd=24,id=hostnet0-device

rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:55:9b:d6,bus=pci.0,addr=0x3

-chardevpty,id=charserial0-deviceisa-

serial,chardev=charserial0,id=serial0-vnc146.20.141.158:0-device

cirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-

pci,id=balloon0,bus=pci.0,addr=0x4

root@kvm:~#

Noticefromtheprecedingoutputthe-chardevsocket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm1.monitorandthe-monchardev=charmonitor,id=monitor,mode=controlparameterspassedtotheQEMUprocess.

2. Wecanaccessthissocketintwoways,eitherbyconnectingtoitusingtoolssuchasncandsocatorwiththevirshcommand,asfollows:

root@kvm:~#virshqemu-monitor-commandkvm1--hmp"info"

infoballoon--showballooninformation

infoblock[-v][device]--showinfoofoneblockdeviceorallblock

devices(anddetailsofimageswith-voption)

infoblock-jobs--showprogressofongoingblockdeviceoperations

infoblockstats--showblockdevicestatistics

infocapture--showcaptureinformation

infochardev--showthecharacterdevices

infocpus--showinfosforeachCPU

infocpustats--showCPUstatistics

infohistory--showthecommandlinehistory

infoirq--showtheinterruptsstatistics(ifavailable)

infojit--showdynamiccompilerinfo

infokvm--showKVMinformation

infomem--showtheactivevirtualmemorymappings

infomice--showwhichguestmouseisreceivingevents

infomigrate--showmigrationstatus

infomigrate_cache_size--showcurrentmigrationxbzrlecachesize

infomigrate_capabilities--showcurrentmigrationcapabilities

infomtree--showmemorytree

infoname--showthecurrentVMname

infonetwork--showthenetworkstate

infonuma--showNUMAinformation

infopci--showPCIinfo

infopcmcia--showguestPCMCIAstatus

infopic--showi8259(PIC)state

infoprofile--showprofilinginformation

infoqdm--showqdevdevicemodellist

infoqtree--showdevicetree

inforegisters--showthecpuregisters

inforoms--showroms

infosnapshots--showthecurrentlysavedVMsnapshots

infospice--showthespiceserverstatus

infostatus--showthecurrentVMstatus(running|paused)

infotlb--showvirtualtophysicalmemorymappings

infotpm--showtheTPMdevice

infotrace-events--showavailabletrace-events&theirstate

infousb--showguestUSBdevices

infousbhost--showhostUSBdevices

infousernet--showusernetworkstackconnectionstates

infouuid--showthecurrentVMUUID

infoversion--showtheversionofQEMU

infovnc--showthevncserverstatus

root@kvm:~#

3. ToobtaininformationaboutthenetworkinterfaceoftheKVMinstance,wecanrunthefollowingcode:

root@kvm:~#virshqemu-monitor-commandkvm1--hmp"infonetwork"

net0:index=0,type=nic,model=rtl8139,macaddr=52:54:00:55:9b:d6

\hostnet0:index=0,type=tap,fd=24

root@kvm:~#

QEMUprovidesaguestagentdaemonthatcanbestartedinsidetheKVMinstanceandthencanbeconnectedtofromthehostOS.Wecancollectadditionaldataorupdatecertainsettingsforthevirtualmachinedirectlyfromthehost.

Let'sseeanexampleofinstallingandusingtheQEMUguestagent:

1. CreatetherequireddirectorythatwillcontainthecommunicationsocketbetweenlibvirtonthehostandtheguestagentrunninginsidetheKVMguest:

root@kvm:~#mkdir-p/var/lib/libvirt/qemu/channel/target

root@kvm:~#chownlibvirt-qemu:kvm

/var/lib/libvirt/qemu/channel/-R

root@kvm:~#

2. EdittheconfigurationfortherunningVMandaddthefollowingdefinitionunderthe<devices>section:


...

<devices>

...

<channeltype='unix'>

<sourcemode='bind'

path='/var/lib/libvirt/qemu/channel/target/kvm1.org.qemu.guest_agent.0'/>

<targettype='virtio'name='org.qemu.guest_agent.0'/>

<aliasname='channel0'/>

<addresstype='virtio-serial'controller='0'bus='0'

port='1'/>

</channel>

...

</devices>

...


root@kvm:~#



Domainkvm1destroyed


Domainkvm1started

root@kvm:~#

4. ConnecttotheVMandinstallandstarttheQEMUguestagent,asfollows:



Escapecharacteris^]

root@debian:~#aptupdate&&aptinstallqemu-guest-agent

...

root@debian:~#serviceqemu-guest-agentstart

root@debian:~#serviceqemu-guest-agentstatus

●qemu-guest-agent.service-LSB:QEMUGuestAgentstartupscript

Loaded:loaded(/etc/init.d/qemu-guest-agent)

Active:active(running)sinceWed2017-04-1914:44:08CDT;

33minago

Process:397ExecStart=/etc/init.d/qemu-guest-agentstart

(code=exited,status=0/SUCCESS)

CGroup:/system.slice/qemu-guest-agent.service

└─425/usr/sbin/qemu-ga--daemonize-mvirtio-serial-p

/dev/virti...

Apr1914:44:08debiansystemd[1]:StartingLSB:QEMUGuestAgent

startups.....

Apr1914:44:08debiansystemd[1]:StartedLSB:QEMUGuestAgent

startupscript.

Hint:Somelineswereellipsized,use-ltoshowinfull.

root@debian:~#

5. Backonthehost,wecanseethenewsocketfile:

root@kvm:~#ls-la/var/lib/libvirt/qemu/channel/target/

total8

drwxr-xr-x2libvirt-qemukvm4096Apr1919:43.

drwxr-xr-x3libvirt-qemukvm4096Apr1919:43..

srwxr-xr-x1libvirt-qemukvm0Apr1919:43

kvm1.org.qemu.guest_agent.0

root@kvm:~#

6. Let'sconnecttotheguestagentfromthehostandlisttheavailablecommandsthattheguestagentaccepts:

root@kvm:~#virshqemu-agent-commandkvm1--pretty--cmd

'{"execute":"guest-info"}'

{

"return":{

"version":"2.1.2",

"supported_commands":[

{

"enabled":true,

"name":"guest-set-vcpus",

"success-response":true

},

{

"enabled":true,

"name":"guest-get-vcpus",


},

{

"enabled":true,

"name":"guest-network-get-interfaces",


},

...

]

}

}

root@kvm:~#

7. GetinformationabouttheguestvCPUbyrunningthefollowingcode:

root@kvm:~#virshqemu-agent-commandkvm1--pretty--cmd

'{"execute":"guest-get-vcpus"}'

{

"return":[

{

"online":true,

"can-offline":false,

"logical-id":0

}

]

}

root@kvm:~#

Usingthemonitoringandguestagentsocketsprovidesanadditionalwayofcollectingmoreinformationabouttherunningvirtualmachinesonthelibvirthostthatwecanlateraddasmonitoringchecks.

MonitoringKVMinstanceswithSensuSensuisacompletemonitoringsolutionthatusestheclient-servermodel;theserverpublisheschecksinamessagequeueprovidedbytheRabbitmqservice.Theclients/agentssubscribetotopicsinthequeueandexecutethespecifiedchecksonthehosttheyrunon.StateandhistoricaldataisstoredinaRedisserver.

Inthisrecipe,wearegoingtoinstalltheSensuserver,Rabbitmqmessagequeue,andtheRedisserverononehost,writeasimplemonitoringcheckusingtheinformationweobtainedfromtheResourceusagecollectionwithlibvirtrecipeearlier,andinstalltheSensuagentinsidetheKVMguest.


ALinuxhostwithlibvirtinstalledandrunningAKVMinstancerunningonthelibvirthostNetworkconnectivitybetweentheKVMinstanceandthelibvirthost

Howtodoit...TosetupanewSensudeploymentanddefinevariousmonitoringchecks,performthefollowingsteps:

1. InstalltheRedisserverandensurethatitisrespondingtorequests:

root@kvm:~#apt-getinstall-yredis-server

...

root@kvm:~#redis-cliping

PONG

root@kvm:~#

2. InstalltheRabbitmqserver:

root@kvm:~#apt-getinstall-yrabbitmq-server

...

root@kvm:~#

3. CreatethevirtualhosttheSensuagentswillsubscribetoandthecredentialsandpermissionsfortheRabbitmqclients:

root@kvm:~#rabbitmqctladd_vhost/sensu

Creatingvhost"/sensu"...

...done.

root@kvm:~#rabbitmqctladd_usersensusecret

Creatinguser"sensu"...

...done.

root@kvm:~#rabbitmqctlset_permissions-p/sensusensu".*"".*"".*"

Settingpermissionsforuser"sensu"invhost"/sensu"...

...done.

root@kvm:~#

4. AddtheSensuupstreamrepositoryanditskeyandtheninstalltheSensupackage:

root@kvm:~#wget-qhttps://sensu.global.ssl.fastly.net/apt/pubkey.gpg-

O-|apt-keyadd-

OK

root@kvm:~#echo"debhttps://sensu.global.ssl.fastly.net/aptsensu

main"|tee/etc/apt/sources.list.d/sensu.list

debhttps://sensu.global.ssl.fastly.net/aptsensumain


...

root@kvm:~#apt-getinstall-ysensu

...

root@kvm:~#

5. SensuusesJSON-basedconfigurationfilesforitsconfiguration.CreatetheSensuAPIconfigurationfile:

root@kvm:/etc/sensu/conf.d#catapi.json

{

"api":{

"host":"localhost",

"bind":"0.0.0.0",

"port":4567

}

}

root@kvm:/etc/sensu/conf.d#

6. ConfigurethetransporttypeforSensu;weareusingRabbitmqforthisdeployment:

root@kvm:/etc/sensu/conf.d#cattransport.json

{

"transport":{

"name":"rabbitmq",

"reconnect_on_error":true

}

}


7. ConfigurewheretheRabbitmqserviceisacceptingconnections,thevirtualhost,andcredentials:

root@kvm:/etc/sensu/conf.d#catrabbitmq.json

{

"rabbitmq":{

"host":"0.0.0.0",

"port":5672,

"vhost":"/sensu",

"user":"sensu",

"password":"secret"

}

}


8. SpecifythehostandporttheRedisserviceislisteningon:

root@kvm:/etc/sensu/conf.d#catredis.json

{

"redis":{

"host":"localhost",

"port":6379

}

}


9. ConfiguretheSensuclient:

root@kvm:/etc/sensu/conf.d#catclient.json

{

"client":{

"name":"ubuntu",

"address":"127.0.0.1",

"subscriptions":[

"base"

],

"socket":{

"bind":"127.0.0.1",

"port":3030

}

}

}


FormoreinformationonSensu,pleaserefertohttps://sensuapp.org/docs/.

10. InstallthewebfrontendforSensu,namedUchiwa:

root@kvm:/etc/sensu/conf.d#apt-getinstall-yuchiwa

...

root@kvm:/etc/sensu/conf.d

11. ConfiguretheUchiwafrontend:

root@kvm:/etc/sensu/conf.d#cat/etc/sensu/uchiwa.json

{

"sensu":[

{

"name":"KVMguests",

"host":"localhost",

"ssl":false,

"port":4567,

"path":"",

"timeout":5000

}

],

"uchiwa":{

"port":3000,

"stats":10,

"refresh":10000

}

}


12. StarttheSensuserver,API,client,andfrontendcomponents:

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-serverstart

*Startingsensu-server[OK]

https://sensuapp.org/docs/

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-apistart

*Startingsensu-api[OK]

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-clientstart

*Startingsensu-client[OK]

root@kvm:/etc/sensu/conf.d#/etc/init.d/uchiwarestart

uchiwastarted.


13. ConnecttotheKVMinstanceconsole;installandconfiguretheSensuclient:

root@kvm:/etc/sensu/conf.d#virshconsolekvm1


Escapecharacteris^]

root@debian:~#wget-q

https://sensu.global.ssl.fastly.net/apt/pubkey.gpg-O-|apt-keyadd-

OK

root@debian:~#echo"debhttps://sensu.global.ssl.fastly.net/aptsensu

main"|tee/etc/apt/sources.list.d/sensu.list

debhttps://sensu.global.ssl.fastly.net/aptsensumain

root@debian:~#aptinstallapt-transport-https

root@debian:~#aptupdate&&aptinstallsensu

...

root@debian:~#cd/etc/sensu/conf.d/

root@debian:/etc/sensu/conf.d#catclient.json

{

"client":{

"name":"monitor_kvm",

"address":"10.10.10.92",

"subscriptions":["base"]

}

}

root@debian:/etc/sensu/conf.d#catrabbitmq.json

{

"rabbitmq":{

"host":"10.10.10.1",

"port":5672,

"vhost":"/sensu",

"user":"sensu",

"password":"secret"

}

}

root@debian:/etc/sensu/conf.d#cattransport.json

{

"transport":{

"name":"rabbitmq",

"reconnect_on_error":true

}

}

root@debian:/etc/sensu/conf.d#

ReplacetheIPaddressoftheclientwiththeIPaddressconfiguredinsidetheKVMinstance.UpdatetheIPaddressoftheRabbitmq

serverwiththeIPaddressconfiguredonthehostbridge.EnsurethattheKVMguestcanpingthebridgeIPonthehostOS.

14. StarttheSensuclient:

root@debian:/etc/sensu/conf.d#/etc/init.d/sensu-clientstart

Startingsensu-client:.


15. ConnecttotheUchiwainterfaceandensurethatthehostSensuclientandtheKVMguestSensuclientarelistedintheCLIENTSsection:

TheUchiwafrontendshowingtheconnectedclients

16. WhilestillconnectedtotheKVMguest,installamemorycheckfromthegemrepositoryandtestit:

root@debian:/etc/sensu/conf.d#aptinstallrubygems

...

root@debian:/etc/sensu/conf.d#gemsearchsensu|grepplugins|grep

memory

sensu-plugins-memory(0.0.2)

sensu-plugins-memory-checks(2.1.0)


root@debian:/etc/sensu/conf.d#geminstallsensu-plugins-memory-checks

...

root@debian:/etc/sensu/conf.d#/etc/init.d/sensu-clientrestart

configurationisvalid

Stoppingsensu-client:.

Startingsensu-client:.

oot@debian:/etc/sensu/conf.d#/usr/local/bin/check-memory-percent.rb-w

80-c9

MEMOK-systemmemoryusage:11%


17. BackonthehostOS,definethenewmemorycheckfortheKVMguest:

root@kvm:/etc/sensu/conf.d#catcheck_memory.json

{

"checks":{

"memory_check":{

"command":"/usr/local/bin/check-memory-percent.rb-w80-c90",

"subscribers":["base"],

"handlers":["default"],

"interval":300

}

}

}


18. RestarttheSensucomponents:

root@kvm:/etc/sensu/conf.d#/etc/init.d/uchiwarestart

Killinguchiwa(pid15350)withSIGTERM

Waitinguchiwa(pid15350)todie...

Waitinguchiwa(pid15350)todie...

uchiwastopped.

uchiwastarted.

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-serverrestart


*Stoppingsensu-server[OK]

*Startingsensu-server[OK]

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-apirestart


*Stoppingsensu-api[OK]

*Startingsensu-api[OK]


19. Thememory_checkfortheKVMinstanceisnowshowingintheUchiwadashboard:

TheUchiwafrontendshowingthememorycheckfortheKVMguest

Howitworks...Intheprevioussection,weinstalledaSensuserverandalltherequiredinfrastructurecomponentsforittorunonthehypervisorhost.ThenweinstalledtheclientinsideaKVMinstance,installedthememoryrubycheck,anddefineditonthehost.Let'sexamineallthestepsinmoredetail.

Instep1,weinstalltheRedisserverandensurethatitisacceptingconnections.Redisisakey-valuestoreservicethatSensuusestostorethehistoricalinformationaboutthechecks,currentstate,andconnectedclients.

WiththeRedisserverinplace,weproceedtoinstallandconfigureRabbitmqinsteps2through9.RabbitmqisamessagebusconformingtotheAdvancedMessageQueuingProtocol(AMQP)standard.TheSensuserverandclientsproduceandconsumemessagesfromthequeuetotriggermonitoringactions.

Althoughnotrequired,insteps10and11,weinstallandconfigureawebfrontendfortheSensuservernamedUchiwa.WecanusethewebinterfacetocheckonthestatusofdifferentchecksfortheKVMguestswemonitor.

Instep13,weinstalltheSensuclientinsidetheKVMguestinstanceandproceedtoinstallamemorymonitoringscriptfromageminstep16.Amonitoringscriptcanbewritteninanylanguage(RUBYinthiscase)aslongasitreturnstheexpectederrorcodesthatSensuexpects.Inthenextsection,wearegoingtowriteanewcheckfromscratchusingBash.

There'smore...Intheprevioussection,wesawanexampleofhowtousearubycheckinsidetheKVMinstanceandmonitorthememoryutilization.SensuprovidesstandalonechecksthatcanbetriggeredfromtheSensuclient,independentlyfromtheSensuserverschedulingmechanism.Let'susethatfeatureandwriteasimplecheckinBashthatwillrunfromthehostOS,insteadoftheKVMguest,andusethevirshcommandtocheckthestatusofaKVMinstance:

1. Writeastandalonecheckdefinitionwiththecustomscriptthatsensu-clientwillexecutetoperformthecheck:

root@kvm:/etc/sensu/conf.d#catcheck_kvm_instance_status.json

{

"checks":{

"check_kvm_instance_status":{

"command":"check_kvm_instance_status.sh-nkvm1",

"standalone":true,

"subscribers":["base"],

"interval":60

}

}

}


2. IntheSensupluginsdirectory,writethissimpleBashscript:

root@kvm:/etc/sensu/conf.d#cd../plugins/

root@kvm:/etc/sensu/plugins#catcheck_kvm_instance_status.sh

#!/bin/bash

#ChecksifaKVMinstanceisrunning

usage()

{

echo"Usage:`basename$0`-n|--namekvm1"

exit2

}

sanity_check()

{

if["$INSTANCE_NAME"==""]

then

usage

fi

}

report_result()

{

if["$INSTANCE_STATE"=="shutoff"]

then

echo"CRITICAL-KVMinstance$INSTANCE_NAMEisnotrunning"

exit2

else

echo"OK-KVMinstance$INSTANCE_NAMEisrunning"

exit0

fi

}

check_instance_state()

{

declare-gINSTANCE_STATE="shutoff"

INSTANCE_STATE=$(sudo/usr/bin/virshdomstate$INSTANCE_NAME)

}

main()

{

sanity_check

check_instance_state

report_result

}

while[[$#>1]]

do

key=$1

case$keyin

-n|--name)

INSTANCE_NAME=$2

shift

;;

*)

usage

;;

esac

shift

done

main

root@kvm:/etc/sensu/plugins#

3. Makethescriptexecutable,addtheSensuuserinasudoersfile,andtestthecheckbyexecutingit:

root@kvm:/etc/sensu/plugins#chmodu+xcheck_kvm_instance_status.sh

root@kvm:/etc/sensu/plugins#chownsensu:sensu

check_kvm_instance_status.sh

root@kvm:/etc/sensu/plugins#echo"sensuALL=(ALL)NOPASSWD:ALL">

/etc/sudoers.d/sensu

root@kvm:/etc/sensu/plugins#sudo-usensu

./check_kvm_instance_status.sh--namekvm1

OK-KVMinstancekvm1isrunning

root@kvm:/etc/sensu/plugins#virshdestroykvm1

Domainkvm1destroyed

root@kvm:/etc/sensu/plugins#sudo-usensu

./check_kvm_instance_status.sh--namekvm1

CRITICAL-KVMinstancekvm1isnotrunning

root@kvm:/etc/sensu/plugins#virshstartkvm1

Domainkvm1started

root@kvm:/etc/sensu/plugins#

4. RestarttheSensuclientonthehost;checkthelogsandtheUchiwadashboardforthenewstandalonecheck:

root@kvm:/etc/sensu/conf.d#/etc/init.d/sensu-clientrestart


*Stoppingsensu-client[OK]

*Startingsensu-client[OK]

root@kvm:/etc/sensu/conf.d#cat/var/log/sensu/sensu-client.log|grep

check_kvm_instance_status

{"timestamp":"2017-04-

20T17:37:48.409805+0000","level":"warn","message":"loadingconfig

file","file":"/etc/sensu/conf.d/check_kvm_instance_status.json"}

{"timestamp":"2017-04-

20T17:38:16.746861+0000","level":"info","message":"publishingcheck

result","payload":{"client":"ubuntu","check":

{"command":"check_kvm_instance_status.sh-n

kvm1","standalone":true,"subscribers":

["base"],"interval":60,"name":"check_kvm_instance_status","issued":1492709896,"executed":1492709896,"duration":0.016,"output":"OK

-KVMinstancekvm1isrunning\n","status":0}}}


TheUchiwafrontendshowingthestandaloneinstancecheck

UsingtheexamplesfromtheResourceusagecollectionwithlibvirtrecipe,youshouldnowbeabletowriteavarietyofSensumonitoringchecksexecutedfromthehypervisororinsidetheKVMguests.

FormoreinformationonhowSensucanexecutescriptswhenanalertistriggered,pleaserefertothehandlerssectionoftheofficialdocumentationathttps://sensuapp.org/docs/latest/reference/handlers.html.

https://sensuapp.org/docs/latest/reference/handlers.html

SimpleKVMbackupswithtarandrsyncInthisrecipe,wearegoingtocreateabackupofaKVMinstanceusingtarandrsyncandstoreitonaremoteserver.ThisistheeasiestwaytobackupaKVMinstance.Inthenextfewrecipes,wearegoingtocreatesnapshotsandusethemasacoldbackup.

GettingreadyForthisextremelysimplerecipe,wearegoingtoneed:

AlibvirthostwitharunningKVMinstance,usinganimagefileasitsbackingstoreThetarandrsyncLinuxutilitiesAremoteservertotransferthebackup

Howtodoit...Tobackupavirtualmachineusingtarandrsync,performthefollowingsteps:

1. Createthebackupdirectoryandchangetoit:

root@kvm:~#mkdirbackup_kvm1&&cdbackup_kvm1

root@kvm:~/backup_kvm1#

2. FindthelocationoftheimagefileoftheKVMguest:

root@kvm:~/backup_kvm1#virshdumpxmlkvm1|grep"sourcefile"

<sourcefile='/var/lib/libvirt/images/kvm1.img'/>


3. Savethecurrentinstanceconfigurationtodisk:

root@kvm:~/backup_kvm1#virshdumpxmlkvm1>kvm1.xml


4. StoptheKVMguestandcopytheimagefiletothebackupdirectory:

root@kvm:~/backup_kvm1#virshdestroykvm1

Domainkvm1destroyed

root@kvm:~/backup_kvm1#cp/var/lib/libvirt/images/kvm1.img.

root@kvm:~/backup_kvm1#ls-lah

total2.4G

drwxr-xr-x2rootroot4.0KApr2018:37.

drwx------7rootroot4.0KApr2018:36..

-rwxr-xr-x1rootroot8.0GApr2018:37kvm1.img

-rw-r--r--1rootroot3.0KApr2018:36kvm1.xml


5. CreateasinglearchivefortheVM'sconfigurationandimagefiles:

root@kvm:~/backup_kvm1#tarjcvfkvm1_backup.tar.bz.

./

./kvm1.img

./kvm1.xml

root@kvm:~/backup_kvm1#rmkvm1.imgkvm1.xml


6. Transferthebackuparchivetoaremoteserver:

root@kvm:~/backup_kvm1#rsync-vazkvm1_backup.tar.bzkvm2:/tmp

sendingincrementalfilelist

kvm1_backup.tar.bz

sent842,977,610bytesreceived35bytes26,761,195.08bytes/sec

totalsizeis845,671,214speedupis1.00


7. Torestorefromthebackup,logintotheremoteserverandextractthearchive:

root@kvm2:~#cd/tmp/

root@kvm2:/tmp#tarjxfvkvm1_backup.tar.bz

./

./kvm1.img

./kvm1.xml

root@kvm2:/tmp#

8. Copytheimagefiletotheconfiguredlocationanddefinetheinstance:

root@kvm2:/tmp#cpkvm1.img/var/lib/libvirt/images/

root@kvm2:/tmp#virshdefinekvm1.xml

Domainkvm1definedfromkvm1.xml

root@kvm2:/tmp#virshlist--all|grepkvm1

-kvm1shutoff

root@kvm2:/tmp#

Howitworks...Aftercreatingthebackupdirectoryinstep1,wesavethecurrentguestdefinitiontodiskinstep3.Instep4,afterstoppingthevirtualmachine,wecopyitsimagefiletothebackupdirectory.Instep5,wecreateabzip2compresseddatafilewhichwetransfertoaremoteserverinstep6.

Ontheremoteserver,weextractthearchiveinstep7andcopytherawimagefiletowheretheXMLdefinitionoftheinstanceisexpectingit,thendefinetheinstanceinstep8.

Notethatinordertopreservetheconsistencyandintegrityofthedatawhilecopyingtheimagefiletothebackupdirectory,wehadtofirststoptheKVMguest.

CreatingsnapshotsAvirtualmachinesnapshotpreservesthecurrentstateofarunningorstoppedinstanceataspecificpointintime.Itcanlaterbeusedtorestoretheinstancefromthatpoint.Snapshotscanbeusedasbackupsorastemplatesforbuildingnewvirtualmachinesthatwillbecopiesoftheoriginalinstance.

Totakeadvantageofsnapshots,thebackingstoremustfirstsupportit.IfyourecallfromtheManagingDiskimageswithqemu-imgrecipeinChapter1,GettingStartedwithQEMUandKVM,wecreatedarawimagetypefortheKVMguest.Inthisrecipe,wearegoingtousetheQEMUCopy-On-Write(QCOW2)imageformatasthebackingstorefortheKVMinstance,becausetherawimageformatdoesnotsupportsnapshots.

UsingtheQCOW2imageformat,wecancreateabaseimagecontainingtheguestOSandeverythingelseweneedforthevirtualmachine,andthencreateseveralcopy-on-writeoverlaydiskimagesontopoftheoriginalbaseimage.Thesenewoverlayimagescanbeusedinnewvirtualmachinesrightaway,bycreatingnewXMLdefinitionfilespointingtothenewimage.

Let'sseeanexampleofusingQEMUtocreateimageoverlaysbeforeweproceedwithmakinglibvirtsnapshots:

1. TocollectinformationaboutaQCOW2image,wecanusetheqemu-imgutility:

root@kvm:~#qemu-imginfokvm1.qcow2

image:kvm1.qcow2

fileformat:qcow2


disksize:2.4G

cluster_size:65536


compat:1.1

lazyrefcounts:false

root@kvm:~#

ToconvertanexistingrawimagetoQCOW2,run:root@kvm:~#qemu-imgconvert-fraw-Oqcow2

/var/lib/libvirt/images/kvm1.img/var/lib/libvirt/images/kvm1.qcow2

2. Let'screateanewoverlayimagebasedontheqcow2precedingimage:

root@kvm:~#qemu-imgcreate-fqcow2-b

/var/lib/libvirt/images/kvm1_copy.qcow2

/var/lib/libvirt/images/kvm1_copy_2.qcow2

Formatting'/var/lib/libvirt/images/kvm1_copy_2.qcow2',fmt=qcow2

size=8589934592backing_file='/var/lib/libvirt/images/kvm1_copy.qcow2'

encryption=offcluster_size=65536lazy_refcounts=off

root@kvm:~#

3. Gettinginformationaboutthenewoverlayimagenowshowsthebackingfileit'sbasedon:

root@kvm:~#qemu-imginfokvm1_copy.qcow2

image:kvm1_copy.qcow2

fileformat:qcow2


disksize:196K

cluster_size:65536

backingfile:/var/lib/libvirt/images/kvm1.qcow2


compat:1.1

lazyrefcounts:false

root@kvm:~#

4. Wecancreateanewoverlayfilefromthepreviousoverlayfile:

root@kvm:~#qemu-imgcreate-fqcow2-b

/var/lib/libvirt/images/kvm1_copy.qcow2


Formatting'/var/lib/libvirt/images/kvm1_copy_2.qcow2',fmt=qcow2

size=8589934592backing_file='/var/lib/libvirt/images/kvm1_copy.qcow2'

encryption=offcluster_size=65536lazy_refcounts=off

root@kvm:~#qemu-imginfo/var/lib/libvirt/images/kvm1_copy_2.qcow2

image:/var/lib/libvirt/images/kvm1_copy_2.qcow2

fileformat:qcow2


disksize:196K

cluster_size:65536

backingfile:/var/lib/libvirt/images/kvm1_copy.qcow2


compat:1.1

lazyrefcounts:false

root@kvm:~#

5. Let'slisttheentireimagechainforthelastoverlayfile:

root@kvm:~#qemu-imginfo--backing-chain


image:/var/lib/libvirt/images/kvm1_copy_2.qcow2

fileformat:qcow2


disksize:196K

cluster_size:65536

backingfile:/var/lib/libvirt/images/kvm1_copy.qcow2


compat:1.1

lazyrefcounts:false

image:/var/lib/libvirt/images/kvm1_copy.qcow2

fileformat:qcow2


disksize:196K

cluster_size:65536



compat:1.1

lazyrefcounts:false

image:/var/lib/libvirt/images/kvm1.qcow2

fileformat:qcow2


disksize:2.4G

cluster_size:65536


compat:1.1

lazyrefcounts:false

root@kvm:~#

LibvirtusestheabilityoftheQCOW2imageformattocreateachainofoverlaysnapshotsthatcanbeusedasbackupsorastemplatesfornewvirtualmachines.Onceanoverlayiscreated,theoriginalbaseimageistreatedasread-only.Modificationstothebaseimages(inthisexample,kvm1.qcow2andkvm1_copy.qcow2becausebotharebaseimagesforthekvm1_copy_2.qcow2image)arenotrecommended.Here'sadiagrammaticrepresentationofthechainofoverlayimagefileswecreatedearlier:

ThechainofoverlayQCOW2images,eachoneservingasabaseimageforthenext


AlibvirthostwithanexistingQCOW2image,withnosnapshotsattachedArunningKVMinstanceTheQEMUtoolset

Howtodoit...TocreateanewKVMsnapshot,followthesesteps:

1. Createaninternalsnapshotoftherunninginstance:

root@kvm:~#virshsnapshot-createkvm1

Domainsnapshot1492797458created

root@kvm:~#

2. Examinethelocationofthenewsnapshotconfiguration:

root@kvm:~#ls-la/var/lib/libvirt/qemu/snapshot/

total12

drwxr-xr-x3libvirt-qemukvm4096Apr2114:05.

drwxr-x---6libvirt-qemukvm4096Apr2114:04..

drwxr-xr-x2rootroot4096Apr2117:57kvm1

root@kvm:~#ls-la/var/lib/libvirt/qemu/snapshot/kvm1/

total12


drwxr-xr-x3libvirt-qemukvm4096Apr2114:05..

-rw-------1rootroot3089Apr2117:571492797458.xml

root@kvm:~#

3. ExaminethesnapshotXMLdefinition:

root@kvm:~#cat/var/lib/libvirt/qemu/snapshot/kvm1/1492797458.xml



<domainsnapshot>

<name>1492797458</name>

<state>running</state>

<creationTime>1492797458</creationTime>

<memorysnapshot='internal'/>

<disks>

<diskname='hda'snapshot='internal'/>

</disks>

<domaintype='kvm'>

<name>kvm1</name>

...

</domain>

<active>1</active>

</domainsnapshot>

root@kvm:~#

4. Collectinformationaboutthebaseimage:

root@kvm:~#qemu-imginfo/var/lib/libvirt/images/kvm1.qcow2

image:/var/lib/libvirt/images/kvm1.qcow2

fileformat:qcow2


disksize:2.4G

cluster_size:65536

Snapshotlist:

IDTAGVMSIZEDATEVMCLOCK

11492797458155M2017-04-2117:57:3803:41:16.790


compat:1.1

lazyrefcounts:false

root@kvm:~#

5. Obtaininformationaboutthediskdeviceonthevirtualmachine:

root@kvm:~#virshdomblklistkvm1

TargetSource

------------------------------------------------

hda/var/lib/libvirt/images/kvm1.qcow2

root@kvm:~#

6. Createanexternal,disk-onlysnapshot:

root@kvm:~#virshsnapshot-create-askvm1kvm1_ext_snapshot"Diskonly

externalsnapshotforkvm1"--disk-only--diskspec

hda,snapshot=external,file=/var/lib/libvirt/images/kvm1_disk_external.qcow2

Domainsnapshotkvm1_ext_snapshotcreated

root@kvm:~#

7. Getinformationabouttheexternalsnapshot:

root@kvm:~#qemu-imginfo

/var/lib/libvirt/images/kvm1_disk_external.qcow2

image:/var/lib/libvirt/images/kvm1_disk_external.qcow2

fileformat:qcow2


disksize:196K

cluster_size:65536


backingfileformat:qcow2


compat:1.1

lazyrefcounts:false

root@kvm:~#

Howitworks...Therearetwomaintypesofsnapshot:

Aninternalsnapshot:ThebaseimagefileitselfcontainsthesavedstateandallsubsequentchangestothevirtualmachineAnexternalsnapshot:Thebaseimagewillcontainthesavedstateofthevirtualmachinethusbecomingaread-onlybaseimage,andanewoverlayimageiscreatedtotrackanyfuturechanges

Bothtypesofsnapshotscanbeperformedonjustthediskorthememoryofthevirtualmachine,eitheronaliveorstoppedinstance.

Intheprecedingstep1,wecreateaninternalsnapshotofthevirtualmachine.Afterthesnapshot,there'sonlyoneimagefile:theoriginalimage,nowcontainingthesnapshot.Wecanseethattheimageisasnapshotinstep4,undertheSnapshotlistsectionoftheoutput.

Instep6,weperformanexternal,disk-onlysnapshot,byspecifyingthevirtualmachinedisk,name,andlocationforthesnapshot.Notethatafterthesnapshot,anewimagefilehasbeencreatedtotrackanyfurtherchanges.Weexaminethatfileinstep7.Notehowthebackingfileistheoriginalqcow2image.

Toperformthedisksnapshots,libvirtleveragestheQEMUfunctionality,liketheqemu-imgcommandwesawearlierwhencreatingtheoverlayimages.

Wecannowsavethesnapshotsasbackupsorusethemtostartnewvirtualmachines.Inthefollowingrecipes,wearegoingtoseeexamplesonhowtouseandmanipulatethesnapshots.

ListingsnapshotsInthepreviousrecipe,wecreatetwosnapshotsofthesameKVMinstance:oneinternalandonedisk-only,externalsnapshot.Inthisrecipe,wearegoingtolearnhowtolistexistingsnapshots.

GettingreadyForthisrecipe,wearegoingtoneed:

AlibvirthostwiththeQEMUtoolsetArunningKVMinstanceThesnapshotswecreatedintheCreatingsnapshotsrecipe

Howtodoit...Tolistallexistingsnapshots,followthenextsteps:

1. ListallsnapshotsforthespecifiedKVMinstance:

root@kvm:~#virshsnapshot-listkvm1

NameCreationTimeState

------------------------------------------------------------

14927974582017-04-2117:57:38+0000running

kvm1_ext_snapshot2017-04-2118:08:49+0000disk-snapshot

root@kvm:~#

2. Listonlythedisk-basedsnapshots:

root@kvm:~#virshsnapshot-list--disk-onlykvm1


------------------------------------------------------------


root@kvm:~#

3. Listonlytheinternalsnapshots:

root@kvm:~#virshsnapshot-list--internalkvm1


------------------------------------------------------------

14927974582017-04-2117:57:38+0000running

root@kvm:~#

4. Listtheexternalsnapshotsonly:

root@kvm:~#virshsnapshot-list--externalkvm1


------------------------------------------------------------


root@kvm:~#

5. Listallimagesinahierarchicaltreeformat:

root@kvm:~#virshsnapshot-list--treekvm1

1492797458

|

+-kvm1_ext_snapshot

root@kvm:~#

Howitworks...Weusedtheversatilevirshsnapshot-listcommandtolistallinternalandexternalsnapshotsforthespecifiedvirtualmachine.Notehowwecangetsimilarinformationusingtheqemu-imgcommanddirectlyontheimagefiles,aswesawearlierinthischapter.However,theAPIcallsthatlibvirtprovidesforlistingsnapshotsaremuchmoreconvenient.Inthenextchapter,wearegoingtoseeexamplesonhowtousethelibvirtPythonbindingstomanipulateKVMinstancesandtheirsnapshots.

InspectingsnapshotsInthisshortrecipe,wearegoingtoseeexamplesonhowtoobtainmoreinformationonexistingvirtualmachinesnapshots.


AlibvirthostwiththeQEMUtoolsetThesnapshotswecreatedintheCreatingsnapshotsrecipe

Howtodoit...Toinspectasnapshot,runthefollowingcommands:

1. ListallavailablesnapshotsforthespecifiedKVMinstance:



------------------------------------------------------------

14927974582017-04-2117:57:38+0000running


root@kvm:~#

2. Getinformationabouttherunningsnapshot:

root@kvm:~#virshsnapshot-infokvm1--snapshotname1492797458

Name:1492797458

Domain:kvm1

Current:no

State:running

Location:internal

Parent:-

Children:1

Descendants:1

Metadata:yes

root@kvm:~#

3. Getinformationaboutthedisksnapshot:

root@kvm:~#virshsnapshot-infokvm1--snapshotnamekvm1_ext_snapshot

Name:kvm1_ext_snapshot

Domain:kvm1

Current:yes

State:disk-snapshot

Location:external

Parent:1492797458

Children:0

Descendants:0

Metadata:yes

root@kvm:~#

4. DumptheXMLconfigurationforthedisksnapshot:

root@kvm:~#virshsnapshot-dumpxmlkvm1--snapshotnamekvm1_ext_snapshot

--security-info

<domainsnapshot>

<name>kvm1_ext_snapshot</name>

<description>Diskonlyexternalsnapshotforkvm1</description>

<state>disk-snapshot</state>

<parent>

<name>1492797458</name>

</parent>

<creationTime>1492798129</creationTime>

<memorysnapshot='no'/>

<disks>

<diskname='hda'snapshot='external'type='file'>

<drivertype='qcow2'/>

<sourcefile='/var/lib/libvirt/images/kvm1_disk_external.qcow2'/>

</disk>

</disks>

<domaintype='kvm'>

<name>kvm1</name>

...

<devices>



<drivername='qemu'type='qcow2'/>

<sourcefile='/var/lib/libvirt/images/kvm1.qcow2'/>


<addresstype='drive'controller='0'bus='0'target='0'

unit='0'/>

</disk>

...

</devices>


</domain>

</domainsnapshot>

Howitworks...Instep1,welistalltheavailablesnapshotsforthekvm1virtualmachine.Insteps2and3,weobtaininformationaboutthesnapshots.OfparticularinterestistheParentandChildrenfields,showingusthehierarchyofthesnapshots.

Instep4,weexaminetheXMLdefinitionoftheKVMguestandthedisk-only,externalsnapshot.Wecanobservethesnapshot='external'typeandthebaseimagelocationspecifiedwiththe<sourcefile='/var/lib/libvirt/images/kvm1_disk_external.qcow2'/>stanza.

EditingsnapshotsInthisrecipe,wearegoingtoedittheXMLdefinitionofanexistingsnapshotandexaminethechanges.



Howtodoit...Toeditasnapshot,runthefollowing:

1. ListallavailablesnapshotsforthespecifiedKVMinstance:



------------------------------------------------------------

14927974582017-04-2117:57:38+0000running


root@kvm:~#

2. Editthedisksnapshotandchangeitsnameanddescription:

root@kvm:~#virshsnapshot-editkvm1--snapshotnamekvm1_ext_snapshot--

rename

<domainsnapshot>

<name>kvm1_ext_snapshot_renamed</name>

<description>Diskonlyexternalsnapshotforkvm1</description>

...

root@kvm:~#

3. Listthesnapshotsaftertheupdate:



------------------------------------------------------------

14927974582017-04-2117:57:38+0000running

kvm1_ext_snapshot_renamed2017-04-2118:08:49+0000disk-snapshot

root@kvm:~#

Howitworks...Libvirtprovidesawaytoeditthesnapshotdefinitionforavirtualmachine.WecanchangevariousXMLattributes,suchasthesnapshotname,description,orthelocationofthebackingimagefile.Instep1,welistallavailablesnapshotsforthespecifiedKVMinstance,thenproceedtoupdatethenameanddescriptionofthediskimage.Finally,instep3,wecanseethechangednamefortheexternalsnapshot.

RevertingsnapshotsInthisrecipe,wearegoingtocreateaninternalsnapshotofarunninginstance,introduceachange,thenrestorebacktotheoriginalinstancestateusingthesnapshot.


AlibvirthostwithanexistingQCOW2imageArunningKVMinstance,usingtheQCOW2imageTheQEMUtoolset

Howtodoit...TorevertthestateofaKVMinstancetoanolderstate,fromanexistingsnapshot,runthefollowing:

1. ConnecttotheKVMinstanceandcreateanewfile:



Escapecharacteris^]

root@debian:~#touchSNAPSHOT

root@debian:~#

root@kvm:~#

2. Createaninternalsnapshotofthevirtualmachine:

root@kvm:~#virshsnapshot-createkvm1

Domainsnapshot1492802417created

root@kvm:~#

3. Getinformationaboutthesnapshot:

root@kvm:~#virshsnapshot-infokvm1--snapshotname1492802417

Name:1492802417

Domain:kvm1

Current:yes

State:running

Location:internal

Parent:1492797458

Children:0

Descendants:0

Metadata:yes

root@kvm:~#

4. Connectbacktothevirtualmachineanddeletethefilewecreatedinstep1:



Escapecharacteris^]

root@debian:~#rm-fSNAPSHOT

root@debian:~#

root@kvm:~#

5. Restoretheinstancefromthelatestsnapshot:

root@kvm:~#virshsnapshot-revertkvm1--snapshotname1492802417

root@kvm:~#

6. Connecttothevirtualmachineandconfirmthatfilewedeletedinthepreviousstepexistsagain:



Escapecharacteris^]

root@debian:~#ls-laSNAPSHOT

-rw-r--r--1rootroot0Apr2114:08SNAPSHOT

root@debian:~#

root@kvm:~#

Howitworks...Instep1,weconnecttotheKVMinstanceusingtheconsoleandcreateanemptyfile.Wearegoingtousethefiletotrackchangesonthevirtualmachine.Instep2,wecreateaninternalsnapshotandobtainmoreinformationaboutitinstep3.Instep4,weconnecttotheKVMguestagainanddeletethefile.Instep5,werestorefromthesnapshot,confirmingthatthestateoftheinstancehasbeenindeedrevertedtobeforethesnapshot,asshownbythepresenceoftheoriginalfilewecreatedearlier.

DeletingsnapshotsInthisquickrecipe,wearegoingtodeletethesnapshotswecreatedearlierintheCreatingsnapshotsrecipe,usinglibvirt.

GettingreadyForthisrecipe,weareonlygoingtoneedthefollowing:


Howtodoit...Todeleteasnapshot,followthesesteps:

1. Listallsnapshotspresentonthehost:



------------------------------------------------------------

14927974582017-04-2117:57:38+0000running

14928024172017-04-2119:20:17+0000running


root@kvm:~#

2. Deletethelatestsnapshotbasedonthecreationtime:

root@kvm:~#virshsnapshot-deletekvm1--snapshotname1492802417

Domainsnapshot1492802417deleted

root@kvm:~#

3. Listtheremainingsnapshots:



------------------------------------------------------------

14927974582017-04-2117:57:38+0000running


root@kvm:~#

4. Deletethelatestsnapshot:

root@kvm:~#virshsnapshot-deletekvm1--current

Domainsnapshot1492797458deleted

root@kvm:~#

Howitworks...Instep1,welistallsnapshotsonthehostOS.Wethendeletethelatestsnapshot,specifyingitsname,instep2.Instep3,weverifythatthesnapshothasbeenindeeddeleted.Finally,instep4,wedeletethelatestimagebyspecifyingthe--currentflag.

Pleasenotethatinordertodeleteorrestoreanexternalsnapshotalibvirtversionnewerthan1.2.2isrequired.IfyourLinuxdistributiondoesnotprovideanewerversioninitsrepositories,youwillhavetocompilelibvirtfromsource.

DeployingKVMInstanceswithOpenStack

Inthischapter,wearegoingtocoverthefollowingtopics:

PreparingthehostfortheOpenStackdeploymentInstallingandconfiguringtheOpenStackKeystoneidentityserviceInstallingandconfiguringtheOpenStackGlanceimageserviceInstallingandconfiguringtheOpenStackNovacomputeserviceInstallingandconfiguringtheOpenStackNeutronnetworkingserviceBuildingandinspectingKVMinstanceswithOpenStackStoppingKVMinstanceswithOpenStackTerminatingKVMinstanceswithOpenStack

IntroductionOpenStackisacloudoperatingsystemthatsimplifiesthedeploymentandmanagementofvirtualmachinesorcontainersinascalableandhighlyavailableway.Itoperatesonpoolsofcomputeresources(physicalorvirtualservers)andprovidesanintelligentschedulingmechanismtoselectappropriatehosts,andtobuildormigrateVMs.

OpenStackallowsaneasiermanagementofvirtualimagesandprovidesacentralizedwayofcreatingandmanagingsoftware-definednetworks.Itintegrateswellwithavarietyofexternalandinternalprojectsinordertodeliveruserandserviceauthentication.OpenStackmodulardesignallowsaddingandremovingservicesasneededwhereaminimalproductiondeploymentmayconsistofasfewastwoprojectsanimageandcomputeservice.

Thefollowingdiagramshowstheever-growinglistofOpenStackprojectsandtheinteractionbetweenthem:

TheOpenStackcomponentsandhowtheyinteractwitheachother

Inthischapter,wearegoingtocreateasimpleOpenStackdeploymentontwocomputenodesusingtheKeystone,Glance,Nova,andNeutronprojectsfromtheNewtonreleaseofOpenStack,onanUbuntuXenial16.04server.

NewtonreleaseofOpenStack,onanUbuntuXenial16.04server.

FormoreinformationontheOpenStackproject,pleasevisithttps://www.openstack.org/software/.

https://www.openstack.org/software/

PreparingthehostfortheOpenStackdeploymentInthisrecipe,wearegoingtoinstalltheinfrastructurecomponentsthatOpenStackdependson,suchasthedatabaseserver,themessagequeue,andthecachingservice.Theprojectsthatwearegoingtousethroughoutthischapterdependontheseservicesforcommunicationandpersistentstorage.

GettingreadyForthisrecipe,wearegoingtoneedthefollowingcomponents:

AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallation

Howtodoit...InordertokeepthedeploymentsimpleandfocusontheprovisioningaspectofOpenStack,wearegoingtouseasinglephysicalservertohostallservices.Inproductionenvironments,itisacommonapproachtoseparateeachserviceontotheirownsetofservers,forscalabilityandhighavailability.Byfollowingthestepsoutlinedinthischapter,youshouldbeabletodeployallservicesonmultiplehosts,byreplacingtheIPaddressesandhostnamesintheconfigurationfiles,asneeded.

1. UpdatethehostandinstallthepackagerepositoryfortheNewtonrelease:

root@controller:~#aptinstallsoftware-properties-common

root@controller:~#add-apt-repositorycloud-archive:newton

root@controller:~#aptupdate&&aptdist-upgrade

root@controller:~#reboot

root@controller:~#aptinstallpython-openstackclient

2. InstalltheMariaDBdatabaseserver:

root@controller:~#aptinstallmariadb-serverpython-pymysql

root@controller:~#cat/etc/mysql/mariadb.conf.d/99-openstack.cnf

[mysqld]

bind-address=10.208.130.36

default-storage-engine=innodb

innodb_file_per_table

max_connections=4096

collation-server=utf8_general_ci

character-set-server=utf8

root@controller:~#

ReplacetheIPaddressofthenetworkinterfacethedatabaseserverbindsto,asperyourhost.

3. Restarttheserviceandsecuretheinstallation:

root@controller:~#servicemysqlrestart

root@controller:~#mysql_secure_installation

Forsimplicity,wearegoingtouselxcpasswordasapasswordforallservicesthroughoutthechapter.

4. InstalltheRabbitMQmessagingservice,createanewuser,password,andsetpermissions:

root@controller:~#aptinstallrabbitmq-server

root@controller:~#rabbitmqctladd_useropenstacklxcpassword

Creatinguser"openstack"...

root@controller:~#rabbitmqctlset_permissionsopenstack".*"".*"".*"

Settingpermissionsforuser"openstack"invhost"/"...

root@controller:~#

5. Installandconfigurethememcachedservice:

root@controller:~#aptinstallmemcachedpython-memcache

root@controller:~#sed-i's/127.0.0.1/10.208.130.36/g'

/etc/memcached.conf

root@controller:~#cat/etc/memcached.conf|grep-vi"#"|sed

'/^$/d'

-d

logfile/var/log/memcached.log

-m64

-p11211

-umemcache

-l10.208.130.36

root@controller:~#servicememcachedrestart

root@controller:~#

Howitworks...OpenStackusesaSQLdatabase,suchasMysql/MariaDB/Percona,tostoreinformationaboutitsservices.Inthefollowingrecipes,wearegoingtocreatedatabasesfortheKeystone,Glance,Nova,andNeutronprojects.WeinstallandconfigureMariaDBinsteps1through3.

Themessagingqueueweinstallandconfigureinstep4providesacentralizedwayfortheservicestocommunicatewitheachotherbyproducingandconsumingmessages.OpenStacksupportsafewdifferentmessagebusimplementations,suchasRabbitMQ,Qpid,andZeroMQ.

TheidentityserviceKeystonecachesauthenticationtokensusingthememcacheddaemon.Weinstallandconfigureitinstep5.

InstallingandconfiguringtheOpenStackKeystoneidentityserviceTheidentityserviceprovidedbytheKeystoneprojectisacentralizedpointinordertomanageauthenticationandauthorization,usedbyotherOpenStackcomponents,suchasNovacomputeandtheimageserviceGlance.Keystonealsokeepsacatalogofservicesandtheendpointstheyprovidethattheusercanlocatebysendingqueriestoit.

Inthisrecipe,wearegoingtoinstallandconfigureKeystone,createtwoprojects(aunitofownership)forourservicesandassignusersandrolestothoseprojects.


AnUbuntuServerwithGreatvirtualizationcapabilitiesAccesstotheInternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe

Howtodoit...Toinstall,configure,createnewprojects,userroles,andcredentials,performthefollowingstepsintheorderpresentedhere:

1. Createthekeystonedatabaseandgrantpermissionstothekeystoneuser:

root@controller:~#mysql-uroot-plxcpassword

MariaDB[(none)]>CREATEDATABASEkeystone;

QueryOK,1rowaffected(0.01sec)

MariaDB[(none)]>GRANTALLPRIVILEGESONkeystone.*TO

'keystone'@'localhost'IDENTIFIEDBY'lxcpassword';

QueryOK,0rowsaffected(0.00sec)

MariaDB[(none)]>GRANTALLPRIVILEGESONkeystone.*TO

'keystone'@'%'IDENTIFIEDBY'lxcpassword';


MariaDB[(none)]>exit

Bye

root@controller:~#

2. InstalltheidentityserviceKeystonefromtherepositoryweconfiguredearlier:

root@controller:~#aptinstallkeystone

3. CreatethefollowingminimalKeystoneconfiguration:

root@controller:~#cat/etc/keystone/keystone.conf

[DEFAULT]

log_dir=/var/log/keystone

[assignment]

[auth]

[cache]

[catalog]

[cors]

[cors.subdomain]

[credential]

[database]

connection=mysql+pymysql://keystone:lxcpassword@controller/keystone

[domain_config]

[endpoint_filter]

[endpoint_policy]

[eventlet_server]

[federation]

[fernet_tokens]

[identity]

[identity_mapping]

[kvs]

[ldap]

[matchmaker_redis]

[memcache]

[oauth1]

[os_inherit]

[oslo_messaging_amqp]

[oslo_messaging_notifications]

[oslo_messaging_rabbit]

[oslo_messaging_zmq]

[oslo_middleware]

[oslo_policy]

[paste_deploy]

[policy]

[profiler]

[resource]

[revoke]

[role]

[saml]

[security_compliance]

[shadow_users]

[signing]

[token]

provider=fernet

[tokenless_auth]

[trust]

[extra_headers]

Distribution=Ubuntu

root@controller:~#

4. PopulatetheKeystonedatabase:

root@controller:~#su-s/bin/sh-c"keystone-managedb_sync"keystone

...

root@controller:~#

5. InitializetheFernetkeyrepositories:

root@controller:~#keystone-managefernet_setup--keystone-userkeystone

--keystone-groupkeystone

root@controller:~#keystone-managecredential_setup--keystone-user

keystone--keystone-groupkeystone

root@controller:~#

6. BootstraptheKeystoneservice:

root@controller:~#keystone-managebootstrap--bootstrap-password

lxcpassword--bootstrap-admin-urlhttp://controller:35357/v3/--

bootstrap-internal-urlhttp://controller:35357/v3/--bootstrap-public-

urlhttp://controller:5000/v3/--bootstrap-region-idRegionOne

root@controller:~#

7. AddthefollowingstanzainApacheandrestarttheservice:

root@controller:~#cat/etc/apache2/apache2.conf

...

ServerNamecontroller

...

root@controller:~#serviceapache2restart

8. DeletethedefaultSQLitedatabasethatKeystoneispackagedwith:

root@controller:~#rm-f/var/lib/keystone/keystone.db

9. Createanadministrativeaccountbydefiningthefollowingenvironmentvariables:

root@controller:~#exportOS_USERNAME=admin

root@controller:~#exportOS_PASSWORD=lxcpassword

root@controller:~#exportOS_PROJECT_NAME=admin

root@controller:~#exportOS_USER_DOMAIN_NAME=default

root@controller:~#exportOS_PROJECT_DOMAIN_NAME=default

root@controller:~#exportOS_AUTH_URL=http://controller:35357/v3

root@controller:~#exportOS_IDENTITY_API_VERSION=3

root@controller:~#

10. CreateaprojectinKeystonefortheservicestouseandlistit:

root@controller:~#openstackprojectcreate--domaindefault--

description"KVMProject"service

+-------------+-----------------------------------+

|Field|Value|

+-------------+-----------------------------------+

|description|KVMProject|

|domain_id|default|

|enabled|True|

|id|9a1a863fe41b42b2955b313f2cca0ef0|

|is_domain|False|

|name|service|

|parent_id|default|

+-------------+-----------------------------------+

root@controller:~#openstackprojectlist

+----------------------------------+---------+

|ID|Name|

+----------------------------------+---------+

|06f4e2d7e384474781803395b24b3af2|admin|

|9a1a863fe41b42b2955b313f2cca0ef0|service|

+----------------------------------+---------+

root@controller:~#

11. CreateanunprivilegedprojectandauserthatwillbeusedbyregularclientsinsteadoftheOpenStackservices:

root@controller:~#openstackprojectcreate--domaindefault--

description"KVMUserProject"kvm

+-------------+----------------------------------+

|Field|Value|

+-------------+----------------------------------+

|description|KVMUserProject|

|domain_id|default|

|enabled|True|

|id|eb9cdc2c2b4e4f098f2d104752970d52|

|is_domain|False|

|name|kvm|

|parent_id|default|

+-------------+----------------------------------+

root@controller:~#

root@controller:~#openstackusercreate--domaindefault--password-

promptkvm

UserPassword:

RepeatUserPassword:

+---------------------+----------------------------------+

|Field|Value|

+---------------------+----------------------------------+

|domain_id|default|

|enabled|True|

|id|1e83e0c8ca194f2e9d8161eb61d21030|

|name|kvm|

|password_expires_at|None|

+---------------------+----------------------------------+

root@controller:~#

12. CreateauserroleandassociateitwiththeKVMprojectanduser:

root@controller:~#openstackrolecreateuser

+-----------+----------------------------------+

|Field|Value|

+-----------+----------------------------------+

|domain_id|None|

|id|331c0b61e9784112874627264f03a058|

|name|user|

+-----------+----------------------------------+

root@controller:~#openstackroleadd--projectkvm--userkvmuser

root@controller:~#

13. ConfiguretheWebServiceGatewayInterface(WSGI)middlewarepipelineforKeystone:

root@controller:~#cat/etc/keystone/keystone-paste.ini

#KeystonePasteDeployconfigurationfile.

[filter:debug]

use=egg:oslo.middleware#debug

[filter:request_id]

use=egg:oslo.middleware#request_id

[filter:build_auth_context]

use=egg:keystone#build_auth_context

[filter:token_auth]

use=egg:keystone#token_auth

[filter:admin_token_auth]

use=egg:keystone#admin_token_auth

[filter:json_body]

use=egg:keystone#json_body

[filter:cors]

use=egg:oslo.middleware#cors

oslo_config_project=keystone

[filter:http_proxy_to_wsgi]

use=egg:oslo.middleware#http_proxy_to_wsgi

[filter:ec2_extension]

use=egg:keystone#ec2_extension

[filter:ec2_extension_v3]

use=egg:keystone#ec2_extension_v3

[filter:s3_extension]

use=egg:keystone#s3_extension

[filter:url_normalize]

use=egg:keystone#url_normalize

[filter:sizelimit]

use=egg:oslo.middleware#sizelimit

[filter:osprofiler]

use=egg:osprofiler#osprofiler

[app:public_service]

use=egg:keystone#public_service

[app:service_v3]

use=egg:keystone#service_v3

[app:admin_service]

use=egg:keystone#admin_service

[pipeline:public_api]

pipeline=corssizelimithttp_proxy_to_wsgiosprofilerurl_normalize

request_idbuild_auth_contexttoken_authjson_bodyec2_extension

public_service

[pipeline:admin_api]


request_idbuild_auth_contexttoken_authjson_bodyec2_extension

s3_extensionadmin_service

[pipeline:api_v3]


request_idbuild_auth_contexttoken_authjson_bodyec2_extension_v3

s3_extensionservice_v3

[app:public_version_service]

use=egg:keystone#public_version_service

[app:admin_version_service]

use=egg:keystone#admin_version_service

[pipeline:public_version_api]

pipeline=corssizelimitosprofilerurl_normalize

public_version_service

[pipeline:admin_version_api]

pipeline=corssizelimitosprofilerurl_normalizeadmin_version_service

[composite:main]

use=egg:Paste#urlmap

/v2.0=public_api

/v3=api_v3

/=public_version_api

[composite:admin]

use=egg:Paste#urlmap

/v2.0=admin_api

/v3=api_v3

/=admin_version_api

root@controller:~#

14. RequestatokenfortheadminandKVMusers:

root@controller:~#openstack--os-auth-urlhttp://controller:35357/v3--

os-project-domain-namedefault--os-user-domain-namedefault--os-

project-nameadmin--os-usernameadmintokenissue

+------------+----------------------------------+

|Field|Value|

+------------+----------------------------------+

|expires|2017-04-2618:29:03+00:00|

|id|gAAAAABZMIdwefsdfB8e4rFk5IALgM4U|

|project_id|123c1e6f33584dd1876c0a34249a6e11|

|user_id|cc14c5dbbd654c438e52d38efaf4f1a6|

+------------+----------------------------------+

root@controller:~#openstack--os-auth-urlhttp://controller:5000/v3--

os-project-domain-namedefault--os-user-domain-namedefault--os-

project-namekvm--os-usernamekvmtokenissue

+------------+----------------------------------+

|Field|Value|

+------------+----------------------------------+

|expires|2017-04-2618:29:52+00:00|

|id|gAAAAABZANkQmInUifl6Up_PzdH_9OHd|

|project_id|10a92eccbad9439d9e56c4edda6b211f|

|user_id|a186b226ed1e4717b25bb978f2bc9958|

+------------+----------------------------------+

root@controller:~#

15. Createthefilesthatwillcontaintheadminandusercredentials:

root@controller:~#catrc.admin

exportOS_PROJECT_DOMAIN_NAME=default

exportOS_USER_DOMAIN_NAME=default

exportOS_PROJECT_NAME=admin

exportOS_USERNAME=admin

exportOS_PASSWORD=lxcpassword

exportOS_AUTH_URL=http://controller:35357/v3

exportOS_IDENTITY_API_VERSION=3

exportOS_IMAGE_API_VERSION=2

root@controller:~#

root@controller:~#catrc.kvm

exportOS_PROJECT_DOMAIN_NAME=default

exportOS_USER_DOMAIN_NAME=default

exportOS_PROJECT_NAME=kvm

exportOS_USERNAME=kvm

exportOS_PASSWORD=lxcpassword

exportOS_AUTH_URL=http://controller:5000/v3

exportOS_IDENTITY_API_VERSION=3

exportOS_IMAGE_API_VERSION=2

root@controller:~#

16. Sourcetheadmincredentialsfile:

root@controller:~#.rc.admin

root@controller:~#

17. Requestanauthenticationtokenfortheadminuser:

root@controller:~#openstacktokenissue

+------------+----------------------------------+

|Field|Value|

+------------+----------------------------------+

|expires|2017-04-2618:30:41+00:00|

|id|gAAAAABZANlBdsu-DTmz6ME2Z8JFKjJM|



+------------+----------------------------------+

root@controller:~#

Howitworks...WestartbycreatingtheKeystonedatabaseinMariaDBwiththenecessaryuserpermissionsinstep1.Instep2,weinstalltheKeystonepackage.

Instep3,wecreatetheconfigurationfilefortheservice.Asyoucanseefromtheoutput,mostoftheoptionshavebeenomitted,anddefaultonesareassumed.

Instep4,werunascriptthatpopulatestheKeystonedatabasebycreatingthedatabaseschema.

Keystoneusestokenstoauthenticateandauthorizeusersandservices.Therearedifferenttokenformatsavailableforuse,suchasUUID,PKI,andFernettokens.Forthisdeployment,wearegoingtousetheFernettokens.TheFernettokensdonotneedtobepersistedinabackendstore.Instep5,weinitializetheFernetkeyrepository.

Formoreinformationontheavailableidentitytokens,refertohttp://docs.openstack.org/admin-guide/identity-tokens.html.

Instep6,webootstrapKeystoneandupdatetheApacheconfigurationinstep7andperformsomecleanupinstep8.

Instep9,weexportalistofenvironmentvariablescontainingtheKeystoneuser,password,andendpoint.

Instep10,wecreateourfirstprojectinKeystonethatwillbeusedbytherestoftheservices.Projectsrepresentaunitofownership,whereallresourcesareownedbyaproject.Insteps11and12,wecreateanunprivilegedprojectandassociateduser.

Instep13,weconfiguretheWSGImiddlewarepipelineforKeystone.

Instep14,werequestandobtaintokensfortheadminandKVMusers,andinstep15,wecreatetwoenvironmentvariablefilesthatwecansourcewhenweneedtoswitchbetweenusers.

http://docs.openstack.org/admin-guide/identity-tokens.html

needtoswitchbetweenusers.

Insteps16and17,wesourcetheadmincredentialsandprojectendpointandobtainanauthorizationtoken.

InstallingandconfiguringtheOpenStackGlanceimageserviceTheGlanceimageserviceprovidesanAPIthatwecanusetodiscover,register,andobtainimagesforvirtualmachines.WhenwelateruseNovacomputetobuildanewKVMinstance,theNovaservicewillsendarequesttoGlancetoobtaintherequestedimagetype.

Inthisrecipe,wearegoingtoinstallGlanceandregisteranewUbuntuimage.

GettingreadyForthisrecipe,wearegoingtoneedthefollowingthings:

AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe

Howtodoit...Toinstall,configure,andregisteranimagewithGlance,followthestepsoutlinedhere:

1. CreatetheGlancedatabaseanduser:


MariaDB[(none)]>CREATEDATABASEglance;


MariaDB[(none)]>GRANTALLPRIVILEGESONglance.*TO

'glance'@'localhost'IDENTIFIEDBY'lxcpassword';


MariaDB[(none)]>GRANTALLPRIVILEGESONglance.*TO'glance'@'%'

IDENTIFIEDBY'lxcpassword';



Bye

root@controller:~#

2. CreatetheGlanceuserandaddittotheadminrole:


promptglance

UserPassword:

RepeatUserPassword:

+---------------------+----------------------------------+

|Field|Value|

+---------------------+----------------------------------+

|domain_id|default|

|enabled|True|

|id|e566c6e2012148daa374cd68077b38df|

|name|glance|


+---------------------+----------------------------------+

root@controller:~#openstackroleadd--projectservice--userglance

admin

root@controller:~#

3. CreatetheGlanceservicedefinition:

root@controller:~#openstackservicecreate--nameglance--description

"OpenStackImage"image

+-------------+----------------------------------+

|Field|Value|

+-------------+----------------------------------+

|description|OpenStackImage|

|enabled|True|

|id|d4d42a586551461c8b445b927f2144e1|

|name|glance|

|type|image|

+-------------+----------------------------------+

root@controller:~#

4. CreatetheGlanceAPIendpointsinKeystone:

root@controller:~#openstackendpointcreate--regionRegionOneimage

publichttp://controller:9292

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|618af0c845194f508752f230364d6e0e|

|interface|public|

|region|RegionOne|

|region_id|RegionOne|

|service_id|d4d42a586551461c8b445b927f2144e1|

|service_name|glance|

|service_type|image|

|url|http://controller:9292|

+--------------+----------------------------------+


internalhttp://controller:9292

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|991a1b03f7194139b98bafe19acf3518|

|interface|internal|

|region|RegionOne|






+--------------+----------------------------------+


adminhttp://controller:9292

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|991a1b03f7194139b98bafe19acf3322|

|interface|admin|

|region|RegionOne|






+--------------+----------------------------------+

root@controller:~#

5. InstalltheGlanceservice:

root@controller:~#aptinstallglance

6. Configuretheservice:

root@controller:~#cat/etc/glance/glance-api.conf

[DEFAULT]

[cors]

[cors.subdomain]

[database]

connection=mysql+pymysql://glance:lxcpassword@controller/glance

[glance_store]

stores=file,http

default_store=file

filesystem_store_datadir=/var/lib/glance/images/

[image_format]

disk_formats=ami,ari,aki,vhd,vhdx,vmdk,raw,qcow2,vdi,iso,root-tar

[keystone_authtoken]

auth_uri=http://controller:5000

auth_url=http://controller:35357

memcached_servers=controller:11211

auth_type=password

project_domain_name=default

user_domain_name=default

project_name=service

username=glance

password=lxcpassword

[matchmaker_redis]

[oslo_concurrency]





[oslo_middleware]

[oslo_policy]

[paste_deploy]

flavor=keystone

[profiler]

[store_type_location_strategy]

[task]

[taskflow_executor]

root@controller:~#

root@controller:~#cat/etc/glance/glance-registry.conf

[DEFAULT]

[database]

connection=mysql+pymysql://glance:lxcpassword@controller/glance





auth_type=password




username=glance


[matchmaker_redis]





[oslo_policy]

[paste_deploy]

flavor=keystone

[profiler]

root@controller:~#

7. PopulatetheGlancedatabase:

root@controller:~#su-s/bin/sh-c"glance-managedb_sync"glance

...

root@controller:~#

8. StarttheGlanceservicedaemons:

root@controller:~#serviceglance-registryrestart

root@controller:~#serviceglance-apirestart

root@controller:~#

9. DownloadaQCOW2imagefortheUbuntudistribution:

root@controller:~#wgethttps://uec-

images.ubuntu.com/releases/16.04/release-20170330/ubuntu-16.04-server-

cloudimg-amd64-disk1.img

Savingto:‘ubuntu-16.04-server-cloudimg-amd64-disk1.img’

ubuntu-16.04-server-cloudimg-amd64-disk1.img100%

[===================================================>]309.75M31.1MB/s

in13s

2017-04-2617:40:21(24.5MB/s)-‘ubuntu-16.04-server-cloudimg-amd64-

disk1.img’saved[324796416/324796416]

root@controller:~#

10. AddtheimagetotheGlanceservice:

root@controller:~#openstackimagecreate"ubuntu_16.04"--fileubuntu-

16.04-server-cloudimg-amd64-disk1.img--disk-formatqcow2--container-

formatbare--public

+------------------+----------------------------------------------------

--+

|Field|Value

|

+------------------+----------------------------------------------------

--+

|checksum|87b0b7a4b03dd0bb2177d5cc02c80720

|

|container_format|bare

|

|created_at|2017-04-26T17:41:44Z

|

|disk_format|qcow2

|

|file|/v2/images/abce08d2-2f9f-4545-a414-

32019d41c0cd/file|

|id|abce08d2-2f9f-4545-a414-32019d41c0cd

|

|min_disk|0

|

|min_ram|0

|

|name|ubuntu_16.04

|

|owner|123c1e6f33584dd1876c0a34249a6e11

|

|protected|False

|

|schema|/v2/schemas/image

|

|size|324796416

|

|status|active

|

|tags|

|

|updated_at|2017-04-26T17:41:45Z

|

|virtual_size|None

|

|visibility|public

|

+------------------+----------------------------------------------------

--+

root@controller:~#

11. Listtheavailableimagesandtheirlocationonthefilesystem:

root@controller:~#openstackimagelist

+--------------------------------------+--------------+--------+

|ID|Name|Status|

+--------------------------------------+--------------+--------+

|abce08d2-2f9f-4545-a414-32019d41c0cd|ubuntu_16.04|active|

+--------------------------------------+--------------+--------+

root@controller:~#ls-lah/var/lib/glance/images/

drwxr-xr-x2glanceglance4.0KApr2617:51.

drwxr-xr-x4glanceglance4.0KApr2617:32..

-rw-r-----1glanceglance310MApr2617:41abce08d2-2f9f-4545-a414-

32019d41c0cd

root@controller:~#qemu-imginfo/var/lib/glance/images/abce08d2-2f9f-

4545-a414-32019d41c0cd

image:/var/lib/glance/images/abce08d2-2f9f-4545-a414-32019d41c0cd

fileformat:qcow2


disksize:310M

cluster_size:65536


compat:0.10

refcountbits:16

root@controller:~#

Howitworks...WestartbycreatingtheGlancedatabaseinMariaDBinstep1.

Insteps2and3,wecreatetheuser,role,andservicefortheGlanceproject.Instep4,wedefinetheGlanceAPIserviceendpointsinKeystone.TheNovaserviceandtheOpenStacktoolcanusetheseendpointstoqueryGlanceforavailableimages.

Instep5,weinstalltheGlancepackageandcreateaminimalconfigurationfileinstep6.

Wethencreatethedatabaseschemasinstep7,byexecutingtheglance-managePythonscriptandrestarttheGlanceserviceinstep8.

Instep9,wedownloadaQCOW2Ubuntuimageandaddittotheglanceregistryinstep10.

Finally,instep11,welistthenewlyaddedimageandexamineitonthehostfilesystem.

InstallingandconfiguringtheOpenStackNovacomputeserviceTheOpenStackComputeservice,codenamedNova,managesapoolofcomputeresourcesandthevirtualmachinesrunningonthem.Novaisasuiteofservicestocreateandmanagethelifecycleofvirtualmachines.WewilluseNovatocreate,examine,stop,delete,andmigrateKVMinstances.

FormoreinformationonthevariousNovaservices,referto:http://docs.openstack.org/developer/nova/.

Inthisrecipe,wearegoingtoinstallandconfigurethefollowingNovacomponents:

nova-api:ThisistheservicethatacceptsandrespondstouserrequeststhroughaRESTfulAPI.Wewilluseitwhencreating,running,stopping,andmigratingKVMinstances.nova-scheduler:Thisistheservicethatmakesdecisionsonwheretoprovisioninstance,basedonfilters,suchasavailablememory,disk,andCPUresources.nova-compute:ThisistheservicethatrunsonthecomputehostsandisresponsibleformanagingthelifecycleoftheKVMinstance,fromprovisioningtodeletion.nova-conductor:ThisistheservicethatsitsbetweentheNovadatabasewecreatedearlierandthenova-computeservice.nova-consoleauth:Thisistheservicethatauthorizestokensforusersthatwanttousevariousconsolestoconnecttothevirtualmachines.nova-novncproxy:ThisistheservicethatgrantsaccesstoinstancesrunningVNC.

http://docs.openstack.org/developer/nova/


AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipeTheGlanceservicewedeployedintheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe

Howtodoit...ToinstallandconfiguretheNovaservicesoutlinedearlier,performthefollowingsteps:

1. CreatetheNovadatabaseanduserinMariaDB:


MariaDB[(none)]>CREATEDATABASEnova_api;


MariaDB[(none)]>CREATEDATABASEnova;


MariaDB[(none)]>GRANTALLPRIVILEGESONnova_api.*TO

'nova'@'localhost'IDENTIFIEDBY'lxcpassword';


MariaDB[(none)]>GRANTALLPRIVILEGESONnova_api.*TO'nova'@'%'



MariaDB[(none)]>GRANTALLPRIVILEGESONnova.*TO'nova'@'localhost'



MariaDB[(none)]>GRANTALLPRIVILEGESONnova.*TO'nova'@'%'




Bye

root@controller:~#

2. CreatetheNovauserandaddittotheadminroleintheIdentityservice:


promptnova

UserPassword:

RepeatUserPassword:

+---------------------+----------------------------------+

|Field|Value|

+---------------------+----------------------------------+

|domain_id|default|

|enabled|True|

|id|038aa8840aca449dbd3e653c5d2c5a08|

|name|nova|


+---------------------+----------------------------------+

root@controller:~#openstackroleadd--projectservice--usernova

admin

root@controller:~#

3. CreatetheNovaserviceandendpoints:

root@controller:~#openstackservicecreate--namenova--description

"OpenStackCompute"compute

+-------------+----------------------------------+

|Field|Value|

+-------------+----------------------------------+

|description|OpenStackCompute|

|enabled|True|

|id|04132edd7f654f56ba0cc23ac182c9aa|

|name|nova|

|type|compute|

+-------------+----------------------------------+

root@controller:~#openstackendpointcreate--regionRegionOnecompute

publichttp://controller:8774/v2.1/%(tenant_id)s

+--------------+-------------------------------------------+

|Field|Value|

+--------------+-------------------------------------------+

|enabled|True|

|id|5fc54236c324412db135dff88807e820|

|interface|public|

|region|RegionOne|


|service_id|04132edd7f654f56ba0cc23ac182c9aa|

|service_name|nova|

|service_type|compute|

|url|http://controller:8774/v2.1/%(tenant_id)s|

+--------------+-------------------------------------------+


internalhttp://controller:8774/v2.1/%(tenant_id)s

+--------------+-------------------------------------------+

|Field|Value|

+--------------+-------------------------------------------+

|enabled|True|

|id|a0f623ed345e4bdb8fced929b7fe6b3f|


|region|RegionOne|



|service_name|nova|



+--------------+-------------------------------------------+


adminhttp://controller:8774/v2.1/%(tenant_id)s

+--------------+-------------------------------------------+

|Field|Value|

+--------------+-------------------------------------------+

|enabled|True|

|id|3964db0d281545acbaa6c18abc44a216|

|interface|admin|

|region|RegionOne|



|service_name|nova|



+--------------+-------------------------------------------+

root@controller:~#

4. InstalltheNovapackagesthatwillprovidetheAPI,theconductor,the

console,andtheschedulerservices:

root@controller:~#aptinstallnova-apinova-conductornova-consoleauth

nova-novncproxynova-scheduler

5. CreatetheNovaconfigurationfile:

root@controller:~#cat/etc/nova/nova.conf

[DEFAULT]

dhcpbridge_flagfile=/etc/nova/nova.conf

dhcpbridge=/usr/bin/nova-dhcpbridge

log-dir=/var/log/nova

state_path=/var/lib/nova

force_dhcp_release=True

verbose=True

ec2_private_dns_show_ip=True

enabled_apis=osapi_compute,metadata

transport_url=rabbit://openstack:lxcpassword@controller

auth_strategy=keystone

my_ip=10.208.132.45

use_neutron=True

firewall_driver=nova.virt.firewall.NoopFirewallDriver

[database]

connection=mysql+pymysql://nova:lxcpassword@controller/nova

[api_database]

connection=mysql+pymysql://nova:lxcpassword@controller/nova_api

[oslo_concurrency]

lock_path=/var/lib/nova/tmp

[libvirt]

use_virtio_for_bridges=True

[wsgi]

api_paste_config=/etc/nova/api-paste.ini





auth_type=password




username=nova


[vnc]

vncserver_listen=$my_ip

vncserver_proxyclient_address=$my_ip

[glance]

api_servers=http://controller:9292

root@controller:~#

6. Createthedatabasetables:

root@controller:~#su-s/bin/sh-c"nova-manageapi_dbsync"nova

...

root@controller:~#su-s/bin/sh-c"nova-managedbsync"nova

...

root@controller:~#

7. StarttheNovaservices:

root@controller:~#servicenova-apirestart

root@controller:~#servicenova-consoleauthrestart

root@controller:~#servicenova-schedulerrestart

root@controller:~#servicenova-conductorrestart

root@controller:~#servicenova-novncproxyrestart

root@controller:~#

8. Installthenova-computeservice,whichwillprovisionKVMinstances:

root@controller:~#aptinstallnova-compute

9. UpdatetheNovaconfigurationfile,asfollows:


[DEFAULT]






verbose=True





my_ip=10.208.132.45

use_neutron=True


compute_driver=libvirt.LibvirtDriver

[database]


[api_database]


[oslo_concurrency]


[libvirt]


[wsgi]






auth_type=password




username=nova


[vnc]

enabled=True



novncproxy_base_url=http://controller:6080/vnc_auto.html

[glance]


root@controller:~#

10. Specifythevirtualizationdrivertobeused:

root@controller:~#cat/etc/nova/nova-compute.conf

[DEFAULT]


[libvirt]

virt_type=kvm

root@controller:~#

11. Restartthenova-computeserviceandlisttheavailableservices:

root@controller:~#servicenova-computerestart

root@controller:~#openstackcomputeservicelist

+----+------------------+------------+----------+---------+-------+-----

-----------------+

|ID|Binary|Host|Zone|Status|State|

UpdatedAt|

+----+------------------+------------+----------+---------+-------+-----

-----------------+

|8|nova-consoleauth|controller|internal|enabled|up|

2017-04-26T17:58|

|9|nova-scheduler|controller|internal|enabled|up|

2017-04-26T17:58|

|10|nova-conductor|controller|internal|enabled|up|

2017-04-26T17:58|

|15|nova-compute|controller|nova|enabled|up|None

|

+----+------------------+------------+----------+---------+-------+-----

-----------------+

root@controller:~#pgrep-lfnova|uniq-f1

14110nova-consoleaut

14176nova-conductor

14239nova-novncproxy

20877nova-api

20994nova-scheduler

21065nova-compute

root@controller:~#

Howitworks...Insteps1and2,wecreatetheNovadatabaseanduserinMariaDB.Instep3,wecreatetheserviceandendpointsthatwecanusetosendAPIcallsto.

Insteps4and5,weinstallthepackagesfortheNovaservicesandproceedtocreateasimpleconfigurationfile.

Wecreatethedatabasetableschemasinstep6andstarttheNovaservicesinstep7.

Forthisexampledeployment,weareusingasinglenodetorunalloftheOpenStackservicesweareinterestedin.However,youcanuseasecondnodejustforthenova-computeservicethatwillprovisiontheKVMvirtualmachines.Weinstallthenova-computeserviceinstep8,updatetheconfigurationfile,andexaminethenova-computeserviceexternalconfigurationinsteps9and10.

WefinishtherecipebymakingsurethatallNovaserviceshavebeenconfiguredandrunninginstep11.

InstallingandconfiguringtheOpenStackNeutronnetworkingserviceTheOpenStackNeutronprojectprovidesnetworkingasaservicetomanagethenetworkingbetweenvirtualinstances.Itisresponsibleforsettingupvirtualinterfaces,configuringasoftwarebridge,creatingroutes,andmanagingIPaddressing.

FormoreinformationonthevariousNeutronservices,refertohttps://docs.openstack.org/security-guide/networking/architecture.html.

Inthisrecipe,wearegoingtoinstallandconfigurethefollowingNeutroncomponents:

neutron-server:ThisistheservicethatprovidesAPItodynamicallyrequestandconfigurevirtualnetworksneutron-plugin-ml2:Thisistheframeworkthatenablestheuseofvariousnetworktechnologies,suchastheLinuxBridge,OpenvSwitch,GRE,andVXLAN,thatwesawinearlierchaptersneutron-linuxbridge-agent:ThisistheservicethatprovidestheLinuxbridgepluginagentneutron-l3-agent:ThisisthedaemonthatperformsforwardingandNATfunctionalitybetweensoftware-definednetworks,bycreatingvirtualroutersneutron-dhcp-agent:ThisistheservicethatcontrolstheDHCPdaemon,whichassignsIPaddressestotheinstancesrunningonthecomputenodesneutron-metadata-agent:ThisistheservicethatpassesinstancemetadatatoNeutron

Inearlierrecipes,weconfiguredandusedtheLinuxbridgeandOpenvSwitchmanuallyandlaterdelegatedthemanagementofthenetworkingtolibvirt.OpenStackNeutronintegrateswithlibvirtandautomatesthisprocessevenfurtherbyexposingAPIcallsthatotherserviceslikeNovacanutilize.

https://docs.openstack.org/security-guide/networking/architecture.html

furtherbyexposingAPIcallsthatotherserviceslikeNovacanutilize.


AnUbuntuserverwithgreatvirtualizationcapabilitiesAccesstotheinternetforpackageinstallationAdatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipeTheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipeTheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe

Howtodoit...Toinstall,configure,andcreateanetworkmanagedbyNeutron,executethefollowingsteps:

1. CreatetheNeutrondatabase:


MariaDB[(none)]>CREATEDATABASEneutron;


MariaDB[(none)]>GRANTALLPRIVILEGESONneutron.*TO

'neutron'@'localhost'IDENTIFIEDBY'lxcpassword';


MariaDB[(none)]>GRANTALLPRIVILEGESONneutron.*TO'neutron'@'%'




Bye

root@controller:~#

2. CreatetheNeutronuserandaddittotheadminroleinKeystone:


promptneutron

UserPassword:

RepeatUserPassword:

+---------------------+----------------------------------+

|Field|Value|

+---------------------+----------------------------------+

|domain_id|default|

|enabled|True|

|id|02934ad74c94461482b95fff32d36894|

|name|neutron|


+---------------------+----------------------------------+

root@controller:~#openstackroleadd--projectservice--userneutron

admin

root@controller:~#

3. CreatetheNeutronserviceandendpoints:

root@controller:~#openstackservicecreate--nameneutron--description

"OpenStackNetworking"network

+-------------+----------------------------------+

|Field|Value|

+-------------+----------------------------------+

|description|OpenStackNetworking|

|enabled|True|

|id|24b32d32d4b54e3ab2d785a1817b8e7e|

|name|neutron|

|type|network|

+-------------+----------------------------------+

root@controller:~#openstackendpointcreate--regionRegionOnenetwork

publichttp://controller:9696

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|544821d511e04847869fc601f2ebf0f7|

|interface|public|

|region|RegionOne|


|service_id|24b32d32d4b54e3ab2d785a1817b8e7e|

|service_name|neutron|

|service_type|network|


+--------------+----------------------------------+


internalhttp://controller:9696

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|05e276ec603f424f85be8705ce7fe86a|


|region|RegionOne|






+--------------+----------------------------------+


adminhttp://controller:9696

+--------------+----------------------------------+

|Field|Value|

+--------------+----------------------------------+

|enabled|True|

|id|836b4309186146fb9143544490cd0bc1|

|interface|admin|

|region|RegionOne|






+--------------+----------------------------------+

root@controller:~#

4. InstalltheNeutronpackages:

root@controller:~#aptinstallneutron-serverneutron-plugin-ml2

neutron-linuxbridge-agentneutron-l3-agentneutron-dhcp-agentneutron-

metadata-agent

...

root@controller:~#

5. CreatetheNeutronconfigurationfile:

root@controller:~#cat/etc/neutron/neutron.conf

[DEFAULT]

core_plugin=ml2

service_plugins=router

allow_overlapping_ips=True



notify_nova_on_port_status_changes=True

notify_nova_on_port_data_changes=True

[agent]

root_helper=sudo/usr/bin/neutron-rootwrap/etc/neutron/rootwrap.conf

[cors]

[cors.subdomain]

[database]

connection=mysql+pymysql://neutron:lxcpassword@controller/neutron





auth_type=password




username=neutron


[matchmaker_redis]

[nova]


auth_type=password



region_name=RegionOne


username=nova


[oslo_concurrency]





[oslo_policy]

[qos]

[quotas]

[ssl]

root@controller:~#

6. DefinethenetworktypeandextensionsthatwearegoingtousewithNeutron:

root@controller:~#cat/etc/neutron/plugins/ml2/ml2_conf.ini

[DEFAULT]

[ml2]

type_drivers=flat,vlan,vxlan

tenant_network_types=vxlan

mechanism_drivers=linuxbridge,l2population

extension_drivers=port_security

[ml2_type_flat]

flat_networks=provider

[ml2_type_geneve]

[ml2_type_gre]

[ml2_type_vlan]

[ml2_type_vxlan]

vni_ranges=1:1000

[securitygroup]

enable_ipset=True

root@controller:~#

7. DefinetheinterfacethatwillbeaddedtothesoftwarebridgeandtheIPthebridgewillbeboundto,replacingtheIPaddressandinterfacename(eth1inthisexample)asneeded:

root@controller:~#cat/etc/neutron/plugins/ml2/linuxbridge_agent.ini

[DEFAULT]

[agent]

[linux_bridge]

physical_interface_mappings=provider:eth1

[securitygroup]

enable_security_group=True

firewall_driver=

neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

[vxlan]

enable_vxlan=True

local_ip=10.208.132.45

l2_population=True

root@controller:~#

8. ConfiguretheLayer3agentasfollows:

root@controller:~#cat/etc/neutron/l3_agent.ini

[DEFAULT]

interface_driver=neutron.agent.linux.interface.BridgeInterfaceDriver

[AGENT]

root@controller:~#

9. ConfiguretheDHCPagent:

root@controller:~#cat/etc/neutron/dhcp_agent.ini

[DEFAULT]

interface_driver=neutron.agent.linux.interface.BridgeInterfaceDriver

dhcp_driver=neutron.agent.linux.dhcp.Dnsmasq

enable_isolated_metadata=True

[AGENT]

root@controller:~#

10. Createaconfigurationforthemetadataagent:

root@controller:~#cat/etc/neutron/metadata_agent.ini

[DEFAULT]

nova_metadata_ip=controller

metadata_proxy_shared_secret=lxcpassword

[AGENT]

[cache]

root@controller:~#

11. UpdatetheconfigurationfilefortheNovaservicestoincludeNeutron.Acompletelyminimalworkingexamplefollowstolookasthefollowing:


[DEFAULT]






verbose=True





my_ip=10.208.132.45

use_neutron=True



scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,

RamFilter,ComputeFilter,ComputeCapabilitiesFilter,

ImagePropertiesFilter,ServerGroupAntiAffinityFilter,

ServerGroupAffinityFilter

[database]


[api_database]


[oslo_concurrency]


[libvirt]


[wsgi]






auth_type=password




username=nova


[vnc]

enabled=True



novncproxy_base_url=http://controller:6080/vnc_auto.html

[glance]


[libvirt]

virt_type=kvm

[neutron]

url=http://controller:9696


auth_type=password



region_name=RegionOne


username=neutron


service_metadata_proxy=True

metadata_proxy_shared_secret=lxcpassword

root@controller:~#

12. PopulatetheNeutrondatabases:

root@controller:~#su-s/bin/sh-c"neutron-db-manage--config-file

/etc/neutron/neutron.conf--config-file

/etc/neutron/plugins/ml2/ml2_conf.iniupgradehead"neutron

INFO[alembic.runtime.migration]ContextimplMySQLImpl.

INFO[alembic.runtime.migration]Willassumenon-transactionalDDL.

Runningupgradeforneutron...

INFO[alembic.runtime.migration]ContextimplMySQLImpl.

INFO[alembic.runtime.migration]Willassumenon-transactionalDDL.

INFO[alembic.runtime.migration]Runningupgrade->kilo,kilo_initial

...

root@controller:~#

13. RestartallNeutronservicesandNova:

root@controller:~#serviceneutron-serverrestart

root@controller:~#serviceneutron-linuxbridge-agentrestart

root@controller:~#serviceneutron-dhcp-agentrestart

root@controller:~#serviceneutron-metadata-agentrestart

root@controller:~#serviceneutron-l3-agentrestart

root@controller:~#servicenova-apirestart

root@controller:~#servicenova-computerestart

root@controller:~#

14. VerifythattheNeutronserviceshavebeenregistered:

root@controller:~#openstacknetworkagentlist

+--------------------------------------+--------------------+------+----

---------------+-------+-------+---------------------------+

|ID|AgentType|Host|AvailabilityZone|Alive|State|Binary|

+--------------------------------------+--------------------+------+----

---------------+-------+-------+---------------------------+

|9242d71d-de25-4b3e-8aa8-62691ef72001|Linuxbridgeagent|kvm2|

None|True|UP|neutron-linuxbridge-agent|

|92b601de-06df-4b10-88c7-8f27bc48f6ab|L3agent|kvm2|nova|True|

UP|neutron-l3-agent|

|d249f986-9b26-4c5d-8ea5-311daf3b395d|DHCPagent|kvm2|nova|True

|UP|neutron-dhcp-agent|

|f3cac79b-a7c3-4672-b846-9268f2d58706|Metadataagent|kvm2|None|

True|UP|neutron-metadata-agent|

+--------------------------------------+--------------------+------+----

---------------+-------+-------+---------------------------+

root@controller:~#

15. Createanewnetwork:

root@controller:~#openstacknetworkcreatenat

+---------------------------+--------------------------------------+

|Field|Value|

+---------------------------+--------------------------------------+

|admin_state_up|UP|

|availability_zone_hints||

|availability_zones||

|created_at|2017-04-26T18:17:24Z|

|description||

|headers||

|id|b7ccb514-21fc-4ced-b74f-026e7e358bba|

|ipv4_address_scope|None|

|ipv6_address_scope|None|

|mtu|1450|

|name|nat|

|port_security_enabled|True|



|provider:network_type|vxlan|

|provider:physical_network|None|

|provider:segmentation_id|37|

|revision_number|3|

|router:external|Internal|

|shared|False|

|status|ACTIVE|

|subnets||

|tags|[]|

|updated_at|2017-04-26T18:17:24Z|

+---------------------------+--------------------------------------+

root@controller:~#

16. DefinetheDNSserver,thedefaultgateway,andthesubnetrangethatwillbeassignedtotheguests:

root@controller:~#openstacksubnetcreate--networknat--dns-

nameserver8.8.8.8--gateway192.168.0.1--subnet-range192.168.0.0/24

nat

+-------------------+--------------------------------------+

|Field|Value|

+-------------------+--------------------------------------+

|allocation_pools|192.168.0.2-192.168.0.254|

|cidr|192.168.0.0/24|

|created_at|2017-04-26T18:17:41Z|

|description||

|dns_nameservers|8.8.8.8|

|enable_dhcp|True|

|gateway_ip|192.168.0.1|

|headers||

|host_routes||

|id|296250a7-f241-4f84-adbb-64a45c943094|

|ip_version|4|

|ipv6_address_mode|None|

|ipv6_ra_mode|None|

|name|nat|

|network_id|b7ccb514-21fc-4ced-b74f-026e7e358bba|



|revision_number|2|

|service_types|[]|

|subnetpool_id|None|

|updated_at|2017-04-26T18:17:41Z|

+-------------------+--------------------------------------+

root@controller:~#

17. UpdatethesubnetinformationinNeutron:

root@controller:~#neutronnet-updatenat--router:external

Updatednetwork:nat

root@controller:~#

18. Createanewsoftwarerouter:

root@controller:~#openstackroutercreaterouter

+-------------------------+--------------------------------------+

|Field|Value|

+-------------------------+--------------------------------------+

|admin_state_up|UP|

|availability_zone_hints||

|availability_zones||

|created_at|2017-04-26T18:18:05Z|

|description||

|external_gateway_info|null|

|flavor_id|None|

|headers||

|id|f9cd8c96-a53c-4585-ad21-0e409f3b4d70|

|name|router|



|revision_number|3|

|routes||

|status|ACTIVE|

|updated_at|2017-04-26T18:18:05Z|

+-------------------------+--------------------------------------+

root@controller:~#

19. Astheadminuser,addthesubnetwecreatedearliertotherouterasaninterface:

root@controller:~#.rc.admin

root@controller:~#neutronrouter-interface-addrouternat

Addedinterface2e1e2fd3-1819-489b-a21f-7005862f9de7torouterrouter.

root@controller:~#

20. ListthenetworknamespacesthatNeutroncreated:

root@controller:~#ipnetns

qrouter-f9cd8c96-a53c-4585-ad21-0e409f3b4d70

qdhcp-b7ccb514-21fc-4ced-b74f-026e7e358bba

root@controller:~#

21. Listtheportsonthesoftwarerouter:

root@controller:~#neutronrouter-port-listrouter

+--------------------------------------+------+-------------------+-----

------------------------------------------------------------------------

-------+

|id|name|mac_address|fixed_ips|

+--------------------------------------+------+-------------------+-----

------------------------------------------------------------------------

-------+

|2e1e2fd3-1819-489b-a21f-7005862f9de7||fa:16:3e:0e:db:14|

{"subnet_id":"296250a7-f241-4f84-adbb-64a45c943094","ip_address":

"192.168.0.1"}|

+--------------------------------------+------+-------------------+-----

------------------------------------------------------------------------

-------+

root@controller:~#

22. ListtheNeutronnetworksandensurethattheonewecreatedearlierispresent:

root@controller:~#openstacknetworklist

+--------------------------------------+------+-------------------------

-------------+

|ID|Name|Subnets

|

+--------------------------------------+------+-------------------------

-------------+

|b7ccb514-21fc-4ced-b74f-026e7e358bba|nat|296250a7-f241-4f84-adbb-

64a45c943094|

+--------------------------------------+------+-------------------------

-------------+

root@controller:~#

Howitworks...WestartthisrecipebycreatinganewdatabaseforNeutroninstep1.WethenproceedtocreatetheuserfortheNeutronserviceandaddittotheadminrolefortheservice.Insteps2and3,wedefinetheserviceendpointsthatwillbeexposedforNovatouse.Insteps4and5,weinstalltheNeutronpackagesandcreateabasicconfigurationfile.Instep6,weselecttheVXLANtypeofnetworkingforthisexampledeployment.Insteps7,8,9,and10,weconfigurethebridgeagent,thelayer3agent,theDHCPagent,andthemetadataagent.

Instep11,weupdatetheNovaconfigurationfiletocontainasectionabouttheNeutronservice.Instep12,wecreatethedatabaseschemaandrestartallNeutronservicesinstep13,includingnova-apiandnova-compute.

Instep14,weverifythattheNeutronserviceshavebeenregisteredandproceedtocreateanewnetworkinstep15.

Instep18,wedefineanewsoftwarerouter.Weaddthesubnetwecreatedearliertoitinstep19,thenverifythenewrouteconfigurationinstep21.

Thelaststep22ensuresthatthenetworkwedefinedearlierisactive.

BuildingandinspectingKVMinstanceswithOpenStackInthisrecipe,wearegoingtobuildourfirstKVMinstanceusingtheOpenStackinfrastructureweputinplaceinthepreviousrecipes.BuildinganewKVMinstanceconsistsofthefollowingsteps:

1. WesendanAPIcalltothenova-apiservice.2. Thenova-apiservicerequestsatargetcomputehostfromthenova-scheduler

service.3. nova-schedulerpicksanavailablecomputehost,basedontheconfigured

filters,suchasavailablememory,disk,andCPUutilization.4. Oncethenova-schedulerselectsanappropriatehost,thenova-computeservice

ontheselectedhost,requeststheimagefromtheGlancerepository,ifnotalreadycachedlocally.Oncetheimageisonthenewserver,nova-computebuildsthenewKVMinstance.

GettingreadyForthisrecipe,wearegoingtoneedthefollowingthings:

Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.

Howtodoit...TobuildanewKVMinstanceusingtheOpenStackcommand-lineinterface(CLI),performthefollowingsteps:

1. EnsurethatwehaveanavailableGlanceimagetouse:

root@controller:~#openstackimagelist

+--------------------------------------+--------------+--------+

|ID|Name|Status|

+--------------------------------------+--------------+--------+

|abce08d2-2f9f-4545-a414-32019d41c0cd|ubuntu_16.04|active|

+--------------------------------------+--------------+--------+

root@controller:~#

2. Createanewinstanceflavortype:

root@controller:~#openstackflavorcreate--id0--vcpus1--ram1024-

-disk5000kvm.medium

+----------------------------+------------+

|Field|Value|

+----------------------------+------------+

|OS-FLV-DISABLED:disabled|False|

|OS-FLV-EXT-DATA:ephemeral|0|

|disk|5000|

|id|0|

|name|kvm.medium|

|os-flavor-access:is_public|True|

|properties||

|ram|1024|

|rxtx_factor|1.0|

|swap||

|vcpus|1|

+----------------------------+------------+

root@controller:~#

root@controller:~#openstackflavorlist

+----+------------+------+------+-----------+-------+-----------+

|ID|Name|RAM|Disk|Ephemeral|VCPUs|IsPublic|

+----+------------+------+------+-----------+-------+-----------+

|0|kvm.medium|1024|5000|0|1|True|

+----+------------+------+------+-----------+-------+-----------+

root@controller:~#

3. CreateanewSSHkey-pair:

root@controller:~#openstackkeypaircreate--public-key

~/.ssh/kvm_rsa.pubkvmkey

+-------------+-------------------------------------------------+

|Field|Value|

+-------------+-------------------------------------------------+

|fingerprint|e9:7e:e6:05:8b:a4:31:c3:5e:41:65:0e:29:23:eb:2a|

|name|kvmkey|


+-------------+-------------------------------------------------+

root@controller:~#openstackkeypairlist

+--------+-------------------------------------------------+

|Name|Fingerprint|

+--------+-------------------------------------------------+

|kvmkey|e9:7e:e6:05:8b:a4:31:c3:5e:41:65:0e:29:23:eb:2a|

+--------+-------------------------------------------------+

root@controller:~#

4. DefinethesecuritygrouprulesthatallowSSHandICMPaccess:

root@controller:~#openstacksecuritygrouprulecreate--protoicmp

default

+-------------------+--------------------------------------+

|Field|Value|

+-------------------+--------------------------------------+

|created_at|2017-04-26T18:17:13Z|

|description||

|direction|ingress|

|ethertype|IPv4|

|headers||

|id|ca28501a-1b3b-448f-8c1b-0fa6f9fa9263|

|port_range_max|None|

|port_range_min|None|



|protocol|icmp|

|remote_group_id|None|

|remote_ip_prefix|0.0.0.0/0|

|revision_number|1|

|security_group_id|050b8174-d961-4706-ab63-1cdd2a25fbdd|

|updated_at|2017-04-26T18:17:13Z|

+-------------------+--------------------------------------+

root@controller:~#

root@controller:~#openstacksecuritygrouprulecreate--prototcp--

dst-port22default

+-------------------+--------------------------------------+

|Field|Value|

+-------------------+--------------------------------------+

|created_at|2017-04-26T18:17:18Z|

|description||

|direction|ingress|

|ethertype|IPv4|

|headers||

|id|334130c3-42b2-4f1b-aba6-c46e91ad203e|

|port_range_max|22|

|port_range_min|22|



|protocol|tcp|

|remote_group_id|None|

|remote_ip_prefix|0.0.0.0/0|

|revision_number|1|

|security_group_id|050b8174-d961-4706-ab63-1cdd2a25fbdd|

|updated_at|2017-04-26T18:17:18Z|

+-------------------+--------------------------------------+

root@controller:~#

5. Listtheavailablenetworks,wedefinedearlier:

root@controller:~#openstacknetworklist

+--------------------------------------+------+-------------------------

-------------+

|ID|Name|Subnets

|

+--------------------------------------+------+-------------------------

-------------+

|b7ccb514-21fc-4ced-b74f-026e7e358bba|nat|296250a7-f241-4f84-adbb-

64a45c943094|

+--------------------------------------+------+-------------------------

-------------+

root@controller:~#

6. BuildanewKVMinstanceandlistitsstatus:

root@controller:~#openstackservercreate--flavorkvm.medium--image

ubuntu_16.04--nicnet-id=b7ccb514-21fc-4ced-b74f-026e7e358bba--

security-groupdefault--key-namekvmkeyubuntu_instance

+--------------------------------------+--------------------------------

-----------+

|Field|Value

|

+--------------------------------------+--------------------------------

-----------+

|OS-DCF:diskConfig|MANUAL

|

|OS-EXT-AZ:availability_zone|

|

|OS-EXT-SRV-ATTR:host|None

|

|OS-EXT-SRV-ATTR:hypervisor_hostname|None

|

|OS-EXT-SRV-ATTR:instance_name|

|

|OS-EXT-STS:power_state|NOSTATE

|

|OS-EXT-STS:task_state|scheduling

|

|OS-EXT-STS:vm_state|building

|

|OS-SRV-USG:launched_at|None

|

|OS-SRV-USG:terminated_at|None

|

|accessIPv4|

|

|accessIPv6|

|

|addresses|

|

|adminPass|Z23yEuDLBjLe

|

|config_drive|

|

|created|2017-04-26T19:11:23Z

|

|flavor|kvm.medium(0)

|

|hostId|

|

|id|0f4745b1-9d4b-4e8a-82f7-

9eaa1f9bb08f|

|image|ubuntu_16.04(abce08d2-a414-

32019d41c0cd)|

|key_name|kvmkey

|

|name|ubuntu_instance

|

|os-extended-volumes:volumes_attached|[]

|

|progress|0

|

|project_id|

123c1e6f33584dd1876c0a34249a6e11|

|properties|

|

|security_groups|[{u'name':u'default'}]

|

|status|BUILD

|

|updated|2017-04-26T19:11:23Z

|

|user_id|

cc14c5dbbd654c438e52d38efaf4f1a6|

+--------------------------------------+--------------------------------

-----------+

root@controller:~#openstackserverlist

+---------------------+-----------------+---------+------------------+--

------------+

|ID|Name|Status|Networks|

ImageName|

+---------------------+-----------------+---------+------------------+--

------------+

|0f4745b1-...-9bb08f|ubuntu_instance|BUILD|nat=192.168.0.11|

ubuntu_16.04|

+---------------------+-----------------+---------+------------------+--

------------+

root@controller:~#

7. Ensurethatthecontainerwasstartedsuccessfully:

root@controller:~#pgrep-lfaqemu

23388/usr/bin/qemu-system-x86_64-nameinstance-00000005-S-machine

pc-i440fx-xenial,accel=kvm,usb=off-cpuHaswell-

noTSX,+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme

-m1024-realtimemlock=off-smp1,sockets=1,cores=1,threads=1-uuid

0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f-smbios

type=1,manufacturer=OpenStackFoundation,product=OpenStack

Nova,version=14.0.4,serial=6d6366d9-4569-6233-dad6-

4927587cc79f,uuid=0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f,family=Virtual

Machine-no-user-config-nodefaults-chardev

socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-

00000005/monitor.sock,server,nowait-mon

chardev=charmonitor,id=monitor,mode=control-rtcbase=utc,driftfix=slew

-globalkvm-pit.lost_tick_policy=discard-no-hpet-no-shutdown-boot

strict=on-devicepiix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive

file=/var/lib/nova/instances/0f4745b1-9d4b-4e8a-82f7-

9eaa1f9bb08f/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none

-devicevirtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-

disk0,id=virtio-disk0,bootindex=1-netdev

tap,fd=26,id=hostnet0,vhost=on,vhostfd=28-devicevirtio-net-

pci,netdev=hostnet0,id=net0,mac=fa:16:3e:3c:c0:0f,bus=pci.0,addr=0x3-

chardevfile,id=charserial0,path=/var/lib/nova/instances/0f4745b1-9d4b-

4e8a-82f7-9eaa1f9bb08f/console.log-deviceisa-

serial,chardev=charserial0,id=serial0-chardevpty,id=charserial1-

deviceisa-serial,chardev=charserial1,id=serial1-deviceusb-

tablet,id=input0-vnc0.0.0.0:0-ken-us-devicecirrus-

vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-balloon-

pci,id=balloon0,bus=pci.0,addr=0x5-msgtimestamp=on


+---------------------------+-----------------+--------+----------------

--+--------------+

|ID|Name|Status|Networks

|ImageName|

+---------------------------+-----------------+--------+----------------

--+--------------+

|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|

nat=192.168.0.11|ubuntu_16.04|

+---------------------------+-----------------+--------+----------------

--+--------------+

root@controller:~#

8. InspecttheKVMinstance:

root@controller:~#openstackservershowubuntu_instance

+--------------------------------------+--------------------------------

-----------+

|Field|Value

|

+--------------------------------------+--------------------------------

-----------+

|OS-DCF:diskConfig|MANUAL

|

|OS-EXT-AZ:availability_zone|nova

|

|OS-EXT-SRV-ATTR:host|controller

|

|OS-EXT-SRV-ATTR:hypervisor_hostname|controller

|

|OS-EXT-SRV-ATTR:instance_name|instance-00000001

|

|OS-EXT-STS:power_state|Running

|

|OS-EXT-STS:task_state|None

|

|OS-EXT-STS:vm_state|active

|

|OS-SRV-USG:launched_at|2017-04-26T19:11:37.000000

|

|OS-SRV-USG:terminated_at|None

|

|accessIPv4|

|

|accessIPv6|

|

|addresses|nat=192.168.0.11

|

|config_drive|

|

|created|2017-04-26T19:11:23Z

|

|flavor|kvm.medium(0)

|

|hostId|

c8d0t2jgdlkasdjg0iu4kjdg3o43045t|

|id|0f4745b1-9d4b-4e8a-82f7-

9eaa1f9bb08f|

|image|ubuntu_16.04(abce08d2-a414-

32019d41c0cd)|

|key_name|kvmkey

|

|name|ubuntu_instance

|

|os-extended-volumes:volumes_attached|[]

|

|progress|0

|

|project_id|

123c1e6f33584dd1876c0a34249a6e11|

|properties|

|

|security_groups|[{u'name':u'default'}]

|

|status|ACTIVE

|

|updated|2017-04-26T19:11:23Z

|

|user_id|

cc14c5dbbd654c438e52d38efaf4f1a6|

+--------------------------------------+--------------------------------

-----------+

root@controller:~#

Howitworks...WestarttherecipebyensuringthatwehaveanavailableGlanceimagetochoosefrom.WelistallavailableimagesinGlanceinstep1.Instep2,wecreateanewinstanceflavor;wespecifytheallocatedCPU,memory,anddiskresourcesforthenewinstancetype.Instep3,althoughitisnotmandatory,wecreateanewSSHkeypairthatcanlaterbeusedtoSSHtothenewinstance.Instep4,wecreatetwonewsecuritygrouprulesthatallowSSHandICMPtraffic.ThisishandyifwewouldliketopingandSSHtothenewinstance.Beforewebuildtheinstance,weneedtolisttheavailablenetworksinNeutron,whichtheguestwillbepartof;wedothisinstep5.

Withalloftheearlierprerequisitesinplace,webuildanewKVMinstanceinstep6,byspecifyingtheinstanceflavor,theGlanceimage,thenetwork,thesecuritygroup,andtheSSHkey.Wethenproceedtolistthestatusoftheinstance.Noticehowthetaskstateshowsasscheduling,meaningthatthenova-schedulerisselectingahosttoprovisiontheinstanceonandthestatusisBUILD.Sinceweareonlyusingasinglehostforthisexampledeployment,theinstanceisgoingtobeprovisionedonthesamecomputeserver.Fromtheoutputofthebuildcommand,wecanalsoseetheIPaddressthatwasassignedtothenewinstance.

Instep7,wecanseethatthenewinstancewassuccessfullyprovisioned,itsstatusnowshowsasACTIVEandanewQEMUprocesshasbeenstarted.

Finally,instep8,weexaminetherunninginstance;notethatthepowerstatefieldnowshowsRunningandthestatusfielddisplaysactive.

StoppingKVMinstanceswithOpenStackInthisshortrecipe,wearegoingtostoparunningKVMinstance,weprovisionedinthelastrecipe,usingthefamiliaropenstackcommandsyntax.


Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.ArunningKVMinstance,provisionedwithOpenStack.

Howtodoit...TostoparunningKVMguestusingOpenStack,performthefollowingsimplesteps:

1. ListtheprovisionedOpenStackinstances:


+---------------------------+-----------------+--------+----------------

--+--------------+


|ImageName|

+---------------------------+-----------------+--------+----------------

--+--------------+


nat=192.168.0.11|ubuntu_16.04|

+---------------------------+-----------------+--------+----------------

--+--------------+

root@controller:~#

2. Stoptheinstance:

root@controller:~#openstackserverstopubuntu_instance

root@controller:~#

3. ListtheKVMguestsusinglibvirt:

root@controller:~#virshlist--all

IdNameState

----------------------------------------------------

-instance-00000001shutoff

root@controller:~#

4. EnsurethattheQEMUprocessfortheinstancehasterminated:

root@controller:~#pgrep-lfaqemu

root@controller:~#

5. CheckthestatusoftheKVMguest:


+---------------------------+-----------------+--------+----------------

--+--------------+


|ImageName|

+---------------------------+-----------------+--------+----------------

--+--------------+

|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|SHUTOFF|

nat=192.168.0.11|ubuntu_16.04|

+---------------------------+-----------------+--------+----------------

--+--------------+

root@controller:~#

Howitworks...WestartbylistingtheavailableKVMinstances,provisionedwithOpenStackinstep1.Instep2,westoptheinstancebyspecifyingitsname.PleasenotethatwecanalsousetheinstanceIDtostopit.SinceOpenStackusesthelibvirttomanagethelifecycleoftheKVMinstances,instep3,weseethattheinstancehasbeenindeeddestroyed.Instep4,weensurethattheQEMUprocessfortheguesthasalsobeenterminated.Inthelaststep,wecanseethattheinstancestateisnowSHUTOFFinsteadofACTIVE.Instancesinthisstatecanbestartedagainbyexecutingthefollowingcommand:

root@controller:~#openstackserverstartubuntu_instance


+---------------------------+-----------------+--------+------------------+-----

---------+

|ID|Name|Status|Networks|

ImageName|

+---------------------------+-----------------+--------+------------------+-----

---------+

|0f4745b1-...-9eaa1f9bb08f|ubuntu_instance|ACTIVE|nat=192.168.0.11|

ubuntu_16.04|

+---------------------------+-----------------+--------+------------------+-----

---------+

root@controller:~#

TerminatingKVMinstanceswithOpenStackInthisrecipe,wearegoingtoterminateaKVMinstanceprovisionedwithOpenStack.Terminatingtheinstancewillundefineitthroughlibvirt,releasetheallocatedCPUmemoryanddiskresourcesbacktothepoolofavailableresource,forthecomputehost,andmarkitsIPaddressasavailableintheNeutrondatabase.


Adatabaseserver,amessagequeue,andmemcachedinstalledandconfigured,asdescribedinthePreparingthehostfortheOpenStackdeploymentrecipe.TheGlanceservicewithanavailableimage.FormoreinformationonhowtodeployGlanceandaddanewimage,refertotheInstallingandconfiguringtheOpenStackGlanceimageservicerecipe.TheKeystoneservicewedeployedintheInstallingandconfiguringtheOpenStackKeystoneidentityservicerecipe.TheNovaservicesweconfiguredintheInstallingandconfiguringtheOpenStackNovacomputeservicerecipe.TheNeutronservicethatwasdeployedintheInstallingandconfiguringtheOpenStackNeutronnetworkingservicerecipe.ArunningKVMinstance,provisionedwithOpenStack.

Howtodoit...Toterminatearunninginstance,performthefollowingsteps:

1. ObtainthenameorIDoftheinstancetobeterminated:

root@controller:~#openstackserverstartubuntu_instance


+---------------------------+-----------------+--------+----------------

--+--------------+


|ImageName|

+---------------------------+-----------------+--------+----------------

--+--------------+


nat=192.168.0.11|ubuntu_16.04|

+---------------------------+-----------------+--------+----------------

--+--------------+

root@controller:~#

2. Deletetheinstancebyprovidingthename:

root@controller:~#openstackserverdeleteubuntu_instance

root@controller:~#

3. Ensurethattheinstancewasundefined:


root@controller:~#virshlist--all

IdNameState

----------------------------------------------------

root@controller:~#

4. Examinethenova-api,neutron-server,andnova-computelogs:

root@controller:~#cat/var/log/nova/nova-api.log|grep-idelete

2017-05-0415:30:07.73320915INFOnova.osapi_compute.wsgi.server[req-

54dbe80f-9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6

123c1e6f33584dd1876c0a34249a6e11-defaultdefault]10.184.226.74

"DELETE/v2.1/123c1e6f33584dd1876c0a34249a6e11/servers/0f4745b1-9d4b-

4e8a-82f7-9eaa1f9bb08fHTTP/1.1"status:204len:339time:0.1859989

root@controller:~#

root@controller:~#cat/var/log/neutron/neutron-server.log|grep-i

delete

2017-05-0415:30:08.40217910INFOneutron.wsgi[req-5c9674d6-c596-4b17-

b975-54625ac7adb2cc14c5dbbd654c438e52d38efaf4f1a6

123c1e6f33584dd1876c0a34249a6e11---]10.184.226.74--[04/May/2017

15:30:08]"DELETE/v2.0/ports/fdaf6ea1-b76a-4895-a028-db15831132fa.json

HTTP/1.1"2041730.320351

root@controller:~#

root@controller:~#cat/var/log/nova/nova-compute.log

...

2017-05-0415:30:07.74721065INFOnova.compute.manager[req-54dbe80f-

9942-43d8-949a-d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6

123c1e6f33584dd1876c0a34249a6e11---][instance:0f4745b1-9d4b-4e8a-

82f7-9eaa1f9bb08f]Terminatinginstance

2017-05-0415:30:07.95221065INFOnova.virt.libvirt.driver[-]

[instance:0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f]Instancedestroyed

successfully.

2017-05-0415:30:07.95321065INFOos_vif[req-54dbe80f-9942-43d8-949a-

d80daa2440a9cc14c5dbbd654c438e52d38efaf4f1a6

123c1e6f33584dd1876c0a34249a6e11---]Successfullyunpluggedvif

VIFBridge(active=True,address=fa:16:3e:3c:c0:0f,bridge_name='brqb7ccb514-

21',has_traffic_filtering=True,id=fdaf6ea1-b76a-4895-a028-

db15831132fa,network=Network(b7ccb514-21fc-4ced-b74f-

026e7e358bba),plugin='linux_bridge',port_profile=<?

>,preserve_on_delete=False,vif_name='tapfdaf6ea1-b7')

2017-05-0415:30:07.97021065INFOnova.virt.libvirt.driver[req-



82f7-9eaa1f9bb08f]Deletinginstancefiles

/var/lib/nova/instances/0f4745b1-9d4b-4e8a-82f7-9eaa1f9bb08f_del

2017-05-0415:30:07.97421065INFOnova.virt.libvirt.driver[req-



82f7-9eaa1f9bb08f]Deletionof/var/lib/nova/instances/0f4745b1-9d4b-

4e8a-82f7-9eaa1f9bb08f_delcomplete

...

root@controller:~#

Howitworks...WebeginbylistingallinstancesthatNovaknowsaboutinstep1,notingthenameoftheinstancewewouldliketodelete.

Instep2,wedeletetheinstancebyspecifyingitsname.NotethatwecanalsouseitsIDinstead.Instep3,weconfirmthattheinstancehasbeenundefinedbylibvirtandisnolongeravailableinOpenStack.

Instep4,wecanseetheAPIcallsthatweresenttothenova-api,neutron-server,andnova-computeservicesandtheactionthatthoseservicestook.

UsingPythontoBuildandManageKVMInstancesInthischapter,wearegoingtocoverthefollowingtopics:

InstallingandusingthePythonlibvirtlibraryDefiningKVMinstanceswithPythonStarting,stopping,anddeletingKVMinstanceswithPythonInspectingKVMinstanceswithPythonBuildingasimpleRESTAPIserverwithlibvirtandbottle

IntroductionThelibvirtlibraryexposesavirtualizationagnosticinterfaceforcontrollingthefulllifecycleofKVM(andothertechnologies,suchasXENandLXC)instances.UsingthePythonbindingswecandefine,start,destroy,anddeletevirtualguests,alongwithanythingelsethevirshuserspacetoolimplements.Infact,wecanseethatthevirshcommandusesvariouslibvirtsharedlibraries,byrunning:

root@kvm:~#ldd/usr/bin/virsh|greplibvirt

libvirt-lxc.so.0=>/usr/lib/x86_64-linux-gnu/libvirt-lxc.so.0

(0x00007fd050d88000)

libvirt-qemu.so.0=>/usr/lib/x86_64-linux-gnu/libvirt-qemu.so.0

(0x00007fd050b84000)

libvirt.so.0=>/usr/lib/x86_64-linux-gnu/libvirt.so.0(0x00007fd050394000)

root@kvm:~#

ThePythonlibvirtmodule,alsoprovidesmethodstomonitorandreporttheuseofCPU,memory,storage,andnetworkresourcesonthehypervisornodeandothercapabilitiesdependingonthetypeofhypervisordriverinuse.

Inthischapter,wearegoingtouseasmallsubsetofthePythonlibvirtAPItodefine,start,inspect,andstopaKVMinstance.

Foracompletelistoffunctions,classes,andmethodsthatthePythonlibvirtmoduleprovides,execute:root@kvm:~#pydoclibvirt

InstallingandusingthePythonlibvirtlibraryInthisrecipewearegoingtoinstallthePythonlibvirtmoduleanditsdependencies,createanewvirtualenvironment,andinstalltheiPythoncommandshellforinteractivecomputing.

GettingreadyForthisrecipewearegoingtoneedthefollowing:

AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMThePython2.7interpreter,usuallyprovidedbythepython2.7package

Howtodoit...ToinstallthePythonlibvirtmodule,theiPythonutility,andtocreateanewvirtualenvironmentforourtests,followthesesteps:

1. InstallthePythondevelopmentpackagespipandvirtualenv:

root@kvm:~#apt-getinstallpython-pippython-devpkg-configbuild-

essentialautoconflibvirt-dev

root@kvm:~#pipinstallvirtualenv

Downloading/unpackingvirtualenv

Downloadingvirtualenv-15.1.0-py2.py3-none-any.whl(1.8MB):1.8MB

downloaded

Installingcollectedpackages:virtualenv

Successfullyinstalledvirtualenv

Cleaningup...

root@kvm:~#

2. CreateanewPythonvirtualenvironmentandactivateit:

root@kvm:~#mkdirkvm_python

root@kvm:~#virtualenvkvm_python/

Newpythonexecutablein/root/kvm_python/bin/python

Installingsetuptools,pip,wheel...done.

root@kvm:~#sourcekvm_python/bin/activate

(kvm_python)root@kvm:~#cdkvm_python/

(kvm_python)root@kvm:~/kvm_python#ls-la

total28

drwxr-xr-x6rootroot4096May917:28.

drwx------8rootroot4096May917:28..

drwxr-xr-x2rootroot4096May917:28bin

drwxr-xr-x2rootroot4096May917:28include

drwxr-xr-x3rootroot4096May917:28lib

drwxr-xr-x2rootroot4096May917:28local

-rw-r--r--1rootroot60May917:28pip-selfcheck.json

(kvm_python)root@kvm:~/kvm_python#

3. Installthelibvirtmodule:

(kvm_python)root@kvm:~/kvm_python#pipinstalllibvirt-python

Collectinglibvirt-python

Usingcachedlibvirt-python-3.3.0.tar.gz

Buildingwheelsforcollectedpackages:libvirt-python

Runningsetup.pybdist_wheelforlibvirt-python...done

Storedindirectory:

/root/.cache/pip/wheels/67/f0/5c/c939bf8fcce5387a36efca53eab34ba8e94a28f244fd1757c1

Successfullybuiltlibvirt-python

Installingcollectedpackages:libvirt-python

Successfullyinstalledlibvirt-python-3.3.0

(kvm_python)root@kvm:~/kvm_python#pipfreeze

appdirs==1.4.3

libvirt-python==3.3.0

packaging==16.8

pyparsing==2.2.0

six==1.10.0

(kvm_python)root@kvm:~/kvm_python#python--version

Python2.7.6


4. InstalliPythonandstartit:

(kvm_python)root@kvm:~/kvm_python#apt-getinstallipython

...

(kvm_python)root@kvm:~/kvm_python#ipython

Python2.7.6(default,Oct262016,20:30:19)

Type"copyright","credits"or"license"formoreinformation.

IPython1.2.1--AnenhancedInteractivePython.

?->IntroductionandoverviewofIPython'sfeatures.

%quickref->Quickreference.

help->Python'sownhelpsystem.

object?->Detailsabout'object',use'object??'forextradetails.

In[1]:

Howitworks...Westartbyinstallingthedependencypackagesinstep1.SincewearegoingtouseaPythonvirtualenvironmentforourdevelopment,weinstallthevirtualenvpackageaswell.ThePythonlibvirtmoduleisgoingtobeinstalledinthevirtualenvironmentwiththepippackagemanager,sincewedon'twanttopollutethehostwithextrapackages.

Instep2,wecreateandactivateanewPythonvirtualenvironmentandinstallthePythonlibvirtmoduleinstep3.

Finallyinstep4,weinstallandstarttheiPythondevelopmenttool,whichwearegoingtousethroughoutthischapter.

DefiningKVMinstanceswithPythonInthisrecipewearegoingtodefineanewKVMinstanceusingthePythonlibvirtmoduleweinstalledinthepreviousrecipe.WearegoingtouseavirtualenvironmentandtheiPythondevelopmenttoolforthefollowingexamples.


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapter

Howtodoit...TodefineanewKVMinstance,usingthePythonlibvirtmodulefollowtheseinstructions:

1. IntheiPythoninterpreter,importthelibvirtmodule:

In[1]:importlibvirt

In[2]:

2. Createtheinstancedefinitionstring:

In[2]:xmlconfig="""


<name>kvm_python</name>




<resource>


</resource>

<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>





<backingStore/>




</disk>


<aliasname='usb'/>


function='0x2'/>

</controller>




</controller>


<aliasname='ide'/>


function='0x1'/>

</controller>


<macaddress='52:54:00:da:02:01'/>

<sourcenetwork='default'bridge='virbr0'/>



<aliasname='net0'/>


function='0x0'/>

</interface>

<serialtype='pty'>


<targetport='0'/>


</serial>





</console>



<graphicstype='vnc'port='5900'autoport='yes'listen='0.0.0.0'>


</graphics>

<video>




function='0x0'/>

</video>




function='0x0'/>

</memballoon>

</devices>

</domain>

"""

In[3]:

3. Obtainaconnectiontothehypervisor:

In[3]:conn=libvirt.open('qemu:///system')

In[4]:

4. Definethenewinstancewithoutstartingit:

In[4]:instance=conn.defineXML(xmlconfig)

In[5]:

5. Listthedefinedinstancesonthehost:

In[5]:instances=conn.listDefinedDomains()

In[6]:print'Definedinstances:{}'.format(instances)

Definedinstances:['kvm_python']

In[7]:

6. Ensuretheinstancehasbeendefined,usingthevirshcommand:

(kvm_python)root@kvm:~/kvm_python#virshlist--all

IdNameState

----------------------------------------------------

-kvm_pythonshutoff


Howitworks...Inthisrecipe,weareusingthepre-existingrawDebianimagewecreatedinChapter1,GettingStartedwithQEMUandKVM,todefinetheKVMinstance.

Instep1,weimportthelibvirtpackageandproceedtodefinethenewKVMinstance.WeassigntheXMLformattedstringtothexmlconfigvariableinstep2.Noticethatthedefinitioncontainsthenameofthenewinstanceandthelocationoftheimagefile.

Instep3,weobtainaconnectionobjectandassignittotheconnvariable.WecannowusetheavailablemethodstodefinetheKVMguest.

TolistallavailablemethodsforanobjectiniPython,typethevariablenamefollowedby.andpresstheTabkeytwice:In[7]:conn.Displayall117possibilities?(yorn)conn.allocPagesconn.getURIconn.nodeDeviceLookupByName

conn.baselineCPUconn.getVersionconn.nodeDeviceLookupSCSIHostByWWN

conn.c_pointerconn.interfaceDefineXMLconn.numOfDefinedDomainsconn.interfaceLookupByMACStringconn.numOfDefinedInterfaces

...

In[7]:conn.

Toobtainhelponamethod,appendthequestionmarkcharacterattheendofthemethod:In[7]:conn.defineXML?

Type:instancemethod

StringForm:<boundmethodvirConnect.defineXMLof<libvirt.virConnectobjectat

0x7fc5e57dc350>>

File:/root/kvm_python/lib/python2.7/site-packages/libvirt.py

Definition:conn.defineXML(self,xml)

Docstring:

Defineadomain,butdoesnotstartit.

Thisdefinitionispersistent,untilexplicitlyundefinedwith

virDomainUndefine().Apreviousdefinitionforthisdomainwouldbe

overriddenifitalreadyexists.

SomehypervisorsmaypreventthisoperationifthereisacurrentblockcopyoperationonatransientdomainwiththesameIDasthedomainbeingdefined;inthatcase,usevirDomainBlockJobAbort()tostoptheblockcopyfirst.

virDomainFreeshouldbeusedtofreetheresourcesafterthedomainobjectisnolongerneeded.

In[7]:

Instep4,weusethedefineXML()methodonthelibvirt.virConnectconnectionobject,passingtheXMLdefinitionstringandassignittotheinstancevariable.Wecanseethetypeofthenewobjectbyrunning:

In[7]:type(instance)

Out[7]:libvirt.virDomain

In[8]:

Instep5,welistthedefinedinstancesonthehost,byusingthelistDefinedDomains()methodandweconfirmtheresultbyusingthevirshcommandinstep6.

There'smore...Let'saddsomesimpleerrorcheckingtotheprecedingPythoncodeandwriteallofittoanewfile.Wearegoingtobeaddingtothisfileinthesubsequentrecipes:

(kvm_python)root@kvm:~/kvm_python#catkvm.py

importlibvirt

xmlconfig="""






<resource>


</resource>

<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>





<backingStore/>




</disk>


<aliasname='usb'/>

<addresstype='pci'domain='0x0000'bus='0x00'slot='0x01'function='0x2'/>

</controller>



</controller>


<aliasname='ide'/>


</controller>


<macaddress='52:54:00:da:02:01'/>

<macaddress='52:54:00:da:02:01'/>




<aliasname='net0'/>


</interface>

<serialtype='pty'>


<targetport='0'/>


</serial>





</console>





</graphics>

<video>




</video>




</memballoon>

</devices>

</domain>

"""


ifconn==None:

print'Failedtoconnectotothehypervizor'

exit(1)

instance=conn.defineXML(xmlconfig)

ifinstance==None:

print'Failedtodefinetheinstance'

exit(1)

instances=conn.listDefinedDomains()

print'Definedinstances:{}'.format(instances)

conn.close()


Toexecutethescript,ensurethatthepython_kmvinstancehasbeenundefinedfirst,thenrun:

(kvm_python)root@kvm:~/kvm_python#pythonkvm.py



Starting,stopping,anddeletingKVMinstanceswithPythonInthisrecipewearegoingtousethecreate()methodontheinstanceobjectwedefinedinthepreviousrecipetostartitandthedestroy()methodtostopit.

Toobtainmoreinformationonthecreate()method,run:

In[1]:instance.create?

Type:instancemethod

StringForm:<boundmethodvirDomain.createof<libvirt.virDomainobjectat

0x7fc5d9b97d90>>

File:/root/kvm_python/lib/python2.7/site-packages/libvirt.py

Definition:instance.create(self)

Docstring:

Launchadefineddomain.Ifthecallsucceedsthedomainmovesfromthe

definedtotherunningdomainspools.Thedomainwillbepausedonly

ifrestoringfrommanagedstatecreatedfromapauseddomain.Formore

control,seevirDomainCreateWithFlags().

In[2]:


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterTheinstanceobjectwecreatedintheDefiningKVMinstanceswithPythonrecipeinthischapter

Howtodoit...TostarttheKVMinstancethatwasdefinedearlier,toobtainitsstatusandfinallytostopit,usethefollowingPythoncode:

1. Invokethecreate()methodontheinstanceobject:

In[1]:instance.create()

Out[1]:0

In[2]:

2. EnsuretheinstanceisinarunningstatebycallingtheisActive()methodontheinstanceobject:

In[2]:instance.isActive()

Out[2]:1

In[3]:

3. CheckthestatusoftheKVMinstancefromthehostOS:


IdNameState

----------------------------------------------------

5kvm_pythonrunning


4. Stoptheinstancewiththedestroy()method:

In[3]:instance.destroy()

Out[3]:0

In[4]:

5. Ensuretheinstancehasbeendestroyed:


Out[4]:0

In[5]:

6. Deletetheinstanceandlistalldefinedguests:

In[5]:instance.undefine()

In[5]:instance.undefine()

Out[5]:0

In[6]:conn.listDefinedDomains()

Out[6]:[]

In[7]:

Howitworks...Instep1,wecallthecreate()methodtolaunchthedefinedinstance.Ifsuccessfultheguestwilltransitionfromshutoffstatetorunningaswecanseeintheoutputofthecommandinstep3.Instep2,weusetheisActive()method,tocheckthestatusoftheinstance.Anoutputof1indicatesthattheinstanceisrunning.

Instep4,westoptheinstanceusingthedestroy()methodandconfirminstep5.

Finallyinstep6,wedeletetheinstanceusingtheundefine()methodandlistalldefinedinstanceswiththelistDefinedDomains()call.

There'smore...Let'saddthenewcodetothePythonscriptwestartedintheDefiningKVMinstanceswithPythonrecipe.Theupdatedscriptshouldlooklikethefollowing:


importlibvirt

importtime

xmlconfig="""






<resource>


</resource>

<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>





<backingStore/>




</disk>


<aliasname='usb'/>


</controller>



</controller>


<aliasname='ide'/>


</controller>


<macaddress='52:54:00:da:02:01'/>




<aliasname='net0'/>


</interface>

<serialtype='pty'>


<targetport='0'/>


</serial>





</console>





</graphics>

<video>




</video>




</memballoon>

</devices>

</domain>

"""


ifconn==None:


exit(1)


ifinstance==None:


exit(1)



time.sleep(5)

ifinstance.create()<0:

print'Failedtostartthe{}instance'.format(instance.name())

exit(1)

ifinstance.isActive():

print'Theinstance{}isrunning'.format(instance.name())

else:

print'Theinstance{}isnotrunning'.format(instance.name())

time.sleep(5)

ifinstance.destroy()<0:

print'Failedtostopthe{}instance'.format(instance.name())

exit(1)

else:

print'Theinstance{}hasbeendestroyed'.format(instance.name())

ifinstance.undefine()<0:

print'Failedtoremovethe{}instance'.format(instance.name())

exit(1)

else:

print'Theinstance{}hasbeenundefined'.format(instance.name())

conn.close()


Startingthescriptshoulddefineanewinstance,startit,stopit,andfinallyremoveit:


Definedinstances:['kvm1','kvm_python']

Theinstancekvm_pythonisrunning

Theinstancekvm_pythonhasbeendestroyed

Theinstancekvm_pythonhasbeenundefined


Intheprecedingscript,weusedtheinstance.name()methodtogetthenameoftheKVMguestandprintit.Wealsocleanup,byclosingtheconnectiontothehypervisorwiththeconn.close()call.

InspectingKVMinstanceswithPythonInthisrecipewearegoingtocollectinstanceinformation,usingmethodsfromthelibvirt.virDomainclass.

FormoreinformationonthelibvirtPythonAPI,pleaserefertotheofficialdocumentationat:http://libvirt.org/docs/libvirt-appdev-guide-python/en-US/html/index.html.

http://libvirt.org/docs/libvirt-appdev-guide-python/en-US/html/index.html


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7,theiPythontool,andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterTheinstanceobjectwecreatedintheDefiningKVMinstanceswithPythonrecipeinthischapter,representingtheKVMguest

Howtodoit...TocollectCPU,memory,andstateinformationaboutarunninginstance,usethefollowingPythonmethods:

1. Getthenameoftheinstance:

In[1]:instance.name()

Out[1]:'kvm_python'

In[2]:

2. Ensuretheinstanceisrunning:


Out[2]:1

In[3]:

3. CollectresourcestatisticsontheKVMinstance:

In[3]:instance.info()

Out[3]:[1,1048576L,1048576L,1,10910000000L]

In[4]:

4. Retrievethemaximumamountofphysicalmemoryallocatedtotheinstance:

In[4]:instance.maxMemory()

Out[4]:1048576L

In[5]:

5. ExtractCPUstatisticsfortheinstance:

In[5]:instance.getCPUStats(1)

Out[5]:

[{'cpu_time':10911545901L,

'system_time':1760000000L,

'user_time':1560000000L}]

In[6]:

6. Checkifthevirtualmachineisusinghardwareacceleration:

In[6]:instance.OSType()

Out[6]:'hvm'

In[7]:

7. Collecttheinstancestate:

In[82]:state,reason=instance.state()

In[83]:ifstate==libvirt.VIR_DOMAIN_NOSTATE:

....:print('Thestateisnostate')

....:elifstate==libvirt.VIR_DOMAIN_RUNNING:

....:print('Thestateisrunning')

....:elifstate==libvirt.VIR_DOMAIN_BLOCKED:

....:print('Thestateisblocked')

....:elifstate==libvirt.VIR_DOMAIN_PAUSED:

....:print('Thestateispaused')

....:elifstate==libvirt.VIR_DOMAIN_SHUTDOWN:

....:print('Thestateisshutdown')

....:elifstate==libvirt.VIR_DOMAIN_SHUTOFF:

....:print('Thestateisshutoff')

....:elifstate==libvirt.VIR_DOMAIN_CRASHED:

....:print('Thestateiscrashed')

....:elifstate==libvirt.VIR_DOMAIN_PMSUSPENDED:

....:print('Thestateissuspended')

....:else:

....:print('Thestateisunknown')

....:

Thestateisrunning

In[84]:

Howitworks...Inthisrecipe,weusedafewnewmethodsfromthelibvirt.virDomainclass.Let'sseewhattheydoinmoredetailandthenaddthemtothesimplekvm.pyPythonscriptwestartedintheDefiningKVMinstanceswithPythonrecipe.

Insteps1and2,wegetthenameoftheKVMinstanceandensureit'sinarunningstate.

Instep3,wecollectthefollowinginstanceinformation,returnedasaPythonlist:

state:Thestateoftheinstance,asdefinedinthevirDomainStateenumeratedtypeathttps://libvirt.org/html/libvirt-libvirt-domain.html#virDomainStatemaxMemory:Themaximummemoryusedbytheguestmemory:ThecurrentamountofmemoryusedbytheinstancenbVirtCPU:ThenumberofallocatedvirtualCPUscpuTime:Thetimeusedbytheinstance(innanoseconds)

Instep4,wecollectthememoryallocatedtotheinstance.Noticehowitmatchestheoutputofthefunctionfromstep3.

Instep5,wecollectinformationabouttheCPUoftheguestinstance.WecanseetheCPU,system,andusertimes.

ThehvmoutputoftheOSType()methodinstep6,indicatesthattheguestOSisdesignedtorunonbaremetal,requiringfullvirtualization,suchasKVM.

Inthelaststepofthisrecipe,wecallthestate()methodtoreturnthecurrentinstancestate.

https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainState

There'smore...Let'sfinishupthischapterwithacompleteexamplescript,containingallofthemethodswehaveusedsofar:


importlibvirt

importtime

defmain():

xmlconfig="""






<resource>


</resource>

<os>


<bootdev='hd'/>

</os>

<features>

<acpi/>

<apic/>

<pae/>

</features>





<devices>





<backingStore/>




</disk>


<aliasname='usb'/>


function='0x2'/>

</controller>



</controller>


<aliasname='ide'/>


function='0x1'/>

</controller>


<macaddress='52:54:00:da:02:01'/>




<aliasname='net0'/>


function='0x0'/>

</interface>

<serialtype='pty'>


<targetport='0'/>


</serial>





</console>





</graphics>

<video>




function='0x0'/>

</video>




function='0x0'/>

</memballoon>

</devices>

</domain>

"""


ifconn==None:


exit(1)


ifinstance==None:


exit(1)



time.sleep(5)

ifinstance.create()<0:

print'Failedtostartthe{}instance'.format(instance.name())

exit(1)

ifinstance.isActive():

print'Theinstance{}isrunning'.format(instance.name())

else:

print'Theinstance{}isnotrunning'.format(instance.name())

print'Theinstancestate,maxmemory,currentmemory,CPUsandtimeis

{}'.format(instance.info())

print'TheCPU,systemandusertimesare{}'.format(instance.getCPUStats(1))

print'TheOStypeforthe{}instanceis{}'.format(instance.name(),

instance.OSType())

time.sleep(5)

ifinstance.destroy()<0:

print'Failedtostopthe{}instance'.format(instance.name())

exit(1)

else:

print'Theinstance{}hasbeendestroyed'.format(instance.name())

ifinstance.undefine()<0:

print'Failedtoremovethe{}instance'.format(instance.name())

exit(1)

else:

print'Theinstance{}hasbeenundefined'.format(instance.name())

conn.close()

if__name__=="__main__":

main()


Executingitprovidesthefollowingoutput,assumingthekvm_pythoninstancehasbeenundefinedfirst:



Theinstancekvm_pythonisrunning

Theinstancestate,maxmemory,currentmemory,CPUsandtimeis[1,1048576L,

1048576L,1,40000000L]

TheCPU,systemandusertimesare[{'cpu_time':42349077L,'system_time':0L,

'user_time':30000000L}]

TheOStypeforthekvm_pythoninstanceishvm

Theinstancekvm_pythonhasbeendestroyed

Theinstancekvm_pythonhasbeenundefined


BuildingasimpleRESTAPIserverwithlibvirtandbottleInthisrecipe,wearegoingtouseallofthelibvirtmethodswesawintheearlierrecipestobuildasimpleRESTfullAPIserver,leveragingthebottlemicroframeworkforPython.

BottleisdescribedasafastandsimpleWebServerGatewayInterface(WSGI)microweb-frameworkforPython,whichisdistributedasasinglemodulefile.

Formoreinformationonthebottlemicroframeworkpleasevisittheofficialwebsiteat:https://bottlepy.org/docs/dev/.

ThesimpleAPIserverweareimplementing,willacceptthefollowingrequests:

list:getmethodthatlistsalldefinedlibvirtinstances.define:postmethodusedtodefineanewKVMinstance.WearegoingtoprovidetheXMLdefinitionasaheaderinthepostrequest.start:postmethodtostartaninstance.Thenameoftheinstancewillbeprovidedintheheaderoftherequest.stop:postmethodtospotaKVMinstance.undefine:postmethodtodeletetheinstance.

https://bottlepy.org/docs/dev/


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredThedebian.imgrawimagefilewebuiltintheInstallingcustomOSontheimagewithdebootstraprecipefromChapter1,GettingStartedwithQEMUandKVMPython2.7andthevirtualenvironmentwecreatedintheInstallingandusingthePythonlibvirtlibraryrecipeinthischapterThecurlcommandlinetoolfortransferringdatawithURLsyntax,usuallyprovidedbythecurlpackage

Howtodoit...ThefollowingstepsdescribehowtoinstallthebottlemoduleandthesimpleRESTfullAPIserverwritteninPython:

1. Installthebottlemodule:

(kvm_python)root@kvm:~/kvm_python#pipinstallbottle

Collectingbottle

...

Downloadingbottle-0.12.13.tar.gz(70kB)

100%|████████████████████████████████|71kB4.5MB/s

...

Successfullyinstalledbottle-0.12.13


2. Createanewfile,importthelibvirtandbottlemodulesandwritethelibvirtconnectionmethod:

(kvm_python)root@kvm:~/kvm_python#vimkvm_api.py

importlibvirt



try:



conn=None

returnconn

3. Implement/definetheAPIrouteandfunction:

defdefineKVMInstance(template):

conn=libvirtConnect()

ifconn==None:

returnHTTPResponse(status=500,body='Errordefininginstance\n')

else:

try:

conn.defineXML(template)

returnHTTPResponse(status=200,body='Instancedefined\n')



@post('/define')

defbuild():

template=str(request.headers.get('X-KVM-Definition'))

status=defineKVMInstance(template)


returnstatus

4. Implement/undefinetheAPIrouteandfunction:

defundefineKVMInstance(name):


ifconn==None:

returnHTTPResponse(status=500,body='Errorundefininginstance\n')

else:

try:

instance=conn.lookupByName(name)

instance.undefine()

returnHTTPResponse(status=200,body='Instanceundefined\n')


returnHTTPResponse(status=500,body='Errorundefining

instance\n')

@post('/undefine')

defbuild():

name=str(request.headers.get('X-KVM-Name'))

status=undefineKVMInstance(name)

returnstatus

5. Implement/starttheAPIrouteandfunction:

defstartKVMInstance(name):


ifconn==None:

returnHTTPResponse(status=500,body='Errorstartinginstance\n')

else:

try:


instance.create()

returnHTTPResponse(status=200,body='Instancestarted\n')



@post('/start')

defbuild():


status=startKVMInstance(name)

returnstatus

6. Implement/stoptheAPIrouteandfunction:

defstopKVMInstance(name):


ifconn==None:

ifconn==None:

returnHTTPResponse(status=500,body='Errorstoppinginstance\n')

else:

try:


instance.destroy()

returnHTTPResponse(status=200,body='Instancestopped\n')



@post('/stop')

defbuild():


status=stopKVMInstance(name)

returnstatus

7. Implement/listtheAPIrouteandfunction:

defgetLibvirtInstances():


ifconn==None:

returnHTTPResponse(status=500,body='Errorlistinginstances\n')

else:

try:


returninstances



@get('/list')

deflist():

kvm_list=getLibvirtInstances()

return"ListofKVMinstances:{}\n".format(kvm_list)

8. Invoketherun()methodtostarttheWSGIserverwhenthescriptisexecuted:

run(host='localhost',port=8080,debug=True)

Howitworks...Let'slookatthecodeinmoredetail.First,savetheprecedingchangesinafileandexecutethescript:

(kvm_python)root@kvm:~/kvm_python#pythonkvm_api.py

Bottlev0.12.13serverstartingup(usingWSGIRefServer())...

Listeningonhttp://localhost:8080/

HitCtrl-Ctoquit.

Inaseparateterminal,defineanewinstance,passingthefollowingXMLdefinition,asaheader:

(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/define--

header"X-KVM-Definition:<domaintype='kvm'><name>kvm_api</name><memory

unit='KiB'>1048576</memory><vcpu>1</vcpu><os><typearch='x86_64'machine='pc-

i440fx-trusty'>hvm</type></os><devices><emulator>/usr/bin/qemu-system-

x86_64</emulator><disktype='file'device='disk'><drivername='qemu'

type='raw'/><sourcefile='/tmp/debian.img'/><targetdev='hda'bus='ide'/></disk>

<interfacetype='network'><macaddress='52:54:00:da:02:01'/><source

network='default'bridge='virbr0'/><targetdev='vnet0'/></interface><graphics

type='vnc'port='5900'autoport='yes'listen='0.0.0.0'><listentype='address'

address='0.0.0.0'/></graphics></devices></domain>"

HTTP/1.0200OK

Date:Fri,12May201720:29:14GMT

Server:WSGIServer/0.1Python/2.7.6

Content-Length:17

Content-Type:text/html;charset=UTF-8

Instancedefined


WeareusingtherawDebianimagewecreatedinChapter1,GettingStartedwithQEMUandKVM.TheXMLdefinitionshouldlookfamiliaraswell;we'vebeenusingitinmostoftherecipesinthischapter.

WeshouldnowhaveanewKVMinstancedefined.Let'susethe/listroutetolistallinstancesandconfirmwiththevirshcommand:

(kvm_python)root@kvm:~/kvm_python#curllocalhost:8080/list

ListofKVMinstances:['kvm_api']


IdNameState

----------------------------------------------------

-kvm_apishutoff


Nowthatwehaveadefinedinstance,let'sstartitusingthe/startrouteandensureit'srunning:

(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/start--

header"X-KVM-Name:kvm_api"

HTTP/1.0200OK



Content-Length:17


Instancestarted


IdNameState

----------------------------------------------------

1kvm_apirunning


Tostoptheinstanceandremoveitcompletely,weusethe/stopand/undefineroutesfromthescript:

(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/stop--


HTTP/1.0200OK



Content-Length:17


Instancestopped



IdNameState

----------------------------------------------------

-kvm_apishutoff


(kvm_python)root@kvm:~/kvm_python#curl-s-i-XPOSTlocalhost:8080/undefine--


HTTP/1.0200OK



Content-Length:19


Instanceundefined



IdNameState

----------------------------------------------------


Letusgothroughthecodeinmoredetail.

Instep1,weinstallthebottlemoduleinthePythonvirtualenvironment.

Afterimportingthelibvirtandbottlepackagesinstep2,wedefinethelibvirtConnect()method.Thefunctionsinourprogramwilluseittoconnecttothehypervisor.

Instep3,weimplementthe/definerouteanditsfunctionality.The@postdecoratorlinksthecodefromthefollowingfunctiontoaURLpath.Inourexamplethe/definerouteisboundtothebuild()function.Passingthe/defineroutetothecurlcommandwillexecutethefunction,whichinturnwillcallthedefineKVMInstance()methodtodefinetheinstance.

Weusethesamecodepatterninsteps4,5,and6tostart,stop,andundefinetheinstance.

Instep7,weusethe@getdecoratortoimplementafunctiontolistalldefinedinstancesonthehost.

Instep8,weusetherunclasswhichprovidestherun()methodweusetostartabuilt-inserver.Inourexampletheserverwillbelisteningonlocalhost,port8080.

Aswesawearlier,executingthescriptwillstartalisteningsocketonport8080,whichwecaninteractwith,usingthecurlcommand.

There'smore...Thefullcodeimplementationfollows:

importlibvirt



try:



conn=None

returnconn

defgetLibvirtInstances():


ifconn==None:


else:

try:


returninstances



defdefineKVMInstance(template):


ifconn==None:


else:

try:

conn.defineXML(template)

returnHTTPResponse(status=200,body='Instancedefined\n')



defundefineKVMInstance(name):


ifconn==None:


else:

try:


instance.undefine()

returnHTTPResponse(status=200,body='Instanceundefined\n')



defstartKVMInstance(name):


ifconn==None:


else:

try:


instance.create()

returnHTTPResponse(status=200,body='Instancestarted\n')



defstopKVMInstance(name):


ifconn==None:


else:

try:


instance.destroy()

returnHTTPResponse(status=200,body='Instancestopped\n')



@post('/define')

defbuild():

template=str(request.headers.get('X-KVM-Definition'))


returnstatus

@post('/undefine')

defbuild():


status=undefineKVMInstance(name)

returnstatus

@get('/list')

deflist():

kvm_list=getLibvirtInstances()

return"ListofKVMinstances:{}\n".format(kvm_list)

@post('/start')

defbuild():


status=startKVMInstance(name)

returnstatus

@post('/stop')

defbuild():


status=stopKVMInstance(name)

returnstatus

run(host='localhost',port=8080,debug=True)

KernelTuningforKVMPerformanceInthischapter,wearegoingtocoverthefollowingperformancetuningrecipes:

TuningthekernelforlowI/OlatencyMemorytuningforKVMguestsCPUperformanceoptionsNUMAtuningwithlibvirtTuningthekernelfornetworkperformance

IntroductionInthischapter,wearegoingtoexplorevariousconfigurationoptionsandtoolsthatcanhelpimprovetheperformanceofthehostOSandtheKVMinstancesrunningonit.

WhenrunningKVMvirtualmachines,it'simportanttounderstandthatfromthehostperspective,theyareregularprocesses.WecanseethatKVMguestsareLinuxprocessesbyexaminingtheprocesstreeonthehypervisor:

root@kvm:~#virshlist

IdNameState

----------------------------------------------------

16kvmrunning


19913/usr/bin/qemu-system-x86_64-namekvm-S-machinepc-i440fx-


1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-no-


socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-

kvm/monitor.sock,server,nowait-monchardev=charmonitor,id=monitor,mode=control

-rtcbase=utc-no-shutdown-bootstrict=on-devicepiix3-usb-


file=/tmp/debian.img,format=raw,if=none,id=drive-ide0-0-0-deviceide-



rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:2f:df:93,bus=pci.0,addr=0x3-

chardevpty,id=charserial0-deviceisa-serial,chardev=charserial0,id=serial0-

vnc0.0.0.0:0-devicecirrus-vga,id=video0,bus=pci.0,addr=0x2-devicevirtio-

balloon-pci,id=balloon0,bus=pci.0,addr=0x4-msgtimestamp=on

root@kvm:~#

ThevirtualCPUsallocatedtotheKVMguestsareLinuxthreads,managedbythehostscheduler:

root@kvm:~#ps-eLf

UIDPIDPPIDLWPCNLWPSTIMETTYTIMECMD

...

libvirt+199131199130314:02?00:00:00/usr/bin/qemu-system-x86_64-name

kvm-S-machinepc-i440fx-trusty,accel=kvm,usb=off-m1024-realtimemlock=off-

smp1,sockets=1,cores=1,threads=1-uuid283c6653-9981-9396-efb4-fb864d87f769-

no-user-config-nodefaul









...

root@kvm:~#

DependingonthetypeofI/Oscheduler,thelibvirtnetworkdriver,andmemoryconfiguration,theperformanceofthevirtualmachinecanvarygreatly.Beforemakinganychangestotheearlier-mentionedcomponents,itisimportanttounderstandthetypeofworktheguestOSwillbeperforming.TuningthehostandguestOSformemory-intensiveworkwillbedifferentfromI/OorCPUboundloads.

BecauseallKVMinstancesarejustregularLinuxprocesses,theQEMUdrivercanapplyanyofthefollowingControlGroup(cgroup)controllers:cpuset,cpu,memory,blkio,anddevicecontrollers.UsingthecgroupcontrollersprovidesmoregranularcontrolovertheallowedCPU,memory,andI/Oresources,aswearegoingtoseeinmoredetailinthefollowingrecipes.

Perhapsthemostimportantpointwhentuningandoptimizinganysystemistoestablishtheperformancebaselinepriortomakinganyadjustments.Startbymeasuringthebaselineperformanceofasubsystem,suchasmemoryorI/O,makesmallincrementaladjustments,thenmeasuretheimpactofthechangesagain.Repeatasnecessaryuntiladesiredeffectisreached.

Therecipesinthischapteraremeanttogivethereaderastartingpointforwhatcanbetunedonthehostandthevirtualmachinestoimproveperformance,oraccountforthesideeffectsofrunningdifferentworkloadsonthesamehost/VMandtheeffectsofmultitenancy.Allresourcesshouldbeadjustedbasedonthetypeofworkload,hardwaresetup,andothervariables.

TuningthekernelforlowI/OlatencyInthisrecipe,wearegoingtocoversomeofthediskperformanceoptimizationtechniquesbyselectinganI/OschedulerandtuningtheblockI/OusingLinuxcontrolgroups,forthevirtualguestandthehost.

TherearethreeI/OschedulerstochoosefromonthehostOSandintheKVMinstance:

noop:Thisisoneofthesimplestkernelschedulers;itworksbyinsertingallincomingI/OrequestsintoasimpleFIFO(FirstIn,FirstOut)queue.ThisschedulerisusefulwhenthehostOSshouldnotattempttoreorderI/Orequestswhenmultiplevirtualmachinesarerunning.deadline:ThisschedulerimposesadeadlineonallI/Ooperationstopreventstarvationofrequests,givingprioritytoreadrequests,duetoprocessesusuallyblockingonreadoperations.cfq:ThemaingoalofCompletelyFairQueuing(CFQ)istomaximizetheoverallCPUutilizationwhileallowingbetterinteractiveperformance.

SelectingtherightI/Oscheduleronthehostandguestsgreatlydependsontheworkloadsandtheunderlyinghardwarestorage.

Asageneralrule,selectingthenoopschedulerfortheguestOSallowsthehosthypervisortobetteroptimizetheI/Orequests,becauseitisawareofallrequestscomingfromthevirtualguests.However,iftheunderlyingstoragefortheKVMmachinesisiSCSIvolumesoranyotherremotestoragesuchasGlusterFS,usingthedeadlinescheduler,mightyieldbetterresults.

OnmostmodernLinuxkernels,thedeadlinescheduleristhedefault,anditmightbesufficientforhostsrunningmultipleKVMvirtualmachines.Aswithanysystemtuning,testingisrequiredwhenchangingtheschedulersonthehostandguestOS.


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachine

Howtodoit...TochangetheI/OscheduleronthehostandtheKVMinstanceandsetanI/Oweight,performthefollowingsteps:

1. OnthehostOS,listtheI/Oschedulercurrentlyinuse,substitutingtheblockdevicewithwhateverisappropriateforyoursystem:

root@kvm:~#cat/sys/block/sda/queue/scheduler

noopdeadline[cfq]

root@kvm:~#

2. ChangetheI/Oschedulerondemandandensureitisinusebyrunning:

root@kvm:~#echodeadline>/sys/block/sda/queue/scheduler

root@kvm:~#cat/sys/block/sda/queue/scheduler

noop[deadline]cfq

root@kvm:~#

3. Tomakethechangepersistentacrossserverrestarts,addthefollowinglinetotheGRUBdefaultconfigurationandupdate:

root@kvm:~#echo'GRUB_CMDLINE_LINUX="elevator=deadline"'>>

/etc/default/grub

root@kvm:~#tail-1/etc/default/grub

GRUB_CMDLINE_LINUX="elevator=deadline"

root@kvm:~#update-grub2


Foundlinuximage:/boot/vmlinuz-3.13.0-107-generic

Foundinitrdimage:/boot/initrd.img-3.13.0-107-generic

done

root@kvm:~#cat/boot/grub/grub.cfg|grepelevator

linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1ro

elevator=deadlinerd.fstab=noacpi=noirqnoapiccgroup_enable=memory

swapaccount=1quiet



swapaccount=1quiet

linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1rorecovery

nomodesetelevator=deadline

root@kvm:~#

4. FortheKVMinstance,setupthenoopI/Oschedulerpersistently:



Escapecharacteris^]

root@kvm1:~#echo'GRUB_CMDLINE_LINUX="elevator=noop"'>>

/etc/default/grub

root@kvm1:~#tail-1/etc/default/grub

GRUB_CMDLINE_LINUX="elevator=noop"

root@kvm1:~#update-grub2




done

root@kvm1:~#cat/boot/grub/grub.cfg|grepelevator


elevator=nooprd.fstab=noacpi=noirqnoapiccgroup_enable=memory

swapaccount=1quiet


elevator=nooprd.fstab=noacpi=noirqnoapiccgroup_enable=memory

swapaccount=1quiet

linux/boot/vmlinuz-3.13.0-107-genericroot=/dev/md126p1rorecovery

nomodesetelevator=noop

root@kvm1:~#

5. Setaweightof100fortheKVMinstanceusingtheblkiocgroupcontroller:

root@kvm:~#virshblkiotune--weight100kvm

root@kvm:~#virshblkiotunekvm

weight:100

device_weight:

device_read_iops_sec:

device_write_iops_sec:

device_read_bytes_sec:

device_write_bytes_sec:

root@kvm:~#

6. Findthecgroupdirectoryhierarchyonthehost:

root@kvm:~#mount|grepcgroup

noneon/sys/fs/cgrouptypetmpfs(rw)

systemdon/sys/fs/cgroup/systemdtypecgroup

(rw,noexec,nosuid,nodev,none,name=systemd)

root@kvm:~#

7. EnsurethatthecgroupfortheKVMinstancecontainstheweightthatwesetupearlierontheblkiocontroller:

root@kvm:~#cat/sys/fs/cgroup/blkio/machine/kvm.libvirt-

qemu/blkio.weight

100

root@kvm:~#

ForadetailedexplanationonhowLinuxcgroupswork,refertotheContainerizationwithLXCbookfromPacktpublishingathttps://www.packtpub.com/virtualization-and-cloud/containerization-lxc.

https://www.packtpub.com/virtualization-and-cloud/containerization-lxc

Howitworks...WecanseewhatI/Oschedulerthekerneliscurrentlyusingbyexaminingtheschedulerfileinthe/sysvirtualfilesystem.Instep1,weseethatit'sthecfqscheduler.WethenproceedtochangetheI/Oschedulerontherunningsysteminstep2.Pleasekeepinmindthatchangingtheschedulerondemandlikethatwillnotpersistserverrestarts.Insteps3and4,wemodifytheGRUBconfigurationwhichwillappendthenewschedulerinformationtothekernelbootinstructions.RestartingtheserverorthevirtualmachinewillnowselectthenewI/Oscheduler.

Ifrunningmultiplevirtualmachinesonthesamehost,itmightbeusefultogivemoreI/Oprioritytosomeofthembasedoncertaincriteria,suchastimeofdayandVMworkload.Instep5,weusetheblkiocgroupcontrollertosetaweightfortheKVMguest.LowerweightwillgivebetterI/Opriority.Insteps6and7,wecanseethatthecorrectcgrouphierarchyhasbeencreatedandtheblkio.weightfilecontainsthenewweightwesetwiththevirshcommand.

MemorytuningforKVMguestsWhenitcomestomemorytuningofKVMgueststherearefewoptionsavailable,dependingontheworkloadofthevirtualmachine.OnesuchoptionisLinuxHugePages.

MostLinuxhostsbydefaultaddressmemoryin4KBsegments,namedpages.However,thekerneliscapableofusinglargerpagesizes.UsingHugePages(pagesbiggerthan4KB)mayimproveperformancebyincreasingtheCPUcachehitsagainstthetransactionLookasideBuffer(TLB).TheTLBisamemorycachethatstoresrecenttranslationsofvirtualmemorytophysicaladdressesforquickretrieval.

Inthisrecipe,wearegoingtoenableandsetHugePagesonthehypervisorandtheKVMguest,thenexaminethetuningoptionsthatthevirshcommandprovides.



Howtodoit...ToenableandsetHugePagesonthehypervisorandtheKVMguestandusethevirshcommandtosetvariousmemoryoptions,followthesesteps:

1. CheckthecurrentHugePagessettingsonthehostOS:

root@kvm:~#cat/proc/meminfo|grep-ihuge

AnonHugePages:509952kB

HugePages_Total:0

HugePages_Free:0

HugePages_Rsvd:0

HugePages_Surp:0

Hugepagesize:2048kB

root@kvm:~#

2. ConnecttotheKVMguestandcheckthecurrentHugePagessettings:

root@kvm1:~#cat/proc/meminfo|grep-ihuge

HugePages_Total:0

HugePages_Free:0

HugePages_Rsvd:0

HugePages_Surp:0

Hugepagesize:2048kB

root@kvm1:~#

3. IncreasethesizeofthepoolofHugePagesfrom0to25000onthehypervisorandverifythefollowing:

root@kvm:~#sysctlvm.nr_hugepages=25000

vm.nr_hugepages=25000

root@kvm:~#cat/proc/meminfo|grep-ihuge

AnonHugePages:446464kB

HugePages_Total:25000

HugePages_Free:24484

HugePages_Rsvd:0

HugePages_Surp:0

Hugepagesize:2048kB

root@kvm:~#cat/proc/sys/vm/nr_hugepages

25000

root@kvm:~#

4. CheckwhetherthehypervisorCPUsupports2MBand1GBHugePagessizes:

root@kvm:~#cat/proc/cpuinfo|egrep-i"pse|pdpe1"|tail-1

flags:fpuvmedepsetscmsrpaemcecx8apicsepmtrrpgemcacmov

patpse36clflushdtsacpimmxfxsrssesse2sshttmpbesyscallnx

pdpe1gbrdtscplmconstant_tscarch_perfmonpebsbtsrep_goodnopl

xtopologynonstop_tscaperfmperfeagerfpupnipclmulqdqdtes64monitor

ds_cplvmxsmxesttm2ssse3fmacx16xtprpdcmpciddcasse4_1sse4_2

x2apicmovbepopcnttsc_deadline_timeraesxsaveavxf16crdrandlahf_lm

abmaratepbxsaveoptplnptsdthermtpr_shadowvnmiflexpriorityept

vpidfsgsbasetsc_adjustbmi1avx2smepbmi2ermsinvpcid

root@kvm:~#

5. Set1GBHugePagessizebyeditingthedefaultGRUBconfigurationandrebooting:

root@kvm:~#cat/etc/default/grub

...

GRUB_CMDLINE_LINUX_DEFAULT="rd.fstab=noacpi=noirqnoapic

cgroup_enable=memoryswapaccount=1quiethugepagesz=1GBhugepages=1"

...

root@kvm:~#update-grub




done

root@kvm:~#cat/boot/grub/grub.cfg|grep-ihuge



swapaccount=1quiethugepagesz=1GBhugepages=1



swapaccount=1quiethugepagesz=1GBhugepages=1

root@kvm:~#reboot

6. InstalltheHugePagespackage:

root@kvm:~#apt-getinstallhugepages

7. CheckthecurrentHugePagessize:

root@kvm:~#hugeadm--pool-list

SizeMinimumCurrentMaximumDefault

2097152250002500025000*

root@kvm:~#

8. EnableHugePagessupportforKVM:

root@kvm:~#sed-i's/KVM_HUGEPAGES=0/KVM_HUGEPAGES=1/g'

/etc/default/qemu-kvm

root@kvm:~#root@kvm:~#/etc/init.d/libvirt-binrestart



root@kvm:~#

9. MounttheHugeTablevirtualfilesystemonthehostOS:

root@kvm:~#mkdir/hugepages

root@kvm:~#echo"hugetlbfs/hugepageshugetlbfsmode=1770,gid=202100"

>>/etc/fstab

root@kvm:~#mount-a

root@kvm:~#mount|grephugepages

hugetlbfson/hugepagestypehugetlbfs(rw,mode=1770,gid=2021)

root@kvm:~#

10. EdittheconfigurationfortheKVMguestandenableHugePages:


Domainkvm1destroyed


...

<memoryBacking>

<hugepages/>

</memoryBacking>

...



Domainkvm1started

root@kvm:~#

IfyouseethefollowingerrorwhenstartingtheKVMinstance:error:internalerror:hugetlbfsfilesystemisnotmountedor

disabledbyadministratorconfig,makesurethattheHugePagesvirtualfilesystemwasmountedsuccessfullyinstep9.

IfyouseethefollowingerrorwhenstartingtheKVMinstance:error:internalerror:processexitedwhileconnectingtomonitor:

file_ram_alloc:can'tmmapRAMpages:Cannotallocatememory,youneedtoincreasetheHugePagespoolinstep3.

11. UpdatethememoryhardlimitfortheKVMinstanceandverify,asfollows:

root@kvm:~#virshmemtunekvm1

hard_limit:unlimited

soft_limit:unlimited

swap_hard_limit:unlimited

root@kvm:~#virshmemtunekvm1--hard-limit2GB

root@kvm:~#virshmemtunekvm1

hard_limit:1953125

soft_limit:unlimited

swap_hard_limit:unlimited

root@kvm:~#

Howitworks...LibvirtandKVMsupportandtakeadvantageofHugePages.Pleasebeawarethatnoteveryworkloadwillbenefitofhavingpageslargerthanthedefault.InstancesrunningdatabasesandmemoryboundKVMinstancesaregoodusecases.Asalways,beforeenablingthisfeature,measuretheperformanceofyourapplicationinsidethevirtualmachinetoensurethatitwillbenefitfromHugePages.

Inthisrecipe,weenabledHugePagesonthehostandtheguestOSandsetahardlimitontheusablememoryfortheguest.Let'sgothroughthestepsinmoredetail.

Insteps1and2,wecheckthecurrentstateofHugePages.Fromtheoutput,wecanseethatthere'snoHugePagespoolcurrentlyallocated,indicatedbytheHugePages_TotalfieldandthecurrentsizeoftheHugePagesof2MB.

Instep3,weincreasetheHugePagespoolsizeto25000.Thechangeisondemandandwillnotpersistserverreboot.Tomakeitpersistent,youcanaddittothe/etc/sysctl.conffile.

InordertousetheHugePagesfeature,weneedtoensurethattheCPUofthehostserverhashardwaresupportforit,asindicatedbythepseandpdpe1flags,asshowninstep4.

Instep5,weconfiguretheGRUBbootloadertostartthekernelwithHugePagessupportandasetsizeof1GB.

Althoughwecanworkdirectlywiththefilesexposedbythe/procvirtualfilesystem,instep6,weinstalltheHugePagespackage,whichprovidesafewusefuluserspacetoolstolistandmanagevariousmemorysettings.Weusethehugeadmcommandinstep7tolistthesizeoftheHugePagespool.

ToenableHugePagessupportforKVM,weupdatethe/etc/default/qemu-kvmfileinstep8,mountthevirtualfilesystemforitinstep9,andfinallyreconfiguretheKVMvirtualmachinetouseHugePagesbyaddingthe<hugepages/>stanzafor

the<memoryBacking>object.

LibvirtprovidesaconvenientwaytomanagetheamountofallocatedmemoryfortheKVMguests.Instep11,wesetahardlimitof2GBforthekvm1virtualmachine.

CPUperformanceoptionsThereareafewmethodstocontrolCPUallocationandtheavailableCPUcyclesforKVMmachines-usingcgroupsandthelibvirt-providedCPUpinningandaffinityfunctions,wearegoingtoexploreinthisrecipe.CPUaffinityisaschedulerpropertythatconnectsaprocesstoagivensetofCPUsonthehostOS.

Whenprovisioningvirtualmachineswithlibvirt,thedefaultbehavioristoprovisiontheguestsonanyavailableCPUcores.Insomecases,Non-UniformMemoryAccess(NUMA)isagoodexampleofwhenweneedtodesignateacoreperKVMinstance(aswearegoingtoseeinthenextrecipe),thatit'sbettertoassignthevirtualmachinetoaspecifiedCPUcore.SinceeachKVMvirtualmachineisakernelprocess(qemu-system-x86_64morespecificallyinourexamples),wecandothisusingtoolssuchastasksetorthevirshcommand.WecanalsousethecgroupsCPUsubsystemtomanageCPUcycleallocation,whichprovidesmoregranularcontroloverCPUresourceutilizationpervirtualmachine.



Howtodoit...TopinaKVMvirtualmachinetoaspecificCPUandtochangetheCPUshares,performthefollowing:

1. ObtaininformationabouttheavailableCPUcoresonthehypervisor:

root@kvm:~#virshnodeinfo

CPUmodel:x86_64

CPU(s):40

CPUfrequency:2593MHz

CPUsocket(s):1

Core(s)persocket:10

Thread(s)percore:2

NUMAcell(s):2

Memorysize:131918328KiB

root@kvm:~#

2. GetinformationabouttheCPUallocationfortheKVMguest:


VCPU:0

CPU:2

State:running

CPUtime:9.1s


root@kvm:~#

3. PintheKVMinstanceCPU(VCPU:0)tothefirsthypervisorCPU(CPU:0)anddisplaytheresult:

root@kvm:~#virshvcpupinkvm100--live


VCPU:0

CPU:0

State:running

CPUtime:9.3s

CPUAffinity:y---------------------------------------

root@kvm:~#

4. ListtheshareofruntimethatisassignedtoaKVMinstance:

root@kvm:~#virshschedinfokvm1

Scheduler:posix

cpu_shares:1024

vcpu_period:100000

vcpu_quota:-1

emulator_period:100000

emulator_quota:-1

root@kvm:~#

5. ModifythecurrentCPUweightofarunningvirtualmachine:

root@kvm:~#virshschedinfokvmcpu_shares=512

Scheduler:posix

cpu_shares:512

vcpu_period:100000

vcpu_quota:-1

emulator_period:100000

emulator_quota:-1

root@kvm:~#

6. ChecktheCPUsharesintheCPUcgroupssubsystem:

root@kvm:~#cat/sys/fs/cgroup/cpu/machine/kvm1.libvirt-qemu/cpu.shares

512

root@kvm:~#

7. ExaminetheupdatedXMLinstancedefinition:


...


<cputune>

<shares>512</shares>

<vcpupinvcpu='0'cpuset='0'/>

</cputune>

...

root@kvm:~#

Howitworks...WebeginbygatheringinformationabouttheCPUresourcesavailableonthehypervisor.Fromtheoutputinstep1,wecanseethatthehostOShas40CPUsononesocket.

Instep2,wecollectinformationaboutthevirtualmachineCPUanditsaffinitywiththehostCPUs.Inthisexample,theKVMguesthasonevirtualCPU,denotedbytheVCPU:0recordandaffinitytoall40hypervisorprocessors,asindicatedbytheCPUAffinity:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyfield.

Instep3,wepin/bindthevirtualCPUtothefirstphysicalprocessoronthehypervisor.Notethechangeintheaffinityoutput:CPUAffinity:y---------------------------------------.

Fromtheoutputofthevirshcommandinstep4,wecanobservethattheCPUsharesallocatedtotheKVMinstancearesetto1024.Thisvalueisaratio,meaningthatifanotherguesthas512shares,itwillhavetwicefewerCPUruntimethanthatofaninstancewith1024shares.Wereducethatvalueinstep5.

Insteps6and7,weconfirmthattheCPUshareswerecorrectlysetintheCPUcgroupsubsystemonthehostOS.Aswementionedearlier,CPUsharesareconfiguredusingcgroupsandcanbeadjusteddirectlyorbytheprovidedlibvirtfunctionality,bymeansofthevirshcommand.

NUMAtuningwithlibvirtNUMAisatechnologythatallowsthesystemmemorytobedividedintozones,alsonamednodes.TheNUMAnodesarethenallocatedtoparticularCPUsorsockets.Incontrasttothetraditionalmonolithicmemoryapproach,whereeachCPU/corecanaccessallthememoryregardlessofitslocality,usuallyresultinginlargerlatencies,NUMAboundprocessescanaccessmemorythatislocaltotheCPUtheyarebeingexecutedon.Inmostcases,thisismuchfasterthanthememoryconnectedtotheremoteCPUsonthesystem.

LibvirtusesthelibnumalibrarytoenableNUMAfunctionalityforvirtualmachines,aswecanseehere:

root@kvm:~#ldd/usr/sbin/libvirtd|grepnuma

libnuma.so.1=>/usr/lib/x86_64-linux-gnu/libnuma.so.1(0x00007fd12d49e000)

root@kvm:~#

LibvirtNUMAsupportsthefollowingmemoryallocationpoliciestoplacevirtualmachinestoNUMAnodes:

strict:Theplacementwillfailifthememorycannotbeallocatedonthetargetnodeinterleave:Memorypagesareallocatedinaround-robinfashionpreferred:Thispolicyallowsthehypervisortoprovidememoryfromothernodesincasethere'snotenoughmemoryavailablefromthespecifiednodes

Inthisrecipe,wearegoingtoenableNUMAaccessforaKVMinstanceandexploreitsimpactontheoverallsystemperformance.


AnUbuntuhost,withlibvirtandQEMUinstalledandconfiguredArunningKVMvirtualmachineThenumastatutility

Howtodoit...ToenableaKVMvirtualmachinetorunonagivenNUMAnodeandCPUusingthestrictNUMApolicy,performthefollowingsteps:

1. Installthenumactlpackageandcheckthehardwareconfigurationofthehypervisor:

root@kvm:~#apt-getinstallnumactl

...

root@kvm:~#numactl--hardware

available:2nodes(0-1)

node0cpus:012345678920212223242526272829

node0size:64317MB

node0free:3173MB

node1cpus:1011121314151617181930313233343536373839

node1size:64509MB

node1free:31401MB

nodedistances:

node01

0:1021

1:2110

root@kvm:~#

2. DisplaythecurrentNUMAplacementfortheKVMguest:

root@kvm:~#numastat-ckvm1

Per-nodeprocessmemoryusage(inMBs)forPID22395(qemu-system-x86)

Node0Node1Total

-----------------

Huge000

Heap112

Stack224

Private392159

------------------------

Total422365

root@kvm:~#

3. EdittheXMLinstancedefinition,setthememorymodetostrict,andselectthesecondNUMAnode(indexingstartsfrom0,sothesecondNUMAnodeislabeledas1),thenrestarttheguest:


...

<vcpuplacement='static'cpuset='10-11'>2</vcpu>

<numatune>

<memorymode='strict'nodeset='1'/>

</numatune>

...



Domainkvm1destroyed


Domainkvm1started

root@kvm:~#

4. GettheNUMAparametersfortheKVMinstance:

root@kvm:~#virshnumatunekvm1

numa_mode:strict

numa_nodeset:1

root@kvm:~#

5. PrintthecurrentvirtualCPUaffinity:


VCPU:0

CPU:11

State:running

CPUtime:8.4s

CPUAffinity:----------yy----------------------------

VCPU:1

CPU:10

State:running

CPUtime:0.3s

CPUAffinity:----------yy----------------------------

root@kvm:~#

6. PrinttheNUMAnodeplacementfortheKVMinstance:

root@kvm:~#numastat-ckvm1

Per-nodeprocessmemoryusage(inMBs)forPID22395(qemu-system-x86)

Node0Node1Total

-----------------

Huge000

Heap033

Stack022

Private0174174

------------------------

Total0179179

root@kvm:~#

Howitworks...WestartbyexaminingtheNUMAsetuponthehostOS.Fromtheoutputofthenumactlcommandinstep1,wecanobservethatthehypervisorhastwoNUMAnodes:node0andnode1.EachnodemanagesalistofCPUs.Inthiscase,NUMAnode1containsCPUsfrom10to19andfrom30to39andcontains64GBofmemory.Thismeansthat64GBofRAMisgoingtobelocaltothoseCPUsandaccesstothememoryfromthoseCPUsisgoingtobemuchfasterthanfromCPUsthatarepartofnode0.ToimprovememoryaccesslatenciesforaKVMguest,weneedtopinthevirtualCPUsassignedtothevirtualmachinetoCPUsthatareapartofthesameNUMAnode.

Instep2,wecanseethattheKVMinstanceusesmemoryfrombothNUMAnodes,whichisnotideal.

Instep3,weedittheguestXMLdefinitionandpintheguestonthe10thand11thCPUs,whichareapartoftheNUMAnode1,usingthecpuset='10-11'parameter.WealsospecifythestrictNUMAnodeandthesecondNUMAnodewiththe<memorymode='strict'nodeset='1'/>parameter.

Afterrestartingtheinstance,instep4,weconfirmthattheKVMguestisnowrunningusingthestrictNUMAmodeonnode1.WealsoconfirmthattheCPUpinningisindeedwhatwespecifiedinstep5.NotethattheCPUaffinityisflaggedonthe10thand11thelementsoftheCPUaffinityelement.

Fromtheoutputinstep6,wecanseethattheKVMguestisnowusingmemoryonlyfromtheNUMAnode1asdesired.

IfyourunamemoryintensiveapplicationbeforeandaftertheNUMAadjustmentandtest,youwillmostlikelyseesignificantperformancegainswhenaccessinglargeamountsofmemoryinsidetheKVMguest,thankstotheCPUandmemorylocalitythatNUMAprovides.

Thereismore...Inthisrecipe,wesawexamplesonhowtomanuallyassignaKVMprocesstoaNUMAnodebyeditingtheXMLdefinitionoftheguest.SomeLinuxdistributionssuchasRHEL/CentOS7andUbuntu16.04providethenumad(NUMAdaemon)service,whichaimsatautomaticallybalancingprocessesbetweenNUMAnodesbymonitoringthecurrentmemorytopology:

1. ToinstalltheserviceonUbuntu16.04,run:

root@kvm:~#lsb_release-a

NoLSBmodulesareavailable.

DistributorID:Ubuntu

Description:Ubuntu16.04.2LTS

Release:16.04

Codename:xenial

root@kvm2:~#aptinstallnumad

...

root@kvm:~#

2. Tostarttheservice,executethefollowingcode:

root@kvm:~#servicenumadstart

root@kvm2:~#pgrep-lfanumad

12601/usr/bin/numad-i15

root@kvm:~#

3. TomanageaspecificKVMguestwithnumad,passtheprocessIDoftheKVMinstance:

root@kvm:~#numad-S0-p$(pidofqemu-system-x86_64)

root@kvm:~#

4. TheservicewillloganyNUMArebalancingattempts:

root@kvm:~#tail/var/log/numad.log

ThuMay2521:06:422017:ChangingTHPscantimein

/sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecsfrom

10000to1000ms.

ThuMay2521:06:422017:Registeringnumadversion20150602PID12601

ThuMay2521:09:252017:AddingPID4601toinclusionPIDlist

ThuMay2521:09:252017:ScanningonlyexplicitPIDlistprocesses

root@kvm:~#

ThenumadservicecanbehelpfulonOpenStackcomputenodes,where

manualNUMAbalancingmaybetooinvolving.

TuningthekernelfornetworkperformanceMostmodernLinuxkernelsshipsufficientlytunedforvariousnetworkworkloads.Somedistributionsprovidepredefinedtuningservices(agoodexampleistunedforRedHat/CentOS),whichincludeasetofprofilesbasedontheserverrole.

Let'sgooverthestepstakenduringdatatransmissionandreception,onatypicalLinuxhost,beforewedelveintohowtotunethehypervisor:

1. Theapplicationfirstwritesthedatatoasocket,whichinturnisputinthetransmitbuffer.

2. ThekernelencapsulatesthedataintoaProtocolDataUnit(PDU).3. ThePDUisthenmovedontotheper-devicetransmitqueue.4. TheNetworkInterfaceCards(NIC)driverthenpopsthePDUfromthe

transmitqueueandcopiesittotheNIC.5. TheNICsendsthedataandraisesahardwareinterrupt.6. Ontheotherendofthecommunicationchannel,theNICreceivesthe

frame,copiesitonthereceivebuffer,andraiseshardinterrupt.7. Thekernelinturnhandlestheinterruptandraisesasoftinterrupttoprocess

thepacket.8. Finally,thekernelhandlesthesoftinterruptandmovesthepacketupthe

TCP/IPstackfordecapsulation,andputsitinareceivebufferforaprocesstoreadfrom.

Inthisrecipe,wearegoingtoexamineafewbestpracticesfortuningtheLinuxkernel,usuallyresultinginbetternetworkperformance,onmultitenantKVMhosts.

Pleasemakesurethatyouestablishabaselinebeforemakinganyconfigurationchanges,bymeasuringthehostperformancefirst.Makesmallincrementalchanges,thenmeasuretheimpactagain.Theexamplesinthisrecipearenotmeanttobecopied/pastedwithoutpriorunderstandingofthepossiblepositiveornegative

withoutpriorunderstandingofthepossiblepositiveornegativeimpacttheymightmake.Usetheexamplespresentedasaguideastowhatcanbetuned-theactualvaluesmustbecarefullyconsidered,basedontheservertypeandtheentireenvironment.



Howtodoit...Totunethekernelforbetternetworkperformance,executethefollowingsteps(formoreinformationonwhatthekerneltunablesare,readtheHowitworks...section):

1. IncreasethemaxTCPsendandreceivesocketbuffersize:

root@kvm:~#sysctlnet.core.rmem_max

net.core.rmem_max=212992

root@kvm:~#sysctlnet.core.wmem_max

net.core.wmem_max=212992

root@kvm:~#sysctlnet.core.rmem_max=33554432

net.core.rmem_max=33554432

root@kvm:~#sysctlnet.core.wmem_max=33554432

net.core.wmem_max=33554432

root@kvm:~#

2. IncreasetheTCPbufferlimits:min,default,andmaxnumberofbytes.Setmaxto16MBfor1GENIC,and32Mor54Mfor10GENIC:

root@kvm:~#sysctlnet.ipv4.tcp_rmem

net.ipv4.tcp_rmem=4096873806291456

root@kvm:~#sysctlnet.ipv4.tcp_wmem

net.ipv4.tcp_wmem=4096163844194304

root@kvm:~#sysctlnet.ipv4.tcp_rmem="40968738033554432"

net.ipv4.tcp_rmem=40968738033554432

root@kvm:~#sysctlnet.ipv4.tcp_wmem="40966553633554432"

net.ipv4.tcp_wmem=40966553633554432

root@kvm:~#

3. EnsurethatTCPwindowscalingisenabled:

root@kvm:~#sysctlnet.ipv4.tcp_window_scaling

net.ipv4.tcp_window_scaling=1

root@kvm:~#

4. TohelpincreaseTCPthroughputwith1GBNICsorlarger,increasethelengthofthetransmitqueueofthenetworkinterface.Forpathswithmorethan50msRTT,avalueof5000-10000isrecommended:

root@kvm:~#ifconfigeth0txqueuelen5000

root@kvm:~#

5. Reducethetcp_fin_timeoutvalue:

root@kvm:~#sysctlnet.ipv4.tcp_fin_timeout

net.ipv4.tcp_fin_timeout=60

root@kvm:~#sysctlnet.ipv4.tcp_fin_timeout=30

net.ipv4.tcp_fin_timeout=30

root@kvm:~#

6. Reducethetcp_keepalive_intvlvalue:

root@kvm:~#sysctlnet.ipv4.tcp_keepalive_intvl

net.ipv4.tcp_keepalive_intvl=75

root@kvm:~#sysctlnet.ipv4.tcp_keepalive_intvl=30

net.ipv4.tcp_keepalive_intvl=30

root@kvm:~#

7. EnablefastrecyclingofTIME_WAITsockets.Thedefaultvalueis0(disabled):

root@kvm:~#sysctlnet.ipv4.tcp_tw_recycle

net.ipv4.tcp_tw_recycle=0

root@kvm:~#sysctlnet.ipv4.tcp_tw_recycle=1

net.ipv4.tcp_tw_recycle=1

root@kvm:~#

8. EnablethereusingofsocketsintheTIME_WAITstatefornewconnections.Thedefaultvalueis0(disabled):

root@kvm:~#sysctlnet.ipv4.tcp_tw_reuse

net.ipv4.tcp_tw_reuse=0

root@kvm:~#sysctlnet.ipv4.tcp_tw_reuse=1

net.ipv4.tcp_tw_reuse=1

root@kvm:~#

9. Startingwithkernelversion2.6.13,Linuxsupportspluggablecongestioncontrolalgorithms.Thecongestioncontrolalgorithmusedissetusingthesysctlvariablenet.ipv4.tcp_congestion_control,whichissettobic/cubicbydefaultonUbuntu.Togetalistofcongestioncontrolalgorithmsthatareavailableinyourkernel(ifyouarerunning2.6.20orhigher),runthefollowing:

root@kvm:~#sysctlnet.ipv4.tcp_available_congestion_control

net.ipv4.tcp_available_congestion_control=cubicreno

root@kvm:~#

10. Toenablemorepluggablecongestioncontrolalgorithms,loadthekernelmodules:

root@kvm:~#modprobetcp_htcp

root@kvm:~#modprobetcp_bic

root@kvm:~#modprobetcp_vegas

root@kvm:~#modprobetcp_westwood

root@kvm:~#sysctlnet.ipv4.tcp_available_congestion_control

net.ipv4.tcp_available_congestion_control=cubicrenohtcpbicvegas

westwood

root@kvm:~#

11. Forlong,fastpaths,itisusuallybettertousecubicorhtcpalgorithms.CubicisthedefaultforanumberofLinuxdistributions,butifitisnotthedefaultonyoursystem,youcandothefollowing:

root@kvm:~#sysctlnet.ipv4.tcp_congestion_control

net.ipv4.tcp_congestion_control=cubic

root@kvm:~#

12. IfthehypervisorisoverwhelmedwithSYNconnections,thefollowingoptionsmighthelpinreducingtheimpact:

root@kvm:~#sysctlnet.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog=2048

root@kvm:~#sysctlnet.ipv4.tcp_max_syn_backlog=16384

net.ipv4.tcp_max_syn_backlog=16384

root@kvm:~#sysctlnet.ipv4.tcp_synack_retries

net.ipv4.tcp_synack_retries=5

root@kvm:~#sysctlnet.ipv4.tcp_synack_retries=1

net.ipv4.tcp_synack_retries=1

root@kvm:~#

13. Havingasufficientnumberofavailablefiledescriptorsisquiteimportant,sinceprettymucheverythingonLinuxisafile.Eachnetworkconnectionusesafiledescriptor/socket.Tocheckyourcurrentmaxandavailablefiledescriptors,runthefollowingcode:

root@kvm:~#sysctlfs.file-nr

fs.file-nr=1280013110746

root@kvm:~#

14. Toincreasethemaxfiledescriptors,executethefollowing:

root@kvm:~#sysctlfs.file-max=10000000

fs.file-max=10000000

root@kvm:~#sysctlfs.file-nr

fs.file-nr=1280010000000

root@kvm:~#

15. Ifyourhypervisorisusingstatefuliptablerules,thenf_conntrackkernelmodulemightrunoutofmemoryforconnectiontrackingandanerrorwillbelogged:nf_conntrack:tablefull,droppingpacket.Inordertoraisethatlimitandthereforeallocatemorememory,youneedtocalculatehowmuchRAMeachconnectionuses.Youcangetthatinformationfromtheprocfile

/proc/slabinfo.Thenf_conntrackentryshowstheactiveentries,howbigeachobjectis,andhowmanyfitinaslab(eachslabfitsinoneormorekernelpage,usually4KifnotusingHugePages).Accountingfortheoverheadofthekernelpagesize,youcanseefromtheslabinfothateachnf_conntrackobjecttakesabout316bytes(thiswilldifferondifferentsystems).Sototrack1Mconnections,you'llneedtoallocateroughly316MBofmemory:

root@kvm:~#sysctlnet.netfilter.nf_conntrack_count

net.netfilter.nf_conntrack_count=23

root@kvm:~#sysctlnet.netfilter.nf_conntrack_max

net.netfilter.nf_conntrack_max=65536

root@kvm:~#sysctl-wnet.netfilter.nf_conntrack_max=1000000

net.netfilter.nf_conntrack_max=1000000

root@kvm:~#echo250000>/sys/module/nf_conntrack/parameters/hashsize#

hashsize=nf_conntrack_max/4

root@kvm:~#

Howitworks...Instep1,weincreasethemaximumsendandreceivesocketbuffers.ThiswillallocatemorememorytotheTCPstack,butonserverswithalargeamountofmemoryandmanyTCPconnections,itwillensurethatthebuffersizeswillbesufficient.AgoodstartingpointforselectingthedefaultvaluesistheBandwidthDelayProduct(BDP)basedonameasureddelay,forexample,multiplythebandwidthofthelinktotheaverageroundtriptimetosomehost.

Instep2,weincreasethemin,default,andmaxnumberofbytesusedbyTCPtoregulatesendbuffersizes.TCPdynamicallyadjuststhesizeofthesendbufferfromthedefaultvalues.

Instep3,wemakesurethatwindowscalingisenabled.TCPwindowscalingautomaticallyincreasesthereceivewindowsize.

Formoreinformationonwindowscaling,pleaserefertohttps://en.wikipedia.org/wiki/TCP_window_scale_option.

Instep5,wereducethetcp_fin_timeoutvaluewhichspecifieshowmanysecondstowaitforafinalFINpacketbeforethesocketisforciblyclosed.Insteps6and7,wereducethenumberofsecondsbetweenTCPkeep-aliveprobesandfastrecyclingofsocketsintheTIME_WAITstate.

Asarefresher,thefollowingdiagramshowsthevariousTCPstatesaconnectioncanbein:

https://en.wikipedia.org/wiki/TCP_window_scale_option

TCPstatediagram

Instep8,weenablethereuseofsocketsintheTIME_WAITstateonlyfornewconnections.OnhostswithlargenumbersofKVMinstances,thismighthaveasignificantimpactonhowfastnewconnectionscanbeestablished.

Insteps9and10,weenablevariouscongestioncontrolalgorithms.Thechoiceofcongestioncontrolalgorithmsisselectedwhenthekernelisbuilt.Instep11,weselectthecubicalgorithm,inwhichthewindowisacubicfunctionoftimesincethelastcongestionevent,withtheinflectionpointsettothewindowpriortothatevent.

Formoreinformationaboutnetworkcongestion-avoidancealgorithms,pleaserefertohttps://en.wikipedia.org/wiki/TCP_congestion_control.

https://en.wikipedia.org/wiki/TCP_congestion_control

OnsystemsexperiencinganoverwhelmingamountofSYNrequests,adjustingthemaximumnumberofqueuedconnectionrequeststhathavestillnotreceivedanacknowledgementfromtheconnectingclient,usingthetcp_max_syn_backlogandtcp_synack_retriesoptions,mighthelp.Wedothatinstep12.

Insteps13and14,weincreasethemaximumnumberoffiledescriptorsonthesystem.Thishelpswhenalargenumberofnetworkconnectionsarepresentbecauseeachconnectionrequiresafiledescriptor.

Inthelaststep,wehavethenf_conntrack_maxoption.Thisisusefulifwearetrackingconnectionsonthehypervisorusingthenf_conntrackkernelmodule.

KVM Virtualization Cookbook · 2018-11-02 · administering large-scale, highly available Linux ... Installing and configuring the OpenStack Neutron networking service Getting ...

Documents