Talend Metadata Manager Reduce Risk and Friction in your Information Supply Chain
TalendMetadataManager
ReduceRiskandFrictioninyourInformationSupplyChain
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage2Tel:+1(650)5393200
TalendMetadataManagerTalend Metadata Manager provides a comprehensive set of capabilities for all facets ofmetadata management. At the heart of Talend Metadata Manager is a repository whichcontains repository objects, such asmodels andmappings that are organized into folders.Models can be harvested from TalendData Integrationmodels, DataModeling tools, DataWarehouses, external metadata repositories for relational databases (RDBMS), and DataIntegration and Business Intelligence tools. A particular type of repository object calledConfiguration,canconnect“metadatastitching”modelsandmappingstogethertorepresentanEnterpriseArchitecture,includingfullsupportfordataflowlineageandimpactanalysis,aswellassemanticlineagedefinitions.
TalendMetadataManagerconsistsoffourmajorcomponents:
• MetadataBridge(metadataimport)• MetadataManager• DataGovernance• MetadataAuthoringwithForwardEngineering(metadataexport)
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage3Tel:+1(650)5393200
MetadataBridge
Metadataiseverywhere.Datawarehousing,businessintelligence,CASEandETLtoolsallhavetheirownrepositories.Justabouteveryapplicationhasitsowndatadictionary.XMLcarriesthe metadata with it in the message or document, and enterprise application integrationenvironmentshavetheirownrepositoriesandmetadatamappingandintegrationfacilities.Inordertosucceed,onemusthaveagoodenterpriserepositoryintegrationenvironmentthatcanintegratethedifferentformatofmetadatafromalltools.TheTalendMetadataManagerrepositorybridgesthetechnicalandnon-technicalaspectsofmetadata,whilesimultaneouslyaddressing the chasm between the different metadata source and target systems thatconstituteanymoderninformationmanagementenvironment.The Metadata Bridge imports all metadata via “bridges” (metadata import components),including Extract, Transformation and Load (ETL)/ Data Integration tools, BusinessIntelligencetools,DataModelingtools,databases,mostallmetadataexchangestandards,andnumerousdataformatsincludingXML.
ImportingmetadatafromTalendStudiowithTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage4Tel:+1(650)5393200
MetadataManager(MM)
VersionandConfigurationManagementNotonlymusttherepositorybeabletoimportondemandinanyformatandtoanytoolorimportmetadatamanytimesasneeded,itmustbeabletomanagetheversionscreatedbythiscontinuous activity. It must also be fundamental to the repository organization foradministrators to then organize, publish and selectively present the information inappropriateconfigurationsofmetadata,asisrequiredforthecorrectandpreciseanswerstoawiderangeof“cuts”acrossthismetadata.TalendMetadataManagerwasdesignedfromthegroundupwithversionandconfigurationmanagementasakeycapability.
MetadataComparisonAllmetadataisrepresentedbyanintegratedmetamodelinTalendMetadataManager.Thisfeatureprovidescomparisonsacrossmetadatafromdatasourceformatssupported,includingdesigntools,databases,etc.,notsimplyamongversionsofagivenmodel.
ComparingmodelsormodelversionswithTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage5Tel:+1(650)5393200
DataMappingSpecificationsOnceimported,metadatacanbemappedinamyriadofwaystoanyothermetadatawithinTalendMetadataManager.Thisabilityiscriticaltothesuccessofanymetadatamanagementsolution. Inparticular, youcandefinedata flowmappings describingdatamovement typerelationships,e.g.whenadatabaseisreadandtheresultswrittentoanotherdatabase,aswellas semanticmappingswhich identify semantic relationships between elements, oftentimesconceptualorlogicalinnature,suchasforadatadictionaryorconceptualmodelsuchasaUMLmodel.
MetadataStitchingMetadatastitchingisfundamentaltothecorrectandautomatedanalysisofthedataflowandsemanticlineageofmetadataintherepository.Italsosupportsversionmanagementacrosstheconstantrateofupdatesandchangesinarepository.TalendMetadataManagerkeepscompleteversionsofallimportedmetadatainself-contained“models”,whicharethenrelatedviastitching’s(simpleconnectionmappings). Inthisway,versionmanagementandconfigurationmanagement isnotonlyentirelycleanandisolatedfromthedefinitionandmaintenanceofmappings,italsoautomaticallysupportsupdatesandchangesintothefuture.
Gettingahighlevelviewofinformationflowsacrosssystemswithmetadatastitching
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage6Tel:+1(650)5393200
In this way, the enterprise architecture is correctly modeled, and data flow lineage iscompletelyandaccuratelyderivable.
Thedifferentrolesandtheirneedswithrespecttodataandrelatedmetadata
LineageandImpactAnalysisOncemetadata ismanaged,metadata is then available for detailed technical and businessanalysis. TalendMetadata Manager supports full technical and business level lineage andimpactanalysisprovidingyounewinsightacrossalltheconnectedmetadatasources.
BusinessUser–LineageReportinganalysisisthetypicalusecase,withquestionssuchas:
• Givenanitemonareport,whatdataentrysystemfieldsimpacttheseresults?• Whyarethenumbersonthisreportthewaytheyare?• HowdoIchangethesystemdatatocorrecttheresultsofthisreport?
DatalineagewithTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage7Tel:+1(650)5393200
TechnicalUser–ImpactAnalysisOfhighinteresttothetechnicaluserarequestionslike:
• IfImustchangetheseelements(datatype,codesets,etc.)inmyoperationaldatastore,whatisthedownstreamimpact?
• ThisnewETLprocessispopulatingmystagingwarehouseinnewways,howdoesthisimpacttheOLAPmodelinmyreportingservices?
TechnicalUser–LineageReverselineagetypequestionsmayalsobeaskedbymoretechnicalusers,suchas:
• HowmanysystemsarerequiredtodeterminethedimensionsforthisportionoftheOLAPmodel?
• Abusinessreportusecase isaskingthe lineageforparticularvaluesonareport,sowheredoesthedatacomefromandhowisitmanipulated?
BusinessUsers–ImpactAnalysisFinally,businessusersmayasktheforwardlineageorimpactanalysisquestions,suchas:
• IfImakeachangetothisfield,whatreportswillbeimpacted?• How is this identity informationmergedwith the personnel system information on
theseotherreports?
ImpactanalysiswithTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage8Tel:+1(650)5393200
DataGovernance(DG)
Critical to thedevelopmentandmanagementofa completedataarchitecture isaBusinessGlossary. Talend Metadata Manager provides an ISO 11179-based Business Glossary tocapture,define,maintainandimplementanenterpriseBusinessGlossaryofterminology,datadefinitions,codesets,domains,validationrules,etc.Inaddition,semanticmappingsdescribehowelementsinasourceModel(moreconceptualliketheBusinessGlossary)defineelementsinadestinationModel(closertoanimplementationorrepresentation).TheBusinessGlossaryhelpsanenterprisereachagreementbetweenallstakeholdersontheirbusiness assets (e.g. terms) and how they relate to data assets (e.g. database tables) andtechnology assets (e.g. ETL mappings). The Business Glossary can be used to documentlogical/physicaldataentitiesandattributesacrossITcollaboratively.Again,itinvolvestracingdependenciesbetweenbusinessandtechnicalassets.InTalendMetadataManager,aBusinessGlossaryisaself-containedcollectionofcategoriesand the terms sub-categories containedwithin each category. In turn, the termsmay besemantically mapped to objects throughout the rest of the repository, such as tables andcolumns inadatamodel. Oncemapped,onemayperformsemantic lineage tracessuchasdefinitionlookupsandtermsemanticusageacrossanyconfigurationscontainingtheBusinessGlossary,mappingsandmappedobjects.
AuthoringthecommonbusinesstermsusedintheorganizationwiththeBusinessGlossary
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage9Tel:+1(650)5393200
BootstrappingaBusinessGlossaryBuildingaBusinessGlossarycanbeassimpleasdragginginanexistingwell-documenteddatamodel,viaimportfromothersources(aCSVfileformat),orcanbepopulateddirectlyviatheuserinterfaceduringtheprocessofclassifyingobjectsinotherdatastoremodels.Ingeneral,acombinationofsuchmethodsareemployedinconjunctionwithoneanother.
WorkflowInordertoensurethattheBusinessGlossaryisaccurate,up-to-date,availabletoallwhoneedaccesstoit,andintegratedproperlywiththerestofthemetadataintherepository,TalendMetadata Manager also provides a robust collection of Data Governance tools andmethodologies. The Business Glossary provides a very flexible workflow and publicationprocessthatcanaddressbothbasicandcomplexneeds.Inaddition,onemaymaintainanynumberofbusinessglossaries,eachwithdifferentworkflowandpublicationcharacteristics.TheBusinessGlossarymaybepartofyourlineage.Itwillappearintherepositorypanelandwhen you open a Business Glossary, youwill be presentedwith a different UI than other(imported)Models.
Workflow-drivensearchcriteriaareavailableallowingonetoefficientlyorganizetermsandidentifywhatactionsarerequiredatanygiventime.Whenworkingwith individual terms,whichareatsomepoint intheworkflowprocess,workflowtransitionbuttonspromptyouwithpossibleactions.
SemanticMappingA SemanticMapping describes how elements in a sourcemodel (more conceptual) defineelementsinadestinationmodel(closertoanimplementationorrepresentation).Putanotherway, elements in the destination model are representations or implementations of theassociatedelementinthesourcemodel.Theyarethreeprimaryusesforsemanticmapping:
• DataStandardizationandCompliance• Multi Level Modeling of semantic relationships from conceptual to logical, and to
physicaldatamodelwithafewsubcases• BusinessGlossarytermclassification
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage10Tel:+1(650)5393200WP208-EN
MetadataAuthoring(MA)withForwardEngineering(MetadataExport)Note:ThefollowingfeaturesonlycomewithTalendMetadataManagerwithAuthoring.
RDBMSandBigDataDocumenterandPhysicalDataModelerThe Talend Metadata Manager Data Documenter allows users to document existing datastores, like databases, big data sources, and imported models, and publish the resultingdocumenteddatastorestotheenterprise.TheDataDocumenteroffersadifferentapproachthantraditionaldatamodelingtools:
• The Business Glossary-driven Data Documentermethodology allows for immediatereuseandcreationoftermsandnamingstandardsonthefly,fasttrackingthedatastoredocumentationprocessensuringcompletesemanticsynchronizationamongyourdatamodelsanddatagovernanceenvironment.
• Web-enabledDataDocumenteroffersbetteraccesstousersthandesktoptools• DataModeling anddiagramming capabilities of theDataDocumenter are similar to
conventionaldatamodelingtools.• Fullintegration(import/export)tomostpopulardatamodelingtoolsisprovided.
VisualizingDataModelswithTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage11Tel:+1(650)5393200WP208-EN
LogicalDataModelerTalend Metadata Manager provides a completely web-enabled logical data modelingenvironmentforproducinglogicalandconceptualmodels:
• TheBusinessGlossary-drivenmethodology allows for immediate reuse (creating ofentities,attributesanddomains)andcreationoftermsandnamingstandardsonthefly, fast tracking the modeling process and ensuring complete semanticsynchronizationamongyourmodelsanddatagovernanceenvironment.
• TheWeb-enabledmodeleroffersbetteraccesstousersthandesktoptools.• TheDataModelingcapabilitiesarecompetitivewithconventionaldatamodeling
tools.• Fullintegration(import/export)withmostpopulardatamodelingtoolsisprovided.
DataMappingDesignerData Mapping Designs represents data integration process designs containing all thenecessarydatamovementdesigndetails, such as lookups, filters, joins and transformationexpressions. TheseDataMappingDesignsare completeenough that theymaybe forwardengineered into Talend Data Integration using the Metadata Bridge. In this way, TalendMetadataManagerprovidesacompletelyweb-baseddatamappingdesigntoolthatcanreuseandbesynchronizedwithallothermetadataartifactsintherepositoryandyourcompletedatagovernanceenvironment.
DefiningthemappingsdirectlyinTalendMetadataManager
TalendInc. 800BridgeParkway,Suite200,RedwoodCity,California94065USPage12Tel:+1(650)5393200WP208-EN
VisualizingtheendtoendinformationflowswithTalendMetadataManager