-
Welcome to the Learning Objekt Operational Data
Provisioning&Extraction.After the explanations of the entire
ODP Architecture and typical scenarios forOperational Analytics
this unit will focus on ODP based extraction scenariosintroducing
new capabilities in SAP Business Objects Data Services.
1
-
The Objectives of this unit are to show a lot of details and
specific benefitsusing Data Services ERP Extractors based on
ODPs.You will learn how:
Extractor Support of Data Services works in general
Business Suite Extractors are presented from the Data Services
ETL Perspective
Typical Dataflow Examples with Delta Support look alike
-
Therefore the Agenda looks like it is shown here:
3
-
With Extractors, BW customers can use one tool to define all
extraction, validation, andcleansing rules to load SAP and non-SAP
data into BW. Non-BW customers can getnative access to the Business
Suite without going through BW.
What are Extractors?? Basically nothing else than ABAP Function
Modules the ERP team has to write? Whenever BW needs access to
data, a new Extractor is written by SAP (or the
customer)? Provide some user exits, but in general are
considered not modifiable? Include the majority of the
transformation logic for BW many BW data flows are a
simple copy of Extractor to BW object? Many sources have
multiple Extractors: some obsolete, some used for different
needs;
for example, one returning the data aggregated, one for the
detailed transactions? As the ERP team is responsible for them,
they know how to get delta or implemented
ways in the ERP system to log changes? Extractors usually do not
have much metadata, no foreign key relationships, primary
keys not marked as such BW and help.sap.com are good sources for
information
? For us all of that does not matter Typically a customer will
ask I want the data forxyz, BW gets it from the Extractor called
abc
4
-
Data Services has many ways to connect to the SAP Business
Suiteapplications (ERP).
Read tables via ABAP - what you do in an ABAP dataflow: an ABAP
programis executed on the ERP server, which generates a data file
that is transferred tothe Data Services server (ftp, shared
directory) and read by the Data Servicesengine for further
processing. Today, it is the recommended approach to readlarge
volumes of data from SAP via Data Services.
RFC_Read_Table - used when you use a table in a regular data
flow.-RFC BAPI calls are like stored procedures in databases. They
can also beused to write data back to SAP.
IDOCs - mostly used for real-time messages, but are also used in
ourmigration offering to load data back to SAP.
And finally:
Extractors - These were not supported in Data Services versions
prior to 4.0;they were used by BW only, but provide a lot of
benefits as you will see in thissession.Exactly this extractor
technology will be opened up through the new ODP APIthat will make
this technology available for external use in ETL processes.
-
Based on the new ODP API SAP Datastores in Busineaa Objects
DataServices have a new Extractors category that you can browse,
import by name,and drag onto a dataflow. The first call returns all
data; subsequent calls giveyou the delta, or you can specify a
datetime.
The ODP API does not show all Extractors, it only shows the
releasedones. The idea is that over the time multiple Extractors
have beendeveloped by SAP, some became obsolete, some might not
work withthis API. So along with the ODP API a new table is created
in thedictionary called ROOSATTR containing all the Extractors the
API andhence DataServices 4.0 supports. Initially these will be a
very limited listfocusing only on the most important Extractors and
will grow over thetime. And for sure it will not include customer
written Extractors.So there is a SAP program - transaction code
unknown as of today -named RODPS_OS_EXPOSE with which customer
written Extractorscan be added to above table. More information on
the availability ofstandard extractors are found in SAP Note
1558737.
-
The graphic shows the main features of the ODP API
implementation in DataServices. In the ERP system all extractors
that are available for external use are describedin the table
ROOSATTR. So called Datastores in Data Services describe source
system connections suchas databases, applications, SAP BW Systems
(OpenHub API) and most recentlySAP Business Suite systems (ODP
API). After connecting an SAP ERP system asa new Datastore in Data
Services a node Extractors will appear in the Datastoretree that
allows to browse into the list of published extractors in the
ROOSATTRtable of the particular ERP system.At the current moment
only single ODP extractors are supported for from the ODPAPI. In
future complex ODPs will be supported as well. Data Services also
supports Metadata Browsing for the SAP ERP world offeringso called
External Metadata for extractors from SAP ERP source systems. In
thistree meta information from all supported Application
Components.
7
-
ODP Extractors can be used in regular data flows and ABAP data
flows, butregular data flows are preferred because they enable data
streaming. The usercan set the Initialize flag to Yes to return all
data, or No to return only thechanges since the last time.
Using the Extractors is (almost) no difference to using tables.
Both arequeried, return data and then used in the ETL context as
usually. Thefirst major difference is, other to SAP table reading,
it is actuallyrecommended to add the Extractor to a regular
dataflow. Remember,when a SAP table is added to a regular dataflow
we read it via theRFC_READ_TABLE function module which returns all
the data in onefunction call. Good for a few rows, certainly not
the way to read millionsof rows. Extractors are different, the API
is more like a SQL dataset. TheExtractor is Prepared and Executed,
then you fetch one array of recordsafter the other putting the
fetched data to the downstream transformimmediately.
So apart from adding the Extractor into a dataflow, there are
just twomore settings to make for an initial load, the
initialization flag should beset to yes and the Extractor date-time
to a global start date
-
Extractors give you the delta, but still as a Designer you need
to implementhow to use the delta information.
The delta behavior of the particular extractor has to be
considered in the detailsof the transformation and load part of the
data flow, as delta logic can beunderstand different in separate
application contexts.
-
The initialization flag is to tell the Extractor we want to read
all its data, notjust the delta. It is kind of the same as the
truncate table flag used in thetarget table for an initial load. As
in the first run the target table is emptyanyhow, truncating
(emptying) the table is not required. But what if theinitial load
should be re-executed? The dataflow makes sure the data istruncated
first. Extractor behave in the same way. If the read neverhappened,
the setting of the initialization flag is of no importance. But if
were-run the initial load and the init flag would be set to No, we
would get thedelta only what would be not acceptable.
Does that mean you need to have two data flows, one with the
initial flagset to yes and one with no for the delta?No, you can
use a global variable here as well. However in most cases youwill
have two data flows anyhow, the initial load will bulk load the
data asquickly as possible, the delta load with all its logic to
deal with changes,deletes etc.The Extractor date-time is used to
associate an Extractor run with a date.Although this is not needed
for the Initial load, this date will help us buildingdelta loads
later.
10
-
You should now be able to Explain Extractor Support of Data
Services in general
Understand the role of Business Suite Extractors seen from the
Data ServicesETL Perspective
Elaborate Typical Dataflow Examples with Delta Support
-
You can find further information on Additional ODP-specific
Administration Tasks inthe SAP Library for SAP NetWeaver 7.0
including Enhancement Package 3 underhttp://help.sap.com.
12
-
13