✒️SAP BI / Fuentes de Datos Por Mary Galicia

Uses and Advantages of the Data Flow with SAP NetWeaver 7.0

DataSource object type RSDS (new DataSource):

The new DataSource of object type RSDS enables real-time data acquisition, as well as direct access to source systems of type File and DB Connect.

Data Transfer Process:

The data transfer process (DTP) makes the transfer processes in the data warehousing layers more transparent. The performance of the transfer processes increases when you optimize parallelization. With the DTP, delta processes can be separated for different targets and filtering options can be used for the persistent objects on different levels. Error handling can also be defined for DataStore objects with the DTP. The ability to sort out incorrect records in an error stack and to write the data to a buffer after the processing steps of the DTP simplifies error handling. When you use a DTP, you can also directly access each DataSource in the SAP source system that supports the corresponding mode in the metadata (also master data and text DataSources).

Transformation:

Transformations simplify the maintenance of rules for cleansing and consolidating data. Instead of two rules (transfer rules and update rules), as in the past, only the transformation rules are still needed. You edit the transformation rule in an intuitive graphic user interface. InfoSources are no longer mandatory; they are optional and are only required for certain functions. Transformations also provide additional functions - such as quantity conversion, performance-optimized reading of master data and DataStore objects - as well as the option to create an end routine or expert routine.

SAP HANA-optimized DataStore object as update source:

Important if you are using the SAP HANA database: the update rules cannot read from an SAP HANA-optimized DataStore object due to its architecture. Therefore, a DataStore object used as a source of update rules in a 3.x data flow cannot be converted to an SAP HANA-optimized object. Once you have migrated the 3.x data flow with the DataStore object used as the source for the update rules, you can perform the conversion and use the SAP HANA-optimized DataStore object.

Data Flow Modeling Tool

You model data flows and elements in the Modeling functional area of the Data Warehousing Workbench. The graphical user interface here helps you to create top-down model and use the best practice models (data flow templates provided by SAP). With top-down modeling, you create a model blueprint on the BW system, which you can use later on to create a persistent data flow

Data Warehousing Workbench Purpose

The Data Warehousing Workbench (DWB) is the central tool for performing the tasks in the data warehousing process. It provides data modeling functions as well as functions for control, monitoring and maintenance of all processes in SAP NetWeaver BI having to do with data procurement, data retention, and data processing.

DB Connect is used to define other database connection in addition to the default connection and these connections are used to transfer data into the BI system from tables or views.

To connect an external database, you should have the following information −

• Tools
• Source Application knowledge
• SQL syntax in Database
• Database functions

Perquisites

In case the source of your Database management system is different from BI DBMS, you need to install database client for source DBMS on the BI application server.

DB Connect key feature includes loading of data into BI from a database that is supported by SAP. When you connect a database to BI, a source system requires creating a direct point of access to the external relational database management system.

DB Architecture

SAP NetWeaver component’s multiconnect function allows you to open extra database connections in addition to the SAP default connection and you can use this connection to connect to external databases.

DB Connect can be used to establish a connection of this type as a source system connection to BI. The DB Connect enhancements to the database allows you to load the data to BI from the database tables or views of the external applications.

For default connection, the DB Client and DBSL are preinstalled for the database management system (DBMS). To use DB Connect to transfer data into the BI system from other database management systems, you need to install database-specific DB Client and database-specific DBSL on the BI application server that you are using to run the DB Connect.

Creating DBMS as Source System

Go to RSA1 → Administration workbench. Under the Modeling Tab → Source Systems

Go to DB Connect → Right click → Create.

Enter the logical system name (DB Connect) and description. Click on Continue.

Enter the database management system (DBMS) that you want to use to manage the database. Then enter the database user under whose name you want the connection to be opened and the DB Password has to enter for authentication by the database.

In the Connection Info, you have to enter the technical information required to open the database connection.

Permanent Indicator

You can set this indicator to keep a permanent connection with the database. If the first transaction ends, then each transaction is checked to see if the connection has been reinitiated. You can use this option if the DB connection has to be accessed frequently.

Save this configuration and you can Click Back to see it in the table.

SAP BW - Flat File Data Transfer

ou can load the data from an external system to BI using these flat files. SAP BI supports data transfer using flat files, files in ASCII format, or in the CSV format.

The data from a flat file can be transferred to BI from a workstation or from an application server.

Following are the steps involved in a Flat File Data Transfer −

• Define a file source system.

• Create a DataSource in BI, defining the metadata for your file in BI.

• Create an InfoPackage that includes the parameters for data transfer to the PSA.

Important Points about Flat File Data Transfer

• If there are character fields that are not filled in a CSV file, they are filled with a blank space and with a zero (0) if they are numerical fields.

• If separators are used inconsistently in a CSV file, the incorrect separator is read as a character and both fields are merged into one field and may be shortened. Subsequent fields are then no longer in the correct order.

• A line break cannot be used as part of a value, even if the value if enclosed with an escape character.

A couple of pointers with regard to the CSV and ASCII files

• The conversion routines that are used to determine whether you have to specify leading zeros. More information − Conversion Routines in the BI-System.

• For dates, you usually use the format YYYYMMDD, without internal separators. Depending on the conversion routine being used, you can also use other formats.

Define a File Source System

Before you can transfer data from a file source system, the metadata must be available in BI in the form of a DataSource. Go to Modeling tab → DataSources.

Right click in context area → Create DataSource.

Enter the technical name of the data source, type of data source and then click on Transfer.

Go to General tab → Select the General Tab. Enter descriptions for the DataSource (short, medium, long).

If required, specify whether the DataSource is initial non-cumulative and might produce duplicate data records in one request.

You can specify whether you want to generate the PSA for the DataSource in the character format. If the PSA is not typed it is not generated in a typed structure but is generated with character-like fields of type CHAR only.

The next step is to click on the Extraction tab page and enter the following details −

Define the delta process for the DataSource. Specify whether you want the DataSource to support direct access to data (Real-time data acquisition is not supported for data transfer from files).

Select the adapter for the data transfer. You can load text files or binary files from your local work station or from the application server. Select the path to the file that you want to load or enter the name of the file directly.

In case you need to create a routine to determine the name of your file. The system reads the file name directly from the file name field, if no, then the routine is defined.

As per the adapter and the file to be loaded, the following setting has to be made −

• Binary files − Specify the character record settings for the data that you want to transfer.

• Text-type files − For text files, determine the rows in your file are header rows and they can therefore be ignored when the data is transferred. Specify the character record settings for the data that you want to transfer.

• For ASCII files − To load the data from an ASCII file, the data is requested with a fixed data record length.

• For CSV files − To load data from an Excel CSV file, mention the data separator and the escape character.

  The next step is to go to the Proposal tab page, this is required only for CSV files. For files in different formats, define the field list on the Fields tab page.

The next step is to go to Fields tab −

You can edit the fields that you transferred to the field list of the DataSource from the Proposal tab. If you did not transfer the field list from a proposal, you can define the fields of the DataSource here as shown in the following screenshot.

You can then perform check, save and activate the DataSource.

You can also select the Preview tab. If you select read Preview Data, the number of data records you specified in your field selection is displayed in a preview.