SAP Integration with Talend Components / Connectors (BAPI, RFC, IDoc, BW, SOAP)

talend has several connectors to integrate sap systems. however, this guide is no introduction to talend’s sap components. instead, this guide helps to

• understand different alternatives to integrate sap systems with talend
• set up a local sap system
• configure talend studio for using sap components
• use talend’s sap wizard
• run a first talend job which connects to sap

all further required information and example use cases for talend’s sap components should be available in the talend component guide at www.help.talend.com . if that’s not the case, please create a jira documentation ticket ( https://jira.talendforge.org/browse/doct )!
now let’s take a look at different alternatives for integration of sap systems with talend.

alternatives for sap integration

three protocols exist for communication between sap and external programs:

• dynamic information and action gateway (diag): e.g. used by sap gui
• remote function call (rfc): a function call with input and output parameters (like a java interface)
• hypertext transfer protocol (http): internet standard

the following alternatives are available for integrating sap systems using some of these protocols.

file

sap supports the direct import of files (call-transaction-program, batch-input, direct input). files have to be in a specific format to be imported. transformation and integration can be realized with talend’s various file components such as tfileinputdelimited.

rfc

remote function call is the proprietary sap ag interface for communication between a sap system and other sap or third-party compatible system over tcp/ip or cpi-c connections. remote function calls may be associated with sap software and abap programming, and provide a way for an external program (written in languages such as php, asp, java, or c, c++) to use data returned from the server. data transactions are not limited to getting data from the server, but can insert data into server records as well. sap can act as the client or server in an rfc call.

a remote function call (rfc) is the call or remote execution of a remote function module in an external system. in the sap system, these functions are provided by the rfc interface system. the rfc interface system enables function calls between two sap systems, or between a sap system and an external system.
tsapinput and tsapoutput are talend’s components to use rfcs.

business application programming interface (bapi)

a bapi is an object-oriented view on most data and transactions of a sap system (called “business objects”). object types of the business objects are stored in the business object repository (bor).
bapis are always implemented as rfcs and therefore can be called the same way. additionally, they have the following characteristics (compared to rfcs):

• stable interface
• no view layer
• no exceptions, instead export parameter: “return”

most business objects offer the following standard bapis:

• getlist
• getdetail
• change
• creationfromdata

tsapinput and tsapoutput are talend’s components to use bapis.

application link enabling (ale)

application link enabling (ale) is used for asynchronous messaging between different systems via “intermediate documents (idoc)”. idoc is a sap document format for business transaction data transfers. it is used to realize distributed business processes.
idoc is similar to xml in purpose, but differs in syntax. both serve the purpose of data exchange and automation in computer systems, but the idoc technology takes a different approach. while xml allows having some metadata about the document itself, an idoc is obligated to have information at its header like its creator, creation time, etc. while xml has a tag-like tree structure containing data and meta-data, idocs use a table with the data and meta-data. idocs also have a session that explains all the processes which the document passed or will pass, allowing one to debug and trace the status of the document.
an idoc consists of

• control record (it contains the type of idoc, port of the partner, release of sap r/3 which produced the idoc, etc.)
• data records of different types. the number and type of segments is mostly fixed for each idoc type, but there is some flexibility (for example an sd order can have any number of items).
• status records containing messages such as 'idoc created', 'the recipient exists', 'idoc was successfully passed to the port', 'could not book the invoice because...'

different idoc types are available to handle different types of messages. for example, the idoc format orders01 may be used for both purchase orders and order confirmations.
tsapidocinput and tsapidocoutput are talend’s components to use ale / idoc.
bapis can also be called asynchronously via ale. all new idocs are even based on bapis.

soap web services

sap supports soap web services. not just sap as java, but also sap as abap! integration can be realized with talend’s esb / web service components such as tesbrequest, tesbresponse, or tesbconsumer.

installation of sap server and client

installation can take about 6 to 8 hours, but it is an “all in one installation”, i.e. you can install it overnight. steps for installation:

• get yourself a windows 7 64 bit laptop or vm with 8+ gb ram and 50+gb free disc space
• get a sap community account (for free, just register): http://scn.sap.com/welcome
• download sap netweaver (software downloads --> sap netweaver main releases: http://www.sdn.sap.com/irj/scn/nw-downloads

download current version of  sap netweaver application server abap 64-bit trial

• install sap server: follow installation guide – a html website included in the download in root of extracted download folder  (start.htm --> there click on “installation” link)
• install sap gui (rich client frontend): start.htm --> there click on “install sap gui” link and follow instructions
• download the sap jco for the operating system on which your connector is running. the sap jco is available for download from sap's website at http://service.sap.com/connectors . you must have an sapnet account to access the sap jco (if you do not already have one, contact your local sap basis administrator).

usage of sap server

• hint: you have to use a windows user which has a password (as you need to enter windows credentials when stopping sap). if you have a windows user without a password (for instance if you use windows within a vm on your mac), sap cannot process these credentials (i.e. it cannot process an empty password field) --> change your windows password before starting sap
• start the management console (windows startmenu --> programs --> sap management console)
• start and stop the sap server (right click on “nsp” --> start / stop)
• default user: sap* (sap system super user)
• password: the one which you entered at installation of sap netweaver, e.g. admin123

usage of sap client

a sap client should be used to get information about the sap system (functions, data, etc.) similarly to using e.g. mysql workbench to get information from a mysql database. sap gui (view layer) communicates with sap as abap (business logic layer). the application server communicates with the relational database (db layer).
different clients are available for sap:

• sap gui windows
• sap gui java
• web browser
• external rfc-program

for local development demos, sap gui windows is probably the best alternative. start sap gui windows by:

• clicking shortcut “windows start menu --> sap frontend --> sap logon”
• entering username  and password
• clicking logon

sap transactions

in sap, you call sap programs via sap transaction codes. important transactions codes are for example:

• bapi: bapi explorer, view all sap bapi's
• se16: data browser, view/add table data
• se38: program editor

here is a list of several other important transaction codes: http://www.sapdev.co.uk/tcodes/tcodes.htm

installation of demo data

the sap installation includes some demo data. as most people do not want to install “real” sap modules such as sap fi, sap crm or sap bi on their local system, this demo data is perfect for demos using talend’s sap connectors.
to install the flight demo on a local sap system, you just have to open the abap editor (transaction: se38) and execute the program sapbc_data_generator. this program generates example data within the flight tables and does some further initializations.

here is a good tutorial with more information and how to test the flight application: http://help.sap.com/saphelp_erp60_sp/helpdata/de/db/7c623cf568896be10000000a11405a/content.htm

configuration of talend studio to use sap components

talend’s sap components are already included in the studio. however, two further steps are required to be able to use them:

• copy sapjco3.dll to the directory c:/windows/system32
• sap java connector jar must be added
• copy sapjco3.jar to the directory “talend/studio/lib/java”
• (re-) start talend studio
• check if sap library is added successfully
• open view “talend modules” (eclipse --> windows --> show view --> talend --> modules)
• sort by column “context”
• look for “tsap*” contexts and check if sapjco3.jar has status “installed”

usage of sap components with talend studio

this section describes how to use talend’s sap components and the sap wizard in general (using one specific example for calling a bapi). detailed descriptions of all sap components (for using bapis, rfcs, idocs, bw, etc.) are available in the documentation talend_components_rg_x.y.z.pdf at www.help.talend.com .

connection to a sap system

a connection to a sap system can be done “built-in” or via “metadata --> sap connections” (the latter only in enterprise version). using the latter has several advantages:

• reuse connection configuration
• quick check if connection to sap works
• wizards for retrieving functions from sap (instead of handwriting without wizard)
• quick test with test parameters if function works before finishing

development lifecycle for a sap job

development lifecycle for sap job:

• create connection (if not existing yet)
• right click on metadata --> sap connection
• create sap connection
• follow wizard

sap jco version: 3
client: “001”
userid: “sap*”
password: “admin123” --> as you defined it while installation
language: “en”
hostname: “localhost”
system number: “00”

• retrieve function (bapi / rfc)
• right click on created connection
• click on “retrieve sap function”
• enter search filter (e.g. bapi_fl*)
• click on “search”
• select and double click on your function (e.g bapi_flcust_getlist)
• you see all input, output and table parameters for this sap function
• click on “test in” --> here you see parameters in more detail:

• you now have to define which input and output parameters you want to use --> remove all other by selecting them and clicking “remove” button
• hint: if you do not remove an input parameter, you usually have to enter a value for it!
• select the output type - can be a single (single record), a table (list of records), or a structure output
• hint: difference between table and structure in sap: http://www.sapfans.com/forums/viewtopic.php?f=12&t=119794
• if you want to do a quick test: enter values for input parameters (if there are any for your function call), then click “launch” button
• in this example, there is only an optional input parameter max_rows
• you should see data in the output fields
• in this example, you see the record with custname “sap ag” and street “neurottstr. 16”

• click “finish” button
• under “metadata --> sap connections --> “your connection” --> sap functions: there you can now see your function (in this example: bapi_flcust_getlist)
• create sap job
• drag&drop the created function into a job (without the wizard, you also can enter all data by hand)
• tsapinput component is proposed automatically. click ok to add it to your job
• go to “initialize input” and add parameter values
• in this example, there is just the parameter “max_rows”
• hint: the parameter value can be changed from a hardcoded value to a variable, of course (just click control space on your keyboard to get access to all available variables via code completion in your studio)

• go to the tsapinput component and add the desired output mapping (i.e. which values you want to process further with other components
• scroll to the bottom to “outputs”
• add the correct table / structure name (in this example: "customer_list")
• click on mapping (which is empty and has to be filled)

• click on “mapping”, then click on “…”
• add the wanted output columns of your sap function
• add the same names at the column “schema xpathqueries” (do not forget the double quotes here!)
• click “ok” button

• connect the tsapinput component to a tlogrowcomponent and synchronize the schema
• hint: always try out if this works before adding further logic to your job!
• run and test your job (you will see five rows logged (as you have configured max_rows = 5

that's it. now enjoy talend's sap components :-)

best regards,

kai wähner (twitter: @kaiwaehner)

content from my blog: http://www.kai-waehner.de/blog/2013/03/03/sap-integration-with-talend-components-connectors-bapi-rfc-idoc-bw-soap/