Using LoopBack Connectors With IBM App Connect Enterprise

IBM® App Connect Enterprise provides a LoopBackRequest node, which enables you to issue synchronous requests to backend data sources by using LoopBack® connectors such as MongoDB, Cloudant®, or PostgreSQL.

In this article, we provide detailed information on the installation and configuration of the required LoopBack® connectors for use with IBM® App Connect Enterprise in container-based deployment in Cloud Pak for Integration. We will primarily cover Cloudant and CouchDB databases; however, a similar technique can be applied to other databases like MongoDB and PostgreSQL.

Prerequisites:

A cloud infrastructure with OpenShift installed and Cloud Pak for Integration.

Environment:

CP4I 2021.2.1.

Build an ACE Custom Image With Loopback Connectors

First, you need to create a custom ACE container image by installing the required LoopBack connectors. Following is a sample Dockerfile that shows how to install Cloudant and CouchDB LoopBack connectors on top of the base ACE server image.

FROM cp.icr.io/cp/appc/ace-server-prod@sha256:fad57087e71d4630e9321ea9f913f8d3bc7ab41ae6287ac1d8fb69bafab23159
USER root
RUN export PATH=$PATH:/opt/ibm/ace-11/common/node/bin \
    && cd /opt/ibm/ace-11/node_modules \
    && npm install loopback-connector-cloudant --save \
    && npm install loopback-connector-couchdb2 --save \
    && chown -R aceuser:mqbrkrs /opt/ibm/ace-11/node_modules
USER 1000

The directory /opt/ibm/ace-11/ in the example above is with reference to the use of an ACE V11 based image. For ACE V12, this path will change to /opt/ibm/ace-12.
 

Now build the custom ACE image using the below Dockerfile:

docker build -t ace-lb:latest -f <Dockerfilename> .

(The last period is important.)

Once the image is built, push it to your OpenShift repository.

HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}')
docker login -u $(oc whoami) -p $(oc whoami -t) $HOST
docker tag ace-lb $HOST/cp4i-ace/ace-lb:latest
docker push $HOST/cp4i-ace/ace-lb:latest

Note: If you want to use your own registry, then make sure the credentials to access your registry are added to the cluster’s secret.

At this point, you should have an image in the OCP Registry that you can use to deploy an Integration Server with the BAR file that contains the message flow with the LoopBack connector.

We now have the ACE image with the required LoopBack connector modules to work with Cloudant and CouchDB databases. Let's look at the details on how to develop message flows with a LoopBack request node and deploy them to the ACE Integration Server in CP4I.

Cloudant Database

In this section, we will see the configuration steps for the LoopBack connector for the Cloudant database.

Step 1: Develop the Integration Flow With LoopBackRequest Node

Below is an example of a simple flow developed for creating a record in the Cloudant database.

HTTP Request --> LoopBackRequest node --> HTTP Reply

Step 2: Prepare Configuration Objects

To deploy to the CP4I Container environment, you will need to inject a certain configuration to the Integration Server at the time of deployment — for example: userid/password for database access, datasource.json file, model definition, etc.

First, create a setdbparms definition:

 mqsisetdbparms  -n loopback::CLOUDANT_SEC_ID -u <username> -p <password>

Note the value for security ID, CLOUDANT_SEC_ID, should be the same as what is configured in the message flow node property.

Then, create a LoopBack connection definition. You need to upload the file for datasource and any models defined as a zip. A sample datasource.jsonfile is shown below for Cloudant.

{
"CLOUDANT":  {
"database": "share_price”,
"name": "CLOUDANT",
"host": "<Host name of Cloudant>",
"port": 443,
"connector": "cloudant"
 }
}    

Finally, package this datasource.json and models (if any) into a zip file.

Step 3: Deployment

• Prepare artifacts for deployment to the CP4I cluster.
  • Create a BAR file for the message flow from Step 1.
  • Keep the Cloudant config objects ready as per Step 2.
• Keep the custom Docker image link created handy.
• Login to the ACE Dashboard and click on Create Server.
• Select Toolkit Flow and click next.
• Upload the BAR file and click next.
• Create configurations.
• Create a setdbparms.txt config – paste the mqsisetdbparms command that you prepared in Step 2 in the text box, as shown in the figure below.
  
• Create one more configuration of type ‘LoopBack Datasource.’ You need to upload the file for datasource and any models defined as a zip that you prepared in Step 2.
  
• Select both the configurations and click next.
  
• Give the server a name, turn on the advanced config toggle on the left, select Runtime containers and provide the link of the custom ACE image that you created in Step 1 (make sure the secret to download the image from artifactory is already part of the secrets of the cluster).
  For non-UI-based deployed, set this value to change the image location: cr.Spec.Pod.Containers.Runtime
  
• Click Create and wait for the integration server to be ready. 
• Inspect the logs tab of Integration Server POD to see the config applied and flow connected to datasource.
  Java

   

  2021-08-13T16:31:18.818Z Setup configuration files - configuration names: lb-ds,lb-setdb,is-01-toolkit-is-adminssl 
  2021-08-13T16:31:18.843Z Construct a configuration on the filesystem - configuration name: 
  lb-ds type: loopbackdatasource
  2021-08-13T16:31:18.868Z Construct loopback connector files on the filesystem
   
  2021-08-13T16:31:18.872Z Construct a configuration on the filesystem - configuration name: 
  lb-setdb type: setdbparms
  2021-08-13T16:31:18.872Z Execute mqsisetdbparms command
  
  An http endpoint was registered on port '7800', path '/LoopBack_Cloudant'.
  2021-08-13 17:33:59.186440: BIP3132I: The HTTP Listener has started listening on port '7800'
  for 'http' connections.
  2021-08-13 17:33:59.186536: BIP1996I: Listening on HTTP URL '/LoopBack_Cloudant'.
  Started native listener for HTTP input node on port 7800 for URL /LoopBack_Cloudant
  2021-08-13 17:33:59.261612: BIP2269I: Deployed resource 'Loopback_Cloudant' 
  (uuid='Loopback_Cloudant',type='MessageFlow')started successfully.
  
  2021-08-13T17:33:59.267Z Integration server not ready yet
  ..2021-08-13 17:34:02.772950: BIP2866I: IBM App Connect Enterprise administration security is
  authentication, authorization file.
  2021-08-13 17:34:02.789816: BIP3132I: The HTTP Listener has started listening on port '7600'
  for 'RestAdmin http' connections.
  . 
  2021-08-13 17:34:02.863875: BIP1991I: Integration server has finished initialization.
  2021-08-13T17:34:04.279Z Integration server is ready

  
  

  Once POD is up, log in to the terminal and inspect the files datasource and setdbparams secret.
  

Step 4: Testing the Message Flow To Insert a Record Into Cloudant DB

• Get the HTTPS route to IS POD to invoke the HTTP request of the flow.
  
  Java

   

  curl --location --request POST 'http://test-lb-http-ace-lb.apps.acecc-shared-46-eus-2.cp.fyre.ibm.com/LoopBack_Cloudant' \--header 'Content-Type: application/json' \ --data-raw '{"companyID":"102","company":"IBM UK Ltd","price":198}'

• Verify record created in Cloudant. Open the Database URL in the browser and check if the record is created. https://<cloudant-host>/dashboard.html#/database/share_price/_all_docs
  

CouchDB Database

• Create a sample flow with HTTPInput --> LoopBackRequestNode --> HTTPReply as shown below.
  
• Set the URL: testcouchdbcreate and Input Message Parsing --> Message domain: JSON on HTTPINPUT NODE properties as shown below.
  
• Provide the below details in LoopBack request node Properties
  mycouchdb -> datasource name in the datasources.json
  LoopBack object -> name collection in the database
  Operation -> the operation like create, retrieve, update, delete we need to do. Eg: Create.
• As explained in the above section (Step 2), create datasources.json for the CouchDB database.
  
• Explanation of the fields in the above datasource definition:
  Java

   

  mycouchdb -> datasource name in the loopbackrequest node
  couchdb2 -> connector used to connect to CouchDB
  url -> url to connect to couch db
  username -> username to connect to CouchDB
  password -> password to connect to CouchDB
  Hostip -> host ip we are trying to connect where CouchDB is installed.  (127.0.0.1)
  Port -> default port used to connect to CouchDB.   (5984)
  Database_name -> Name of the couchdb database name. (testcouchdb)

• Follow the same process for deployment as per Step 3 in the previous section. Upon successful deployment, invoke the message flow using the route URL. For example: http://<route>/testcouchdbcreate in REST Client.
  
• Open the Database URL in the browser and check if the collection is created.
  http://127.0.0.1:5984/_utils/#database/testcouchdb/_all_docs