Connecting to Microsoft Dynamics 365 for Operations Using Java and Mule

This is the first of a three-part series on connecting to Microsoft Dynamics 365 for finance and operations. In this article, we will understand different integration patterns offered by Dynamics 365 and take a deep dive on using Batch data API — Recurring Integrations. In the second article, I will explain how to use Mulesoft Dynamics 365 connector for Inbound operations.

The following table lists the integration patterns that are available for Finance and Operations.

A synchronous pattern is a blocking request and response pattern where the caller is blocked until the callee has finished running and gives a response. An asynchronous pattern is a non-blocking pattern, where the caller submits the request and then continues without waiting for a response.

A more detailed explanation of different scenarios and integration patterns is given here.

Let's see how we can use Java to invoke REST API exposed by Dynamics 365 Recurring Jobs (Integrations) to push files into their queues. Mule connector for Dynamics 365 falls under the “Select” connector category, meaning enterprise subscription is needed in order to use it in production. If you are using “Community” edition, then this article on using Java will help you connect to Dynamics 365 Operations.

How Does It Work?

At a high level, using Java, we invoke the REST API exposed by Dynamics 365 and send files (CSV, TXT) as HTTP StreamEntity/FileEntity objects. As dynamics recurring jobs are async, we instantly get a 200 Ok response (most of the cases) or 401 Unauthorized or 500 Internal server error. Inside Dynamics 365, behind REST API is basically a queue where all the incoming files are stored/queued. Inside Dynamics 365, we have recurring jobs that are scheduled to run at a specified time (every minute/hour/daily), which pulls these files from the queue to staging area/database for validation/processing before finally moving to a production database.

Authorization

Dynamics authorization is based on Bearer Token (Security/OAuth 2.0 Token).

Before any clients can communicate with Dynamics 365 for operations services, they must be registered in Azure Active Directory (AAD). Adding/Registering a new application (client app) in Azure Active Directory is a simple process. Follow this link — “Adding an application” topic and select “Native” as the application type.

Step 1: We are creating/registering a new client application with application type as Native. We can also select Web app/API, however, is not needed for Java-based connectivity.

Native Client Application — This flow uses a username and password for authentication and authorization. “User Name/Password” that we are using is a service account created just for dynamics connectivity, which is created in Active Directory (AD) just like any organization user and also set up as a user in Dynamics 365 with suitable permissions.

Step 2: Settings — Required Permissions — click the add button to “Select an API.” Here, select Microsoft Dynamics ERP (SAAS for which we are requesting token). Select “Delegated Permissions” and save. This way, everyone in the organization including the service account, which is set up in AD, gets token.

Step 3: Copy Application ID (will look something like: 956118f1-02db-4f0c-a143-ad8a0a63cebf), which is also known as client ID, needs to be set up for every single Dynamics recurring job (Manage authorization policy — of a recurring job).

We have registered a new application by selecting Dynamics 365 ERP (SAAS) as the API endpoint (for which token will be generated) and setting delegated permission so users in the organization AD including our service account can access it with their credentials.

Authorization Code Grant Flow — Native client applications and websites use a flow in which an authorization code represents the resource owner's consent to allow the application to access a resource. The application gets the authorization code from Azure AD and then exchanges it for an access token that provides access to the resource. The application never sees the user's credentials, and the user's agent or browser environment never sees the access token.

Azure Active Directory Authentication Library (ADAL)

The Azure Active Directory Authentication Library (ADAL) enables application developers to authenticate users to cloud or on-premises Active Directory (AD) and obtain tokens for securing API calls.

TenantId — The unique ID of the organization in Azure AD that has granted access for your app. This parameter will only be specified if the customer granted access.

"AADTenant": "ABC1.onmicrosoft.com",

"AzureAuthEndpoint": "https://login.windows.net",

"AADAuthEndpoint": https://login.windows.net/ABC1.onmicrosoft.com”,

"AzureClientId": "956118f1-02db-4f0c-a143-ad8a0a63cebf",

"D365EndPoint": "https://abc-qa.sandbox.operations.dynamics.com",

"Username": "D365_USERNAME",(Service Account)

"Password": "D365_PASSWORD",

"D365Path": "/api/connector/enqueue/C11C79FB-E6BC-1EBD-2B9F-Z50FA1D6EDC6?&"

With TokenService Java class, we can complete Authentication/Authorization and get access/bearer Token

import com.microsoft.aad.adal4j.AuthenticationContext;
import com.microsoft.aad.adal4j.AuthenticationResult;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

@Component
public class TokenService {

    @Value("${abc.activedirectory.userName}")
    private String userName;
    @Value("${abc.activedirectory.password}")
    private String password;
    @Value("${abc.activedirectory.clientId}")
    private String clientId;
    @Value("${abc.activedirectory.authEndPoint}")
    private String authEndPoint;
    @Value("${abc.activedirectory.d365EndPoint}")
    private String d365EndPoint;
    @Value("${abc.activedirectory.d365Path}")
    private String d365Path;

    public String getAccessToken() throws Exception {

        AuthenticationContext context = null;
        AuthenticationResult result = null;
        ExecutorService service = null;

        try {
            service = Executors.newFixedThreadPool(1);
            context = new AuthenticationContext(authEndPoint, false, service);
            Future<AuthenticationResult> future = context.acquireToken(d365EndPoint, clientId, userName,
                    password, null);
            result = future.get();
        } catch (Exception exception) {
        LOGGER.error("Exception in TokenService in getAccessToken : " + exception.getMessage());
        } finally {
            service.shutdown();
        }
        return result.getAccessToken();
    }

    public String getURL() {

    StringBuilder urlBuilder = new StringBuilder(); 
    LOGGER.info("Executing TokenService getURL()");

    urlBuilder.append(d365EndPoint);
    urlBuilder.append(d365Path);

    return urlBuilder.toString();
    }
}

Maven Dependency:

<dependencies>
<!-- Azure Active Directory (AAD) - OAuth/Bearer/Security Token Dependency -->
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>adal4j</artifactId>
<version>1.2.0</version>
</dependency>
<!-- OAuth 2.0 SDK with OpenID Connection extensions -->
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>oauth2-oidc-sdk</artifactId>
<version>5.24</version>
</dependency>
<!-- Apache HttpClient. We use HttpClients, HttpPost class from this library -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.3</version>
</dependency>
<!-- Needed for Spring annotations -->
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
<version>3.2.2</version>
<scope>provided</scope>
</dependency>
</dependencies>

Http Verb: POST
D365 URI: https://abc-dev.sandbox.ax.dynamics.com
Activity Id (Enqueue Job Id): C11C79FB-E6BC-1EBD-2B9F-Z50FA1D6EDC6
Entity Name: Vendors
Company: AV01

Also, we need to send two headers as part of the request:
1.Authorization Header:
a. Name: Authorization
b. Value: Bearer TOKEN_VALUE (Received from above TokenService class)
2.External Correlation:
a. Name: x-ms-dyn-externalidentifier
b. Value: ENQUEUE_FILE_NAME (Actual File Name including the extension.
Example: ABCSupplierList.csv)

With DynamicsFileUploadService Java class, we can send files/data to Dynamics 365 recurring jobs.

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOUtils;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.FileEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.mule.api.MuleEventContext;
import org.mule.api.MuleMessage;
import org.mule.api.lifecycle.Callable;
import org.mule.api.transport.PropertyScope;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

@Component
public class DynamicsFileUploadService implements Callable {

    @Autowired
    private TokenService tokenService;

   public Object onCall(MuleEventContext eventContext) throws Exception {

        LOGGER.info("DynamicsFileUploadService onCall method executing");
        CloseableHttpClient client = null;
        CloseableHttpResponse response = null;
        File tempFile = null;
        String responseString = null;
        String logMessage = "Dynamics Response - Message ID: ";
        String logcode = "; Http Status Response Code: ";
        String logPhrase = "; Response Phrase: ";
        String logOutput = "";
        String responsePhrase = "";
        int responseStatusCode = 0;
        StringBuilder tokenBuilder = null;
        StringBuilder logMessageBuilder = null;
        InputStreamEntity inputStreamEntity = null; 
        BufferedHttpEntity bufferedHttpEntity = null;
        InputStream bufferedInputStream = null;
        HttpPost httpPost = null;

try {

            MuleMessage message = eventContext.getMessage();
            String filename = ResourceConstants.FILE_NAME;

            byte[] fileInputStreamByteArray = message.getPayloadAsBytes();
            client = HttpClients.createDefault();
            httpPost = new HttpPost(tokenService.getURL());

            tokenBuilder = new StringBuilder();
            tokenBuilder.append(ResourceConstants.BEARER);
            tokenBuilder.append(tokenService.getAccessToken().trim());

            httpPost.addHeader(ResourceConstants.AUTHORIZATION, tokenBuilder.toString());
            httpPost.addHeader(ResourceConstants.FILEIDENTIFIER, filename);

            FileOutputStream fileOutputStream = new FileOutputStream(tempFile);
            IOUtils.write(fileInputStreamByteArray, fileOutputStream);

            FileEntity multipart = new FileEntity(tempFile);
            httpPost.setEntity(multipart);

            response = client.execute(httpPost);

            responseString = EntityUtils.toString(response.getEntity(), ResourceConstants.UTF8);

            responseStatusCode = response.getStatusLine().getStatusCode();
            responsePhrase = response.getStatusLine().getReasonPhrase();            

            logMessageBuilder = new StringBuilder();
            logMessageBuilder.append(logMessage);
            logMessageBuilder.append(responseString);
            logMessageBuilder.append(logcode);
            logMessageBuilder.append(responseStatusCode);
            logMessageBuilder.append(logPhrase);
            logMessageBuilder.append(responsePhrase);            

            logOutput = logMessageBuilder.toString(); 

} catch (Exception exception) {
        LOGGER.error("Exception in DynamicsFileUploadService: " + exception.getMessage());
        } finally {
            tempFile.delete();

      if(client != null)
      client.close();
        }

Version 2 — Same code until header. Here we are using InputStreamEntity instead of FileOutputStream. This helps to get rid of creating a temp. file on disk.

inputStreamEntity = new InputStreamEntity(new ByteArrayInputStream(fileInputStreamByteArray), fileInputStreamByteArray.length);
             inputStreamEntity.setContentType(ResourceConstants.BINARY_STREAM);
             inputStreamEntity.setChunked(false);

bufferedHttpEntity = new BufferedHttpEntity(inputStreamEntity);
httpPost.setEntity(bufferedHttpEntity);

The following table lists the possible Dynamics 365 recurring (scheduled data) jobs status values.

I hope you enjoyed reading this article. I'm looking forward to your comments! In the next article, I will explain how to use Mulesoft Dynamics 365 Connector to connect to recurring jobs.