Process Files on the Basis of TimeStamp Attached to File Name in Mule 4

Earlier we were directly polling on one location and getting data in the form of the files. After some transformation, we were sending those data (in XML) to a queue. After that, they came to us saying that they need a solution so that they can control the processing through control-m.

You may also like: Mule 3 to Mule 4 Migrations Tool: Real-Time API

In this process, we designed a solution where we will give them one API where they will send us the request and will start picking the files from that location after receiving the request. In Mule 3 we had to use mule requester component but now in Mule 4, we have a file connector that is capable of doing this.

Another problem was they will put the files in a particular location having naming convention DDMMYYYYHHMMSSfilename. We have to pick the file in descending order of the timestamp attached to the file name.

Another problem was the files that we need to pick from the location were also having some files so first, we have to extract the files and then sent it to the queue for further processing.

In Mule 4 we have a file connector that supports below operations.

1. Copy.
2. Create Directory.
3. Delete.
4. List.
5. Move.
6. Read.
7. Rename.
8. Write.

Below I have attached the flow diagram that I have used to solve the above problem.

In the above flow, we are receiving the request via control-m which will initiate the request to pick files from the given location.

List operation of file connector gives you the capability to get metadata as well as the content of the files in the given directory in the list format. After the list component, I have used one transform component which will sort the files according to timestamp attached to file names. I have used below transformation logic to do that.

After this will check whether the file is having extension zip or not because if the file is having zip extension we need to extract it and then publish it to the queue.

If the file is having zip extension then we are calling another flow ‘extract-And-Publishfiles’.

This flow is having the first component as an extract. This extract operation is from the compression module of Mule 4. It has the following configuration.

Our payload is having a compressed file in binary format. The output of this component will give the list of the object having the key as file name and value as the content of the file.

After this, we have used each to iterate over the output we received from the extract component. For each, I have used the below logic to separate filename and content.

Now we have payload having two keys fileName and value. Since we are only working on XML files so the first transform inside foreach will convert the payload into XML format using the following logic.

Now the last flow that we have used is publish_to_queue.

This flow will be used to publish the transformed XML to a queue.

Foreach is used for iterating over the multiple files we receive after the extraction of zip files.

Further Reading

Mule 4: Hello World

MUnit Testing With Mule 4