DataWeave Use Case

This blog is all about a use case that has some complex transformation and complex data structure. Here, we have an input JSON sample and the expected JSON sample.

Muralidhar Gumma

Jan. 19, 24 · Code Snippet

Likes (1)

Comment

Save

2.6K Views

Hi Muleys,

In this blog, I am sharing my experience with a use case that has some complex transformations and complex data structures.

Let's see the transformation for complex data structures (made easy and simple for the article) and data code for achieving the expected output.

Input Data

    JSON
   
 

   {
    "data" : [
        {
            "Employee Department" : "Marketing" ,
            "Country Name" : "India" ,
            "Employee Details" : {
                "Full Name" : "K Ram Kumar" ,
                "Age" : "34" ,
                "Id" : "abc123Ind"
            } ,
            "Employee Status" : "Active" ,
            "Start Year" : "2008" ,
            "End Year" : ""
        } ,
        {
            "Employee Department" : "Finance" ,
            "Country Name" : "United States" ,
            "Employee Details" : {
                "Full Name" : "John Jim Martin" ,
                "Age" : "34" ,
                "Id": "123abcUS"
            } ,
            "Employee Status" : "Inactive" ,
            "Start Year" : "2008" ,
            "End Year" : "2011"
        } ,
        {
            "Employee Department" : "Finance" ,
            "Country Name" : "United States" ,
            "Employee Details" : {
                "Full Name" : "Tom Tim David" ,
                "Age" : "35" ,
                "Id" : "123xyzUS"
            } ,
            "Employee Status" : "Active" ,
            "Start Year" : "2008" ,
            "End Year" : ""
        } ,
        {
            "Employee Department" : "Marketing" ,
            "Country Name" : "India" ,
            "Employee Details" : {
                "Full Name" : "M Mohan Krishna" ,
                "Age" : "36" ,
                "Id" : "xyz123Ind"
            },
            "Employee Status" : "Inactive" ,
            "Start Year" : "2008" ,
            "End Year" : "2015"
        }
    ]
} 
  

Requirements for Expected Output

If we have the status as "Inactive" and from the "Finance" department, it should create two objects. 1st with the start year and 2nd with the end year.
If we have the status as "Inactive" and from the "Marketing" department, it should create one object with all details w/o the start or end year.
If we have the status as "Active" and from the "Finance" department, it should create one object with all details with the start year.
If we have the status as "Active" and from the "Marketing" department, it should create one object with all details w/o the start or end year.
All the data should be at the same level, i.e., in one array, irrespective of different structures.

Expected Final Output

    JSON
   
 

   {
  "employeeData" : [
    {
      "empName" : "John Jim Martin" ,
      "empId" : "123abcUS" ,
      "age" : "34" ,
      "country" : "United States" ,
      "department" : "Finance" ,
      "startYear" : "2008"
    } ,
    {
      "empName" : "John Jim Martin" ,
      "empId" : "123abcUS" ,
      "age" : "34" ,
      "country" : "United States" ,
      "department" : "Finance" ,
      "endYear" : "2011"
    } ,
    {
      "empName" : "M Mohan Krishna" ,
      "empId" : "xyz123Ind" ,
      "age" : "36" ,
      "country" : "India" ,
      "department" : "Marketing"
    } ,
    {
      "empName" : "K Ram Kumar" ,
      "empId" : "abc123Ind" ,
      "age" : "34" ,
      "country" : "India" ,
      "department" : "Marketing"
    } ,
    {
      "empName" : "Tom Tim David" ,
      "empId" : "123xyzUS" ,
      "age" : "35" ,
      "country" : "United States" ,
      "department" : "Finance" ,
      "startYear" : "2008"
    }
  ]
}
  

Observation From the Expected Output

From the given expected output, we can clearly see below points

We have all the data at the same level in one array, irrespective of different structures.
We have one extra object for the end year in comparison to source input and expected output data.

DataWeave Code

    JSON
   
 

   %dw 2.0
output application/json
// Filter Objects with "Inactive" Status
var a = payload.data filter ( $."Employee Status" == "Inactive" )

// With variable "a" as input, filter and map Objects related to "Finance Department"
var b = a filter ( $."Employee Department"  == "Finance" ) map (( item, index ) -> [ { "empName" : item."Employee Details"."Full Name" , "empId" : item."Employee Details".Id , "age" : item."Employee Details".Age ,"country" : item."Country Name" , "department" : item."Employee Department" , "startYear" : item."Start Year"} , { "empName" : item."Employee Details"."Full Name" , "empId" : item."Employee Details".Id , "age" : item."Employee Details".Age , "country" : item."Country Name" ,"department" : item."Employee Department" , "endYear" : item."End Year" }])

//With variable "a" as input, filter and map Objects related to "Marketing Department"
var c = a filter ( $."Employee Department"  == "Marketing" ) map (( item, index ) -> {"empName" : item."Employee Details"."Full Name" , "empId" : item."Employee Details".Id , "age" : item."Employee Details".Age , "country" : item."Country Name" , "department" : item."Employee Department" })

//From original payload or Input Data, filter objects with status "Inactive" and map the objects
var d = payload.data filter ( $."Employee Status" != "Inactive" ) map (( item, index ) ->
if( item."Employee Department" == "Finance" ) { "empName" : item."Employee Details"."Full Name" ,"empId" : item."Employee Details".Id ,"age" : item."Employee Details".Age , "country" : item."Country Name" , "department" : item."Employee Department" , "startYear" : item."Start Year" } else  { "empName" : item."Employee Details"."Full Name" , "empId" : item."Employee Details".Id ,"age" : item."Employee Details".Age ,"country" :item."Country Name" , "department" :item."Employee Department" } )

---
//From all the variable , transform the final expected output
{ "employeeData": (if(flatten(b) != null)(flatten(b)) else [{"error": "Data not found"}])  ++ (if(c != null) c else [{"error": "Data not found"}] ) ++ (if(d != null)d else [{"error": "Data not found"}] ) -- [{"error": "Data not found"}]}        
  

Point To Be Noted

All the data was handled in the same transform message.
To make it more precise / better understood, local variables were created, and all were called in the end.
1. Local variable "a" filter objects with "Inactive" status.
2. The local variable "b" takes the output of variable "a" as input, filters, and maps Objects related to the "Finance Department."
3. The local variable "c" takes the output of variable "a" as input, filters, and maps Objects related to the "Marketing Department."
4. Local variable "d" filters objects with the status "Inactive" and maps the objects from the original payload or Input Data.
All the local variables "a," "b," "c," and "d" transform the final expected output.
The "Flatten" operation was used to meet requirement 1, i.e., printing at the same level as other objects. Flatten operation converts an array of arrays to a single array.
Error was handled in processing part in same component so as to increase the performance and quick response.
In the end, an error message was removed so the expected output doesn't showcase the extra/error fields if any condition doesn't match.

Conclusion

As a MuleSoft Developer/Arch, the above illustration may or may not comply with your requirement, but it can enlighten new ways of writing the data codes.

More use cases to come.

Happy Learning!!

Data structure

Opinions expressed by DZone contributors are their own.

Related

Trending