DataWeave code can be difficult to debug for those that are new to the language. This is especially true for those who are also new to the functional programming paradigm. You probably know by now that you cannot use the Mule debugger to step into DataWeave code and run it line by line, which makes the problem of how to debug DataWeave code even worse.
Luckily, there are a few properties inherent in functional languages like DataWeave that can make them easier to debug and reason about. Using those properties, we can break up our code and make our DataWeave scripts easier to debug and understand. Lastly, the DataWeave language comes with some utilities that make the lack of debugger less of an issue.
Refactoring using %var
and %function
%var
and %function
Most DataWeave code that I read uses none of the organizational features of the language. Two of these features,%var
and%function
, are powerful tools for refactoring your code so that it is less repetitive, more easily understood by others, and easier to debug. Here’s a piece of code that I see quite often:
Repetitive Code
%dw 1.0
%output application/java
---
payload map {
firstName: $.firstName unless $.firstName == null otherwise "",
lastName: $.lastName unless $.lastName == null otherwise ""
}
We have a repetitive pattern here:<field>
unless<field> != null otherwise "
. If we ignore the fact that we can get this functionality with the default operator (i.e.default ""), we can refactor this repetitive code into a function so we can DRY out the code, and get consistent behavior whenever we want to do a null check and default a value. We also get the advantage of naming the behavior (i.e. naming the function) so that its intent is more obvious:
Refactored Code
%dw 1.0
%output application/java
%function defaultIfNull(field, defaultValue) field unless field !=
null otherwise defaultValue
---
payload map {
firstName: defaultIfNull($.firstName, ""),
lastName: defaultIfNull($.lastName, "")
}
%var
is great for giving meaning to things that don’t currently have a descriptive name. For example, in the DataWeave code above we have payload. When I see a code like this I ask myself: payload of what? What does the payload represent? This is probably more of a personal preference, but I find that I like using%function
to name behavior, giving meaning to values that don’t currently have a descriptive name makes the intent of the code more obvious. I’d write the above code like this instead:
Using %var
to Give Meaning
%var
to Give Meaning
%dw 1.0
%output application/java
%var people = payload
%function defaultIfNull(field, defaultValue)
field unless field != null otherwise defaultValue
---
people map {
firstName: defaultIfNull($.firstName, ""),
lastName: defaultIfNull($.lastName, "")
}
Taking Advantage of Referential Transparency
%var
is also great for storing intermediate values during longer calculations. For example, let’s say we have the following DataWeave script:
Confusing Mess
%dw 1.0
%output applications/java
---
(flatten (payload.products map $.availabilities)) map $.region filter
$.id == 1
It can be a bit hard to understand what’s going on here, and it’s going to be even more difficult to determine how small changes in the code will affect the output. We can use %var
to break up this long chain of expressions into smaller expressions. We can then use the debugger or a logger after the DataWeave transformer to view what these expressions return.
Refactored using %var
%var
%dw 1.0
%output applications/java
%var availablilities = flatten (payload.products map $.availabilities)
%var regions = availabilities map $.region
%var wantedRegions = regions filter $.id == 1
---
{
// Test if availabilities returns what we expected
availabilities: availabilities,
//Test if regions returns what we expected
regions: regions,
//Test if wantedRegions returns what we expected
wantedRegions: wantedRegions
}
With the code this way, it is incredibly easy to identify what part of the code might be incorrect, as each piece is isolated and named. If you’re paying close enough attention, or are familiar with other functional languages, you may have noticed that the expressionflatten (payload.products map $.availabilities) map $.region
andavailabilities map $.region
return the same thing whenavailabilities
in the second expression is set to the appropriate value (i.e. what is returned from the expressionflatten (payload.products map $.availabilities
)). With functional languages like DataWeave, you’re free to make these substitutions because DataWeave expressions are referentially transparent. All of the repercussions of this are outside the scope of this article, but for now you can know that “an expression is said to be referentially transparent if it can be replaced with its corresponding value without changing the program’s behavior” (according to Wikipedia at the time of writing of this article). Use this principle to your advantage when debugging and writing DataWeave code.
Using log
to Trace Execution
log
to Trace ExecutionDataWeave has a function called log
(documentation here) that takes a string, and value or expression as input. As output, it will log to the console, the string, with the value or result of the expression appended, and return the value of the expression. Here’s a simple example:
log
Expression Example
log
Expression Example
%dw 1.0
%output applications/java
---
log("1 + 2 = ", 1 + 2)
This script will set the payload of the message to3
and will log the following to the console:1 + 2 = 3
. In the last example of the previous section we could’ve used log instead of setting the payload to determine if the values were correct. Here’s a refactored example:
Refactored substitution example using log
log
% dw 1.0
%output applications/java
%var availabilities = log("availabilities is: ", flatten
(payload.products map $.availabilities))
%var regions = log("regions is: ", availabilities map $.region)
%var wantedRegions = log("wantedRegions is", regions filter $.id ==
1)
---
wantedRegions
Which would log the appropriate values to the console when executing the script.
In the above example, the use oflog
was pretty much optional. You could’ve gone with our initial example, or the one refactored forlog
, and been happy either way, but this isn’t always the case. You’ve probably noticed that DataWeave doesn’t have imperative looping constructions like Java’sfor
, enhancedfor
, andwhile
loops. Looping in functional languages is typically accomplished with recursion, and DataWeave is no exception. Let’s say you wanted to split a string into n-sized chunks, all contained in an array. We can write a recursive solution for this. Here’s a working solution:
Recursion Example
% dw 1.0
%output applications/java
%var string = 'HowAreYouFa'
%function chunkStr(str, chunkSize) doChunkStr(str, chunkSize, [])
%function doChunkStr(str, chunkSize, arr)
arr when ((sizeOf str) == 0)
otherwise do ChunkStr(
str[chunkSize to -1]
unless ((sizeOf str) < chunkSize)
otherwise "",
chunkSize,
arr ++ ([str[0 to (chunkSize – 1)]]
when (sizeOf str) > chunkSize
otherwise [str]))
---
chunkStr (string, 3)
Which returns['How', 'Are', 'You', 'Fa']
. When I was developing this solution, I had a difficult time getting the recursion to work the way I wanted it to; I’m certainly more familiar with the imperative looping constructs offered by Java. While making modifications to determine the solution, sometimes I would get backnull
, sometimes I would get back['How', 'Are', 'You', 'null']
. To a programmer with a lot of experience using recursion, these outputs might point to obvious solutions. But for me, without tracing the solution on paper, which I really didn’t want to do, I didn’t see a lot of options for debugging my code. After all, DataWeave is not imperative, you can’t just throw in aSystem.out.println()
call in the middle of the function like you can in Java. It turns out thatlog
was the solution to this problem. Here’s the refactored recursion example usinglog
to output intermediate values as it reaches the final array.
Debugging a Recursive Function
%dw 1.0
%output applications/java
%var string = 'HowAreYouFa'
%function chunkStr(str, chunkSize) doChunkStr(str, chunkSize, [])
%function doChunkStr(str, chunkSize, arr)
arr when ((sizeOf str) == 0)
otherwise doChunkStr(
log("str is: ", str[chunkSize to -1]
unless ((sizeOf str) < chunkSize)
otherwise ""),
log("chunkSize is: ", chunkSize(,
log("arr is: ", arr ++ ([str[0 to (chunkSize – 1)]]
when (sizeOf str) > chunkSize
otherwise [str])))
---
chunkStr(string, 3)
The console output looked like this (formatted for clarity). This display enabled me to easily see where things were going wrong in the recursive loop, or in this case, that things were going according to plan:
Console Output
Str is: - "AreYouFa"
chunkSize is: - 3
arr is: - [
"How"
]
str is: - "YouFa"
chunkSize is: -3
arr is: -[
"How",
"Are"
]
str is: -"Fa"
chunkSize is: - 3
arr is: -[
"How",
"Are",
"You"
]
str is: -""
chunkSize is: - 3
arr is: - [
"How",
"Are",
"You",
"Fa"
]
I hope this article gave you some good insights into how you can organize your DataWeave, take advantage of referential transparency, and use log to making debugging DataWeave code a lot easier.
About the Author
Joshua Erney has been working as a software engineer at Mountain State Software Solutions (ArganoMS3) since November 2016, specializing in APIs, software integration, and MuleSoft products.
This is the best ever blog I have ever read on dataweave in 3 years of my mulesoft’s journey !
Thank you so much I have seen your all blogs and also looking forward with your knowledge I am currently started my career as a mulesoft Developer so will you please guide me? I am sending my Email Address