Tips on Debugging DataWeave Code

DataWeave code can be difficult to debug for those that are new to the language. This is especially true for those who are also new to the functional programming paradigm. You probably know by now that you cannot use the Mule debugger to step into DataWeave code and run it line by line, which makes the problem of how to debug DataWeave code even worse.

Luckily, there are a few properties inherent in functional languages like DataWeave that can make them easier to debug and reason about. Using those properties, we can break up our code and make our DataWeave scripts easier to debug and understand. Lastly, the DataWeave language comes with some utilities that make the lack of debugger less of an issue.

Refactoring using %var and %function

Most DataWeave code that I read uses none of the organizational features of the language. Two of these features, %var and %function, are powerful tools for refactoring your code so that it is less repetitive, more easily understood by others, and easier to debug. Here’s a piece of code that I see quite often:

Repetitive Code

%dw 1.0
%output application/java
---
payload map {
  firstName: $.firstName unless $.firstName == null otherwise "",
  lastName:  $.lastName  unless $.lastName  == null otherwise ""
}

We have a repetitive pattern here: <field> unless <field> != null otherwise ". If we ignore the fact that we can get this functionality with the default operator (i.e.  default ""), we can refactor this repetitive code into a function so we can DRY out the code, and get consistent behavior whenever we want to do a null check and default a value. We also get the advantage of naming the behavior (i.e. naming the function) so that its intent is more obvious:

Refactored Code

%dw 1.0
%output application/java
%function defaultIfNull(field, defaultValue) field unless field !=
null otherwise defaultValue
---
payload map {
  firstName: defaultIfNull($.firstName, ""),
  lastName:  defaultIfNull($.lastName,  "")
}

%var is great for giving meaning to things that don’t currently have a descriptive name. For example, in the DataWeave code above we have payload. When I see a code like this I ask myself: payload of what? What does the payload represent? This is probably more of a personal preference, but I find that I like using %function to name behavior, giving meaning to values that don’t currently have a descriptive name makes the intent of the code more obvious. I’d write the above code like this instead: 

Using %var to Give Meaning

%dw 1.0
%output application/java

%var people = payload

%function defaultIfNull(field, defaultValue) 
  field unless field != null otherwise defaultValue
---
people map {
  firstName: defaultIfNull($.firstName, ""),
  lastName:  defaultIfNull($.lastName,  "")
}

Taking Advantage of Referential Transparency

%var is also great for storing intermediate values during longer calculations. For example, let’s say we have the following DataWeave script:

Confusing Mess

%dw 1.0
%output applications/java
---
(flatten (payload.products map $.availabilities)) map $.region filter
$.id == 1

It can be a bit hard to understand what’s going on here, and it’s going to be even more difficult to determine how small changes in the code will affect the output. We can use %var to break up this long chain of expressions into smaller expressions. We can then use the debugger or a logger after the DataWeave transformer to view what these expressions return.

Refactored using %var

%dw 1.0
%output applications/java

%var availablilities = flatten (payload.products map $.availabilities)


%var regions = availabilities map $.region

%var wantedRegions = regions filter $.id == 1
---

{
  // Test if availabilities returns what we expected
  availabilities: availabilities,

  //Test if regions returns what we expected
  regions: regions,

  //Test if wantedRegions returns what we expected
  wantedRegions: wantedRegions
}

With the code this way, it is incredibly easy to identify what part of the code might be incorrect, as each piece is isolated and named. If you’re paying close enough attention, or are familiar with other functional languages, you may have noticed that the expression flatten (payload.products map $.availabilities) map $.region and availabilities map $.region return the same thing when availabilities in the second expression is set to the appropriate value (i.e. what is returned from the expression flatten (payload.products map $.availabilities)). With functional languages like DataWeave, you’re free to make these substitutions because DataWeave expressions are referentially transparent. All of the repercussions of this are outside the scope of this article, but for now you can know that “an expression is said to be referentially transparent if it can be replaced with its corresponding value without changing the program’s behavior” (according to Wikipedia at the time of writing of this article). Use this principle to your advantage when debugging and writing DataWeave code.

Using log to Trace Execution

DataWeave has a function called log (documentation here) that takes a string, and value or expression as input. As output, it will log to the console, the string, with the value or result of the expression appended, and return the value of the expression. Here’s a simple example:

log Expression Example

%dw 1.0
%output applications/java
---
log("1 + 2 = ", 1 + 2)

This script will set the payload of the message to 3 and will log the following to the console: 1 + 2 = 3. In the last example of the previous section we could’ve used log instead of setting the payload to determine if the values were correct. Here’s a refactored example:

Refactored substitution example using log

% dw 1.0
%output applications/java

%var availabilities = log("availabilities is: ", flatten
(payload.products map $.availabilities))

%var regions =        log("regions is: ", availabilities map $.region)

%var wantedRegions =  log("wantedRegions is", regions filter $.id ==
1)
---
wantedRegions

Which would log the appropriate values to the console when executing the script.

In the above example, the use of log was pretty much optional. You could’ve gone with our initial example, or the one refactored for log, and been happy either way, but this isn’t always the case. You’ve probably noticed that DataWeave doesn’t have imperative looping constructions like Java’s for, enhanced for, and while loops. Looping in functional languages is typically accomplished with recursion, and DataWeave is no exception. Let’s say you wanted to split a string into n-sized chunks, all contained in an array. We can write a recursive solution for this. Here’s a working solution:

Recursion Example

% dw 1.0
%output applications/java

%var string = 'HowAreYouFa'

%function chunkStr(str, chunkSize) doChunkStr(str, chunkSize, [])

%function doChunkStr(str, chunkSize, arr)
  arr when ((sizeOf str) == 0)
	otherwise do ChunkStr(
		    str[chunkSize to -1] 
		      unless ((sizeOf str) < chunkSize)
		      otherwise "",
		    chunkSize,
		    arr ++ ([str[0 to (chunkSize – 1)]] 
		      when (sizeOf str) > chunkSize 
		      otherwise [str]))
---
chunkStr (string, 3)

Which returns ['How', 'Are', 'You', 'Fa']. When I was developing this solution, I had a difficult time getting the recursion to work the way I wanted it to; I’m certainly more familiar with the imperative looping constructs offered by Java. While making modifications to determine the solution, sometimes I would get back null, sometimes I would get back ['How', 'Are', 'You', 'null']. To a programmer with a lot of experience using recursion, these outputs might point to obvious solutions. But for me, without tracing the solution on paper, which I really didn’t want to do, I didn’t see a lot of options for debugging my code. After all, DataWeave is not imperative, you can’t just throw in a System.out.println() call in the middle of the function like you can in Java. It turns out that log was the solution to this problem. Here’s the refactored recursion example using log to output intermediate values as it reaches the final array.

Debugging a Recursive Function

%dw 1.0
%output applications/java

%var string = 'HowAreYouFa'

%function chunkStr(str, chunkSize) doChunkStr(str, chunkSize, [])

%function doChunkStr(str, chunkSize, arr)
  arr when ((sizeOf str) == 0)
	otherwise doChunkStr(
		       log("str is: ", str[chunkSize to -1]
		         unless ((sizeOf str) < chunkSize)
		         otherwise ""),
		       log("chunkSize is: ", chunkSize(,
		       log("arr is: ", arr ++ ([str[0 to (chunkSize – 1)]]
		         when (sizeOf str) > chunkSize
		         otherwise [str])))
		    
---
chunkStr(string, 3)

The console output looked like this (formatted for clarity). This display enabled me to easily see where things were going wrong in the recursive loop, or in this case, that things were going according to plan:

Console Output

Str is: - "AreYouFa"
chunkSize is: - 3
arr is: - [
  "How"
]

str is: - "YouFa"
chunkSize is: -3
arr is: -[
  "How",
  "Are"
]

str is: -"Fa"
chunkSize is: - 3
arr is: -[
  "How",
  "Are",
  "You"
]

str is: -""
chunkSize is: - 3
arr is: - [
  "How",
  "Are",
  "You",
  "Fa"
]

I hope this article gave you some good insights into how you can organize your DataWeave, take advantage of referential transparency, and use log to making debugging DataWeave code a lot easier.

About the Author

Joshua Erney has been working as a software engineer at Mountain State Software Solutions (ArganoMS3) since November 2016, specializing in APIs, software integration, and MuleSoft products.