So, you decided that you would like to explore some of the other frameworks in the Java space for RESTful API implementation. Your company currently has MuleSoft implemented across the entire division, but you’ve been doing some research and came across the ArganoMS3 Camel Accelerator Archetypes. I’ll be your guide through a proof of concept conversion, which you can follow along with on an API of your own. All of the resources used are in the public domain and fully accessible online.

Throughout this guide we will discuss:

Converting our RAML to an OAS 3.0 document

Right out of the gate into a new realm of API development, we will need to migrate the proprietary RAML used in Mulesoft APIs to an industry-standard format. The file in particular we need is an OpenAPI Specification (OAS), formally known as “swagger”. This specification is a means to describe all the resources and sub-resources available to obtain from an underlying API. Traditionally these specs also include detailed examples of response data on GET and a complete representation of the incoming data accepted for POST. As you complete this more often, the process becomes more mainstream, and you can get through a few Small -> Medium specifications per day.

Starting inside the Anypoint online portal navigate to the Exchange:

Within the Mulesoft Anypoint online website, you can enter into your company’s Exchange to access a list of all the APIs available within your company and begin by selecting the API you’re tasked to convert. The example showcases a process API, which we then opted to download. Here in the download sub-menu, MuleSoft recently added an option that attempts to generate an OAS from a set of pre-configured mappings between the two. The downside to this is the feature is only designed to output version swagger 2.0. Once you have selected the OAS option under download, you will see the browser download the contained zip folder shown below.

After completing the download and extraction of the swagger 2.0 specification, the real conversion begins. For starters, we need to convert the specification version from 2.0 to 3.0. I would also recommend learning about the OpenAPI Specification syntax at this time if you have never been exposed to this language before. The route I take for this conversion is to use our friend Google. I know the tooling for the syntax update by heart, but I prefer to work smarter and not harder for time-consuming purposes. A recent resource which has served me well is Mermade Swagger 2.0 to OpenAPI 3.0.0 converter (https://github.com/Mermade/oas-kit), which both performs an almost perfect upgrade on the syntax and converts JSON to YAML. While the newly generated OAS 3.0 specification is near completion, I would highly recommend going back over this document with a fine-tooth comb. Areas that typically need addressing include confirming the following: description tags are exported as x-amf-displayName, response data types are */* instead of an individual one, response codes are not globalized in components, and the OAS3.0 tag for anyOf or oneOf are displayed as x-amf-union. After completing your review, it never hurts to lint your file using swagger.io for personal use, or more industrially options like the Insomnia CLI tool (https://insomnia.rest) provides a command bringing this platform into a build pipeline like Jenkins or Github Actions.

Utilizing the ArganoMS3 Camel Accelerator Archetypes

The next step in the migration from MuleSoft to Camel is facilitated by the ArganoMS3 Camel Accelerator Archetype, which will allow us to completely generate a skeleton of each endpoint described in the specification. Start by placing the YAML file we just created into a folder of its own, that is if the file is just sitting out in the /Downloads or /Documents folders. When we generate the Maven project it will create our project folder in the same directory as the OAS.

Next is our Maven command that performs the project generation:

mvn archetype:generate \
-DarchetypeGroupId=com.ms3-inc.tavros \
-DarchetypeArtifactId=camel-openapi-archetype \
-DarchetypeVersion=0.2.7 \
-DgroupId=com.vikingsasquatch.camel \
-Dversion=0.1.0-SNAPSHOT \
-DartifactId=process-api-ac \
-DspecificationUri=./process-api-oas.yaml

Looking at the command above, you can see we will need to have Maven installed on your developer machine. ArganoMS3 developed this tool to quickstart development following API-first design. Before implementing, it would be helpful to establish the contract beforehand and check out Understanding the API-First Approach to Building Products (https://swagger.io/resources/articles/adopting-an-api-first-approach/). Still referring to the above command, the archetypeGroupId, archetypeArtifactId, and archetypeVersion should be the latest, which can be confirmed via Maven Central, the most up-to-date place to check (https://search.maven.org/artifact/com.ms3-inc.tavros/camel-openapi-archetype). ArganoMS3 also makes this OpenSource and provides an additional artifact for generating a configuration that executes a route based on a cron job, viewable on MS3Inc Github repo (https://github.com/MS3Inc/camel-archetypes). The groupId, artifactId, and version should be set as your company is standard for Maven projects. Lastly, the specificationUri should be configured like above to reference the file we just created and moved into our directory.

On a successful build, you will see the standard Maven BUILD SUCCESS message along with output from the archetype mentioning it parsed the spec and added endpoints to both RoutesImplementation and RoutesGeneration. If you now grab a list of the contents, you should see a new folder created resembling the same name as the artifactId you provided, where you will find a POM, Dockerfile and a src folder. This structure might be familiar if you have ever created a Spring-Boot project using Spring Initializer (https://start.spring.io/).

In the following steps, we will demonstrate how to get started with development in Camel. If you have not already, start your IDE and import our newly created project as a Maven project from the filesystem. This process depends on your development environment. Here are two example articles that will help:

Migrating the Mulesoft EIP to Camel

Understanding a Camel producers and consumers

The Camel enterprise service bus, similar to an event in Mule, begins its journey in the RoutesGenerated.

In Mule, we use the APIKit Router to perform request validation, and a huge part of using the Mule platform is removing boilerplate for functionality like that. The ArganoMS3 Camel Archetype includes an OpenAPI Validator that understands every CRUD operation. In the above example image, we can see the processor on line 39. The rest() function binds to an operation and path. We generate this from a specification to avoid tooling by hand. In the Camel platform, we use consumers and producers to manage the flow of processing. Referring to the image above again, this RoutesGenerated is the initial consumer. The matching of the rest route is performed then the exchange is sent by the to() producer to a direct() consumer with mapping carried out by string names.

At this time, the validation concluded the request is good so, next our request meets the rest router and splits for it to be delivered to the proper processing route. All routes generated by the archetype place the actual implementation in the RoutesImplementation file. By default, the archetype places a stub here to output a test JSON object as a response. This RoutesImplementations tub also serves as an example of using our ArganoMS3 Datasonnet libraries to manipulate data in the messages to a comparable speed found in the proprietary DataWeave library in Mule.

This particular implementation above showcases three “from” consumers, representing the consumers waiting to pass the exchange to begin processing then returning to the previous producer.

The Mulesoft EIP provided a complete set of utility functions for performing. The above screenshot shows a flow you might have with a simple resource API. The Mule flow shown in the bottom right describes a flow of obtaining a HTTP header value, making a SQL call to a database, then performing a data transformation on the database output before responding to the client. An implementation sometimes requires additional business logic. in the Mule implementation shown in the corner of the picture below a Choice statement is used to determine a final result after checking conditions then performing a final transformation to return standardized data to the client.

Camel choice components

When needing to direct logic based on the evaluation of an expression, we have the ability to use the choice router. A choice allows us to nest “when” conditions and to ensure only one of the statements are triggered. Above you can see an instance where logically we are accepting either the “state” header or the “zip” header. One or the other must not be null but if they are then we trigger a bad request (HTTP status code 400) back to the client with an error message. Just like with a choice in Mule, the “payload” arriving at the next processor, which in our case is a Message Transformer, would contain the output from the proper choice statement that the traffic was routed to.

Camel loop and async components

When working with arrays or batch data, a simple processing approach to work with each item would be to use a code block that repeats. In Java, we have for-loops and for-each lambda functions which allow us to easily manipulate an object in a position. The Camel framework provides the loop function which is configured to run X amount of times.. The index position is stored within the exchange properties which exist for the lifetime of the request. In our above example, the payload data was a JSON array of objects with K/V needed for the database. Because this is not multithreaded, we can just save the data we need to properties, then make our SQL call and overwrite that data on the next pass. In this exact situation, we are processing the data linearly because our end result needs to be returned as a response; however, if the data only needs to be saved, then the loop processor can be bundled with an asynchronous producer.

DataWeave to DataSonnet

Camel itself is an extremely powerful framework for manipulating data through an HTTP interface. With the power of Java and Maven you can have access to thousands of resources at will. The remaining piece of the puzzle is a quick processing language that can integrate directly with our message exchange. In this regard, we have bundled the Camel Archetype with DataSonnet, which extends the JSonnet data-templating language.

In this script that I transformed when migrating a Mule application to Camel, we proxied data with our API to a SOAP service. In the background, our Camel application is responsible for receiving a specific JSON body, which determines the OAS specification. Once that body is validated, we hand it to our DataSonnet function that will run our functions. In this specific example above, the data is needed for further processing so, we must save that as a property. Regarding o XML transformations, the indistinguishable similarities between the scripts are shown above.

Continuing to discuss message flow for our JSON to XML payload, once the request to the upstream API was complete, we processed the response before responding to our client. Now employ DataSonnet again to perform our XML back to a JSON object. The transformation shown above represents the comparison of this XML to JSON script and the many similarities within them. You can see in DataSonnet we are resolving a variable at the top to the inline if statement used by DataWeave. If you also notice the “county” value being returned in this response because we are using the variables in the Mule exchange. Through the Camel archetype, we can use “exchangeProperties” or just “properties” for accessing these event or system properties.

Protecting our deployed API with the Kong Gateway

After having completed API development, the process of deploying and securing the sensitive data served is a daunting task. Once the application is up and running, virtually nothing stops an attacker from penetrating all combinations on the API trying to obtain the data hosted on the database. The Kong gateway is a platform for exposing and administering backend services, such as web servers or microservices. Although it provides a reservoir of features, the core Kong service is a web UI that allocates monitoring, vitals, configuration, and management tools for overseeing all HTTP operating within the business organization. The benefit of using Kong over other gateways is its highly configurable environment that enables complete modularity for all features, including deployment options for allowing horizontal scaling of the proxy or admin/documentation management services individually.

At the core of Kong, we’re basically manipulating Nginx through the OpenResty platform to provide a simple and hot-swappable platform for routing Layer 7 traffic. Kong does this in two steps. The first is configuring services by constructing an upstream proxy to a service running within the same network or cluster as Kong. Configuring just the service does not expose any applications through the gateway. Lastly, by completing the configuring by creating a route to allow path matching in a URL and/or specific host matching can be done.

Services

Services can be configured to expose a variety of applications from within its local network. The job of the service is to just keep this record of what it connects to, and you provide a custom name to help identify it within the interface. This is more powerful than manual configurations with Nginx because, without even interrupting the flow of traffic, this upstream URL could be set to v2. Although more likely, you can use the plethora of plugins to employ a deployment strategy to balance traffic between upgrades like a Canary Release.

Routes

As I mentioned before, routes are allowexposing a host and/or path combination from the Kong HTTP listener. In this above route configuration, you can see very simple configurations that you have to assign an internal upstream service configured above and named for use in other parts of the UI. The matchers responsible for binding to a request are the protocols, which for HTTP, we can select HTTP or HTTPS(port 80 or 443). We can also match on a Host, which would be helpful when forwarding all network traffic to Kong for routing by subdomain. Lastly, we can supply a path for accessing the specific service resource.

Compare to a legacy example

If you’re familiar with Nginx this configuration would look like:

# The Service from Kong.
upstream helloWorld {
    # Configuring the upstream Service.
    server v1.hello-world-exp-api-ac-dev.svc.cluster.local;
}

# The Route from Kong.
server {
    # The "protocol" config option is set to http (80).
    listen 80;
    # The "server_name" config option is set to the "Host".
    server_name api-dev.kong.ms3-inc.com;
    # The "path" config determines the location to listen on.
    location /v1/ac/hello-world/ {
        proxy_pass http://helloWorld;
    }
}

The issue creating static redirects in Nginx is the constant change and depreciation over time as the organization upgrades projects. In the above Nginx example, the ability to rewrite the URL to direct to the proper API endpoint is not implemented because that is a more complex use of Nginx. Kong employs the OpenResty Lua library to assist in managing this rewrite. In addition, Kong allows third-party plugins to create the plugin development kit built into the Lua library. Kong comes with a suite of pre-installed resources such as additional logging plugins, enabling CORS with one-click, authorizing OIDC, basic-auth, or key-auth as traffic is proxied through Kong.

Requesting our API

Now that Kong is completely deployed and our route is allowing traffic through a web server to a proxied service running safely unexposed directly to the internet, we are ready to test the connectivity. To do so, we will use the Postman graphical client or curl the route directly.

As seen in the deployment above, I was able to successfully request, from the proxy endpoint, the data that was built into our example API. The only way to know this request came from Kong is to check our response headers. Kong includes some health metrics about the system; X-Kong-Upstream-Latency describes how long Kong waited for a response from the configured service, and X-Kong-Proxy-Latency is the processing time of the Kong proxy.

Ultimately, Camel makes building APIs much simpler, allowing many developers to experience a significant time reduction during implementation and upkeep because they no longer are tied to a proprietary platform with lots of overhead background tasks. The instruction for the Kong Gateway provided is an eye-opener to the capabilities of software that provide a host of better features when compared to the Anypoint API Manager. Whether you are reading this as an integration engineer working in the Mule environment or looking for the most efficient way to get off the ground developing APIs, I hope you found value from this information. For more guidance, check out the resources below.

Resources

About the Author

Aaron Renner, Software Engineer

Aaron Renner has been leading client engagements here at ArganoMS3 for the last year, helping to modernize and improve our client’s architecture. As a Certified MuleSoft Developer and experienced Java Engineer, Aaron works hands-on with developing API infrastructure and deploying secure gateways for clients to manage the entire business stack.

Mulesoft to Tavros Migration