Integration Archives - MS3 | Cloud Integration Solutions

In this video blog, MS3’s Director of Technology, Joe Rice, discusses how Kong Konnect enables instant access to purpose-built multi-cloud connectivity across APIs and microservices. Learn more at https://konghq.com/kong-konnect

Watch by clicking below:

Instant Multi-Cloud API and Microservice Connectivity

Financial institutions have spent millions of dollars and many years cycling through different API governance and integration ecosystems using traditional rest-centric API gateways, complex service mesh technology or even homegrown approaches that eventually are too difficult to manage or maintain.

But with the advent of the Cloud, containers can provide a lightweight and flexible way to add value (not latency) and scale without adding single points of failure. The traditional monolithic, shared gateways that I’ve seen don’t do that. They’re not container- and cloud-ready. They’re not lightweight and easy to distribute.

I’ve seen implementations of Apigee, Mulesoft, API Connect, cloud native gateways and even appliance-based solutions that at the end of the day are monolithic and essentially monotone. They do one thing well, such as a REST API, but lack the flexibility that’s needed for the modern enterprise. They are complex and heavyweight, albeit feature-rich. While they do deliver the base routing, bridging transformation and connectivity, they have limits in formats and impose lock-in via a high-friction, vendor-specific process.

But the Kong service connectivity platform, via the connect ecosystem, is different. It allows you to outsource the management of the control plane, hosting it in data centers while still being able to locally control policy enforcement and data, whether it be an Amazon, Google, Azure or in your local data centers.

Being able to deliver vital observability, zero-trust security, pluggable extensibility with well-organized ways to find and reuse APIs, and a catalog along with industry standard controls and guardrails to authenticate, authorize, identify and only allow legitimate access to trusted resources.

So what else can it do? It’s not a one-trick pony. It cannot just do REST but gRPC, GraphQL and events. It can run anywhere – standalone Kubernetes or the cloud.

It’s full-lifecycle with DevOps automation, and it’s container-optimized with the observability and flexibility needed via either distributed service mesh or lightweight gateways.

With the latest release, it’s even satisfying the need for multi-geography support (US, UK, Germany, Japan and Australia) to provide and comply with data sovereignty and local regulations.

And it’s available either as a free open source solution through Kong Konnect Free, Kong Konnect Plus (a pay-as-you-go approach), or Kong Konnect Enterprise to deliver enterprise-grade security, scalability, observability and more.

But is it easy to set up? Watch this video where I walk you through what I was able to do in a very brief period of time by just signing up for Konnect. They’ll set up a Kong Konnect environment for you, which is basically the control plane of the Kong Gateway ecosystem.

There’s no faster or easier way to set up robust, cloud native, container-ready Zero-trust security that’s format- and protocol-agnostic. Check it out today.

Our vision for Tavros, “a cost-effective, cloud-native, integration platform composed of best-of-breed and seamlessly integrated open-source components”, started to form long before the 2020 global pandemic mandated such a platform. The first signs of Tavros started sometime around 2018 with OpenTracing (now OpenTelemetry).

With many decades of combined integration experience, we wanted to address the areas of an integrated platform where organizations chronically fell short at providing enterprise-ready software and alleviate the pain points. These included things such as observability of integrations for easier tracing and troubleshooting, integrated DevOps to aid in faster delivery, integrated configuration management to ensure deployability across all environments, Cloud Native Software to support ANY cloud provider, and a way to connect and transform data with ease and efficiency. These areas became the focus areas in which Tavros would be built.

Given the proliferation of the microservices architectural style, accelerated by the maturity and value of Kubernetes as a container orchestration platform, we were actively trying to find better ways to address the need for observability in our customers. We eventually found the Distributed Tracing (Dapper) whitepaper by Google, and the amazing open-source work done by Uber, Lightstep, and Red Hat, on the OpenTracing framework, all backed by the emerging Cloud Native Computing Foundation. We saw tremendous value and sound engineering that we wanted to bring to our customers. So we went to work and made our own open-source contributions in the form instrumentation for Mule and Apache Camel.

From this point we started tracking the expansion of valuable, independently developed open-sourced components addressing pain points we’ve identified time and time again in our customers, all growing organically with the support of the CNCF.

We started to have sporadic conversations like:

X platform is really great at some functions, but not so much at others. It would be great if we could just plug-in component A for that.
Could we somehow pick and choose the best piece for each concern?
What’s the potential comparative return on investment vs established offerings?
Can you imagine if we all had to stay at home indefinitely in a couple of years because of a crazy virus?

A couple of years later we were introduced to DataSonnet which was the “missing link” that was needed before we felt we could really commit to this effort. Data transformation is at the heart of enterprise integration (identify your bounded contexts!) We always keep an eye out for solid open-source alternatives to support our client base. With the introduction of DataSonnet, a data transformation DSL initiated by our partner ModusBox, in early 2020 we immediately saw the value in an expanded version of the tool’s capabilities. Based on the incredible work in Google’s JSonnet and DataBricks’ super fast Scala re-implementation we knew the engineering was solid and that ModusBox had already done a lot of the work in facilitating the extensibility needed. After a few months of engineering efforts, we co-released DataSonnet v2. In addition to the DataSonnet release, we constructed a native component to intertwine the language within the Apache Camel DSL. With a few tweaks, we were able to get inclusion as an official expression language within Apache Camel’s latest Long Term Support version 3.7.

With these two pieces figured out, observability and data transformation, the train started rolling. After 10+ years of enterprise integration work at hundreds of different customers and projects utilizing different technologies and components, we have a solid understanding of what supplemental tooling always proves valuable, which components are usually punted down the road that could otherwise accelerate time-to-value for a project; and the experience configuring those components! So we defined and committed a dedicated team to our vision.

Today, Tavros is an integration platform that not only compares to existing enterprise offerings but that we believe is better in many respects.

Here are some features to highlight:

Provisioned and configured on-demand by Kubernetes Operations (kOps) and Ansible
Infrastructure and Platform As Code (IaC), backed by GitOps and Continuous Delivery after provisioning
Out Of the Box application Continuous Delivery pipelines
Out Of the Box best-practices configuration for cross-cutting concerns like Logging, Tracing, and Mutual TLS
Single Pane of Glass observability
Many time-to-value accelerators for integration applications
Single Sign On (SSO) experience across the supported components
A Bring-Your-Own-Component design
Out Of the Box API catalog and developer portals tied into the Continuous Delivery pipelines
Commercial support available if needed
Minimal to zero licensing cost
No risk of vendor lock-in

The components that currently power the Tavros platform include: kOps, Ansible, Kubernetes, Flux v2, Helm, Kubeseal, Kong, Kuma, Keycloak, Jaeger, Elastic Cloud, Gitea, Jenkins, Nexus, Apache Camel, Spring Boot, DataSonnet, and OpenTracing. With plans for many others as part of our Bring-Your-Own-Component design goals.

With quick uptake already occurring by early adopters, we’re excited to bring this unbelievable amount of value to our customers, partners, and the open-source community in general. We hope to start engaging with the community to make Tavros even better.

For more information, check out our GitHub repository and our website.

About the Author: Jose Montoya

Tavros Principal Product Manager, Sr. Cloud Architect

As a Solutions Architect at MS3, Jose Montoya is in charge of delivering tailor-made software to our clients. Throughout his 5 year career with MS3 he has worked for multiple companies of every size and of various industries, always delivering business value by working alongside stakeholders at different organizational levels.

In his time off Jose likes working on community-driven open-sourced projects, staying up to date with the latest software engineering practices, watching sports, playing videogames, or tending to his garden in South Texas.

Education: Bachelor of Science in Computer Engineering from the University of Texas.

In early July of 2020, Mulesoft released it’s Mule Migration Assistant (MMA) which automates some aspects of migration from Mule 3.x to 4.x. Promised well over a year ago, better late than never. The documentation can be found at https://docs.mulesoft.com/mule-runtime/4.3/migration-intro

I took the time to try out the migration process on a very simple project and see what lessons might be learned. This article can be viewed as a description of what I did and a review of the tools.

The MMA application can be downloaded in compiled form from https://github.com/mulesoft/mule-migration-assistant/releases

It’s available as a tarball and a zip. After decompressing it, you are left with an executable JAR called mule-migration-assistant-runner-1.1.0.jar, plus a collection of support libraries.

I created a trivial Mule project using Studio 6 called “mma-hello” using the 3.9.2 runtime. A simple project demonstrates basic operation of the migration tool without complicating the results. The mma-hello application logs a hello message and sets the response to hello world.

The code looks like this:

And the XML source:

That’s about as simple as it gets.

To run the assistant, CD into your installed MMA directory. Then:

java -jar mule-migration-assistant-runner-*1.1.0.jar [parameters]

Where documented parameters consist of:

For me, this works out to:

java -jar mule-migration-assistant-runner-1.1.0-SNAPSHOT.jar \

-projectBasePath C:/Users/markj/AnypointStudio/personal-workspace/mma-hello \

-destinationProjectBasePath C:/Users/markj/AnypointStudio/personal-workspace/mma-hello-v4 \

-muleVersion 4.3.0

Note that this command is not fully described on docs.mulesoft.com, nor is it in the README file on GitHub. Rather, you need to drill down a bit into the /docs of the cloned sources, but you can also use the -help parameter to provide more help information:

Typically, the documentation is out of date regarding the actual application. Note the addition of several proxy parameters that allow projects to be migrated in a secured environment.

Running the command above, I get:

Executing migrator 1.1.0-SNAPSHOT…

================================================================

MIGRATION ASSISTANT RUN SUCCESSFULLY

================================================================

Total time: 1.567 s

Migration report: C:\Users\markj\AnypointStudio\personal-workspace\mma-hello-v4\report\summary.html

The report is formatted as HTML:

Warnings include:

To have a look at the results, I imported it into Studio 7.4. The resulting code looks like this:

There are several references to inbound and outbound properties here. I think these are standard warnings regarding such properties, though none are set or used in the mma-hello example. Still, this clutters up the XML code.

Also notice the use of the DW function, migration::HttpListener::httpListenerResponseHeaders(vars). This looks like it’s pushing vars into headers, but the original code doesn’t do that. Perhaps this reflects the default behavior of the 3.9.2 HTTP connector. In order for that information to continue to be present in the migrated application, it needs to be pushed into headers.

Finally, the original code in my 3.9.2 code comes across without any problem.

While these additional warnings are useful the first few times the migration assistant is used, it just adds clutter to the code. This would be annoying if you needed to migrate 100 APIs and needlessly increases file sizes.

There are some things that the MMA won’t handle. For example, it’s not all that surprising that Java classes don’t migrate since custom Java classes used in a Mule project would have a lot of references to the older object model. Mapping the old model to the new one would be quite difficult. Also unsupported are:

DataMapper (manual migration to DataWeave is recommended)
Custom types defined in Studio
The <expression-component> element
Any unlisted connectors or modules.

I had problems when I tried to include a Mule domain, but didn’t take the time to figure it out. Caveat emptor!

Overall, the Mule Migration Assistant can do a lot of the heavy lifting involved in migration. The documentation suggests that you “prepare the code” for migration, like converting MEL expressions into Dataweave, but that kinda misses the point of a migration tool. If you used DataMapper in your old projects, you are going to have to migrate them by hand to Mule 4 – no getting around that. MMA will migrate standard connectors, but not third party or custom ones. The report generated is helpful in pinning down where the problems are. Warnings are too verbose, in my opinion, and there should be an option to reduce/eliminate them. Warning comments will seriously clutter up converted XML.

Bear in mind that this review covers a VERY simple example. More complex applications will be more difficult to migrate. The support provided by the MMA only covers the mechanical parts of the conversion, which is a small part of a large application or set of interlocking APIs. This is where assistance from experts like those provided by MS3 can help to make your migration process smooth, timely, reliable, and within budget. Send an email to contact@ms3-inc.com to contact our migration team.

About the Author: Mark Norton

*TEAM LEAD & PROPOSAL MANAGER · SENIOR SOFTWARE ENGINEER & ARCHITECT*

Mark Norton is a Senior Contributing Engineer focused on providing business integration solutions using Mulesoft (Certified Developer), AWS (Certified Business Provider), and other technologies. As one of the first team leaders, he has helped MS 3 grow and guide the members of his team on their career path and client work. As a proposal manager, Mark has led and supported several significant proposal efforts at federal and state levels, in addition to commercial opportunities. Before joining MS 3 in 2015, Mark managed his independent consulting firm that developed integration solutions for the Higher Education sector.

For 10 Years, MS³ has been delivering Innovative & Cutting-Edge Solutions to our clients. Watch “The MS³ Story” to hear from MS³ Founder & CEO, Aaron Weikle to see how it all started, what the journey was like, and what is up next for us.

In 2009, MS³ began its journey, and in those 10 years it has grown into a leader in Innovative & Cutting-Edge Software Solutions. In Celebration of our 10th Anniversary, here is a scrapbook looking back on where we began, and where we are now. Enjoy!

For more information, check out our GitHub repository and our website.

About the Author: Jose Montoya

About the Author: Mark Norton

About

Solutions

Partners

Popular Links