Integration Archives - MS3 | Cloud Integration Solutions

Our vision for Tavros, “a cost-effective, cloud-native, integration platform composed of best-of-breed and seamlessly integrated open-source components”, started to form long before the 2020 global pandemic mandated such a platform. The first signs of Tavros started sometime around 2018 with OpenTracing (now OpenTelemetry).

With many decades of combined integration experience, we wanted to address the areas of an integrated platform where organizations chronically fell short at providing enterprise-ready software and alleviate the pain points. These included things such as observability of integrations for easier tracing and troubleshooting, integrated DevOps to aid in faster delivery, integrated configuration management to ensure deployability across all environments, Cloud Native Software to support ANY cloud provider, and a way to connect and transform data with ease and efficiency. These areas became the focus areas in which Tavros would be built.

Given the proliferation of the microservices architectural style, accelerated by the maturity and value of Kubernetes as a container orchestration platform, we were actively trying to find better ways to address the need for observability in our customers. We eventually found the Distributed Tracing (Dapper) whitepaper by Google, and the amazing open-source work done by Uber, Lightstep, and Red Hat, on the OpenTracing framework, all backed by the emerging Cloud Native Computing Foundation. We saw tremendous value and sound engineering that we wanted to bring to our customers. So we went to work and made our own open-source contributions in the form instrumentation for Mule and Apache Camel.

From this point we started tracking the expansion of valuable, independently developed open-sourced components addressing pain points we’ve identified time and time again in our customers, all growing organically with the support of the CNCF.

We started to have sporadic conversations like:

X platform is really great at some functions, but not so much at others. It would be great if we could just plug-in component A for that.
Could we somehow pick and choose the best piece for each concern?
What’s the potential comparative return on investment vs established offerings?
Can you imagine if we all had to stay at home indefinitely in a couple of years because of a crazy virus?

A couple of years later we were introduced to DataSonnet which was the “missing link” that was needed before we felt we could really commit to this effort. Data transformation is at the heart of enterprise integration (identify your bounded contexts!) We always keep an eye out for solid open-source alternatives to support our client base. With the introduction of DataSonnet, a data transformation DSL initiated by our partner ModusBox, in early 2020 we immediately saw the value in an expanded version of the tool’s capabilities. Based on the incredible work in Google’s JSonnet and DataBricks’ super fast Scala re-implementation we knew the engineering was solid and that ModusBox had already done a lot of the work in facilitating the extensibility needed. After a few months of engineering efforts, we co-released DataSonnet v2. In addition to the DataSonnet release, we constructed a native component to intertwine the language within the Apache Camel DSL. With a few tweaks, we were able to get inclusion as an official expression language within Apache Camel’s latest Long Term Support version 3.7.

With these two pieces figured out, observability and data transformation, the train started rolling. After 10+ years of enterprise integration work at hundreds of different customers and projects utilizing different technologies and components, we have a solid understanding of what supplemental tooling always proves valuable, which components are usually punted down the road that could otherwise accelerate time-to-value for a project; and the experience configuring those components! So we defined and committed a dedicated team to our vision.

Today, Tavros is an integration platform that not only compares to existing enterprise offerings but that we believe is better in many respects.

Here are some features to highlight:

Provisioned and configured on-demand by Kubernetes Operations (kOps) and Ansible
Infrastructure and Platform As Code (IaC), backed by GitOps and Continuous Delivery after provisioning
Out Of the Box application Continuous Delivery pipelines
Out Of the Box best-practices configuration for cross-cutting concerns like Logging, Tracing, and Mutual TLS
Single Pane of Glass observability
Many time-to-value accelerators for integration applications
Single Sign On (SSO) experience across the supported components
A Bring-Your-Own-Component design
Out Of the Box API catalog and developer portals tied into the Continuous Delivery pipelines
Commercial support available if needed
Minimal to zero licensing cost
No risk of vendor lock-in

The components that currently power the Tavros platform include: kOps, Ansible, Kubernetes, Flux v2, Helm, Kubeseal, Kong, Kuma, Keycloak, Jaeger, Elastic Cloud, Gitea, Jenkins, Nexus, Apache Camel, Spring Boot, DataSonnet, and OpenTracing. With plans for many others as part of our Bring-Your-Own-Component design goals.

With quick uptake already occurring by early adopters, we’re excited to bring this unbelievable amount of value to our customers, partners, and the open-source community in general. We hope to start engaging with the community to make Tavros even better.

For more information, check out our GitHub repository and our website.

About the Author: Jose Montoya

Tavros Principal Product Manager, Sr. Cloud Architect

As a Solutions Architect at MS3, Jose Montoya is in charge of delivering tailor-made software to our clients. Throughout his 5 year career with MS3 he has worked for multiple companies of every size and of various industries, always delivering business value by working alongside stakeholders at different organizational levels.

In his time off Jose likes working on community-driven open-sourced projects, staying up to date with the latest software engineering practices, watching sports, playing videogames, or tending to his garden in South Texas.

Education: Bachelor of Science in Computer Engineering from the University of Texas.

In early July of 2020, Mulesoft released it’s Mule Migration Assistant (MMA) which automates some aspects of migration from Mule 3.x to 4.x. Promised well over a year ago, better late than never. The documentation can be found at https://docs.mulesoft.com/mule-runtime/4.3/migration-intro

I took the time to try out the migration process on a very simple project and see what lessons might be learned. This article can be viewed as a description of what I did and a review of the tools.

The MMA application can be downloaded in compiled form from https://github.com/mulesoft/mule-migration-assistant/releases

It’s available as a tarball and a zip. After decompressing it, you are left with an executable JAR called mule-migration-assistant-runner-1.1.0.jar, plus a collection of support libraries.

I created a trivial Mule project using Studio 6 called “mma-hello” using the 3.9.2 runtime. A simple project demonstrates basic operation of the migration tool without complicating the results. The mma-hello application logs a hello message and sets the response to hello world.

The code looks like this:

And the XML source:

That’s about as simple as it gets.

To run the assistant, CD into your installed MMA directory. Then:

java -jar mule-migration-assistant-runner-*1.1.0.jar [parameters]

Where documented parameters consist of:

For me, this works out to:

java -jar mule-migration-assistant-runner-1.1.0-SNAPSHOT.jar \

-projectBasePath C:/Users/markj/AnypointStudio/personal-workspace/mma-hello \

-destinationProjectBasePath C:/Users/markj/AnypointStudio/personal-workspace/mma-hello-v4 \

-muleVersion 4.3.0

Note that this command is not fully described on docs.mulesoft.com, nor is it in the README file on GitHub. Rather, you need to drill down a bit into the /docs of the cloned sources, but you can also use the -help parameter to provide more help information:

Typically, the documentation is out of date regarding the actual application. Note the addition of several proxy parameters that allow projects to be migrated in a secured environment.

Running the command above, I get:

Executing migrator 1.1.0-SNAPSHOT…

================================================================

MIGRATION ASSISTANT RUN SUCCESSFULLY

================================================================

Total time: 1.567 s

Migration report: C:\Users\markj\AnypointStudio\personal-workspace\mma-hello-v4\report\summary.html

The report is formatted as HTML:

Warnings include:

To have a look at the results, I imported it into Studio 7.4. The resulting code looks like this:

There are several references to inbound and outbound properties here. I think these are standard warnings regarding such properties, though none are set or used in the mma-hello example. Still, this clutters up the XML code.

Also notice the use of the DW function, migration::HttpListener::httpListenerResponseHeaders(vars). This looks like it’s pushing vars into headers, but the original code doesn’t do that. Perhaps this reflects the default behavior of the 3.9.2 HTTP connector. In order for that information to continue to be present in the migrated application, it needs to be pushed into headers.

Finally, the original code in my 3.9.2 code comes across without any problem.

While these additional warnings are useful the first few times the migration assistant is used, it just adds clutter to the code. This would be annoying if you needed to migrate 100 APIs and needlessly increases file sizes.

There are some things that the MMA won’t handle. For example, it’s not all that surprising that Java classes don’t migrate since custom Java classes used in a Mule project would have a lot of references to the older object model. Mapping the old model to the new one would be quite difficult. Also unsupported are:

DataMapper (manual migration to DataWeave is recommended)
Custom types defined in Studio
The <expression-component> element
Any unlisted connectors or modules.

I had problems when I tried to include a Mule domain, but didn’t take the time to figure it out. Caveat emptor!

Overall, the Mule Migration Assistant can do a lot of the heavy lifting involved in migration. The documentation suggests that you “prepare the code” for migration, like converting MEL expressions into Dataweave, but that kinda misses the point of a migration tool. If you used DataMapper in your old projects, you are going to have to migrate them by hand to Mule 4 – no getting around that. MMA will migrate standard connectors, but not third party or custom ones. The report generated is helpful in pinning down where the problems are. Warnings are too verbose, in my opinion, and there should be an option to reduce/eliminate them. Warning comments will seriously clutter up converted XML.

Bear in mind that this review covers a VERY simple example. More complex applications will be more difficult to migrate. The support provided by the MMA only covers the mechanical parts of the conversion, which is a small part of a large application or set of interlocking APIs. This is where assistance from experts like those provided by MS3 can help to make your migration process smooth, timely, reliable, and within budget. Send an email to contact@ms3-inc.com to contact our migration team.

About the Author: Mark Norton

*TEAM LEAD & PROPOSAL MANAGER · SENIOR SOFTWARE ENGINEER & ARCHITECT*

Mark Norton is a Senior Contributing Engineer focused on providing business integration solutions using Mulesoft (Certified Developer), AWS (Certified Business Provider), and other technologies. As one of the first team leaders, he has helped MS 3 grow and guide the members of his team on their career path and client work. As a proposal manager, Mark has led and supported several significant proposal efforts at federal and state levels, in addition to commercial opportunities. Before joining MS 3 in 2015, Mark managed his independent consulting firm that developed integration solutions for the Higher Education sector.

For 10 Years, MS³ has been delivering Innovative & Cutting-Edge Solutions to our clients. Watch “The MS³ Story” to hear from MS³ Founder & CEO, Aaron Weikle to see how it all started, what the journey was like, and what is up next for us.

In 2009, MS³ began its journey, and in those 10 years it has grown into a leader in Innovative & Cutting-Edge Software Solutions. In Celebration of our 10th Anniversary, here is a scrapbook looking back on where we began, and where we are now. Enjoy!

Mountain State Software Solutions (MS³) selected to the Navy Financial Management (FM) API Initiatives team

On November 4, 2019, Mountain State Software Solutions (MS³) was chosen to be a part of an elite team for the execution of Navy Financial Management (FM) API Initiatives. The objective of this project is to provide digital modernization across the Navy’s financial systems. MS³’s responsibilities for this three year-long-project will be the implementation of a strategic Enterprise Integration Platform (EIP), which supports Financial Tracking, Audibility, and Interconnectivity between FM Systems.

The Office of Financial Policy and Systems (FMP) strives to continually provide the highest quality of information, data, and technology to the Navy FM community, so they must continue to improve the efficiency of the IT systems for all associated parties.

MS3 provides a fresh perspective as a leader in commercial API Implementations. We have implemented cutting-edge technology for many large-scale commercial enterprises such as Ralph Lauren, Chick-Fil-A, and Harvard University, just to name a few. Through our vast experience, MS³ will be bringing valuable lessons learned and knowledge of industry best practices to the table to support Navy as they work to track and manage their financial data.

“MS³ is excited to have been chosen to implement the Navy FMP API Initiatives as part of this contract award,” said MS³ Founder and CEO Aaron Weikle. “MS³ is renowned in the commercial world for the implementation of next-generation API solutions. We will be bringing these experiences to the government sector which will allow us to make a huge difference for the U.S. Department of Navy and the U.S. Taxpayer!”

About MS³

MS³ is a global IT consulting firm based in the Washington DC metropolitan area that specializes in engineering FUTURE PROOF solutions for both commercial and federal customers. Our focus is on business acceleration and providing API-driven for today’s most complex integration challenges. With knowledge and experience in everything from MuleSoft Integration to AWS Cloud Services to DevOps, IoT and RPA capabilities, and so much more. MS³ can leverage various innovative technologies to create a custom solution that is right for your business.

The MS³ team is excited about this opportunity to partner on this 3-year initiative, and to support Navy in their IT endeavors going forward!

2

3

…

6

Next Page »