Always up-to-date, reliable API and documentation for multiple microservices

inteam blog post cover

Case’s Context

It all started with Equals Money, a company founded in 2005 aiming to simplify money management for everyone using modern technology. To achieve this goal, they offer a wide range of financial solutions for personal and business use, including software for payment processing, card handling, and internal budget management.

Moreover, there is an even wider range of external systems connected to Equals Money’s systems through API.


The challenge at hand

The situation described above presented several challenges:

  • All definitions of data sent or received from external sources integrated through API were scattered across different projects. There were multiple repositories, each defining its own sets of rules, contracts, and data.
  • There were no safeguards to ensure that what we declared was consistent with reality. This led to inconsistencies and errors that only became apparent when the client attempted to use a specific feature. For example, we specified that we needed a customer ID and a company or product name to send information about cards. Later, it turned out that we needed additional information, but because the knowledge of those systems was scattered, we were unable to specify it upfront. These inconsistencies only became apparent at this late stage, which is frustrating for the client.
  • There were no solutions to automatically detect no longer used or out-of-date mechanisms. As a result, we maintained unnecessary code.
  • The company adopted the Contract First principle: initially, we defined the sent and received data model following the OpenAI standard, directly in .yaml files, and then we coded it. Documentation was always created first, using the .jml file extension. This solution didn’t support developers’ work and caused frustration. There was no solid type, syntax, or correctness checking. Moreover, .yaml files made documentation harder to read.
  • From a broader business perspective, changing pure .yaml OpenAI schema files to any Typescript solution that had been used for a long time was a demanding challenge. It was a commonly used standard, so many such files were functioning within the company. It was unrealistic to change all of them at once; we needed the right solution to migrate safely and gradually, step by step, to a new technology.
  • The old solutions did not support managing consistency, easy automation, or creating informative diagrams about those systems and their connections. Modern technology could introduce much more functionality to APIs, documentation, and speed-up the development process.

Time for a Change

Problems with the API and documentation were well-known across the company. Our first challenge was to propose an optimal solution and convince developers and business decision-makers that we could achieve it gradually, within a reasonable time frame, or revert if necessary.

Furthermore, the solution had to be consistent with the company’s overall security protocols. We needed to ensure that the documentation contained exactly what we intended to show, no more and no less.

We also cooperated with the sales team to gain knowledge about what we wanted to share publicly, to demonstrate the services we offer.

Complexity of the Process

It was the kind of project that revealed itself as it progressed. We didn’t know the exact details of what we needed to do or what the final outcome would look like.

We started with a technology overview, checking what was available on the market and creating a prototype. After several iterations and revisions, we reached a point where everyone was pleased.

Central Contract Repository

First, we needed a central contract repository—a separate element within the whole system to collect data in one place and define all the documentation and information flow.

Consequently, every contract would be defined only once, in contrast to many solutions that were popular a few years ago, such as contract tests. These tests were meant to check every data schema for correctness and compliance with declarations between different applications.

Our solution ensured that contract tests were no longer necessary. With a single central repository used in every internal application, it became impossible to make such mistakes.

We could catch potential problems faster on many levels. From a business perspective, this is a priceless solution—it allows the company to save significant amounts of money and prevent image problems by finding errors and inconsistencies before a client can spot them.

Implementing TypeScript

Additionally, we proposed replacing .yaml files with API and documentation in the code. Typescript and zod. libraries were chosen for this task as a more natural and developer-friendly solution.

All API and documentation would be in code, making it easy to import information from the repository for different applications or duplicate certain fragments for other projects.

We deliberately picked TypeScript and zod. so that the new and old technology would have a common area: OpenAPI. Having this common element was crucial for migration, allowing us to arrange everything together.

Moreover, systems used by Equals Money require creating documentation in the OpenAPI format to automatically generate all the API descriptions. It’s a common practice in financial services to automatically generate documentation, especially for developers, because it ensures the documentation is almost always up to date with no human interference needed. When something changes, it’s reflected in the documentation almost immediately. This helps ensure there are no errors or inconsistencies.

Positive Side Effects

This type of migration has another benefit—during the process, we examine every service in detail. Every endpoint is tested both automatically and manually. Testers thoroughly check the entire system, effectively creating a solid inventory.

Additionally, we can identify and address any missing elements. We can also spot outdated components or areas for improvement.

Throughout the process, we aimed to create additional benefits by leaving the code better than it was.

Transitioning Between Technologies

The whole process was rather complicated. We needed to integrate the old technology with the new one, along with a few different technologies, using a common standard: OpenAPI—a consistent and precise standard for defining contracts and APIs.

We utilized this standard to produce schemas in the OpenAPI format. Furthermore, we then combined them into one final contract.

As a result, everything is now in more recent technology without losing any functionality. Features such as contract validation, incoming and outgoing data validation, error handling, and logging are at least as good as, if not better than, the old solution.

During the process, we needed to verify the correctness of the emerging documentation by checking every endpoint, making it a slow process. Transferring complicated schemas to new technology requires accuracy and time to ensure uninterrupted service for clients.

Moreover, developers needed to learn and get accustomed to the new technology. This transition required them to step out of their comfort zones, necessitating a patient approach. There were many questions about how to do things and how the new system worked.

We had to thoroughly and properly describe the new technology to enable everyone to use it in their everyday work. It had to be simple, intuitive, and, most importantly, better than the previous solution.

Every change carries risk, and a change of this magnitude in the API requires a very responsible approach, especially in the financial industry.


First Results

The whole project is not finished yet, but we are already seeing the first results. The most visible effect now is how much work it saves, especially for front-end teams, who often duplicate fragments of code from the documentation.

Thanks to the central repository, they always know what kind of data will be received, what needs to be sent to the server, and what will come in the response. They don’t need to replicate it based on .yaml files; they just have a prepared component to embed on the site. There’s no need to check or worry about data correctness and consistency.

All of this also means that there are far fewer errors in the code.

Another result we are seeing is automated documentation. The central contract repository makes it much easier to create tools that describe system architecture or automatically check which services are exchanging data. Based on such a tool, we plan to create diagrams of data exchange across our entire system network.

These automatically created diagrams will always be up-to-date, accurately showing the flow of data. This will be especially useful for software architects when designing new functionalities.

Another benefit we are already seeing is the detection of components and services that are no longer used but still maintained. Now we can remove those parts of the code. Less code means easier work, lower costs, and less wasted time—all of which result in significant savings for the company.

Additionally, by checking all the endpoints, we found and corrected a few inconsistencies right away.

Moreover, we started to validate not only incoming data but also outgoing data, allowing us to detect potential errors as soon as possible.


The Final Results We Aim For

Our goal is to achieve an always up-to-date, reliable API and documentation for all the company’s services, ensuring compatibility between what we declare in the documentation and the data and information that we send and receive in reality.

Internally, the central contract repository means easier API and documentation management, automation, tracking of how individual services communicate, in which order, how often, and creating diagrams based on that information.

As a result, we will always detect unused services that are unnecessarily maintained.


Final Thoughts and Conclusion

An organized API poses a common challenge for companies with multiple microservices.

Although there are a few solutions designed to support consistent API and documentation, such as contract tests or hexagonal architectures, there is no standard solution to this problem that has gained widespread acceptance in the corporate world.

Our idea of a central contract repository to be shared between applications is definitely a good solution for small and medium projects.


Leader of the project from IN Team:
Paweł Górecki / Software Developer
“To sum up: tidy up your API!”

Get in touch

Save time and resources by securing a flexible IT team extension with top-notch, dependable developers.

Reliable technology teams
Registered data
VAT-ID:
KRS:
PL5272733624
0000933665
Connect
IN Team © 2024  |
Privacy Policy
|
Whistleblower