Our team at Pandium has collectively spent a century building and scaling integrations. Below, we have outlined some processes, tools, and tips that we recommend to maximize the time to delivery when building and launching integrations.
Insufficient facts always invites danger. Resolving authentication should be the very first step before programming an integration! It is best to test the API endpoints for authentication access. Without doing this, anything you build will almost certainly have to be rewritten, as it will be based on assumptions. You have to know what you’re working with.
%2520(1).jpeg)
It can take days to gain access to systems, so resolving authentication should be done first. Writing code without access is a good way to have to rewrite it later when your assumptions are wrong.
Similarly, even after having access, it is a good idea to first see if you can hit all the endpoints you may need to use. Some things may require feature settings or other provisioning steps that need to be carried out. This can take days to resolve, so it's best to do it early to not lose momentum.
Before integrating platforms, you’ve got to read the API docs of the resources you will be using, even if they might not always be up to date. Conduct thorough research and try to get ahead of any API constraints. Don’t be the guy/gal shooting a file cabinet into the sun because you didn’t read the manual.
Related content: An Engineers Guide to Integrating with NetSuite’s API
You may not generally see API constraints until you are running your integration in production where you find that it can't keep up with either the API rate limits or even your platform's rate limits.
Looking for rate limits in documentation can allow you to try to implement concurrency first instead of discovering it later and trying to implement concurrency over your existing code.
This includes performing a competitive analysis, reviewing the available API documentation, and identifying any discrepancies between user requirements and technical limitations. These preliminary steps are crucial in determining the scope of the integrations and ensuring a positive user experience.
Engineers are often the most expensive and scarce members of your team. Therefore, understanding what is feasible early on in the product development process can save the team and the company valuable time, money, and resources.
As a young boy growing up in Alphabet City, my dog Toaster was attacked by a local raccoon one evening; I resolved to counter-attack the next night. I propped a Ruffles chip under a cardboard refrigerator box with a stick tied to a string that I held.
Related Content: What's Wrong with Low and No Code Platforms?
That black-and-white, furry b*stard couldn’t keep his little mouth shut long enough to realize he was trapped by the time he ate the chip. What an idiot!
However, my next move did not go well. Have you ever stepped into a box to engage in a cage match with a feral raccoon? Me too, and it went how you would expect. It took years to grow out of my Phantom of the Opera look (with none of the forbidden love appeal).
Please don’t be like me: Use the right tool for the job.
If you wouldn’t use a refrigerator box to catch a raccoon, you wouldn’t skip code in writing an important integration.
When writing integrations, it's best to use code to have more control over when data is exchanged, how it flows, and under what conditions. Using third-party, no-code solutions can be useful for solving simple use cases or addressing the long-tail of integrations that aren’t core to your product and customers. A cardboard box for catching a snail or baby, if you will.
Native integrations are integrations written in code that are built directly between two or more software and are built by one or both of the system owners.
These typically use each system’s APIs, and they give organizations the most control over their integrations.
Native integrations provide the ability to set up workflows in the exact ways an organization’s customers need, and are only limited by the partner API.
More practical reasons for using code to write integrations include:
Data Mapping enables you to better understand which endpoints will have to be grabbed during the development process. Then, you can further investigate the API Docs to see what's possible.
Postman is an invaluable resource for building integrations. It allows our team to share workspaces during development, which accelerates development time and helps in the maintenance of integrations. And it's not just for engineers - project managers can also use it to get access to data models.
Whether catching a raccoon or building an integration, a plan will always help your results.
For project managers and engineers writing integrations, it's recommended to start by architecting the project rather than diving into the code right away. Have a specification of your flows and segment them differently as needed.
For instance, are you syncing orders, tracking, or tickets? It's advisable to write your integrations modularly. That way, when you implement your configurations, you can quickly toggle them on and off.
This document doesn't have to be incredibly complex. A few paragraphs about what you're trying to achieve and why, along with some high-level flow diagrams, can go a long way to keep everyone on the same page. You may not have everything figured out upfront, but as you set up the various mappings, make sure to update the plan.
Related Content: Designing and Scoping User-Facing SaaS Integrations with ShipBob
This document can serve as an ongoing artifact of decisions, and it helps keep everyone working on the integration in the loop. And, this document can become a valuable resource later on when creating help documentation or during post-launch maintenance.
The insights you gain from users are invaluable! Trying to build integrations into complex systems without having a real user ready to use it can be an easy road to failure.
Without a beta period with real customers' data, there is no way to validate use cases and scale.
Instead of building an integration before having any users sign up to use it, it's recommended to collaborate with sales and CX teams to gauge interest in an integration or to create a listing on your marketplace that allows your users to express interest.
Either of these strategies can help source a list of potential users interested in testing the integration.
Real data is needed to ensure you have uncovered edge cases before your users do. No matter how good your test data is, when your integration hits real users' data, it will need to be tweaked and sometimes reworked, with new understandings that can only come with actual experience.
If you can conduct capacity testing, that's a plus, especially if you can get sample data sets together in your system at high volume. Prior to launch, running tests and seeing how partner systems perform under a high load can be helpful.
Sometimes, it's not necessarily the integration's fault, but rather inefficiencies in the APIs on either side that can slow things down. Understanding these limitations can help you express them to customers or talk to your partners about potential fixes.
In addition to having a living integration plan/spec document, take the time to create user stories and identify the user journey. This will help ensure that what you're building addresses the business solution needed vs realizing that you’ve implemented the wrong thing, or used the wrong endpoint or mapping.
Conducting user interviews is a valuable step in gaining insight into your end-users.
By asking targeted questions about their work behaviors, constraints, and potential workarounds they use to achieve their objectives, you can gain a better understanding of their personas, requirements, and limitations.
Investing time in user interviews is well worth the effort, as it allows you to identify suggestions and feedback that can be leveraged to improve your integration and product overall.
For more information on why integration user stories are essential, check our our article on Why You Need Integration User Stories.
This ensures easy communication and viability of launch. Building an integration requires an in depth knowledge of both of the systems you're connecting to. Having someone who knows the platform you're building with well can help with defining use cases and functionality , as well as, with support when you need it.
This includes taking into account, rate limits, run time limits, throughput (single object endpoints vs. bulk).
You may not see issues with API constraints come about until you are running your integration in production, and you find that it can't keep up with either the API rate limits or even your platform's rate limits.
Generally, even third-party integration platforms have a runtime limit. And so, try to get ahead of this and look for the rate limits in the documentation. Then, you can work to implement concurrency first instead of trying to implement concurrency over existing code.
The recommendation here is to plan ahead of time with certain constraints that you might see.
Designing integrations that can scale to 1000s of users requires thinking about throughput. Favor bulk endpoints over single endpoints, and if an API supports sub-resources being created with the main resource, do that in one call as opposed to hitting individual requests.
Maximize the amount of data you are sending using concurrency, but also be mindful of rate limits.
If you need assistance with an integration project, or just don't want to deal with it yourself, reach out to us at Pandium! Our team has the experience and knowledge developing complex, native integrations across a variety of systems, and we can help you get the most out of the systems you're looking to connect to in order to accomplish your business goals.
