This article is written by Harri Kirik, Software Engineer @ Mobi Lab
I am a software engineer and part of a team at Mobi Lab, a design agency. In Mobi Lab, we build successful digital products and augmented reality solutions for people on the go.
In this post, I will share what we have learned about the importance of continuous integration and delivery (“CI/CD”) and how Codemagic improves our workflow and makes our clients happy.
I’ll also talk about using Codemagic to build the SplitKey SDK, an eIDAS authentication and digital signature service powering millions of monthly transactions, which sets higher-than-usual requirements for a secure build infrastructure.
What is CI/CD?
Great apps are always the result of collaboration, be it between a multi-talented and diverse team or a solo author and their app’s user base. One of the cornerstones of enabling this strong collaboration is maintaining a short feedback loop between implementing, testing, and validating new ideas. This can be more time-consuming and error-prone than it seems, especially as the number of stakeholders with different focuses and needs increases. This is where a good CI/CD comes in.
A Continuous Integration and Delivery (“CI/CD”) service is a tool that enables merging the work of all the team members several times a day. Typically, in a way that triggers an automated validation, build, and testing flow that results in new binaries and reports becoming available for the team, stakeholders, and clients. This process enables a tight collaboration and a fast feedback loop.
Confidence for engineers
Firstly, the CI process gives confidence to an engineer that their changes are compatible and compliant with the standards of the existing work. If they aren’t, the CI/CD tool creates a report about the issues and notifies the engineer to enable a quick fix. As an additional benefit, the feedback is so fast that the engineer will not have time to start on a new idea, so returning to fix the previous issues will not incur a context switch.
Continuous delivery for fast feedback
Secondly, CI/CD simplifies and speeds up the delivery of the app to other stakeholders, allowing them to try it out and give immediate feedback (“CD” or Continuous Delivery). This fact can sometimes get lost for engineers, as they can easily build the newest version from their machine. However, the value becomes apparent when downstream engineering and quality assurance teams get blocked or feedback is not timely enough to enable even small tweaks without incurring high costs for rewriting things.
While CI/CDs are used both by product companies and agencies, for an agency, it is a lot more crucial to get this process working flawlessly. On this, I’ll elaborate below.
A solid CI strategy is a big win for an agency
As an agency, we at Mobi Lab have stream-aligned teams delivering value to multiple clients. The profile of each client, their involvement level, technical competency, and the nature of their product are widely different. To create value effectively and via unified service design, we have standardized our whole client journey. The use of CI is an important part of it.
We learned that letting every team have their unique CI setup was a technical nightmare.
Already in a very early agency stage, we learned that letting every team have their unique CI setup was a technical nightmare. It is hard to ensure security, update software, move projects or team members between teams, or hand over projects to clients. If a software project starts with engineers spending the first one to two weeks getting the project base created and the CI up and running, then there is little to nothing visual to show the client for the first sprints. The timeline and budget of any smaller project will go over the agreement. There is no feedback loop and a real danger of clients losing their trust in the team.
The way to solve this correctly is to create a focus group by including all interested parties and representatives of different engineering teams (in our case Android and iOS native, Flutter, React Native, backend, AR/VR) and then agree on and document a set of standards for all teams inside the company. The documentation should give straightforward answers to questions like which CI I should use, how to handle accesses, and which flows I should enable.
Now, with this principle in mind, there is just the “simple” question of which CI to use. One that would make both the engineers and the clients happy. One that would not only streamline setup for simple projects but would also be usable for complicated custom setups.
Our CI misadventures with Jenkins - free but expensive
While a design agency, Mobi Lab has a strong engineering background. In hindsight, this easily explains why we started with the most custom CI solution possible - Jenkins (then still called “Hudson”). Jenkins allows you to self-host the CI, build almost any project imaginable, apply custom plugins, and use the CI via APIs to set up new projects automatically. Also, the license is free. Understandably, it looks like a perfect tool for tech-savvy, roll-your-own mentality engineers.
Turns out that Jenkins has some disadvantages, especially for an agency with multiple teams, platforms, and clients. By far, the biggest is the setup cost.
Turns out that Jenkins has some disadvantages, especially for an agency with multiple teams, platforms, and clients. By far, the biggest is the setup cost. The high customizability of Jenkins comes with the cost of a more complicated initial configuration, not only when setting up Jenkins but also when adding a new app build. Mobile application engineers had it the worst, as these platforms had the most complicated build configurations, especially when it comes to iOS apps as Apple is notorious for not being CI-friendly. While a lot of this could be mitigated via automation and the use of per-platform templates, it still took an engineer about 1 work-week per platform to set up a new build.
The second (and the scariest) disadvantage is the maintenance cost. When using a self-hosted solution, you are also responsible for keeping the software up-to-date and secure. The same applies to the machines the whole self-hosted system runs on. Our initial hope was to delegate this to our system administrator. However, it quickly became clear that they would not be able to help with problems on specific platforms, such as native Android or iOS. As a solution, we delegated this task to some talented platform engineers. This created a lot of regular and emergency hours spent on keeping the system running and usable by all our teams and forced a set of platform engineers to maintain a CI instead of building apps.
In the end, while we had somewhat happier clients, we also had unhappy engineers—and high costs.
Our Codemagic era
As an agency, we have one really important requirement for the choice of the CI service provider - they needed to support using custom self-hosted code repositories. We have a lot of different clients, and their projects use different infrastructures, not only GitHub and GitLab. Forcing all of our clients to move to these two would have surely made them very unhappy.
Secondly, the pricing structure of the service was important. As an agency, we have a lot of active projects at a time, most with different stakeholders. Projects are usually small, with frequent builds and short build times. A pricing structure that worked either per user or per application count would have been very costly to us and to our clients, compared to one that used build time to price the CI service.
There was only one that fulfilled both of these requirements - Codemagic (then still called “Greenhouse CI”).
Based on these two criteria, we looked into about 4-5 different CI services, disregarding the self-hosted ones. There was only one that fulfilled both of these requirements - Codemagic (then still called “Greenhouse CI”). It allowed custom repositories and had a suitable pricing structure. Plus, it had a convenient support channel for our developers.
Now, when we had a partner that took care of the maintenance and engineering support problem, we also tackled the setup one. Together with the focus team, we created extensive documentation on how to access the company’s teams on Codemagic, how to configure new users, add new builds, and publish artifacts. Codemagic had guidance on most of this, but these often offered alternatives for doing the same thing, so we supplied our opinionated defaults. As an additional benefit, we immediately noticed that having user support provided by Codemagic via their web-based Intercom-powered widgets reduces the amount of internal support we have to handle ourselves. It creates a more frictionless workflow and happier engineers.
In the end, we had a new CI provider that solved our primary goal - to make both our engineers and clients happy by providing easy-to-understand and fast-to-set-up flows for engineers and allowing a fast feedback loop with relatively low costs to your clients.
Building SplitKey SDK in Codemagic
During our Codemagic usage, we have had the chance to set up some interesting CI builds. While most of our builds are smaller apps with simple setups focused on delivering information to the end-users, we have had some more demanding library projects.
SplitKey is actively used in multiple countries, including by 3.4 million people in Estonia, Latvia, and Lithuania, powering roughly 80 million monthly transactions.
One of these is the SplitKey SDK, a digital ID technology by Cybernetica. SplitKey is a technology that, when integrated into an application, enables anyone to allow end-users to digitally sign documents following European Union regulation, the eIDAS directive. SplitKey is actively used in multiple countries, including by 3.4 million people in Estonia, Latvia, and Lithuania, powering roughly 80 million monthly transactions. It is used to power secure authentication and signing, thus facilitating access to online banking, signing digital contracts, and providing authentication to government and 3rd-part services.
The demands for building the SplitKey SDK are non-trivial. Firstly, the SDK is built for Android and iOS platforms and consists of four interdependent sublibraries, bringing the total number of build projects across platforms to eight. To solve this, we have defined a pre-build script in the project’s YAML build configuration that resolves any needed dependencies between the components and ensures correct versions and release types are used. The script is versioned in the same repository as the rest of our code, making any updates to the script versionable and trackable.
Secondly, there is a strict requirement that the whole SDK must be buildable in an air-gapped computer (a machine with no Internet or networking access). Thus, all build dependencies for all builds need to be available without any online access. To better verify this requirement, we keep all dependencies in a separate repository, which is set up in the Codemagic build environment via a pre-build script in the project’s YAML. The projects of the SDK components are directed to only resolve dependencies from this source. This, along with an additional release-time test in a real air-gapped environment, ensures we support the air-gapped machine requirement.
Thirdly, releases are different than usual for the SplitKey SDK. In this project, we release the SDK source code, and the actual binary releases are built at the client’s site. Our client uses this process to apply the strictest control on building the actual binaries used in production.
These tools give us the confidence to do fast and agile source code releases while knowing the production binaries our client builds will meet the standards our client expects.
To support this, we run and build verification tooling for all components in Codemagic, consisting of unit tests and a GUI-enabled integration tester. These tools give us the confidence to do fast and agile source code releases while knowing the production binaries our client builds will meet the standards our client expects.
While this custom setup took time, Codemagic enabled us to do this in a maintainable and auditable way, bringing confidence and making our engineers and the client happy.
A good CI/CD is the key to success
A well-chosen CI/CD tool is a key to success and a way to make both engineers and clients happy. It gives engineers confidence and enables fast feedback for your stakeholders and clients. It allows you to build a unified CI strategy for your agency and optimize the setup and processes across your apps and teams. It saves time and builds trust to do what you do best – build world-class apps.
