With so many new software quality processes released in the last few years, how can you know if implementing Continuous Integration and Continuous Delivery is worth it for your apps?
Lewis Cianci looks at some of the use cases, and some of the reasons why it might be worth your time.
As a software developer, I think we can agree that we have seen an incredible amount of change over the last five years. In these new technologies and methodologies, there’s simply not enough time to decide if something is right for you, or your software development needs. So, everything runs the risk of fading into background noise.
One of these things is the rise of Continuous Integration/Continuous Deployment (CI/CD) tooling in this time. In this article, we’ll take more of a practical approach to considering CI/CD, and what it can do for you.
>> Back to basics: The ultimate guide to continuous integration and delivery (CI/CD)
The value in CI/CD
There’s a saying in the advertising world, that goes something like this: Advertising shouldn’t ever cost you money. The meaning of it is that if you have an advertising campaign for your product or service, and you pay money for that campaign, you should attract more than that amount of money in sales for your product or service. After all, if you don’t, then what are you paying for? To make the advertising company rich?😌
Anything you adopt in your development pipeline should more or less follow the same principle. It shouldn’t add a cost, whatever it may be (increase in build time, more licensing fees, etc) without ultimately saving you more in time, or money, then if you didn’t have that particular thing. That’s essentially the pitch of any piece of software, or service that you purchase today.
So CI/CD offers to make us more efficient software developers. How?
To answer this question, let’s look at how someone might develop an application without a CI/CD solution.
Click, wait, click, wait, a life without CI/CD
Let’s take a trivial web application as an example. This trivial example will be called Pictures of Cats. There are two components in this example.
Web Server: Written in ASP .NET Core and C#, featuring an API where people can download a random picture of a cat, and people can favourite a particular picture of a cat. Every night the web server runs a scheduled task to aggregate the cats that were favourited that day and emails a report to people on a subscription list with the “cat stats” for that day.
Client App: We’re not messing around with this app, we’ve decided that just a website or even a Single Page App won’t be performant enough for this app, no, we need a native experience! Our cats deserve the best. So we have a phone app written in Flutter that queries the web server shown above for a picture of a cat. It can also show a leader board, etc.
That’s our million dollar idea. And if we strike out, we still get to look at pictures of cats, so that’s gotta be worth it, right?
What we would normally do
So, we’d spec out a website to perform that function, set up a database to store users details and logins (securely! of course). And then make a phone app. And we’re done!
But we’ve lost in two key areas.
Running the build process manually
Every time we make an update for our app, we’ll have to carry out the following steps manually.
- Build the app in release mode for iOS and Android
- Increment the Android version number (otherwise, the store will reject it because there’s already an app bundle/apk with the same version uploaded)
- Increment the iOS bundle version (for the same reason as above)
- Go to the Play Console
- Upload APK/App Bundle
- Fill out required fields (what’s new in this version etc) *
- Send the APK to the appropriate release channel
- Open XCode
- Publish app to App Store Connect from xcode
- Complete the app encryption question to get your app released to Testflight *
* you’d have to do this in some capacity in a CI/CD solution, but it can be automated.
So, that’s okay for our first app. But we have to release updates in the future. Doing the above steps over and over again will get tedious. Plus, we’re manually managing the version code for our app with each deployment. If we get the versions wrong per deployment, this can make troubleshooting difficult in the future, especially if your iOS and Android releases are different versions for no good reason.
Also! We went straight from building our app to deploying it. We didn’t write tests, or use any tests to ensure quality! Let’s take a minute to think about what this means.
Every app we create exists to solve a problem. But how do we know that our app actually works? If your answer is that you know it works because you don’t get many crash reports, then aren’t you effectively using your users as testers?
Anyway, back to our super exciting app release.
We release Pictures of Cats v1.0 to the store…
…and it goes viral. There’s a surprising amount of money in this catatonic app idea. This increased influx of users bring with them a wide variety of devices, countries and bugs.
Simple things, like a change of timezone, could introduce an issue where comments on the cat photos are out by a few hours. Or literally anything could happen in our app because we haven’t written any tests.
Fixing our first bug
An issue crops up. People are using the app and the times are all over the place. Pretty quickly it becomes apparent that the reason for this is because DateTime’s are being shown in their UTC format, not a local time based on where the user is. We would check who it was affecting, and then try to work out a fix. This bug report process isn’t as polite as lovely emails from users coming into your inbox, no, it’s usually more like one star Google Play reviews (“rubbish app! can’t even get the time of when a cat was uploaded correct! 1 star…”. Or something like that. You get the drift.)
We track it down to how our JSON is being deserialised, and we change our JSON de/serialising configuration in this instance, and re-run our tests to validate that we haven’t broken anything else. Ah, wait, those tests…
We’ll get around to writing them one day. For now, let’s just shake it down lightly by using the app and confirming that everything looks okay. That’ll do, right? Plus, you’re on a timeframe! People are using your app and it’s not working which is unacceptable.
Once you’re happy with this you’d push it out to your beta channel, and then onto production as well.
Little do we know though, by changing how our DateTime’s are de/serialised, we’ve introduced another bug. You didn’t have any tests to validate this use case. So when the client tries to retrieve the DateTime value from another source, the conversion fails, or the time is wrong.
This sounds like a pretty unlikely bug to run in to I know that, but you’d be surprised the amount of times something like DateTime’s can make you come unstuck. It’s unlikely that it would have a huge impact on the Pictures of Cats app, but another app I wrote had resources that could be booked. For example, someone would log into it and book a resource for use, at a certain time of day.
It’s simple, right? Everything worked fine locally. Then I deployed it to Azure onto a server on the cheapest instance available. Of course, it was a different time zone to where I was in the world.
Then the reports started coming in. People booking the resources noted a weird issue. When they booked a resource, it would actually put the booking in for 10 hours in the past. 10 am bookings became 2 am bookings. And so on.
In my personal experience, fixing issues without a solid test foundation is just as likely to introduce more bugs, potentially even worse than the one you were trying to fix in the first place.
So what now?
We’ve inadvertently introduced a bug into our app, and we’ve deployed it. Reports of issues start trickling in in the form of one star reviews, emails from our customers, etc. If you have Crashlytics or Sentry.io set up, you might get some good stack traces to help you work out the issue. But then, the amount of effort involved in fixing the is still pretty big. You have to patch the issue, build for iOS, build for Android, and then deploy it and get past the store approval process. By that time, people may have uninstalled your app, and especially if it is paid or had In-App Purchases (IAP), you may have some trust issues with the quality of your apps.
Now you’re just kind of caught in the never-ending process of bug whack-a-mole, fixing issues and potentially introducing more as you go along. You have no time - every time another issue pops up you gain more one star reviews. It’s not a good outcome. You still have no tests, you still have no way of answering the question “How do I know my app works?”
What went wrong?
In the above example, we made a mistake with our initial implementation. This isn’t actually the main issue though - mistakes happen all the time and it’s how we become better developers.
The main issue is that we didn’t test our change thoroughly enough, and we didn’t have a good CI/CD solution to rely on to avoid these kinds of issues. We would have had a much better chance of avoiding the above situation had we implemented a CI/CD workflow.
What is Continuous Integration?
The destination for Continuous Integration is to simply find an automated way to build, test, and then package those applications.
What is Continuous Deployment?
This essentially takes the output of the above process and makes it available to the people who should have it (your customers, testers, etc).
Let’s have a do-over
So, let’s re-visit the idea above, but this time, we’re going to use our CI/CD solution. We’ll have to spend more time before deploying the app to get our CI/CD setup working correctly, but let’s see if this investment pays off.
We write our app again. It’s the same thing, same app, same amount of cats. But this time we follow an aspect of Test Driven Design, and instead of just sitting down and penning the UI and business logic from scratch, we actually write tests first, and then write the code that satisfies these tests.
That might sound like a roundabout way of doing things, but consider for a moment. Your getCats() function in your provider/service will always return a list of cats. Your getCatsRanked() will always return a list of cats, ordered from high to low. So your test cases just assert what you yourself already know, that these functions will return these types of data. This way, as you build out your app, your test suite grows with it. Instead of getting all the way to the end and trying to write the unit tests then, you build tests as you add functionality.
In Dart, these are just unit tests, for testing the services in your app. You could write a unit test to assert that a List has cats in it, and that the counter is always reverse descending order. And then you could write a widget test to ensure that they are laid out in the correct order, by checking the text value of a constructed Text field. All of this you can accomplish without even so much as starting an emulator.
We have a well tested, high quality app now. Let’s plug it into our CI/CD solution.
Connecting it to Codemagic
I know what you’re thinking. Ugh, of course, you’re connecting it to Codemagic. I’m on the Codemagic blog, what did I expect.
Are you just a paid shill, recommending Codemagic for my CI/CD solution?
I mean, it would make sense. You’re on the Codemagic blog, reading about Codemagic CI/CD, and here I am, you guessed it, recommending you to use Codemagic.
Before I started contributing articles for Codemagic’s blog, I chose them as my CI/CD provider for my apps. The reasons why I did this are still completely valid today. If I had experienced a better Flutter CI/CD build solution than Codemagic I would recommend them instead, but Codemagic is the easiest, most straightforward CI/CD provider I have found for Flutter. If I thought someone else was better I would write about them instead (but it might make things awkward for me and the people that run this blog…)
Plus, it does cool things. You can deploy your app to web by checking a box in the deployment pipeline, and it just works.
You can put your apps up for free, and their standard build pipeline will run tests on your Flutter app. If the tests fail, your app won’t deploy to the stores. If they pass, you can configure it to only deploy to your pre-approved list of testers. Neat, huh?
>> Read more about getting started with Codemagic CI/CD for Flutter
The initial scenario, but with tests and CI/CD
You push your build, Codemagic runs the build, and then afterwards, runs your tests for your app. Codemagic automatically increments the version for our Play Store and iOS versions, so they are kept in sync automatically. So our crash reports and other reporting makes sense as both of our app versions are the same between platforms.
Also, the handy repository of tests that we have built up over time are run against our latest build. If any of our tests fail, then the app doesn’t get deployed to the testers. Instead, we just get a report telling us what tests have failed, for us to action.
However we choose to build this out is completely up to us. We could use codemagic.yaml to trigger the deployment of our environment on a remote Azure instance, so we can instantiate a test instance of our web app for our test phone app to connect to, and then run full end-to-end integration tests. The complexity of this will grow as the complexity as your app itself grows.
If you don’t want to go all-in with a CI/CD provider like Codemagic today, or you think you might in the future, that’s perfectly fine too. One of the main tenants of DevOps (used as a buzzword far too frequently) is to automate as much stuff as you can. You can use tools like fastlane to build, test, and release your apps locally from your computer before committing to a CI/CD provider. Whatever you learn in that process will benefit you later.
Using CI/CD creates a safer, more reliable development experience. Writing tests and implementing a CI/CD provider does take time initially, but in almost all cases, you gain this time back by simply knowing that your app works as you expect. And after all, the last thing you want is to be up at night worrying about your Pictures of Cats app 🐱
Lewis Cianci is a software developer in Brisbane, Australia. His first computer had a tape drive. He’s been developing software for at least ten years, and has used quite a few mobile development frameworks (like Ionic and Xamarin Forms) in his time. After converting to Flutter, though, he’s never going back. You can reach him at his blog, read about other non-fluttery things at Medium, or maybe catch a glimpse of him at your nearest and most fanciest coffee shop with him and his dear wife.
More articles by Lewis:
