CI/CD Build Speed Benchmark: Codemagic vs GitHub Actions vs Bitrise

Written by Masaki Sato

For teams using CI/CD, the specs of the build machine can have a significant impact on development productivity. Faster builds mean shorter fix-and-verify cycles, which speed up the overall development process. However, it’s hard to know how fast each CI/CD service actually is without comparing them under the same conditions.

In this article, I compare the iOS build speeds of GitHub Actions, Bitrise, and Codemagic using the same Flutter project, and compare them in terms of cost-performance as well.

Benchmark Conditions

Test Subject

For this benchmark, I used the open-source meditation app Medito. It has many reviews on the store, making it a production-level Flutter app of sufficient scale.

Measurement Method

I ran an iOS release build equivalent to flutter build ios --release --no-codesign on each service, and compared the elapsed time from build start to finish.

Build logs for each run are available via the public links below.

Build Speed Benchmark

Let’s look at the benchmark results for each service.

GitHub Actions

GitHub Actions is a CI/CD service provided by GitHub, known for its seamless integration with GitHub repositories. I tested two runner types: the standard runner and the larger runner.

Standard Runner Benchmark (Apple M1, 3-core CPU, 7GB RAM)

The standard macOS runner, available to all accounts.

Build time: 12 min 18 sec

See the build log for details.

Larger Runner Benchmark (Apple M2 Pro, 5 vCPU, 14GB RAM)

A larger macOS runner, available only to organizations on the GitHub Team or GitHub Enterprise Cloud plan.

Build time: 6 min 31 sec

See the build log for details.

Bitrise

Bitrise is a CI/CD service specialized for mobile development, featuring a GUI-based workflow editor. The machine used for this test was the M2 Pro Large (6 CPU, 14GB RAM), available with the Pro plan.

Build time: 8 min 13 sec

See the build log for details.

Codemagic

Codemagic is a CI/CD service for mobile development, featuring extensive third-party integrations and the ability to preview apps in the browser. I tested with two different machine types.

Mac mini M2 Benchmark (8-core CPU, 8GB RAM)

Available on the free tier or pay-as-you-go plan.

Build time: 7 min 20 sec

See the build log for details.

Mac mini M4 Benchmark (10-core CPU, 16GB RAM)

Available on the pay-as-you-go plan.

Build time: 6 min 58 sec

See the build log for details.

Benchmark Comparison Summary

Speed comparison is calculated relative to the GitHub Actions standard runner as the baseline.

This article focused on iOS builds, but benchmarks for other platforms and higher-tier machines are also available here.

Cost-Performance Comparison

In addition to build speed, cost is an important factor when choosing a CI/CD service. Let’s take a look at the pricing models for each service and then compare the build costs.

Pricing Models

GitHub Actions

Standard Runner (M1)

• Pay-as-you-go: $0.062/min
• Free for public repositories
• Private repositories get 2,000 free minutes/month (macOS consumes at 10x rate, so effectively 200 minutes)

Larger Runner (M2 Pro)

• Pay-as-you-go: $0.102/min
• Available only to organizations on the GitHub Team or GitHub Enterprise Cloud plan

Bitrise

• Pro plan: from $218/month ($200/month if billed annually), which includes up to 250 builds per month (required to use M2 Pro Large)
• For higher build volumes or other plans, please refer to Bitrise’s pricing page and contact them for details

Codemagic

• Pay as you go: $0.095/min (M2), $0.114/min (M4)
• Fixed price: $3,990/year — unlimited M2 builds

Pricing is as of the time of writing. Please check each service’s official website for the latest information.

Cost Per Build

Here’s what each iOS build in our benchmark costs on each service.

Monthly Cost by Build Frequency

Based on the build times and per-minute rates above, here’s the estimated monthly cost by number of builds. For the GitHub Actions standard runner (M1), the free tier for private repositories (effectively 200 macOS minutes) is taken into account.

Key takeaways:

• The GitHub Actions standard runner (M1) has a generous free tier, making it cost-effective for projects with low build frequency. However, its build speed was the slowest in this comparison, and costs increase as the number of builds grows. The larger runner (M2 Pro) was the fastest in this comparison and is also competitive on pay-as-you-go cost.
• Bitrise’s Pro plan includes up to 250 builds per month at a flat rate, making costs predictable for teams with a consistent build volume. However, it can be expensive for teams with low build frequency.
• Codemagic really pays off when build volume is high. With its fixed-price plan ($3,990/year, roughly $333/month) offering unlimited builds on a capable M2 machine, costs stay flat as build volume grows. Based on the build times in this benchmark, at around 500 builds per month, the fixed-price plan becomes cheaper than every other service, and the gap only grows from there.

Conclusion

In this article, we compared the build speed benchmarks of three CI/CD services using the same Flutter project. The GitHub Actions larger runner (M2 Pro) is an excellent choice for both speed and cost. If you want to keep costs to a minimum and have low build frequency, the GitHub Actions standard runner with its generous free tier is also a viable option. And for teams with a high build volume, Codemagic’s fixed-price plan, which offers unlimited builds on a capable M2 machine, is a great way to optimize costs.

If you’re interested in Codemagic, give it a try here!

If you have any questions or need help, feel free to reach out to the support community:

• GitHub Discussion: https://github.com/orgs/codemagic-ci-cd/discussions
• Discord community: https://discord.com/invite/pefznye93R