TL;DR: M2 machines are up to 33% faster than M1 machines! For macOS and iOS builds, we see build time improvements of ~50% compared with the Mac Pro.
CI/CD with M1 and M2 machines
One year ago, Codemagic introduced support for M1 machines, which revolutionized CI/CD by reducing build times by 70% compared with Intel Mac mini machines (see Flutter builds are way faster with M1 machines: A comparison of VMs).
But how do the new M2 chips, introduced last summer by Apple, compare? Codemagic is the first CI/CD provider to offer M2 support. At the moment M2 machines are only available on request. If you’re interested, then reach out to us.
Now, let’s examine these chips’ performance and determine if an upgrade is worthwhile.
This article is written by Nils Reichardt
Comparison setup
To ensure a comprehensive comparison, we considered the following factors:
- Apps used for the builds
- Flutter and Xcode versions
- Target platforms
- Build modes
Apps
We used two apps for the comparison:
- The default Counter app from Flutter
- Sharezone — an open-source school planner for students, parents, and teachers to organize themselves together
We primarily focused on Sharezone, as it is a real production app with 400,000+ registered users and a variety of dependencies (see Sharezone’s dependencies). This ensures a realistic evaluation of build times.
Platforms
We targeted three platforms:
- iOS
- macOS
- Tests (unit and widget tests)
System
For the test, we used Codemagic’s virtual machines, which are available for CI/CD. These machines have the following configurations:
- M2 Mac mini:
- M1 Mac mini:
- Mac Pro:
Software configuration
- Flutter: 3.7.12
- Xcode: 14.3 (Mac Pro uses 14.2)
To obtain a realistic view, all builds used Flutter’s release mode.
For tests, we simply used the flutter test command.
Dependency caching
We compared builds without and with caching. For caching, we cached the following paths:
- Dart cache:
$FLUTTER_ROOT/.pub-cache - CocoaPods cache:
HOME/Library/Caches/CocoaPods
For more information on using caching in Codemagic, consult the documentation: Dependency caching.
codemagic.yaml
The codemagic.yaml file contains all configurations used for the comparison.
Flutter Counter app: codemagic.yaml Sharezone app: codemagic.yaml
Accuracy
We executed each build three times and calculated the average. For example, the build time for the iOS Counter app represents the average time of three builds.
In total, we ran 63 builds. You can also view this spreadsheet, which shows all the raw data and links to the builds.
Results of comparing M2 mini vs. M1 mini vs. Mac Pro
Now let’s examine the results. You will notice a significant difference in build times between the Counter app and the Sharezone app, emphasizing the importance of using a real app for comparison.
All build times are measured in seconds.
Counter app results
| Platform | Mac Pro | M1 mini | M2 mini |
|---|---|---|---|
| iOS | 67 | 59 | 41 |
| macOS | 58 | 48 | 38 |
Mac Pro vs. M2: -38.6% (iOS) and -33.5% (macOS)
M1 vs. M2: -29.5% (iOS) and -20.7% (macOS)
Sharezone app results
iOS
| Caching | Mac Pro | M1 mini | M2 mini |
|---|---|---|---|
| No | 617 | 482 | 302 |
| Yes | 493 | 290 | 244 |
The improvement from M1 to M2 isn’t as significant as that from Mac Pro to M2. Nonetheless, M2 builds are still 20% faster than M1 (without caching). Compared with the Mac Pro, the M2 machines cut the build time in half.
Mac Pro vs. M2: -51.0% (without caching) and -50.5% (with caching)
M1 vs. M2: -20.8% (without caching) and -15.7% (with caching)
macOS
| Caching | Mac Pro | M1 mini | M2 mini |
|---|---|---|---|
| No | 631 | 410 | 291 |
| Yes | 485 | 332 | 223 |
The macOS builds show greater improvements than the iOS builds: 29% without caching and 33% with caching (M2 vs. M1). When comparing M2 and Mac Pro, build times are 53% to 54% faster.
Mac Pro vs. M2: -53.8% (without caching) and -54.1% (with caching)
M1 vs. M2: -29.0% (without caching) and -33.0% (with caching)
Unit and widget tests
| Mac Pro | M1 mini | M2 mini |
|---|---|---|
| 82 | 40 | 35 |
When comparing the unit and widget tests, there’s almost no difference between the M1 and M2 machines but a 57% improvement compared with the Mac Pro machine.
Mac Pro vs. M2: -57.9%
M1 vs. M2: -12.61%
As a side note, Sharezone has 211 unit and widget tests.
Overall
Summary
In conclusion, M2 machines are approximately 15% to 33% faster than their M1 counterparts. The most significant difference is observed in macOS builds, with M2 machines being up to 33% faster. iOS builds see an improvement of up to 20%.
M2 machines are only available on request at the moment. If you’re interested, then reach out to us:
Overall, this comparison shows how Apple’s M2 chip performs better than the M1 chip, especially in macOS builds. When considering whether to upgrade their CI/CD pipeline machines, developers and teams must take these distinctions into account. Faster build times provide advantages that can result in speedier iteration rates, more effective development cycles, and ultimately better final products for users.
In the upcoming years, we anticipate more enhancements in performance and efficacy thanks to Apple’s silicon technology’s ongoing breakthroughs. To maintain a competitive edge in the ever-changing software development ecosystem, developers and organizations should closely monitor these advancements and be ready to adopt these changes.
This article is written by Nils Reichardt, Co-Founder of Sharezone, a collaborative school planner for Android, iOS, Web and macOS with +400.000 registered users. He fell in love with Flutter since the first beta release in March 2018. You can find Nils on Twitter, GitHub and LinkedIn.
