Written by Rudrank Riyam.
Have you ever been in situations where you wished your local builds were faster? Like, significantly — up to 70% faster? That’s what Spotify engineering has achieved with their open-source tool called XCRemoteCache.
XCRemoteCache is a remote cache tool for your Xcode projects that reuses target artifacts. The artifacts are generated on a remote machine, preferably on your CI service, that uploads it to a WebDAV capable REST server. Then, your local machine can access it from that server.
Who needs remote caching and what for?
Normally, developers compile the project on their machine every time they want to build it. A remote cache helps you speed up builds by compiling once, and then that cache can be used by everyone. It is particularly useful for big projects with multiple targets or dependencies that don’t often require updates.
You can cache once, for example, on the main branch. As long as the input files and compilation parameters remain the same, the cache can speed up the build process for your local builds.
However, there are a few things you should consider for optimal caching:
- Fetching the cache from the server requires network usage. If you’re caching every unit, the network traffic overhead will offset the build speed gained.
- If you’re caching the whole project, it will eventually result in a full build on the local machine, which violates the tool’s purpose.
This article discusses how you can use the XCRemoteCache tool to store the cache on a remote server using Codemagic and access that cache on your local machine.
How XCRemoteCache works
XCRemoteCache has two modes:
- The Producer mode generates the cache. It generates the artifacts by building the targets and uploading meta and artifact files to your remote cache server.
- The Consumer mode consumes the cache. It tries to reuse the artifacts that are available on your remote server.
The producer mode also creates a metafile containing all the compilation files the compiler used and the full SHA-1 commit identifier it was built against.
The next step is to use a remote server to use these two modes to produce the cache and then consume it from the local machine.
Creating a remote server for XCRemoteCache
XCRemoteCache can use a WebDAV capable REST server that supports PUT, GET, and HEAD methods. For a sample REST cache server, you can use the docker image provided in the repository. It also supports Amazon S3 and Google Cloud Storage buckets, which you can use as cache servers using Amazon v4 Signature Authorization.
For this article, we’ll use Amazon S3, which provides a generous plan to start. You can create your account here. Then, create a new bucket, and choose the closest AWS Region for best performance.
XCRemoteCache requires the Access Key ID and a secret access key. You can find them under the Your Security Credentials section. Create a New Access Key, and save the ID and key for future use.
Creating configuration for XCRemoteCache
We’ll use Codemagic CI/CD to generate and upload the cache to the server. So, if you want to follow these steps and don’t have an account, sign up here.
XCRemoteCache uses the configuration file .rcinfo to change the modes and store information about the cache server, the link to the primary repository, and other configurations. You can find the full list of configuration parameters here.
Create a .rcinfo YAML file. This file should be next to your .xcodeproj file. Add the following entries to it. In this example, the AWS region closest to the developers’ residence has been chosen.
primary_repo: https://linkToTheRepository.git
primary_branch: main
cache_addresses:
- https:// https://bucketName.s3.ap-south-1.amazonaws.com
aws_secret_key: <Secret Access Key>
aws_access_key: <Access Key ID>
aws_region: ap-south-1
aws_service: s3
Next, download the XCRemoteCache ZIP file from the releases page. Recently, they’ve added support for Apple silicon remote machines.
Download the universal binary that includes both arm64 and x86_64 architectures. Rename it to xcremotecache, and unzip the bundle next to your .xcodeproj.
Both the workflow configuration and XCRemoteCache should be in the same folder as the
.xcodeprojfile.
Using Codemagic to generate cache
Next, we’ll create the workflow configuration to upload the cache to the server using Codemagic. We’ll run the automatic integration script for the producer side.
Here’s a sample template that you can use:
workflows:
cache-ios-app:
name: Cache App
environment:
xcode: latest
cocoapods: default
vars:
XCODE_PROJECT: "<NameOfProject>.xcodeproj"
XCODE_TARGET: "<NameOfTarget>"
XCODE_SCHEME: "<NameOfScheme>"
scripts:
- name: Setup XCRemoteCache
script: |
xcremotecache/xcprepare integrate --input "$XCODE_PROJECT" --mode producer --final-producer-target "$XCODE_TARGET"
- name: Build IPA
script: |
xcodebuild build -project "$XCODE_PROJECT" -scheme "$XCODE_SCHEME" CODE_SIGN_INDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO
artifacts:
- build/ios/ipa/*.ipa
- /tmp/xcodebuild_logs/*.log
This uploads the cache to the remote server, which you can then use on your local machine. Run the following command in the directory where .xcodeproj is located:
xcremotecache/xcprepare integrate --input <NameOfProject>.xcodeproj --mode consumer
And then normally build the project to reuse the cache.
Using CocoaPods plugin for XCRemoteCache
XCRemoteCache provides a CocoaPods plugin that integrates XCRemoteCache with the project. This is helpful if you want to cache your CocoaPods dependencies.
If you’re using RubyGems, you can install using the following command:
gem install cocoapods-xcremotecache
In your Podfile, add the plugin reference:
plugin 'cocoapods-xcremotecache'
Configure XCRemoteCache at the top of the definition. As we’re using AWS S3, we provide the secret key as well:
plugin 'cocoapods-xcremotecache'
xcremotecache({
'cache_addresses' => ['https://bucketName.s3.ap-south-1.amazonaws.com/'],
'primary_repo' => 'https://linkToTheRepository.git',
'mode' => 'producer',
'final_target' => '<Name of Target>',
'primary_branch' => 'main',
'aws_secret_key' => '<Secret Access Key>',
'aws_access_key' => '<Access Key ID>',
'aws_region' => 'ap-south-1',
'aws_service' => 's3'
})
target 'XCRemoteCacheExample' do
pod 'SDWebImageSwiftUI'
end
Finally, call pod install and verify that [XCRC] XCRemoteCache enabled is printed in the console.
While using the CocoaPods plugin, you don’t have to call
xcprepare integrateagain.
Here is the updated workflow configuration to support CocoaPods:
workflows:
cache-ios-app:
name: Cache App
environment:
xcode: latest
cocoapods: default
vars:
XCODE_WORKSPACE: "<NameOfWorkspace>.xcworkspace"
XCODE_SCHEME: "<NameOfScheme>"
scripts:
- name: Install CocoaPods plugin
script: |
gem install cocoapods-xcremotecache
pod install
- name: Build IPA
script: |
xcodebuild build -workspace "$XCODE_WORKSPACE" -scheme "$XCODE_SCHEME" CODE_SIGN_INDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO
artifacts:
- build/ios/ipa/*.ipa
- /tmp/xcodebuild_logs/*.log
This caches the dependencies on the remote server, which you can use locally for your builds!
Limitations
Although XCRemoteCache works great for dependencies managed by CocoaPods, Carthage, or any other custom dependency manager, Swift Package Manager (SPM) dependencies are not supported at the time of writing this article.
SPM doesn’t allow customizing Build Settings. And that’s why XCRemoteCache cannot specify clang and swiftc wrappers that control if it should skip the local compilation or not.
Conclusion
Saving developers’ productive time is crucial, and using XCRemoteCache helps them reduce build times by caching targets and dependencies remotely.
To take full advantage of the tool, ensure that the project is multi-target and the remote server’s location is close to the developers’ machines.
You can find the project with starter template workflows with and without Cocoapods dependencies here.
We hope this article helped you integrate XCRemoteCache into your project. If you have any suggestions or feedback, join our Slack community or mention @codemagicio on Twitter!
