How to manage your Flutter monorepos

Monorepos are extremely helpful for larger code bases. But they also come with additional cost of management. In this article, are we going through the process of managing a monorepo with a tool like Melos and set up our repository for CI/CD with Codemagic.

This article is written by Nils Reichardt and updated in November 2022.

Introduction to monorepos

Nowadays, we are seeing many companies and projects that are using the structure of a monorepo. To name a few examples: Flutter itself, FlutterFire (the Flutter packages for Firebase), Riverpod, the Plus Plugins by the Flutter Community, Very Good Ventures in projects like I/O Photo Booth.

But what is a monorepo? A monorepo is a way to manage multiple projects in one version-controlled repository.

Advantages

A monorepo has some useful advantages:

• Code Reuse: Split your codebase into small independent packages, which is great for code reuse and testing
• Better CI: With a monorepo you can easily trigger the CI when changing something else in your repository, like trigger the Flutter Integration tests, when making changes to the backend
• Dependency management: Have local packages without the need of a dependency manager, like pub.dev.
• Enforcing Layered Architecture: Enforce yourself and your team to apply a layered architecture by splitting the layers into multiple packages
• Everything in one place: New developers just clone the monorepo and having in one repository

Disadvantages

Like everything in life a monorepo does have some disadvantages:

• More overhead: You need to set up tools to manage the repository
• No per-project access control: When having everything in one repository, everyone with repository access can access everything.

Note: This is just a selection of advantages and disadvantages. There are many more things to consider when comparing a monorepo and multi-repos.

Scope of this article

After getting the basic understanding for a monorepo, we should set the scope for this article since monorepos are in general a big topic. You can have your frontend, your backend, your internal tools, your website, etc. in your monorepo. Google is known for having the largest code base in the world. They have everything in one repository. Covering everything of a monorepo would be too much.

This article will focus on Flutter/Dart monorepo in which you split your app into small independent packages.

Example of the article

To get a practical view, we take the Flutter counter app with a few adjustments as an example.

apps/
  counter_app
packages/
  counter_widgets
  counter_lint

In apps we have apps that we actually deploy. We could also have here an internal app, website, etc.

In packages we have our local packages.

You can check out the full source here.

Tools

As just mentioned, are tools extremely helpful to manage your monorepo. You will face with challenges like:

• Get the dependencies for all packages
• Check linting for all packages
• Check formatting for all packages
• Run tests for all packages
• Other challenges (like running build_runner in all packages, merge code coverage for all packages, etc.)

You can write your own bash script or your own CLI, which helps you to manage these tasks. However, this costs you some time. In order to be quicker, you can use the tools of the community, like Melos, Very Good CLI, Sidekick. In this article, we are going to use Melos. Melos is also used by repositories like FlutterFire, AWS Amplify (Flutter), Flame and Plus Plugins.

Setting up Melos

First, you need to install Melos. To install Melos, run the following command in your terminal:

dart pub global activate melos

To configure Melos, we need to create a top level melos.yaml file. The structure looks like this now:

apps/
  counter_app
packages/
  counter_widgets
  counter_lint
melos.yaml

Now, we create the melos.yaml with basic configuration:

# The name of the project (required) is used for displaying purposes within IO environments and IDEs.
name: counter

# A list of paths to local packages that are included in the Melos workspace. Each entry can be a specific path or a glob pattern.
packages:
  - "apps/*"
  - "packages/**"

# Recommended option for projects with Dart 2.17.0 or greater.
#
# This enables a new mechanism for linking local packages, which integrates
# better with other tooling (e.g. dart tool, flutter tool, IDE plugins) than the
# mechanism currently being used by default. Please read the documentation for
# usePubspecOverrides before enabling this feature.
#
# See https://melos.invertase.dev/getting-started#setup
command:
  bootstrap:
    usePubspecOverrides: true

After setting up the melos.yaml, run the bootstrap command to initialize Melos for your project:

melos bootstrap

Bootstrapping has 2 primary roles:

1. Installing all package dependencies (internally using pub get).
2. Locally linking any packages together.

In the melos.yaml we can also define our commands which are executed in every Dart/Flutter package inside our defined Melos workspace.

scripts:
  analyze:
    run: melos exec -- "flutter analyze"
    description: Run `flutter analyze` in all packages
  
  format:
    run: melos exec -- "flutter format . --set-exit-if-changed"
    description: Run `flutter format .` in all packages

  test:
    # Only run the test command when the package has a test directory
    run: melos exec --dir-exists=test -- "flutter test"
    description: Run `flutter test` in all packages

We are now able to execute our script with melos run SCRIPT_NAME. To run the flutter analyze command in all packages we can use this command:

melos run analyze

You can add in the melos.yaml file every script you want, like running the build_runner. Check also the Melos documentation to find more about the scripts configuration.

Checkout also the VS Code extension for Melos: melos-code. It helps you to work with Melos and VS Code.

Setting up your Flutter monorepo for CI/CD

You should be able to manage your monorepo locally with Melos. However, you might need to configure your CI/CD environment to fully support your monorepo. We are going to use Codemagic as a CI/CD provider.

Scope of our CI

Our CI pipeline should do the following checks for every pull request:

• Run the melos run analyze command
• Run the melos run format command
• Run the melos run test command
• Uploads the results of failed Golden tests

Configure CI/CD for a monorepo

Setup Codemagic

First, you need a Codemagic account. If you don’t have one already, you can sign up for Codemagic with your Git provider. Set up Codemagic with the Workflow Editor or the codemagic.yaml. If you need a step-by-step guide, you can follow this article to set up your monorepo for Codemagic: Introducing support for monorepos on Codemagic.

The Workflow Editor is simple for a basic app. However, for a monorepo it’s better to use the codemagic.yaml because we run our own commands with Melos. For that reason, this article only covers the setup for the codemagic.yaml.

Setup Melos for CI/CD

Setting up Melos in CI/CD is similar as setting it up for your local machine.

1. Run dart pub global activate melos
2. Run melos bootstrap

Let’s check out the basic configuration in the codemagic.yaml:

workflows:
  ci:
    name: CI
    instance_type: mac_mini
    # Setting the timeout for a build to 15 minutes.
    max_build_duration: 15
    environment:
      # Using the latest Flutter version.
      flutter: stable
    # This workflow should trigger when a new pull request opens or updates.
    triggering:
      events:
        - pull_request
    scripts:

      - name: Melos Bootstrap
         script: |
           dart pub global activate melos
           melos bootstrap           

Run Melos scripts

Let’s add our Melos scripts to the codemagic.yaml. This is how the codemagic.yaml looks now:

workflows:
  ci:
    name: CI
    instance_type: mac_mini
    # Setting the timeout for a build to 15 minutes.
    max_build_duration: 15
    environment:
      # Using the latest Flutter version.
      flutter: stable
    # This workflow should trigger when a new pull request opens or updates.
    triggering:
      events:
        - pull_request
    scripts:

      - name: Melos Bootstrap
         script: |
          dart pub global activate melos
          melos bootstrap          

      - name: Run Analyze
         script: melos run analyze

      - name: Run Format
         script: melos run format

      - name: Run Tests
         script: melos run test

Get the results of failed Golden tests

With Golden tests, you can render a widget and compare this to a screenshot. To understand more about Golden tests, check out this blog article: How to run Flutter Golden (Snapshot) tests with Codemagic CI/CD. If you have Golden tests in your Flutter repo, you may want to access the results of failed golden tests. When having a monorepo, you need to check every package, if there are results of failed Golden tests. To do this, you can just this script:

...
- name: Run Tests
  script: |
    melos run test
    
    # Upload results of failed golden tests if test command failed.
    if [ $? -ne 0 ]; then
      # Finds all "failures" folders and copies them to the export
      # directory. Therefore, we are able to view the results of the
      # failed golden tests.
      #
      # The command will use the exit code 0 (success) even when there are
      # no failures folders.
      find * -path '**/failures' -execdir bash -c "cp -r failures $FCI_EXPORT_DIR" \;
      
      # Because we caught the exit code of the test command, we need to
      # set manually again.
      exit 1
    fi    

Configure path conditions

At the moment, we run our CI for every pull request. No matter what changed in this pull quest. Even when we just change the documentation files or files in our backend (assuming we would also have our backend in our monorepo).

Therefore, you are using unnecessary Codemagic build minutes.

To make your Codemagic build minutes more effective, you can set path conditions. With path conditions, you can define that the CI should only run when changes were made for specific paths.

Just use the when keyword to configure the paths:

...
environment:
  # Using the latest Flutter version.
  flutter: stable
when:
  changeset:
    includes:
      # Only run the CI when a file in one of the following directories
      # changed.
      - "apps/**"
      - "packages/**"
      - "codemagic.yaml"
    excludes:
      # Don't run the CI when only .md files have changed.
      - "**/*.md"
# This workflow should trigger when a new pull request opens or updates.
triggering:
...

You can also check out the Codemagic documentation for run builds and builds steps conditionally for more information about conditional runs.

Conclusion

Monorepos are great for larger code bases. However, as you may have noticed in this article, they come with a bit more effort to manage. Nevertheless , you should now be able to manage your own Flutter monorepo with Melos and configure your CI/CD.

You can check out the full source of the repository here.

How to build Flutter apps on Codemagic

Start now

This article is written by Nils Reichardt, Co-Founder of Sharezone, a collaborative school planner for Android, iOS, Web and macOS with +300.000 registered users. He fell in love with Flutter since the first beta release in March 2018. You can find Nils on Twitter, GitHub and LinkedIn.