GitHub Actions for Monorepos

GitHub Actions for Monorepos

In the world of software development, a monorepo (short for monolithic repository) is a version control strategy where multiple projects, often related, are stored in a single repository. This approach can simplify dependency management, improve code sharing, and streamline collaboration across teams. However, managing CI/CD workflows in a monorepo can present unique challenges, especially when using GitHub Actions. In this lesson, we will delve into the intricacies of configuring GitHub Actions workflows for monorepo projects, exploring best practices, advanced examples, and practical use cases.

Understanding Monorepos

Before we dive into GitHub Actions, it's essential to understand what a monorepo is and its advantages:

• Single Source of Truth: All related projects and libraries are contained within one repository, making it easier to manage dependencies and track changes.
• Simplified Dependency Management: Changes to shared libraries can be made in one place, reducing the risk of version mismatches.
• Improved Collaboration: Teams can work on different projects simultaneously without worrying about separate repositories.
• Consistent Tooling: A unified build and test process can be established across multiple projects, ensuring consistency in development practices.

Challenges of Using GitHub Actions with Monorepos

While monorepos offer many benefits, they also introduce challenges in CI/CD configuration:

• Complex Workflows: With multiple projects in a single repository, workflows can become complex and harder to manage.
• Resource Management: Running tests and builds for all projects on every commit can lead to increased resource consumption and longer build times.
• Selective Execution: You may want to run jobs only for specific projects that have changed, rather than for the entire repository.

Configuring GitHub Actions for Monorepos

To effectively manage CI/CD workflows in a monorepo setup, we can leverage GitHub Actions' features such as paths filters, job dependencies, and conditional execution.

Basic Workflow Structure

A typical workflow for a monorepo might look like the following:

name: CI for Monorepo

on:
  push:
    branches:
      - main
    paths:
      - 'project-a/**'
      - 'project-b/**'
      - 'project-c/**'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '14'

      - name: Install dependencies for Project A
        run: |
          cd project-a
          npm install

      - name: Run tests for Project A
        run: |
          cd project-a
          npm test

      - name: Install dependencies for Project B
        run: |
          cd project-b
          npm install

      - name: Run tests for Project B
        run: |
          cd project-b
          npm test

      - name: Install dependencies for Project C
        run: |
          cd project-c
          npm install

      - name: Run tests for Project C
        run: |
          cd project-c
          npm test

In this example: - The workflow is triggered on pushes to the main branch. - It checks for changes in specific project directories (project-a, project-b, and project-c). - It runs separate build and test steps for each project, ensuring that the workflow only executes relevant commands based on the changes.

Path Filters

Path filters are a crucial feature when working with monorepos. They allow you to specify which files or directories must change for a workflow to trigger. This is particularly useful in a monorepo where you might not want to run the entire CI/CD pipeline for changes in a single project.

Example of Path Filters

Here's an example of how to use path filters to trigger workflows only for specific projects:

on:
  push:
    branches:
      - main
    paths:
      - 'project-a/**':
          types:
            - modified
      - 'project-b/**':
          types:
            - modified

In this configuration: - The workflow is triggered only if files in project-a or project-b are modified. - This selective triggering can significantly reduce unnecessary builds and tests, improving overall performance.

Conditional Job Execution

In some cases, you may want to conditionally execute jobs based on the modified files. For instance, if only project-a changes, you might want to skip jobs related to project-b and project-c.

Example of Conditional Job Execution

You can achieve this by using the if conditional in your jobs:

jobs:
  build-project-a:
    runs-on: ubuntu-latest
    if: ${{ github.event.head_commit.modified contains 'project-a/' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Build Project A
        run: cd project-a && npm run build

  build-project-b:
    runs-on: ubuntu-latest
    if: ${{ github.event.head_commit.modified contains 'project-b/' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Build Project B
        run: cd project-b && npm run build

In this example: - The if condition checks whether the last commit modified files in the specified project directories before executing the corresponding build jobs.

Managing Dependencies in Monorepos

In a monorepo setup, managing dependencies can be complex. Tools like Lerna or Nx can help streamline this process by providing commands to manage dependencies across packages effectively.

Using Lerna for Dependency Management

Lerna is a popular tool for managing JavaScript projects with multiple packages. It helps in bootstrapping projects, managing versions, and publishing packages. Here's an example of how you can integrate Lerna with GitHub Actions:

name: CI with Lerna

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '14'

      - name: Install Lerna
        run: npm install -g lerna

      - name: Bootstrap Lerna
        run: lerna bootstrap

      - name: Run tests
        run: lerna run test

In this workflow: - Lerna is installed and bootstrapped, ensuring all dependencies are linked correctly across packages. - Tests are executed for all packages managed by Lerna.

Advanced Use Cases

Deploying Changes Based on Project

In some scenarios, you might want to deploy changes only for specific projects when they are modified. This can be accomplished using conditional deployment steps in your workflow:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Deploy Project A
        if: ${{ github.event.head_commit.modified contains 'project-a/' }}
        run: cd project-a && ./deploy.sh

      - name: Deploy Project B
        if: ${{ github.event.head_commit.modified contains 'project-b/' }}
        run: cd project-b && ./deploy.sh

This example demonstrates how to conditionally deploy each project based on changes, ensuring that only the affected projects are deployed, thus optimizing deployment times and resource usage.

Performance Considerations

When working with monorepos, performance is a critical aspect to consider. Here are some best practices to improve workflow performance:

• Limit Workflow Triggers: Use path filters and conditional executions to limit the workflows to only those that need to run based on changes.
• Parallel Jobs: If possible, structure your jobs to run in parallel, reducing overall execution time.
• Cache Dependencies: Utilize caching strategies to speed up dependency installation. For example:

      - name: Cache Node.js modules
        uses: actions/cache@v2
        with:
          path: ~/.npm
          key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-npm-

Comparison with Alternative Approaches

While monorepos offer many benefits, they are not the only approach to managing multiple projects. Here’s a brief comparison with multi-repo setups:

Common Interview Questions

1. What are the advantages of using a monorepo?
   Monorepos simplify dependency management, improve collaboration, and provide a single source of truth for related projects.

2. How can you optimize CI/CD workflows in a monorepo?
   Use path filters, conditional job execution, and caching strategies to optimize performance and limit unnecessary builds.

3. What tools can help manage dependencies in a monorepo?
   Tools like Lerna and Nx can help manage dependencies and streamline workflows in monorepo setups.

Mini Project: Setting Up a Monorepo CI/CD Pipeline

For this mini project, you will create a GitHub Actions workflow for a sample monorepo containing two projects: project-a and project-b. Follow these steps:

1. Create a new GitHub repository and add two directories: project-a and project-b.
2. Inside each directory, add a simple Node.js application with a package.json file and a test script.
3. Set up a GitHub Actions workflow that: - Triggers on push events. - Uses path filters to run jobs only for changed projects. - Runs tests for each project. - Deploys the affected projects.

Key Takeaways

• A monorepo is a single repository that contains multiple projects, simplifying dependency management and collaboration.
• GitHub Actions can be configured to handle CI/CD workflows in monorepos using path filters, conditional job execution, and caching strategies.
• Performance optimization techniques include limiting workflow triggers and running jobs in parallel.
• Tools like Lerna can help manage dependencies effectively in a monorepo setup.

As we conclude this lesson, we have explored how to leverage GitHub Actions for monorepos, addressing both the challenges and advantages of this approach. In the next lesson, we will dive into Custom Action Development, where you will learn how to create your own GitHub Actions tailored to your specific needs.

Hands-on Exercises

1. Basic Monorepo Setup: Create a simple monorepo with two projects (project-a and project-b). Set up a basic GitHub Actions workflow that runs tests for both projects on every push.

2. Implement Path Filters: Modify the workflow to include path filters so that tests only run for the project that was changed in the last commit.

3. Conditional Job Execution: Enhance the workflow to conditionally execute deployment steps based on which project was modified.

4. Integrate Lerna: Refactor your monorepo to use Lerna for dependency management and update your GitHub Actions workflow to bootstrap and run tests using Lerna.

5. Mini Project: Create a GitHub Actions workflow for a monorepo containing three projects. Ensure that the workflow uses path filters, conditional job execution, and caching strategies to optimize performance.

• A monorepo contains multiple projects in a single repository, simplifying dependency management and collaboration.
• GitHub Actions workflows can be configured to handle monorepos using path filters and conditional execution.
• Performance optimization techniques are essential for managing CI/CD workflows in monorepos.
• Tools like Lerna can assist in managing dependencies across multiple projects effectively.
• Understanding the unique challenges of monorepos is crucial for setting up efficient workflows.