Introducing GitHub Actions Importer

We’re excited to announce a public preview of GitHub Actions Importer, which helps you forecast, plan, and facilitate migrations from your current CI/CD tool to GitHub Actions.

Doing individual migrations is relatively easy in isolation. However, for companies that have a large and established CI/CD footprint, having tooling available to assist with migrations is key to their ability to adopt Actions at scale. In the time that we’ve been developing and using GitHub Actions Importer in its private preview form, we’ve already encountered numerous customers that have thousands of pipelines—even in excess of 15K—in their legacy environments that need to be migrated. GitHub Actions Importer is designed to help when manual migration is not feasible, such as when customers have a large number of teams that depend on hundreds or thousands of workflows.

The phases of a GitHub Actions Importer migration

Using GitHub Actions Importer to facilitate a migration to Actions can be conceptualized in distinct phases.

Planning

A migration typically starts with an analysis of your existing CI/CD usage and approach, which includes addressing such questions as:

• How many pipelines are currently defined?
• What is the typical makeup of these pipelines? Are they defined using out-of-the-box tasks, or is the product being extended to provide custom behavior?
• Which pipelines are the highest priority to be migrated? Should any pipelines not be migrated?
• What type and how many runners are being used? What mix of GitHub-hosted and self-hosted runners should you select when migrating?

Migrating

Once you begin your migration efforts, you can expect that a small subset of migrated workflows will require manual work to complete. Although the actual conversion rate for a customer will depend on the makeup of each individual pipeline, in practice we’ve found that, on average, GitHub Actions Importer can successfully convert in excess of 90% of the tasks and constructs used in a workflow.

To finalize your migration, you should plan to iterate on the new GitHub Actions workflow to validate that it functions equivalently to the existing pipeline and that constructs that could not be migrated automatically are implemented. This could include—but is not limited to—configuring secrets, self-hosted runners, environments, and branch protection rules. In addition, if you were using any custom tasks or less commonly used public tasks, you will need to port or re-implement that logic manually.

How to get started with GitHub Actions Importer

The user interface for GitHub Actions Importer is an extension to the official GitHub CLI, which delegates to a Docker container. When you get access to GitHub Actions Importer, have installed it and the GitHub CLI, and are ready to try it in your own environment, you’ll typically want to take the following steps.

1. Plan the timeline and complexity of the migration

GitHub Actions Importer provides an audit command that is designed to help analyze the complexity of a potential migration, which can be used to formulate a migration plan. This command will fetch all of the pipelines defined in a specified scope of the existing CI/CD environment, attempt a conversion of these pipelines to their equivalent workflow, and write a summary report with statistics gathered from the audit.

Running an audit on the command line will look something like this:

$ gh actions-importer audit jenkins --output-dir .

The Audit summary report provides details aggregated at the pipeline level, aggregated at the build step level, and identifies manual migration tasks that will need to be completed.

Pipelines

This section contains high-level aggregated statistics on GitHub Actions Importer’s ability to migrate the targeted pipeline automatically.

The Pipelines section provides the following key metrics:

• Successful: identifies the number of pipelines that had all of their constructs and individual items converted automatically to their GitHub Actions equivalent.
• Partially successful: identifies the number of pipelines that had all of their constructs converted, however, there were some individual items (for example,build tasks or build triggers) that could not be converted automatically.
• Unsupported: identifies the number of pipeline definitions that use constructs that are unsupported by GitHub Actions Importer. Please refer to the platform-specific documentation to learn which pipeline types are supported.
• Failed: identifies the number of pipelines where GitHub Actions Importer had a fatal error during an attempt at conversion. This can occur for one of three reasons:
  • The pipeline was misconfigured and not valid in its existing CI/CD platform.
  • GitHub Actions Importer encountered an internal error when converting it.
  • There was an unsuccessful network response, often due to invalid credentials, that caused the pipeline to be inaccessible.

The Job types section summarizes which pipeline types are being used and whether they are supported or unsupported. Please refer to the platform-specific documentation to learn which pipeline types are supported.

Build steps

This section presents an aggregated summary of the individual build steps that are used across all of the target pipelines and how many could be converted automatically.

The Build steps section provides the following details:

• A known build step is one that GitHub Actions Importer can automatically convert to an equivalent action.
• An unknown build step is one that cannot be automatically converted to an equivalent action.
• An unsupported build step is either:
  • Fundamentally not supported by GitHub Actions.
  • Configured in a way that is incompatible with GitHub Actions.
• Actions provides a comprehensive list of actions that would be used in all of the converted workflows. This is important in these scenarios:
  • When you are migrating to GHES and need to sync these actions to your GHES instance.
  • When you are defining an organization-level allowlist.
  • When you need to provide a comprehensive list of actions for your security and/or compliance teams to review and approve.

In addition, the audit summary report includes an equivalent summary of build triggers, environment variables, and other uncategorized items.

Manual tasks

This section presents an overview of the manual tasks that can be identified.

The Manual tasks section can provide the following details:

• Secrets includes a summarized list of the secrets used in the converted pipelines. Because secrets cannot be migrated automatically, they will need to be created manually in order for these workflows to function properly.
• Self-hosted runners summarizes and identifies the labeled runners referenced in the source pipelines. You will need to decide whether a GitHub-hosted or self-hosted runner is appropriate to run these workflows within GitHub Actions.

File manifest

The final section of the audit summary report provides a manifest of all the files generated during the audit. These files can include:

• The original pipeline as it was defined in the source CI/CD platform.
• A log of the network responses used to convert a pipeline.
• The converted workflow file(s).
• A log of any error messages to troubleshoot a failed pipeline conversion.

2. Understand what is not migrated automatically

GitHub Actions Importer cannot migrate everything. It’s important to be mindful of its limitations, which include:

• Secrets and encrypted values are not automatically converted to repository secrets. References to secrets will be converted into context expressions and populating these values will be a manual task.
• Self-hosted build agents are not automatically converted to self-hosted runners. Determining whether to use GitHub-hosted runners or create equivalent self-hosted runners will be a manual task. References to these self-hosted runners will be converted to the same set of labels in a needs statement in the resulting workflow.
• Historical packages are not migrated to GitHub Packages. Any steps that publish or consume pipeline artifacts and caches will be converted using the equivalent actions.
• Permissions for CI/CD pipelines are not migrated automatically and will need to be manually configured.
• Build steps or build triggers that are less commonly used may not be automatically converted by GitHub Actions Importer. This can be a factor for migrations involving Azure DevOps, Jenkins, and CircleCI, all of which can be extended through marketplace customizations.

3. Forecast your Actions usage

The forecast command is designed to help you understand the compute capacity you’re currently using within your CI/CD environment. This command fetches jobs that have been completed over a specified time period and uses that data to calculate usage metrics.

Running a forecast on the command line will look something like this:

$ gh actions-importer audit jenkins –start-date 7/1/22 --output-dir .

The Forecast report includes these metrics:

• Job count: the total number of completed jobs.
• Pipeline count: the number of unique pipelines used.
• Execution time: the amount of time a runner spent on a job. This metric can be used to help set expectations for the cost of GitHub-hosted runners.
• Queue time: the amount of time a job spent waiting for a runner to be available.
• Concurrent jobs: the number of jobs running at any given time. This metric can be used to understand the number of concurrent runners necessary at current levels.

Additionally, these metrics are presented for each runner queue defined in the source CI/CD system. This is useful if you will need a mix of hosted and self-hosted runners and/or if you use a mix of platforms.

4. Customize a workflow’s conversion

You can use the dry-run command to convert an existing pipeline to its equivalent GitHub Actions workflow. The console output of the command will list the path to the file or files that GitHub Actions Importer generated. Before migrating, you should perform a dry run of a pipeline and validate the contents are suitable.

If the conversion of a pipeline was only “partially successful” (that is, it included tasks that could not be converted automatically), the task that was not converted will be included in a commented section. For example, if you were to run the following CLI command:

$ gh actions-importer dry-run jenkins --source-url $SOURCE_URL --output-dir 

The converted workflow will have contents similar to the following.

name: ethanis/universe
on:
  workflow_dispatch:
jobs:
  Build:
    runs-on:
      - self-hosted
      - main
    steps:
    - name: checkout
      uses: actions/checkout@v3
#     # This item has no matching transformer
#     - buildJavascriptApp:
#       - key: deploy
#         value:
#           isLiteral: true
#           value: false

In this situation, you will need to decide how to implement this functionality. If this task is used within multiple pipelines, the recommended approach is to implement a custom transformer that can handle this scenario in every pipeline. If this is a one-off scenario, you can edit the converted workflow file manually.

Let’s assume in this example that the buildJavascriptApp build step is used in multiple workflows and could be easily replicated with a single shell command. This is what that custom transformer would look like:

transform “buildJavascriptApp” do |item|
  {
    name: “Build Javascript App”,
    run: “npm install && npm run build”
  }
end

If you were to add these contents to a file named transformers.rb, they can be provided to GitHub Actions Importer on the command line. For example:

$ gh actions-importer dry-run jenkins --source-url $SOURCE_URL --output-dir . --custom-transformers transformers.rb

5. Perform the production workflow migration

You can use the migrate command to convert an existing pipeline to its equivalent action and open a pull request with the converted workflows and associated files.

$ gh actions-importer migrate jenkins --source-url $SOURCE_URL –target-url $TARGET_URL --output-dir .

Any necessary manual tasks will be included in the description of the pull request. Once these manual tasks and the code reviews are complete, the pull request can be merged and the workflow will have been successfully migrated to GitHub Actions.

Optional: configure self-serve migrations with IssueOps

The GitHub Actions Importer IssueOps template repository provides the foundational functionality required to run GitHub Actions Importer commands through GitHub Actions and Issues. This approach allows you to migrate pipelines to GitHub Actions without needing to have GitHub Actions Importer and Docker installed on a local machine. Organizations that want to enable self-service migrations to GitHub Actions could find this approach useful.

Sign up for GitHub Actions Importer

Request access by completing the waitlist form here.

Written by

Ethan Dennis

@ethanis