Gitlab CI/CD: Passing Environment Variables Between Jobs: A Practical Guide

In continuous integration and continuous deployment (CI/CD) workflows, sharing data between different stages or jobs in a pipeline is often necessary. GitLab CI/CD provides several ways to pass environment variables from one job to another, ensuring that critical information like build artifacts, configurations, or authentication tokens can flow through the pipeline efficiently.

This post will cover different strategies for passing environment variables between jobs in GitLab CI/CD and how to use them effectively.

---

Why Pass Environment Variables Between Jobs?

Environment variables are an essential part of CI/CD pipelines, allowing you to store dynamic data such as:

• API keys or authentication tokens.

• Build information, such as version numbers or commit SHAs.

• Paths to generated artifacts, like compiled binaries or Docker images.

• Configuration details that are required across multiple jobs.

Sometimes, the output or data generated by one job needs to be available in subsequent jobs for successful pipeline execution. Passing environment variables between jobs helps streamline this process, allowing for seamless sharing of dynamic data across different stages in the pipeline.

---

Strategies for Passing Environment Variables Between Jobs

There are several ways to share environment variables between jobs in GitLab CI/CD, depending on your use case and pipeline design. Here are the most common approaches:

1. Using Artifacts to Store Environment Variables

One of the simplest and most robust ways to pass environment variables between jobs is by using artifacts. Artifacts are files generated by one job that can be downloaded and used by another job. This method works well for larger or more complex data that needs to be passed between jobs.

Example: Storing Environment Variables in a File

In the first job, we store environment variables in a file and use artifacts to pass them to the next job.

stages:
  - build
  - deploy

build_job:
  stage: build
  script:
    - echo "API_TOKEN=$API_TOKEN" > env_variables.txt
    - echo "Build number: $CI_PIPELINE_ID"
  artifacts:
    paths:
      - env_variables.txt

deploy_job:
  stage: deploy
  script:
    - source env_variables.txt
    - echo "Deploying with API_TOKEN=$API_TOKEN"
  dependencies:
    - build_job

Explanation:

• The build_job creates a file (env_variables.txt) containing the environment variables.

• These variables are stored as artifacts and passed to the next job, deploy_job.

• In the deploy_job, the environment variables are sourced from the file and used in the deployment script.

2. Using dependencies to Share Artifacts

When passing artifacts between jobs, it’s essential to ensure that subsequent jobs have access to those artifacts. This is where the dependencies keyword comes in handy. It allows jobs to specify which upstream jobs they should inherit artifacts from.

In the example above, deploy_job uses dependencies to inherit the artifacts (the file containing environment variables) from build_job. This allows the deployment stage to access the stored environment variables.

---

---

3. Using needs to Pass Variables Between Jobs in Parallel

In some cases, you may want to run jobs in parallel and still pass information between them. The needs keyword helps define job dependencies while allowing parallel execution. However, to pass environment variables, you can still rely on artifacts.

Example: Passing Variables with needs

stages:
  - build
  - deploy

build_job:
  stage: build
  script:
    - echo "VERSION=1.0.0" > version.txt
  artifacts:
    paths:
      - version.txt

deploy_job:
  stage: deploy
  needs:
    - build_job
  script:
    - source version.txt
    - echo "Deploying version $VERSION"

Explanation:

• The needs keyword allows deploy_job to run while depending on the output of build_job—even in parallel with other jobs.

• The artifact containing the version number is passed and used in the deployment stage.

---

4. Using export and eval to Directly Pass Variables

Another way to pass environment variables from one job to another is by using export and eval commands. While this approach requires careful handling to avoid exposing sensitive data, it can be useful for passing smaller or straightforward variables.

Example: Exporting and Reusing Variables

stages:
  - prepare
  - test

prepare_job:
  stage: prepare
  script:
    - export TEST_VAR="Test Value"
    - echo "TEST_VAR=$TEST_VAR" > variables.env
  artifacts:
    paths:
      - variables.env

test_job:
  stage: test
  script:
    - source variables.env
    - echo "Using TEST_VAR: $TEST_VAR"
  needs:
    - job: prepare_job
      artifacts: true

Explanation:

• In prepare_job, we export the environment variable TEST_VAR and store it in a file (variables.env).

• The artifact (variables.env) is passed to test_job, where the variable is sourced and used.

---

Using GitLab’s Built-In CI/CD Variables

In addition to user-defined variables, GitLab provides a set of predefined CI/CD environment variables that can be useful when passing data between jobs. Some common examples include:

• $CI_COMMIT_REF_NAME: The branch name for the commit.

• $CI_PIPELINE_ID: The ID of the pipeline.

• $CI_JOB_STAGE: The current job’s stage.

• $CI_PROJECT_URL: The URL to the project’s repository.

These built-in variables can be referenced across jobs in the pipeline, helping to provide useful context or configuration details without needing to pass them explicitly.

Example: Using Predefined Variables Across Jobs

stages:
  - build
  - deploy

build_job:
  stage: build
  script:
    - echo "Commit ref: $CI_COMMIT_REF_NAME"
  artifacts:
    paths:
      - build_output/

deploy_job:
  stage: deploy
  script:
    - echo "Deploying from branch: $CI_COMMIT_REF_NAME"

Explanation:

• In this case, the environment variable $CI_COMMIT_REF_NAME is a predefined variable that can be used in both jobs without needing to explicitly pass it through artifacts or other means.

• This makes it easy to access key information about the current state of the pipeline.

---

Best Practices for Passing Variables Between Jobs

When designing pipelines that pass variables between jobs, keep these best practices in mind:

1. Keep Sensitive Data Secure

Be cautious about passing sensitive environment variables, such as API keys or credentials. Use GitLab’s CI/CD Secret Variables feature to securely store and access sensitive information, and avoid exposing such data in job outputs or logs.

2. Use Artifacts for Complex Data

For complex or structured data, such as configuration files or lists of values, it’s best to pass variables via artifacts. This ensures that all necessary data is transferred between jobs in a reliable and reproducible way.

3. Leverage GitLab’s Predefined Variables

GitLab provides many built-in environment variables that contain useful information about the pipeline. Use these whenever possible to avoid redundant variable definitions and simplify your pipeline configuration.

4. Document Your Pipeline

Clearly document how variables are passed between jobs in your .gitlab-ci.yml file, making it easier for other team members to understand the flow of data throughout the pipeline.

---

Conclusion

Passing environment variables between jobs in GitLab CI/CD is a powerful way to share dynamic data, configuration details, and build artifacts across different stages of your pipeline. By leveraging artifacts, dependencies, and built-in variables, you can create more flexible and efficient pipelines that meet the needs of your development process.

With the strategies outlined in this post, you can ensure that your GitLab CI/CD pipelines are optimized for collaboration, automation, and success—whether you're building software, deploying applications, or running tests.

---

Advanced Gitlab Course