Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
1 result
Blame Permalink
To learn more about this project, read the wiki.
README.md 27.48 KiB

GitLab CI template for {{cookiecutter.template_name}}

{%- if cookiecutter.template_type == 'build' %} This project implements a GitLab CI/CD template to build, test and analyse your {{cookiecutter.template_name}} projects.

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/{{cookiecutter.project_slug}}'
    ref: '1.0.0'
    file: '/templates/gitlab-ci-{{cookiecutter.project_slug}}.yml'

Global configuration

The {{cookiecutter.template_name}} template uses some global configuration used throughout all jobs.

Name Description Default value
{{cookiecutter.template_PREFIX}}_IMAGE The Docker image used to run {{cookiecutter.cli_tool}} registry.hub.docker.com/{{cookiecutter.project_slug}}:latest

Jobs

{{cookiecutter.template_prefix}}-build job

This job performs build and tests at once.

It uses the following variable:

Name Description Default value
{{cookiecutter.template_PREFIX}}_BUILD_ARGS Arguments used by the build job build --with-default-args

SonarQube analysis

If you're using the SonarQube template to analyse your {{cookiecutter.template_PREFIX}} code, here are 2 sample sonar-project.properties files.

# see: https://docs.sonarqube.org/latest/analysis/languages/{{cookiecutter.project_slug}}/
# set your source directory(ies) here (relative to the sonar-project.properties file)
sonar.sources=.
# exclude unwanted directories and files from being analysed
sonar.exclusions=output/**,**/*_test.{{cookiecutter.template_prefix}}

# set your tests directory(ies) here (relative to the sonar-project.properties file)
sonar.tests=.
sonar.test.inclusions=**/*_test.{{cookiecutter.template_prefix}}

# tests report (TODO)
sonar.{{cookiecutter.project_slug}}.testExecutionReportPaths=reports/sonar_test_report.xml
# coverage report (TODO)
sonar.{{cookiecutter.project_slug}}.coverage.reportPaths=reports/coverage.cov

More info:

{{cookiecutter.template_prefix}}-lint job

This job performs a lint analysis of your code, mapped to the build stage.

It uses the following variables:

Name Description Default value
{{cookiecutter.template_PREFIX}}_LINT_IMAGE The Docker image used to run the lint tool {{cookiecutter.project_slug}}-lint:latest
{{cookiecutter.template_PREFIX}}_LINT_DISABLED Set to true to disable the lint analysis none (enabled)
{{cookiecutter.template_PREFIX}}_LINT_ARGS Lint options and arguments --serevity=medium

{{cookiecutter.template_prefix}}-depcheck job

This job enables a manual dependency check analysis of your code, mapped to the test stage.

It uses the following variables:

Name Description Default value
{{cookiecutter.template_PREFIX}}_DEPCHECK_IMAGE The Docker image used to run the dependency check tool {{cookiecutter.project_slug}}-depcheck:latest
{{cookiecutter.template_PREFIX}}_DEPCHECK_ARGS Dependency check options and arguments none

{{cookiecutter.template_prefix}}-publish job

This job is disabled by default and performs a publish of your built binaries.

It uses the following variables:

Name Description Default value
{{cookiecutter.template_PREFIX}}_PUBLISH_ENABLED Variable to enable the publish job none (disabled)
{{cookiecutter.template_PREFIX}}_PUBLISH_ARGS Arguments used by the publish job publish --with-default-args
🔒 {{cookiecutter.template_PREFIX}}_PUBLISH_LOGIN Login to use to publish has to be defined
🔒 {{cookiecutter.template_PREFIX}}_PUBLISH_PASSWORD Password to use to publish has to be defined

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

  1. Manage them as project or group CI/CD variables:
    • masked to prevent them from being inadvertently displayed in your job logs,
    • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
  2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
  3. Don't forget to escape special characters (ex: $ -> $$). {%- elif cookiecutter.template_type == 'deploy' %} This project implements a GitLab CI/CD template to deploy your application to {{cookiecutter.template_name}}.

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/{{cookiecutter.project_slug}}'
    ref: '1.0.0'
    file: '/templates/gitlab-ci-{{cookiecutter.project_slug}}.yml'

Understand

This chapter introduces key notions and principle to understand how this template works.

Managed deployment environments

This template implements continuous delivery/continuous deployment for projects hosted on {{cookiecutter.template_name}}.

It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.

The following chapters present the managed predefined environments and their associated Git workflow.

Review environments

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments

Lastly, the template supports 2 environments associated to your production branch (master or main by default):

  • a staging environment (an iso-prod environment meant for testing and validation purpose),
  • the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

  • continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
  • continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Supported authentication methods

The {{cookiecutter.template_name}} template supports the following authentication methods:

  • TODO (document)

Deployment context variables

In order to manage the various deployment environments, this template provides a couple of dynamic variables that you might use in your hook scripts, deployment manifests and other deployment resources:

  • ${environment_type}: the current deployment environment type (review, integration, staging or production)
  • ${environment_name}: a generated application name to use for the current deployment environment (ex: myapp-review-fix-bug-12 or myapp-staging) - details below

Generated environment name

The ${environment_name} variable is generated to designate each deployment environment with a unique and meaningful application name. By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels... It is built from:

  • the application base name (defaults to $CI_PROJECT_NAME but can be overridden globally and/or per deployment environment - see configuration variables)
  • GitLab predefined $CI_ENVIRONMENT_SLUG variable (sluggified name, truncated to 24 characters)

The ${environment_name} variable is then evaluated as:

  • <app base name> for the production environment
  • <app base name>-$CI_ENVIRONMENT_SLUG for all other deployment environments
  • 💡 ${environment_name} can also be overriden per environment with the appropriate configuration variable

Examples (with an application's base name myapp):

$environment_type Branch $CI_ENVIRONMENT_SLUG $environment_name
review feat/blabla review-feat-bla-xmuzs6 myapp-review-feat-bla-xmuzs6
integration develop integration myapp-integration
staging main staging myapp-staging
production main production myapp

Deployment and cleanup

TODO: explain here the supported techniques to deploy and cleanup the environments.

You should also explained clearly what is expected from the template user and what is the lookup policy in case the template implements one.

Example:

The {{cookiecutter.template_name}} template requires you to provide a shell script that fully implements your application deployment and cleanup using the {{cookiecutter.cli_tool}} CLI and all other tools available in the selected Docker image.

The deployment script is searched as follows:

  1. look for a specific {{cookiecutter.template_prefix}}-deploy-$environment_type.sh in the ${{cookiecutter.template_PREFIX}}_SCRIPTS_DIR directory in your project (e.g. {{cookiecutter.template_prefix}}-deploy-staging.sh for staging environment),
  2. if not found: look for a default {{cookiecutter.template_prefix}}-deploy.sh in the ${{cookiecutter.template_PREFIX}}_SCRIPTS_DIR directory in your project,
  3. if not found: the deployment job will fail.

The cleanup script is searched as follows:

  1. look for a specific {{cookiecutter.template_prefix}}-cleanup-$environment_type.sh in the ${{cookiecutter.template_PREFIX}}_SCRIPTS_DIR directory in your project (e.g. {{cookiecutter.template_prefix}}-cleanup-staging.sh for staging environment),
  2. if not found: look for a default {{cookiecutter.template_prefix}}-cleanup.sh in the ${{cookiecutter.template_PREFIX}}_SCRIPTS_DIR directory in your project,
  3. if not found: the cleanup job will fail.

ℹ️ Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ... Part of this complexity can be handled by the lookup policies described above (ex: one script per env) and also by using available environment variables:

  1. deployment context variables provided by the template:
    • ${environment_type}: the current environment type (review, integration, staging or production)
    • ${environment_name}: the application name to use for the current environment (ex: myproject-review-fix-bug-12 or myproject-staging)
    • ${hostname}: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
  2. any GitLab CI variable
  3. any custom variable (ex: ${SECRET_TOKEN} that you have set in your project CI/CD variables)

Environments URL management

The {{cookiecutter.template_name}} template supports two ways of providing your environments url:

  • a static way: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
  • a dynamic way: when the url cannot be known before the deployment job is executed.

The static way can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):

  • ${{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL to define a default url pattern for all your envs,
  • ${{cookiecutter.template_PREFIX}}_REVIEW_ENVIRONMENT_URL, ${{cookiecutter.template_PREFIX}}_INTEG_ENVIRONMENT_URL, ${{cookiecutter.template_PREFIX}}_STAGING_ENVIRONMENT_URL and ${{cookiecutter.template_PREFIX}}_PROD_ENVIRONMENT_URL to override the default.

ℹ️ Each of those variables support a late variable expansion mechanism with the %{somevar} syntax, allowing you to use any dynamically evaluated variables such as ${environment_name}.

Example:

variables:
  {{cookiecutter.template_PREFIX}}_BASE_APP_NAME: "wonderapp"
  # global url for all environments
  {{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
  # override for prod (late expansion of ${{cookiecutter.template_PREFIX}}_BASE_APP_NAME not needed here)
  {{cookiecutter.template_PREFIX}}_PROD_ENVIRONMENT_URL: "https://${{cookiecutter.template_PREFIX}}_BASE_APP_NAME.acme.domain"
  # override for review (using separate resource paths)
  {{cookiecutter.template_PREFIX}}_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"

To implement the dynamic way, your deployment script shall simply generate a environment_url.txt file in the working directory, containing only the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.

Deployment output variables

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

  • $environment_type: set to the type of environment (review, integration, staging or production),
  • $environment_name: the application name (see below),
  • $environment_url: set to the environment URL (whether determined statically or dynamically).

Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

You may also add and propagate your own custom variables, by pushing them to the {{ cookiecutter.project_slug }}.env file in your deployment scripts or hooks.

Configuration reference

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

  1. Manage them as project or group CI/CD variables:
    • masked to prevent them from being inadvertently displayed in your job logs,
    • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
  2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
  3. Don't forget to escape special characters (ex: $ -> $$).

Global configuration

The {{cookiecutter.template_name}} template uses some global configuration used throughout all jobs and environments.

Name Description Default value
{{cookiecutter.template_PREFIX}}_IMAGE the Docker image used to run {{cookiecutter.template_name}} CLI commands registry.hub.docker.com/{{cookiecutter.project_slug}}:latest
{{cookiecutter.template_PREFIX}}_BASE_APP_NAME Base application name $CI_PROJECT_NAME (see GitLab doc)
 {{cookiecutter.template_PREFIX}}_API_URL Default {{cookiecutter.template_name}} API url none
 {{cookiecutter.template_PREFIX}}_API_TOKEN Default {{cookiecutter.template_name}} API token none
 {{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL Default environments url (only define for static environment URLs declaration)
supports late variable expansion (ex: https://%{environment_name}.{{cookiecutter.project_slug}}.acme.com) 		none
{{cookiecutter.template_PREFIX}}_SCRIPTS_DIR Directory where deploy & cleanup scripts are located . (root project dir)

Review environments configuration

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

They are disabled by default and can be enabled by setting the {{cookiecutter.template_PREFIX}}_REVIEW_PROJECT variable (see below).

Here are variables supported to configure review environments:

Name Description Default value
{{cookiecutter.template_PREFIX}}_REVIEW_PROJECT Project ID for review env none (disabled)
{{cookiecutter.template_PREFIX}}_REVIEW_APP_NAME Application name for review env "{{ '${' }}{{cookiecutter.template_PREFIX}}_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}" (ex: myproject-review-fix-bug-12)
 {{cookiecutter.template_PREFIX}}_REVIEW_API_URL API url for review env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_URL
 {{cookiecutter.template_PREFIX}}_REVIEW_API_TOKEN API token for review env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_TOKEN
 {{cookiecutter.template_PREFIX}}_REVIEW_ENVIRONMENT_URL The review environments url (only define for static environment URLs declaration and if different from default) ${{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL
{{cookiecutter.template_PREFIX}}_REVIEW_AUTOSTOP_DURATION The amount of time before GitLab will automatically stop review environments 4 hours

Integration environment configuration

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the {{cookiecutter.template_PREFIX}}_INTEG_PROJECT variable (see below).

Here are variables supported to configure the integration environment:

Name Description Default value
{{cookiecutter.template_PREFIX}}_INTEG_PROJECT Project ID for integration env none (disabled)
{{cookiecutter.template_PREFIX}}_INTEG_APP_NAME Application name for integration env {{ '${' }}{{cookiecutter.template_PREFIX}}_BASE_APP_NAME}-integration
 {{cookiecutter.template_PREFIX}}_INTEG_API_URL API url for integration env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_URL
 {{cookiecutter.template_PREFIX}}_INTEG_API_TOKEN API token for integration env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_TOKEN
 {{cookiecutter.template_PREFIX}}_INTEG_ENVIRONMENT_URL The integration environment url (only define for static environment URLs declaration and if different from default) ${{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL

Staging environment configuration

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (main or master by default).

It is disabled by default and can be enabled by setting the {{cookiecutter.template_PREFIX}}_STAGING_PROJECT variable (see below).

Here are variables supported to configure the staging environment:

Name Description Default value
{{cookiecutter.template_PREFIX}}_STAGING_PROJECT Project ID for staging env none (disabled)
{{cookiecutter.template_PREFIX}}_STAGING_APP_NAME Application name for staging env {{ '${' }}{{cookiecutter.template_PREFIX}}_BASE_APP_NAME}-staging
 {{cookiecutter.template_PREFIX}}_STAGING_API_URL API url for staging env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_URL
 {{cookiecutter.template_PREFIX}}_STAGING_API_TOKEN API token for staging env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_TOKEN
 {{cookiecutter.template_PREFIX}}_STAGING_ENVIRONMENT_URL The staging environment url (only define for static environment URLs declaration and if different from default) ${{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL

Production environment configuration

The production environment is the final deployment environment associated with your production branch (main or master by default).

It is disabled by default and can be enabled by setting the {{cookiecutter.template_PREFIX}}_PROD_PROJECT variable (see below).

Here are variables supported to configure the production environment:

Name Description Default value
{{cookiecutter.template_PREFIX}}_PROD_PROJECT Project ID for production env none (disabled)
{{cookiecutter.template_PREFIX}}_PROD_APP_NAME Application name for production env ${{cookiecutter.template_PREFIX}}_BASE_APP_NAME
 {{cookiecutter.template_PREFIX}}_PROD_API_URL API url for production env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_URL
 {{cookiecutter.template_PREFIX}}_PROD_API_TOKEN API token for production env (only define if different from default) ${{cookiecutter.template_PREFIX}}_API_TOKEN
 {{cookiecutter.template_PREFIX}}_PROD_ENVIRONMENT_URL The production environment url (only define for static environment URLs declaration and if different from default) ${{cookiecutter.template_PREFIX}}_ENVIRONMENT_URL
{{cookiecutter.template_PREFIX}}_PROD_DEPLOY_STRATEGY Defines the deployment to production strategy. One of manual (i.e. one-click) or auto. manual

{%- elif cookiecutter.template_type == 'acceptance' %} This project implements a GitLab CI/CD template to run your automated tests with {{cookiecutter.template_name}}.

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/{{cookiecutter.project_slug}}'
    ref: '1.0.0'
    file: '/templates/gitlab-ci-{{cookiecutter.project_slug}}.yml'

{{cookiecutter.project_slug}} job

This job starts {{cookiecutter.template_name}} (functional) tests.

It uses the following variable:

Name Description Default value
{{cookiecutter.template_PREFIX}}_IMAGE The Docker image used to run {{cookiecutter.template_name}}. registry.hub.docker.com/{{cookiecutter.project_slug}}:latest
{{cookiecutter.template_PREFIX}}_PROJECT_DIR The {{cookiecutter.template_name}} project directory (containing test scripts) .
{{cookiecutter.template_PREFIX}}_EXTRA_ARGS {{cookiecutter.template_name}} extra run options none
REVIEW_ENABLED Set to true to enable {{cookiecutter.template_name}} tests on review environments (dynamic environments instantiated on development branches) none (disabled)

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report Format Usage
${{cookiecutter.template_PREFIX}}_PROJECT_DIR/reports/{{cookiecutter.project_slug}}.xunit.xml xUnit test report(s) GitLab integration

base url auto evaluation

By default, the {{cookiecutter.template_name}} template tries to auto-evaluate its base url (i.e. the variable pointing at server under test) by looking either for a $environment_url variable or for an environment_url.txt file.

Therefore if an upstream job in the pipeline deployed your code to a server and propagated the deployed server url, either through a dotenv variable $environment_url or through a basic environment_url.txt file, then the {{cookiecutter.template_name}} test will automatically be run on this server.

⚠️ all our deployment templates implement this design. Therefore even purely dynamic environments (such as review environments) will automatically be propagated to your {{cookiecutter.template_name}} tests. {%- endif %}