Introduction to environments and deployments

Introduced in GitLab 8.9.

During the development of software, there can be many stages until it's ready for public consumption. You sure want to first test your code and then deploy it in a testing or staging environment before you release it to the public. That way you can prevent bugs not only in your software, but in the deployment process as well.

GitLab CI is capable of not only testing or building your projects, but also deploying them in your infrastructure, with the added benefit of giving you a way to track your deployments. In other words, you can always know what is currently being deployed or has been deployed on your servers.

Overview

With environments, you can control the Continuous Deployment of your software all within GitLab. All you need to do is define them in your project's.gitlab-ci.ymlas we will explore below. GitLab provides a full history of your deployments per every environment.

Environments are like tags for your CI jobs, describing where code gets deployed. Deployments are created whenjobs版本的代码部署到环境中,所以每一个environment can have one or more deployments. GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers. If you have a deployment service such asKubernetesenabled for your project, you can use it to assist with your deployments, and can even access aweb terminalfor your environment from within GitLab!

To better understand how environments and deployments work, let's consider an example. We assume that you have already created a project in GitLab and set up a Runner. The example will cover the following:

We are developing an application
We want to run tests and build our app on all branches
Our default branch ismaster
We deploy the app only when a pipeline onmasterbranch is run

Let's see how it all ties together.

Defining environments

Let's consider the following.gitlab-ci.ymlexample:

stages:-test-build-deploytest:stage:testscript:echo "Running tests"build:stage:buildscript:echo "Building the app"deploy_staging:stage:deployscript:-echo "Deploy to staging server"environment:name:stagingurl:https://staging.example.comonly:-master

We have defined 3stages:

test
build
deploy

The jobs assigned to these stages will run in this order. If a job fails, then the jobs that are assigned to the next stage won't run, rendering the pipeline as failed. In our case, thetestjob will run first, then thebuildand lastly thedeploy_staging. With this, we ensure that first the tests pass, then our app is able to be built successfully, and lastly we deploy to the staging server.

Theenvironmentkeyword is just a hint for GitLab that this job actually deploys to this environment'sname. It can also have aurlwhich, as we will later see, is exposed in various places within GitLab. Each time a job that has an environment specified and succeeds, a deployment is recorded, remembering the Git SHA and environment name.

Note:Starting with GitLab 8.15, the environment name is exposed to the Runner in two forms:$CI_ENVIRONMENT_NAME, and$CI_ENVIRONMENT_SLUG. The first is the name given in.gitlab-ci.yml(with any variables expanded), while the second is a "cleaned-up" version of the name, suitable for use in URLs, DNS, etc.

Note:Starting with GitLab 9.3, the environment URL is exposed to the Runner via$CI_ENVIRONMENT_URL. The URL would be expanded from.gitlab-ci.yml, or if the URL was not defined there, the external URL from the environment would be used.

To sum up, with the above.gitlab-ci.ymlwe have achieved that:

All branches will run thetestandbuildjobs.
Thedeploy_stagingjob will runonlyon themasterbranch which means all merge requests that are created from branches don't get to deploy to the staging server
When a merge request is merged, all jobs will run and thedeploy_stagingin particular will deploy our code to a staging server while the deployment will be recorded in an environment namedstaging.

Let's now see how that information is exposed within GitLab.

Viewing the current status of an environment

The environment list under your project'sPipelines ➔ Environments, is where you can find information of the last deployment status of an environment.

Here's how the Environments page looks so far.

There's a bunch of information there, specifically you can see:

The environment's name with a link to its deployments
The last deployment ID number and who performed it
The job ID of the last deployment with its respective job name
The commit information of the last deployment such as who committed, to what branch and the Git SHA of the commit
The exact time the last deployment was performed
A button that takes you to the URL that you have defined under theenvironmentkeyword in.gitlab-ci.yml
A button that re-deploys the latest deployment, meaning it runs the job defined by the environment name for that specific commit

Notes:

While you can create environments manually in the web interface, we recommend that you define your environments in.gitlab-ci.yml第一。他们将会为您自动创建after the first deploy.

The environments page can only be viewed by Reporters and above. For more information on the permissions, see thepermissions documentation.

Only deploys that happen after your.gitlab-ci.ymlis properly configured will show up in the "Environment" and "Last deployment" lists.

The information shown in the Environments page is limited to the latest deployments, but as you may have guessed an environment can have multiple deployments.

Viewing the deployment history of an environment

GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers. That way you can have the full history of your deployments per every environment right in your browser. Clicking on an environment will show the history of its deployments. Assuming you have deployed multiple times already, here's how a specific environment's page looks like.

We can see the same information as when in the Environments page, but this time all deployments are shown. As you may have noticed, apart from theRe-deploybutton there are nowRollbackbuttons for each deployment. Let's see how that works.

Rolling back changes

You can't control everything, so sometimes things go wrong. When that unfortunate time comes GitLab has you covered. Simply by clicking theRollbackbutton that can be found in the deployments page (Pipelines ➔ Environments ➔环境的名字) you can relaunch the job with the commit associated with it.

Note:Bear in mind that your mileage will vary and it's entirely up to how you define the deployment process in the job'sscriptwhether the rollback succeeds or not. GitLab CI is just following orders.

Thankfully that was the staging server that we had to rollback, and since we learn from our mistakes, we decided to not make the same again when we deploy to the production server. Enter manual actions for deployments.

Manually deploying to environments

Turning a job from running automatically to a manual action is as simple as addingwhen: manualto it. To expand on our previous example, let's add another job that this time deploys our app to a production server and is tracked by aproductionenvironment. The.gitlab-ci.ymllooks like this so far:

stages:-test-build-deploytest:stage:testscript:echo "Running tests"build:stage:buildscript:echo "Building the app"deploy_staging:stage:deployscript:-echo "Deploy to staging server"environment:name:stagingurl:https://staging.example.comonly:-masterdeploy_prod:stage:deployscript:-echo "Deploy to production server"environment:name:productionurl:https://example.comwhen:manualonly:-master

Thewhen: manualaction exposes a play button in GitLab's UI and thedeploy_prodjob will only be triggered if and when we click that play button. You can find it in the pipeline, job, environment, and deployment views.

Pipelines	Single pipeline	Environments	Deployments	jobs

Clicking on the play button in either of these places will trigger thedeploy_prodjob, and the deployment will be recorded under a new environment namedproduction.

Note:Remember that if your environment's name isproduction(all lowercase), then it will get recorded inCycle Analytics. Double the benefit!

Dynamic environments

顾名思义,它可以创建environments on the fly by just declaring their names dynamically in.gitlab-ci.yml. Dynamic environments is the basis ofReview apps.

Note:Thenameandurlparameters can use most of the defined CI variables, including predefined, secure variables and.gitlab-ci.ymlvariables. You however cannot use variables defined underscriptor on the Runner's side. There are other variables that are unsupported in environment name context:

CI_JOB_ID

CI_JOB_TOKEN

CI_BUILD_ID

CI_BUILD_TOKEN

CI_REGISTRY_USER

CI_REGISTRY_PASSWORD

CI_REPOSITORY_URL

CI_ENVIRONMENT_URL

CI_DEPLOY_USER

CI_DEPLOY_PASSWORD

GitLab Runner exposes variousenvironment variableswhen a job runs, and as such, you can use them as environment names. Let's add another job in our example which will deploy to all branches exceptmaster:

deploy_review:stage:deployscript:-echo "Deploy a review app"environment:name:review/$CI_COMMIT_REF_NAMEurl:https://$CI_ENVIRONMENT_SLUG.example.comonly:-branchesexcept:-master

Let's break it down in pieces. The job's name isdeploy_reviewand it runs on thedeploystage. Thescriptat this point is fictional, you'd have to use your own based on your deployment. Then, we set theenvironmentwith theenvironment:namebeingreview/$CI_COMMIT_REF_NAME. Now that's an interesting one. Since the环境的名字can contain slashes (/), we can use this pattern to distinguish between dynamic environments and the regular ones.

So, the first part isreview, followed by a/and then$CI_COMMIT_REF_NAMEwhich takes the value of the branch name. Since$CI_COMMIT_REF_NAMEitself may also contain/, or other characters that would be invalid in a domain name or URL, we use$CI_ENVIRONMENT_SLUGin theenvironment:urlso that the environment can get a specific and distinct URL for each branch. In this case, given a$CI_COMMIT_REF_NAMEof100-Do-The-Thing, the URL will be something likehttps://100-do-the-4f99a2.example.com. Again, the way you set up the web server to serve these requests is based on your setup.

You could also use$CI_COMMIT_REF_SLUGinenvironment:url, e.g.:https://$CI_COMMIT_REF_SLUG.example.com. We use$CI_ENVIRONMENT_SLUGhere because it is guaranteed to be unique, but if you're using a workflow likeGitLab Flow, collisions are very unlikely, and you may prefer environment names to be more closely based on the branch name - the example above would give you an URL likehttps://100-do-the-thing.example.com

Last but not least, we tell the job to runonlyon branchesexceptmaster.

Note:You are not bound to use the same prefix or only slashes in the dynamic environments' names (/), but as we will see later, this will enable thegrouping similar environmentsfeature.

The whole.gitlab-ci.ymllooks like this so far:

stages:-test-build-deploytest:stage:testscript:echo "Running tests"build:stage:buildscript:echo "Building the app"deploy_review:stage:deployscript:-echo "Deploy a review app"environment:name:review/$CI_COMMIT_REF_NAMEurl:https://$CI_ENVIRONMENT_SLUG.example.comonly:-branchesexcept:-masterdeploy_staging:stage:deployscript:-echo "Deploy to staging server"environment:name:stagingurl:https://staging.example.comonly:-masterdeploy_prod:stage:deployscript:-echo "Deploy to production server"environment:name:productionurl:https://example.comwhen:manualonly:-master

A more realistic example would include copying files to a location where a webserver (NGINX) could then read and serve. The example below will copy thepublicdirectory to/srv/nginx/$CI_COMMIT_REF_SLUG/public:

review_app:stage:deployscript:-rsync -av --delete public /srv/nginx/$CI_COMMIT_REF_SLUGenvironment:name:review/$CI_COMMIT_REF_NAMEurl:https://$CI_COMMIT_REF_SLUG.example.com

It is assumed that the user has already setup NGINX and GitLab Runner in the server this job will run on.

Note:Be sure to check out thelimitationssection for some edge cases regarding naming of your branches and Review Apps.

The development workflow would now be:

Developer creates a branch locally
Developer makes changes, commits and pushes the branch to GitLab
Developer creates a merge request

Behind the scenes:

GitLab Runner picks up the changes and starts running the jobs
The jobs run sequentially as defined instages
- First, the tests pass
- Then, the job begins and successfully also passes
- 最后,应用程序部署到一个environment with a name specific to the branch

So now, every branch gets its own environment and is deployed to its own place with the added benefit of having ahistory of deploymentsand also being able torollback changesif needed. Let's briefly see where URL that's defined in the environments is exposed.

Making use of the environment URL

Theenvironment URLis exposed in a few places within GitLab.

In a merge request widget as a link	In the Environments view as a button	In the Deployments view as a button

If a merge request is eventually merged to the default branch (in our casemaster) and that branch also deploys to an environment (in our casestagingand/orproduction) you can see this information in the merge request itself.

Go directly from source files to public pages on the environment

Introduced in GitLab 8.17.

To go one step further, we can specify a Route Map to get GitLab to show us "View on [environment URL]" buttons to go directly from a file to that file's representation on the deployed website. It will be exposed in a few places:

In the diff for a merge request, comparison or commit	In the file view

To get this to work, you need to tell GitLab how the paths of files in your repository map to paths of pages on your website, using a Route Map.

A Route Map is a file inside the repository at.gitlab/route-map.yml, which contains a YAML array that mapssourcepaths (in the repository) topublicpaths (on the website).

This is an example of a route map forMiddlemanstatic websites likehttp://about.gitlab.com:

# Team data-source:'data/team.yml'# data/team.ymlpublic:'team/'# team/# Blogposts-source:/source\/posts\/([0-9]{4})-([0-9]{2})-([0-9]{2})-(.+?)\..*/# source/posts/2017-01-30-around-the-world-in-6-releases.html.md.erbpublic:'\1/\2/\3/\4/'# 2017/01/30/around-the-world-in-6-releases/# HTML files-source:/source\/(.+?\.html).*/# source/index.html.hamlpublic:'\1'# index.html# Other files-source:/source\/(.*)/# source/images/blogimages/around-the-world-in-6-releases-cover.pngpublic:'\1'# images/blogimages/around-the-world-in-6-releases-cover.png

Mappings are defined as entries in the root YAML array, and are identified by a-prefix. Within an entry, we have a hash map with two keys:

source
- a string, starting and ending with', for an exact match
- a regular expression, starting and ending with/, for a pattern match
  - The regular expression needs to match the entire source path -^and$anchors are implied.
  - Can include capture groups denoted by()that can be referred to in thepublicpath.
  - Slashes (/) can, but don't have to, be escaped as\/.
  - Literal periods (.) should be escaped as\..
public
- a string, starting and ending with'.
  - Can include\Nexpressions to refer to capture groups in thesourceregular expression in order of their occurrence, starting with\1.

The public path for a source path is determined by finding the firstsourceexpression that matches it, and returning the correspondingpublicpath, replacing the\Nexpressions with the values of the()capture groups if appropriate.

In the example above, the fact that mappings are evaluated in order of their definition is used to ensure thatsource/index.html.hamlwill match/source\/(.+?\.html).*/instead of/source\/(.*)/, and will result in a public path ofindex.html, instead ofindex.html.haml.

We now have a full development cycle, where our app is tested, built, deployed as a Review app, deployed to a staging server once the merge request is merged, and finally manually deployed to the production server. What we just described is a single workflow, but imagine tens of developers working on a project at the same time. They each push to their branches, and dynamic environments are created all the time. In that case, we probably need to do some clean up. Read next how environments can be stopped.

Stopping an environment

By stopping an environment, you are effectively terminating its recording of the deployments that happen in it.

A branch is associated with an environment when the CI pipeline that is created for this branch, was recently deployed to this environment. You can think of the CI pipeline as the glue between the branch and the environment:branch ➔ CI pipeline ➔ environment.

There is a special case where environments can be manually stopped. That can happen if you provide another job for that matter. The syntax is a little tricky since a job calls another job to do the job.

Consider the following example where thedeploy_reviewcalls thestop_reviewto clean up and stop the environment:

deploy_review:stage:deployscript:-echo "Deploy a review app"environment:name:review/$CI_COMMIT_REF_NAMEurl:https://$CI_ENVIRONMENT_SLUG.example.comon_stop:stop_reviewonly:-branchesexcept:-masterstop_review:stage:deployvariables:GIT_STRATEGY:nonescript:-echo "Remove review app"when:manualenvironment:name:review/$CI_COMMIT_REF_NAMEaction:stop

Setting theGIT_STRATEGYtononeis necessary on thestop_reviewjob so that theGitLab Runnerwon't try to checkout the code after the branch is deleted.

Note:Starting with GitLab 8.14, dynamic environments will be stopped automatically when their associated branch is deleted.

当你有一个环境a stop action defined (typically when the environment describes a review app), GitLab will automatically trigger a stop action when the associated branch is deleted. Thestop_reviewjob must be in the samestageas thedeploy_reviewone in order for the environment to automatically stop.

You can read more in the.gitlab-ci.ymlreference.

Grouping similar environments

Introducedin GitLab 8.14.

As we've seen in thedynamic environments, you can prepend their name with a word, then followed by a/and finally the branch name which is automatically defined by theCI_COMMIT_REF_NAMEvariable.

In short, environments that are named liketype/fooare presented under a group namedtype.

In our minimal example, we name the environmentsreview/$CI_COMMIT_REF_NAMEwhere$CI_COMMIT_REF_NAMEis the branch name:

deploy_review:stage:deployscript:-echo "Deploy a review app"environment:name:review/$CI_COMMIT_REF_NAME

In that case, if you visit the Environments page, and provided the branches exist, you should see something like:

Monitoring environments

Notes:

For the monitoring dashboard to appear, you need to:

Have enabled thePrometheus integration

Configured Prometheus to collect at least onesupported metric

With GitLab 9.2, all deployments to an environment are shown directly on the monitoring dashboard

If you have enabledPrometheus for monitoring system and response metrics, you can monitor the performance behavior of your app running in each environment.

Once configured, GitLab will attempt to retrievesupported performance metricsfor any environment which has had a successful deployment. If monitoring data was successfully retrieved, a Monitoring button will appear for each environment.

Clicking on the Monitoring button will display a new page, showing up to the last 8 hours of performance data. It may take a minute or two for data to appear after initial deployment.

All deployments to an environment are shown directly on the monitoring dashboard which allows easy correlation between any changes in performance and a new version of the app, all without leaving GitLab.

Web terminals

Note:Web terminals were added in GitLab 8.15 and are only available to project masters and owners.

If you deploy to your environments with the help of a deployment service (e.g., theKubernetes service), GitLab can open a terminal session to your environment! This is a very powerful feature that allows you to debug issues without leaving the comfort of your web browser. To enable it, just follow the instructions given in the service integration documentation.

Once enabled, your environments will gain a "terminal" button:

You can also access the terminal button from the page for a specific environment:

Wherever you find it, clicking the button will take you to a separate page to establish the terminal session:

This works just like any other terminal - you'll be in the container created by your deployment, so you can run shell commands and get responses in real time, check the logs, try out configuration or code tweaks, etc. You can open multiple terminals to the same environment - they each get their own shell session - and even a multiplexer likescreenortmux!

Note:Container-based deployments often lack basic tools (like an editor), and may be stopped or restarted at any time. If this happens, you will lose all your changes! Treat this as a debugging tool, not a comprehensive online IDE.

While this is fine for deploying to some stable environments like staging or production, what happens for branches? So far we haven't defined anything regarding deployments for branches other thanmaster. Dynamic environments will help us achieve that.

Checkout deployments locally

Since 8.13, a reference in the git repository is saved for each deployment, so knowing the state of your current environments is only agit fetchaway.

In your git config, append the[remote ""]block with an extra fetch line:

fetch = +refs/environments/*:refs/remotes/origin/environments/*

Limitations

You are limited to use only theCI predefined variablesin theenvironment: name. If you try to re-use variables defined insidescriptas part of the environment name, it will not work.