Skip to content

Migrate GitLab CI to GitLab CE or EE

Beginning with version 8.0 of GitLab Community Edition (CE) and Enterprise Edition (EE), GitLab CI is no longer its own application, but is instead built into the CE and EE applications.

This guide will detail the process of migrating your CI installation and data into your GitLab CE or EE installation.You can only migrate CI data from GitLab CI 8.0 to GitLab 8.0; migrating between other versions (e.g.7.14 to 8.1) is not possible.

We recommend that you read through the entire migration process in this document before beginning.

Overview

In this document we assume you have a GitLab server and a GitLab CI server. It does not matter if these are the same machine.

The migration consists of three parts: updating GitLab and GitLab CI, moving data, and redirecting traffic.

Please note that CI builds triggered on your GitLab server in the time between updating to 8.0 and finishing the migration will be lost. Your GitLab server can be online for most of the procedure; the only GitLab downtime (if any) is during the upgrade to 8.0. Your CI service will be offline from the moment you upgrade to 8.0 until you finish the migration procedure.

Before upgrading

If you have GitLab CI installed using omnibus-gitlab packages butyou don't want to migrate your existing data:

mv/var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds.$(date+%s)

runsudo gitlab-ctl reconfigureand you can reach CI atgitlab.example.com/ci.

If you want to migrate your existing data, continue reading.

0. Updating Omnibus from versions prior to 7.13

If you are updating from older versions you should first update to 7.14 and then to 8.0. Otherwise it's pretty likely that you will encounter problems described in theTroubleshooting.

1. Verify that backups work

Make sure that the backup script on both servers can connect to the database.

# On your CI server:# Omnibussudo chown gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/buildssudo gitlab-ci-rake backup:create# Sourcecd /home/gitlab_ci/gitlab-cisudo -u gitlab_ci -H bundle exec rake backup:create RAILS_ENV=production

Also check on your GitLab server.

# On your GitLab server:# Omnibussudo gitlab-rake gitlab:backup:create SKIP=repositories,uploads# Sourcecd /home/git/gitlabsudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production SKIP=repositories,uploads

If this fails you need to fix it before upgrading to 8.0. Also seehttps://about.gitlab.com/getting-help/

2. Check source and target database types

Check what databases you use on your GitLab server and your CI server. Look for the 'adapter:' line. If your CI server and your GitLab server use the same database adapter no special care is needed. If your CI server uses MySQL and your GitLab server uses PostgreSQL you need to pass a special option during the 'Moving data' part.If your CI server uses PostgreSQL and your GitLab server uses MySQL you cannot migrate your CI data to GitLab 8.0.

# On your CI server:# Omnibussudo gitlab-ci-rake env:info# Sourcecd /home/gitlab_ci/gitlab-cisudo -u gitlab_ci -H bundle exec rake env:info RAILS_ENV=production
# On your GitLab server:# Omnibussudo gitlab-rake gitlab:env:info# Sourcecd /home/git/gitlabsudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

3. Storage planning

Decide where to store CI build traces on GitLab server. GitLab CI uses files on disk to store CI build traces. The default path for these build traces is/var/opt/gitlab/gitlab-ci/builds(Omnibus) or/home/git/gitlab/builds(Source). If you are storing your repository data in a special location, or if you are using NFS, you should make sure that you store build traces on the same storage as your Git repositories.

I. Upgrading

From this point on, GitLab CI will be unavailable for your end users.

1. Upgrade GitLab to 8.0

First upgrade your GitLab server to version 8.0:https://about.gitlab.com/update/

2. Disable CI on the GitLab server during the migration

After you update, go to the admin panel and temporarily disable CI. As an administrator, go toAdmin Area->Settings,在Continuous IntegrationuncheckDisable to prevent CI usage until rake ci:migrate is run (8.0 only).

3. CI settings are now in GitLab

If you want to use custom CI settings (e.g. change where builds are stored), please update/etc/gitlab/gitlab.rb(Omnibus) or/home/git/gitlab/config/gitlab.yml(Source).

4. Upgrade GitLab CI to 8.0

Now upgrade GitLab CI to version 8.0. If you are using Omnibus packages, this may have already happened when you upgraded GitLab to 8.0.

5. Disable GitLab CI on the CI server

Disable GitLab CI after upgrading to 8.0.

# On your CI server:# Omnibussudo gitlab-ctl stop ci-unicornsudo gitlab-ctl stop ci-sidekiq# Sourcesudo service gitlab_ci stopcd /home/gitlab_ci/gitlab-cisudo -u gitlab_ci -H bundle exec whenever --clear-crontab RAILS_ENV=production

II. Moving data

1. Database encryption key

从你的CI服务移动数据库加密密钥er to your GitLab server. The command below will show you what you need to copy-paste to your GitLab server. On Omnibus GitLab servers you will have to add a line to/etc/gitlab/gitlab.rb. On GitLab servers installed from source you will have to replace the contents of/home/git/gitlab/config/secrets.yml.

# On your CI server:# Omnibussudo gitlab-ci-rake backup:show_secrets# Sourcecd /home/gitlab_ci/gitlab-cisudo -u gitlab_ci -H bundle exec rake backup:show_secrets RAILS_ENV=production

2. SQL data and build traces

Create your final CI data export. If you are converting from MySQL to PostgreSQL, addMYSQL_TO_POSTGRESQL=1to the end of the rake command. When the command finishes it will print the path to your data export archive; you will need this file later.

# On your CI server:# Omnibussudo chown gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/buildssudo gitlab-ci-rake backup:create# Sourcecd /home/gitlab_ci/gitlab-cisudo -u gitlab_ci -H bundle exec rake backup:create RAILS_ENV=production

3. Copy data to the GitLab server

If you were running GitLab and GitLab CI on the same server you can skip this step.

Copy your CI data archive to your GitLab server. There are many ways to do this, below we use SSH agent forwarding and 'scp', which will be easy and fast for most setups. You can also copy the data archive first from the CI server to your laptop and then from your laptop to the GitLab server.

# Start from your laptopssh -A ci_admin@ci_server.example# Now on the CI serverscp /path/to/12345_gitlab_ci_backup.tar gitlab_admin@gitlab_server.example:~

4. Move data to the GitLab backups folder

Make the CI data archive discoverable for GitLab. We assume below that you store backups in the default path, adjust the command if necessary.

# On your GitLab server:# Omnibussudo mv /path/to/12345_gitlab_ci_backup.tar /var/opt/gitlab/backups/# Sourcesudo mv /path/to/12345_gitlab_ci_backup.tar /home/git/gitlab/tmp/backups/

5. Import the CI data into GitLab.

This step will delete any existing CI data on your GitLab server. There should be no CI data yet because you turned CI on the GitLab server off earlier.

# On your GitLab server:# Omnibussudo chown git:git /var/opt/gitlab/gitlab-ci/buildssudo gitlab-rake ci:migrate# Sourcecd /home/git/gitlabsudo - u git - h包exec耙ci: RAILS_E迁移NV=production

6. Restart GitLab

# On your GitLab server:# Omnibussudo gitlab-ctl hup unicornsudo gitlab-ctl restart sidekiq# Sourcesudo service gitlab reload

III. Redirecting traffic

If you were running GitLab CI with Omnibus packages and you were using the internal NGINX configuration your CI service should now be available both atci.example.com(the old address) andgitlab.example.com/ci.You are done!

If you installed GitLab CI from source we now need to configure a redirect in NGINX so that existing CI runners can keep using the old CI server address, and so that existing links to your CI server keep working.

1. Update Nginx configuration

To ensure that your existing CI runners are able to communicate with the migrated installation, and that existing build triggers still work, you'll need to update your Nginx configuration to redirect requests for the old locations to the new ones.

Edit/etc/nginx/sites-available/gitlab_ciand paste:

# GITLAB CIserver{listen80default_server;# e.g., listen 192.168.1.1:80;server_nameYOUR_CI_SERVER_FQDN;# e.g., server_name source.example.com;access_log/var/log/nginx/gitlab_ci_access.log;error_log/var/log/nginx/gitlab_ci_error.log;# expose API to fix runnerslocation/api{proxy_read_timeout300;proxy_connect_timeout300;proxy_redirectoff;proxy_set_headerX-Real-IPremote_addr美元;# You need to specify your DNS servers that are able to resolve YOUR_GITLAB_SERVER_FQDNresolver8.8.8.88.8.4.4;proxy_pass$scheme://YOUR_GITLAB_SERVER_FQDN/ci$request_uri;}# redirect all other CI requestslocation/{return301$scheme://YOUR_GITLAB_SERVER_FQDN/ci$request_uri;}# adjust this to match the largest build log your runners might submit,# set to 0 to disable limitclient_max_body_size10m;}

Make sure you substitute these placeholder values with your real ones:

  1. YOUR_CI_SERVER_FQDN: The existing public-facing address of your GitLab CI install (e.g.,ci.gitlab.com).
  2. YOUR_GITLAB_SERVER_FQDN: The current public-facing address of your GitLab CE (or EE) install (e.g.,gitlab.com).

Make sure not to remove the/ci$request_uripart. This is required to properly forward the requests.

You should also make sure that you can:

  1. curl https://YOUR_GITLAB_SERVER_FQDN/from your previous GitLab CI server.
  2. curl https://YOUR_CI_SERVER_FQDN/from your GitLab CE (or EE) server.

2. Check Nginx configuration

sudo nginx -t

3. Restart Nginx

sudo /etc/init.d/nginx restart

Restore from backup

If something went wrong and you need to restore a backup, consult theBackup restorationguide.

Troubleshooting

show:secrets problem (Omnibus-only)

If you see errors like this:

Missing `secret_key_base` or `db_key_base` for 'production' environment. The secrets will be generated and stored in `config/secrets.yml`rake aborted!Errno::EACCES: Permission denied @ rb_sysopen - config/secrets.yml

This can happen if you are updating from versions prior to 7.13 straight to 8.0. The fix for this is to update to Omnibus 7.14 first and then update it to 8.0.

Permission denied when accessing /var/opt/gitlab/gitlab-ci/builds

To fix that issue you have to change builds/ folder permission before doing final backup:

sudo chown -R gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/builds

Then before executing置信区间:迁移you need to fix builds folder permission:

sudo chown git:git /var/opt/gitlab/gitlab-ci/builds

Problems when importing CI database to GitLab

If you were migrating CI database from MySQL to PostgreSQL manually you can see errors during import about missing sequences:

ALTER SEQUENCEERROR: relation "ci_builds_id_seq" does not existERROR: relation "ci_commits_id_seq" does not existERROR: relation "ci_events_id_seq" does not existERROR: relation "ci_jobs_id_seq" does not existERROR: relation "ci_projects_id_seq" does not existERROR: relation "ci_runner_projects_id_seq" does not existERROR: relation "ci_runners_id_seq" does not existERROR: relation "ci_services_id_seq" does not existERROR: relation "ci_taggings_id_seq" does not existERROR: relation "ci_tags_id_seq" does not existCREATE TABLE

To fix that you need to apply this SQL statement before doing final backup:

##OmnibusGitLabgitlab-ci-railsdbconsole<<EOF-- ALTER TABLES - DROP DEFAULTSALTERTABLEONLYci_application_settingsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_buildsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_commitsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_eventsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_jobsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_projectsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_runner_projectsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_runnersALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_servicesALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_taggingsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_tagsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_trigger_requestsALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_triggersALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_variablesALTERCOLUMNidDROPDEFAULT;ALTERTABLEONLYci_web_hooksALTERCOLUMNidDROPDEFAULT;-- ALTER SEQUENCESALTERSEQUENCEci_application_settings_id_seq拥有BYci_application_settings.id;ALTERSEQUENCEci_builds_id_seq拥有BYci_builds.id;ALTERSEQUENCEci_commits_id_seq拥有BYci_commits.id;ALTERSEQUENCEci_events_id_seq拥有BYci_events.id;ALTERSEQUENCEci_jobs_id_seq拥有BYci_jobs.id;ALTERSEQUENCEci_projects_id_seq拥有BYci_projects.id;ALTERSEQUENCEci_runner_projects_id_seq拥有BYci_runner_projects.id;ALTERSEQUENCEci_runners_id_seq拥有BYci_runners.id;ALTERSEQUENCEci_services_id_seq拥有BYci_services.id;ALTERSEQUENCEci_taggings_id_seq拥有BYci_taggings.id;ALTERSEQUENCEci_tags_id_seq拥有BYci_tags.id;ALTERSEQUENCEci_trigger_requests_id_seq拥有BYci_trigger_requests.id;ALTERSEQUENCEci_triggers_id_seq拥有BYci_triggers.id;ALTERSEQUENCEci_variables_id_seq拥有BYci_variables.id;ALTERSEQUENCEci_web_hooks_id_seq拥有BYci_web_hooks.id;-- ALTER TABLES - RE-APPLY DEFAULTSALTERTABLEONLYci_application_settingsALTERCOLUMNidSETDEFAULTnextval('ci_application_settings_id_seq'::regclass);ALTERTABLEONLYci_buildsALTERCOLUMNidSETDEFAULTnextval('ci_builds_id_seq'::regclass);ALTERTABLEONLYci_commitsALTERCOLUMNidSETDEFAULTnextval('ci_commits_id_seq'::regclass);ALTERTABLEONLYci_eventsALTERCOLUMNidSETDEFAULTnextval('ci_events_id_seq'::regclass);ALTERTABLEONLYci_jobsALTERCOLUMNidSETDEFAULTnextval('ci_jobs_id_seq'::regclass);ALTERTABLEONLYci_projectsALTERCOLUMNidSETDEFAULTnextval('ci_projects_id_seq'::regclass);ALTERTABLEONLYci_runner_projectsALTERCOLUMNidSETDEFAULTnextval('ci_runner_projects_id_seq'::regclass);ALTERTABLEONLYci_runnersALTERCOLUMNidSETDEFAULTnextval('ci_runners_id_seq'::regclass);ALTERTABLEONLYci_servicesALTERCOLUMNidSETDEFAULTnextval('ci_services_id_seq'::regclass);ALTERTABLEONLYci_taggingsALTERCOLUMNidSETDEFAULTnextval('ci_taggings_id_seq'::regclass);ALTERTABLEONLYci_tagsALTERCOLUMNidSETDEFAULTnextval('ci_tags_id_seq'::regclass);ALTERTABLEONLYci_trigger_requestsALTERCOLUMNidSETDEFAULTnextval('ci_trigger_requests_id_seq'::regclass);ALTERTABLEONLYci_triggersALTERCOLUMNidSETDEFAULTnextval('ci_triggers_id_seq'::regclass);ALTERTABLEONLYci_variablesALTERCOLUMNidSETDEFAULTnextval('ci_variables_id_seq'::regclass);ALTERTABLEONLYci_web_hooksALTERCOLUMNidSETDEFAULTnextval('ci_web_hooks_id_seq'::regclass);EOF##Sourceinstallationscd/home/gitlab_ci/gitlab-cisudo-ugitlab_ci-Hbundleexecrailsdbconsoleproduction<<EOF...COPYSQLSTATEMENTSFROMABOVE...EOF
Baidu
map