Gitlab 502 ошибка

• How to see GitLab Pages logs
• unsupported protocol scheme \"\""
• 502 error when connecting to GitLab Pages proxy when server does not listen over IPv6
• Intermittent 502 errors or after a few days
• Unable to access GitLab Pages
• Failed to connect to the internal GitLab API
• Pages cannot communicate with an instance of the GitLab API
• Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages
• 500 error with securecookie: failed to generate random iv and Failed to save the session
• The requested scope is invalid, malformed, or unknown
• Workaround in case no wildcard DNS entry can be set
• Pages daemon fails with permission denied errors
• The redirect URI included is not valid. when using Pages Access Control
• 500 error cannot serve from disk
• httprange: new resource 403
• GitLab Pages doesn’t work after upgrading to GitLab 14.0 or above
• GitLab Pages deploy job fails with error “is not a recognized provider”
• 404 error The page you're looking for could not be found

This page contains a list of issues you might encounter when administering GitLab Pages.

How to see GitLab Pages logs

You can see Pages daemon logs by running:

sudo gitlab-ctl tail gitlab-pages

You can also find the log file in /var/log/gitlab/gitlab-pages/current.

unsupported protocol scheme \"\""

If you see the following error:

{"error":"failed to connect to internal Pages API: Get \"/api/v4/internal/pages/status\": unsupported protocol scheme \"\"","level":"warning","msg":"attempted to connect to the API","time":"2021-06-23T20:03:30Z"}

It means you didn’t set the HTTP(S) protocol scheme in the Pages server settings.
To fix it:

1. Edit /etc/gitlab/gitlab.rb:

   gitlab_pages['gitlab_server'] = "https://<your_gitlab_server_public_host_and_port>"
   gitlab_pages['internal_gitlab_server'] = "https://<your_gitlab_server_private_host_and_port>" # optional, gitlab_pages['gitlab_server'] is used as default
   

2. Reconfigure GitLab:

   sudo gitlab-ctl reconfigure
   

502 error when connecting to GitLab Pages proxy when server does not listen over IPv6

In some cases, NGINX might default to using IPv6 to connect to the GitLab Pages
service even when the server does not listen over IPv6. You can identify when
this is happening if you see something similar to the log entry below in the
gitlab_pages_error.log:

2020/02/24 16:32:05 [error] 112654#0: *4982804 connect() failed (111: Connection refused) while connecting to upstream, client: 123.123.123.123, server: ~^(?<group>.*)\.pages\.example\.com$, request: "GET /-/group/project/-/jobs/1234/artifacts/artifact.txt HTTP/1.1", upstream: "http://[::1]:8090//-/group/project/-/jobs/1234/artifacts/artifact.txt", host: "group.example.com"

To resolve this, set an explicit IP and port for the GitLab Pages listen_proxy setting
to define the explicit address that the GitLab Pages daemon should listen on:

gitlab_pages['listen_proxy'] = '127.0.0.1:8090'

Intermittent 502 errors or after a few days

If you run Pages on a system that uses systemd and
tmpfiles.d,
you may encounter intermittent 502 errors trying to serve Pages with an error similar to:

dial tcp: lookup gitlab.example.com on [::1]:53: dial udp [::1]:53: connect: no route to host"

GitLab Pages creates a bind mount
inside /tmp/gitlab-pages-* that includes files like /etc/hosts.
However, systemd may clean the /tmp/ directory on a regular basis so the DNS
configuration may be lost.

To stop systemd from cleaning the Pages related content:

1. Tell tmpfiles.d to not remove the Pages /tmp directory:

   echo 'x /tmp/gitlab-pages-*' >> /etc/tmpfiles.d/gitlab-pages-jail.conf
   

2. Restart GitLab Pages:

   sudo gitlab-ctl restart gitlab-pages
   

Unable to access GitLab Pages

If you can’t access your GitLab Pages (such as receiving 502 Bad Gateway errors, or a login loop)
and in your Pages log shows this error:

"error":"retrieval context done: context deadline exceeded","host":"root.docs-cit.otenet.gr","level":"error","msg":"could not fetch domain information from a source"

1. Add the following to /etc/gitlab/gitlab.rb:

   gitlab_pages['internal_gitlab_server'] = 'http://localhost:8080'
   

2. Restart GitLab Pages:

   sudo gitlab-ctl restart gitlab-pages
   

Failed to connect to the internal GitLab API

If you see the following error:

ERRO[0010] Failed to connect to the internal GitLab API after 0.50s  error="failed to connect to internal Pages API: HTTP status: 401"

If you are Running GitLab Pages on a separate server
you must copy the /etc/gitlab/gitlab-secrets.json file
from the GitLab server to the Pages server after upgrading to GitLab 13.3,
as described in that section.

Other reasons may include network connectivity issues between your
GitLab server and your Pages server such as firewall configurations or closed ports.
For example, if there is a connection timeout:

error="failed to connect to internal Pages API: Get \"https://gitlab.example.com:3000/api/v4/internal/pages/status\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"

Pages cannot communicate with an instance of the GitLab API

If you use the default value for domain_config_source=auto and run multiple instances of GitLab
Pages, you may see intermittent 502 error responses while serving Pages content. You may also see
the following warning in the Pages logs:

WARN[0010] Pages cannot communicate with an instance of the GitLab API. Please sync your gitlab-secrets.json file https://gitlab.com/gitlab-org/gitlab-pages/-/issues/535#workaround. error="pages endpoint unauthorized"

This can happen if your gitlab-secrets.json file is out of date between GitLab Rails and GitLab
Pages. Follow steps 8-10 of Running GitLab Pages on a separate server,
in all of your GitLab Pages instances.

Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages

Connections will time out when using a Network Load Balancer with client IP preservation enabled and the request is looped back to the source server.
This can happen to GitLab instances with multiple servers
running both the core GitLab application and GitLab Pages. This can also happen when a single
container is running both the core GitLab application and GitLab Pages.

AWS recommends using an IP target type
to resolve this issue.

Turning off client IP preservation
may resolve this issue when the core GitLab application and GitLab Pages run on the same host or
container.

500 error with securecookie: failed to generate random iv and Failed to save the session

This problem most likely results from an out-dated operating system.
The Pages daemon uses the securecookie library to get random strings via crypto/rand in Go.
This requires the getrandom system call or /dev/urandom to be available on the host OS.
Upgrading to an officially supported operating system is recommended.

The requested scope is invalid, malformed, or unknown

This problem comes from the permissions of the GitLab Pages OAuth application. To fix it:

1. On the left sidebar, select Search or go to.
2. Select Admin Area.
3. On the left sidebar, select Applications > GitLab Pages.
4. Edit the application.
5. Under Scopes, ensure that the api scope is selected.
6. Save your changes.

When running a separate Pages server,
this setting needs to be configured on the main GitLab server.

Workaround in case no wildcard DNS entry can be set

If the wildcard DNS prerequisite can’t be met, you can still use GitLab Pages in a limited fashion:

1. Move
   all projects you need to use Pages with into a single group namespace, for example pages.
2. Configure a DNS entry without the *.-wildcard, for example pages.example.io.
3. Configure pages_external_url http://example.io/ in your gitlab.rb file.
   Omit the group namespace here, because it automatically is prepended by GitLab.

Pages daemon fails with permission denied errors

If /tmp is mounted with noexec, the Pages daemon fails to start with an error like:

{"error":"fork/exec /gitlab-pages: permission denied","level":"fatal","msg":"could not create pages daemon","time":"2021-02-02T21:54:34Z"}

In this case, change TMPDIR to a location that is not mounted with noexec. Add the following to
/etc/gitlab/gitlab.rb:

gitlab_pages['env'] = {'TMPDIR' => '<new_tmp_path>'}

Once added, reconfigure with sudo gitlab-ctl reconfigure and restart GitLab with
sudo gitlab-ctl restart.

The redirect URI included is not valid. when using Pages Access Control

You may see this error if pages_external_url was updated at some point of time. Verify the following:

1. Check the System OAuth application:

   1. On the left sidebar, select Search or go to.
   2. Select Admin Area.
   3. Select Applications and then Add new application.
   4. Ensure the Callback URL/Redirect URI is using the protocol (HTTP or HTTPS) that
      pages_external_url is configured to use.

2. The domain and path components of Redirect URI are valid: they should look like projects.<pages_external_url>/auth.

500 error cannot serve from disk

If you get a 500 response from Pages and encounter an error similar to:

ERRO[0145] cannot serve from disk                        error="gitlab: disk access is disabled via enable-disk=false" project_id=27 source_path="file:///shared/pages/@hashed/67/06/670671cd97404156226e507973f2ab8330d3022ca96e0c93bdbdb320c41adcaf/pages_deployments/14/artifacts.zip" source_type=zip

It means that GitLab Rails is telling GitLab Pages to serve content from a location on disk,
however, GitLab Pages was configured to disable disk access.

To enable disk access:

1. Enable disk access for GitLab Pages in /etc/gitlab/gitlab.rb:

   gitlab_pages['enable_disk'] = true
   

2. Reconfigure GitLab.

httprange: new resource 403

If you see an error similar to:

{"error":"httprange: new resource 403: \"403 Forbidden\"","host":"root.pages.example.com","level":"error","msg":"vfs.Root","path":"/pages1/","time":"2021-06-10T08:45:19Z"}

And you run pages on the separate server syncing files via NFS, it may mean that
the shared pages directory is mounted on a different path on the main GitLab server and the
GitLab Pages server.

In that case, it’s highly recommended you to configure
object storage and migrate any existing pages data to it.

Alternatively, you can mount the GitLab Pages shared directory to the same path on
both servers.

GitLab Pages doesn’t work after upgrading to GitLab 14.0 or above

GitLab 14.0 introduces a number of changes to GitLab Pages which may require manual intervention.

1. Firstly follow the migration guide.
2. Try to upgrade to GitLab 14.3 or above. Some of the issues were fixed in GitLab 14.1, 14.2 and 14.3.
3. If it doesn’t work, see GitLab Pages logs, and if you see any errors there then search them on this page.

To do that:

1. Describe the issue you’re seeing in the migration feedback issue.

2. Edit /etc/gitlab/gitlab.rb:

   gitlab_pages['use_legacy_storage'] = true
   

3. Reconfigure GitLab.

GitLab Pages deploy job fails with error “is not a recognized provider”

If the pages job succeeds but the deploy job gives the error “is not a recognized provider”:

The error message is not a recognized provider could be coming from the fog gem that GitLab uses to connect to cloud providers for object storage.

To fix that:

1. Check your gitlab.rb file. If you have gitlab_rails['pages_object_store_enabled'] enabled, but no bucket details have been configured, either:

   • Configure object storage for your Pages deployments, following the S3-compatible connection settings guide.
   • Store your deployments locally, by commenting out that line.

2. Save the changes you made to your gitlab.rb file, then reconfigure GitLab.

404 error The page you're looking for could not be found

If you get a 404 Page Not Found response from GitLab Pages:

1. Check .gitlab-ci.yml contains the job pages:.
2. Check the current project’s pipeline to confirm the job pages:deploy is being run.

Without the pages:deploy job, the updates to your GitLab Pages site are never published.

Теги: nginx, gitlab, гитлаб

Нередко пользователи сервиса GitLab сталкиваются с проблемой под названием «Ошибка 502». Как правило, она сопровождается следующей фразой: «Whoops, GitLab is taking too much time to respond». Давайте разберём, в чём может быть проблема.

Ошибку 502, как и вышеупомянутую фразу, вам показывает Nginx (компонент, входящий в GitLab). В общем случае речь идёт о том, что web-сервер не может получить от бэкенда ответ. А раз мы говорим о GitLab, то бэкендом здесь выступает Unix-сокет — /var/opt/gitlab/gitlab-workhorse/socket. Тут стоит упомянуть, что конфигурация Nginx для GitLab находится по адресу /var/opt/gitlab, а конкретно Nginx — здесь: /var/opt/gitlab/nginx/conf.

Почему же бэкенд не отвечает?

Ответить на этот вопрос со 100%-ной точностью нельзя. Но ряд причин всё же имеется:
1. У вас на сервере недостаточно оперативной памяти. Если памяти всего 2 Гб, ошибку 502 вы будете всё равно время от времени видеть, даже работая с GitLab в одиночку. Дело в том, что для работы таких компонентов, как Nginx, PostgreSQL, Redis и прочих требуется много памяти. В качестве решения проблемы можно увеличить либо включить swap.
2. У вас упала служба под названием GitLab-workhorse. Она открывает сокет, который слушает Nginx. А вот почему это произошло — вопрос отдельный. Не менее интересно и то, почему она функционирует, а сокета нет. Чтобы решить проблему, попробуйте просто перезагрузить сервер. Также бывает, что сервис падает из-за занятого порта какой-то службы, относящейся к GitLab. Это случается, если на сервере, кроме GitLab запущены другие службы. Ошибки могут быть и в конфигурации. Также нередко проблемы появляются после обновления.
3. Из-за каких-то причин изменились права доступа к сокету /var/opt/gitlab/gitlab-workhorse/socket, в результате чего Nginx не может получить доступ. Проверьте, от какого именно пользователя работает Nginx и удостоверьтесь, что у него достаточно прав для доступа к сокету.

Пожалуй, это основные причины возникновения ошибки 502 в GitLab, покрывающие большинство случаев.

Более подробно ознакомиться с архитектурой GitLab и освоить нюансы его работы вы можете на курсе CI/CD. Именно этой теме посвящено несколько занятий из первого модуля. Скачать программу курса можно здесь.

По материалам статьи «Установка и настройка Gitlab на Centos и Ubuntu».

If you face 502 error from gitlab server, it means that server is busy for doing something in most of cases. It might be associated with server performance.

Option 1. Wait for 5 minutes if your server is just booted

There are some kind of initialization process in Gitlab, so it may takes 2-5 mins usually, so you don’t need to think that server is crashed or something wrong.

OPTION 2. Adjust Unicorn timeout or the number of workers

You can use the following settings in /etc/gitlab/gitlab.rb. Run sudo gitlab-ctl reconfigure for the change to take effect.

unicorn['worker_processes'] = 3
unicorn['worker_timeout'] = 60