Shallow Discussion on Pipeline Usage

  • 2023/6/27
  • 5-minute read

Summary

Through previous sharing, I have installed gitlab-runner and Jenkins in my own environment. I used to use script for fully automated deployment. All operations were done by the shell executor. Today I will share some understanding of the GitLab-runner executor.

Jenkins and GitLab-CI

Some readers are confused why GitLab-CI is used instead of Jenkins. Here is a simple comparison between GitLab’s pipeline and Jenkins’ pipeline.

  1. Installation and configuration:
  • GitLab CI: As a part of GitLab, the installation is simple and the configuration is relatively simple.
  • Jenkins: It is a standalone tool with complex installation and configuration, requiring configuration of various plugins and environments.
  1. Difficulty of use:
  • GitLab CI: The configuration is simple, using YAML format configuration files, easy to get started, and the learning curve is flat.
  • Jenkins: Powerful features, but tedious configuration, requires programming to implement more complex tasks, and the learning curve is steep.
  1. Integration with source code management:
  • GitLab CI: Natively integrated with Git, very easy to integrate with GitLab repositories, can automatically detect repository changes and run pipelines.
  • Jenkins: Can be integrated with various source code management tools, but triggers need to be manually configured to detect changes.
  1. Scalability:
  • GitLab CI: GitLab Runner can be installed to extend, supports integration with Kubernetes, and is relatively easy to scale horizontally.
  • Jenkins: Supports distributed deployment by itself, with a large number of plugins that can connect different environments, high scalability but complexity.
  1. Preset environment and assets:
  • GitLab CI: No preset environment or assets, an independent environment is created each time the pipeline runs.
  • Jenkins: Rich preset environments, credentials, caches, etc. can be reused, but also increase management difficulty.

So how to choose: GitLab CI is simple to use but lacks some functionality, and extension and management are relatively simple. Suitable for small and medium-sized projects. Jenkins has powerful features but is more complex, requiring more time to manage and extend. Suitable for large projects. The two can work well together, for example, using GitLab CI for daily builds and Jenkins for release management.

In summary, for individual developers or small teams, the GitLab pipeline is sufficient. For larger scales, choose GitLab or Jenkins pipelines according to actual needs, combined use.

GitLab Runner Executor

It is a kind of program that can execute build, test and deployment tasks in GitLab CI/CD. It can run on different operating systems such as Linux, Windows and macOS.

The executor list has been locked and will no longer evolve or accept new executors. The following are the types of executors:

    SSH
    Shell
    Parallels
    VirtualBox 
    Docker
    Docker Autoscaler (alpha)
    Docker Machine (auto-scaling)
    Kubernetes
    Instance (alpha)
    Custom

Currently, ssh, shell and docker as well as k8s are generally used. Simply put, the purpose is to execute remote commands, scripts, docker image operations and deploy to k8s. For specific introductions, you can go directly to the official website:

https://docs.gitlab.cn/runner/executors/#i-am-not-sure

How to choose

When registering, which executors should be chosen? It depends on which tasks are used in your pipeline. If only shell scripts are used, choose shell. If docker operations are used, choose docker.

When registering by default, only one executor type can be selected. However, in the .gitlab-ci.yml file, we can specify different executors for different jobs. Therefore, we can use the shell executor to build the application and the docker executor to deploy the application in the same CI/CD pipeline.

build: 
  stage: build 
  script: 
    - npm install 
    - npm run build 
deploy: 
  stage: deploy 
  image: nginx:stable-alpine 
  script: 
    - mv /dist /usr/share/nginx/html  
  only: 
    - test
  • The build job uses the shell executor to build the Vue app through npm.
  • The deploy job uses the docker executor to deploy the dist directory to the Nginx container.

So when registering GitLab Runner, the choice of executor type does not affect our specification of different executors for different jobs in the .gitlab-ci.yml file. We can specify by:

  • script uses shell executor
  • image uses docker executor

Questions continued

1. Do two GitLab runners need to be registered for the same project?

You can register two or one runner for a project. But registering multiple GitLab Runners has the benefit of avoiding resource competition and conflicts and ensuring the stability and reliability of the project. Registering additional Runner types is usually for the following purposes:

  1. Isolate the execution environment. For example, register the shell Runner for building and the docker Runner for deployment to isolate the build environment and the production environment.
  2. Process more jobs in parallel. If the job queue of a Runner is too long, register more Runners to improve the processing capacity of the CI pipeline.
  3. Different machine resources. For example, register docker Runner on the server and shell Runner on the personal computer.

2. Why do you still need to select the executor type when registering?

Registering GitLab Runner lets us choose the executor type, which seems contradictory to the fact that one Runner can handle jobs of different executor types. This is mainly for two reasons:

  1. Default executor. Although a Runner can handle jobs of different executor types, the executor type selected during registration will become the default executor of the Runner. That is, if a job does not specify image or script, the Runner will use the default executor to run it.
  2. Optimize the environment. Choosing different executor types can optimize the running environment for different types of jobs on that Runner. For example:
  • Choose shell, the Runner will default to installing GIT, OpenSSH, Build Essentials, etc., convenient for handling script type jobs.
  • Choose docker, the Runner will default to supporting Docker commands, convenient for pulling and running various images to handle image type jobs.
CI/CD