Gitlab Pipeline Run Python Script
GitLab is a Git-based platform, similar to Github, that provides an array of tools and technologies for various aspects of the software development lifecycle (SDLC).
Test:pylint: image: python:3.6 script: - pip install pylint - quiet - pylint - ignored - classes=socketobject.py. Test:pylint is simply the name of the job. You can choose whatever you want. The rest of the code indicates that gitlab-runner should use the docker image python:3.6, and run the mentioned commands. See full list on blueskyjunkie.ca. Here is my./ci-cdscripts/getversion.py: import os refName = os.environ.get('CICOMMITREFNAME') piplineID = os.environ.get('CIPIPELINEID') relVersion = refName + '.0.' + piplineID version = relVersion.replace('rel.' , ') print('current version is', version) python output in pipeline log.
Test:pylint: image: python:3.6 script: - pip install pylint - quiet - pylint - ignored - classes=socketobject.py. Test:pylint is simply the name of the job. You can choose whatever you want. The rest of the code indicates that gitlab-runner should use the docker image python:3.6, and run the mentioned commands.
Gitlab allows users to host repositories using the popular version control system: Git. It also allows users to track proposals for new implementations, generate bug reports, review code, etc.
Another essential feature, which we will use in this tutorial, is its build-in continuous integration tool called Gitlab-ci.
In this tutorial, we are going to explore the basics of Gitlab and Gitlab-ci by building a CI/CD pipeline and running it using the UI provided by Gitlab.
Table of contents
Prerequisites
- Basic knowledge of Git and version control:
- Basic understanding of CI/CD:
- Basic knowledge of programming:
Setting up Gitlab
Setting up a Gitlab account is very easy. Head over to this link, create an account, or choose to sign in with Google or GitHub. Once you sign in, it will redirect you to the homepage where you can explore various options such as creating a new project, a group or just explore some public repositories.
Follow these steps to set up a new project on Gitlab:
- On the homepage, click on “create blank project” to initialize a new repository.
- Gitlab will redirect you to a form where you fill in the details of your project, such as title, description, etc. You can also choose to make your project public or private. After you finish adding the details, click on “create project”.
- Click on “clone” and copy the HTTPS URL to clone the project. Open a terminal or command line and enter the following command to clone the project into your local machine:
Now that you have the project on your local machine, we can start building our own CI/CD pipeline and deploy it on Gitlab using Gitlab-ci.
Setting Up Gitlab
Building a CI/CD pipeline
Continuous Integration can be defined as a process, in the software development lifecycle, where engineers and developers push code into a git-based repository. If developers do this frequently, an automated process, to verify and build the code, is included to save time and costs.
According to the official documentation, GitLab CI/CD is a powerful tool built into GitLab that allows you to apply all the continuous methods (Continuous Integration, Delivery, and Deployment) to your software with no third-party application or integration needed.
Users can configure GitLab CI/CD by creating a file called .gitlab-ci.yml in the root directory of the project. This file is written in a simple and easy to use language called YAML. This is a good article on YAML and its basics.
Once you define the .gitlab-ci.yml file, Gitlab creates a pipeline, that runs whenever there is a change to the code in the repository. These pipelines can have single or multiple stages that run one after the other(in series).
Each stage can consist of multiple jobs that are executed in parallel by the gitlab-runner, that is an application that works with GitLab to run jobs in a pipeline.
Code
Let’s build a simple CI pipeline to run a Python script whenever we push changes to our repository.
Create a Python script called test.py
in the repository and add the following lines of code into the file.
The script prints “hello world” and “testing” 5 times. This is just an example to help you understand how to create and run pipelines. You can run complex scripts and commands as well.
Create a file called .gitlab-ci.yml in the repository and add the following snippet of code. The variables
section defines a list of variables that can be used in the pipeline.
I have declared a variable called example
, whose value is set to “this is an example variable”. Variables are declared as key-value pairs (separated by a semicolon) in YAML.
The stages
section defines a list of stages in the pipeline. These stages are executed one after the other, and hence, it’s important to order them based on dependencies. In other words, stages that are dependant on other stages must be declared after all of their dependencies.
The next lines define the actual stages that run in series. They start with the name of the stage followed by the tag stage
, that defines which stage it is a part of. The script
tag defines a shell script or command that is executed by the runner.
We can use the echo
command to print something on the terminal and the cat
command to display the contents of the Python script. We can also use echo
to print the value of the variables that are defined in the file. These variables can be accessed by using the $
symbol.
Run the following commands to push the changes to the repository:
Gitlab Pipeline Run Python Script Examples
You will be prompted to enter your Gitlab username and password. Once you correctly enter the username and password, the changes will be pushed to the repository and Gitlab will instantly run the code in the .gitlab-ci file by creating a pipeline.
Navigate to the CI/CD section on the left navbar to view a list of pipelines. The latest one will be at the top and will have the “latest” tag. It will also show us whether the pipeline passed or failed. Click on “passed” or “failed” to view the stages of the pipeline and all the steps that were executed.
You will see the different stages that you defined and whether each stage passed or not. You can click on the individual stages to view the jobs executed in each stage in detail.
Conclusion
In conclusion, Gitlab-ci is an impeccable tool that makes the lives of software engineers easier by providing a robust solution without requiring any third-party library or tool.
In this tutorial, we understood the basics of GitLab and GitLab-ci by building a simple CI pipeline that automates the process of testing and building your code without the need for human intervention.
Now that you have grasped the basics, you can go ahead and build your own pipeline. Whether you are a software developer, a system administrator, or a DevOps engineer, GitLab-ci is an indispensable part of your toolkit.
Peer Review Contributions by: Gregory Manley
About the author
Adith BharadwajGitlab Pipeline Api
Adith Bharadwaj is a senior at the National Institute of Engineering (NIE) and a Software Engineer Intern at Cisco - India. Adith has a keen interest in solving challenging problems and is a data science and machine learning enthusiast. When he’s not coding, he loves drawing, working out, and watching great TV shows, movies or anime.
Hello everyone,
my name is Roman and I’m trying to set up CI pipeline. This pipeline should consist of 4 stages. First 3 stages (build, test and package) should be run on each push to the git and the last one (post_merge) should be run only when project maintainer is merged merge request into the master branch.
Blueskyjunkie.ca › Python-ci-pipeline-part-2Gitlab CI/CD Pipeline For Open Source Python Projects, Part 2 ...
On the “package” stage I create a docker-image with my application and push this image into the local docker-registry. This docker image has tag equal to the name of current branch in the git. Testers can deploy this image into the testing environment and test this feature manually. When this branch is merged into the master this docker-image is not necessary anymore and I want to remove it on the “post_merge” stage.
It’s possible to remove tag by using Gitlab API something like this:
But I can’t find the way when to make this API call.
To implement this approach I created following config in .gitlab-ci.yml (I skipped some not important parts, such as variables definition):
First 3 steps are working as I want, but the last one is not working. In this step variable $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME is not available and it’s empty. I also tried to replace it with $CI_COMMIT_REF_NAME, but in this context this variable has value “master”, but I need to have the name of source branch here. Also I tried to replace cleanup job with something like this:
But in this case this stage starts as independent pipeline (which is consist of the only one job) and this stage doesn’t have access to artifacts of build stage. Also this pipeline runs not only on the merge request merged, but on merge request is created too.
Does any one know how to set up .gitlab-ci.yml to run stage only when some branch is merged into the master and get the name of source branch?