Harness GitOps PR pipelines
Currently, this feature is behind the feature flags ENV_GROUP
, NG_SVC_ENV_REDESIGN
, and OPTIMIZED_GIT_FETCH_FILES
MULTI_SERVICE_INFRA
. Contact Harness Support to enable the feature.
This topic shows you how to create a Harness PR pipeline in Harness GitOps to make changes to one of an ApplicationSet's target environments.
Often, even though your ApplicationSet syncs one microservice/application to multiple target environments, you might want to change a microservice in just one of the target environments, such as a dev environment. A Harness PR Pipeline enables you to do this.
This topic builds on the ApplicationSet created in Harness GitOps ApplicationSets. Ensure you have read that topic and, ideally, have set up an ApplicationSet in Harness before creating a PR pipeline.
ApplicationSets and PR pipelines summary
Before you begin
This topic extends Harness GitOps features covered in the following topics:
Review these topics before proceeding.
Create a PR pipeline
When you deploy a Harness PR pipeline, you indicate the target environment application and the config.json keys/values you want changed. Harness creates the pull request in your Git repo and merges it for you. Once complete, the target environment application has the new keys/values.
For the PR Pipeline, we'll create two Harness environments, dev and prod. These names are the same as the folder names in the repo:
We use the same names so that when we select a Harness environment we can pass along the same name as the target folder.
Next, we'll create a Harness service that points to the config.json files in these directories.
The path to the config.json files in the service will use the expression <+env.name>: examples/git-generator-files-discovery/cluster-config/engineering/<+env.name>/config.json
.
At runtime, this expression resolves to the Harness environment you selected.
When you run the pipeline, you'll select which environment to use, dev or prod, and Harness will use the corresponding repo folder and update that application only.
Create Harness environments for each target environment
First, let's create the dev environment.
- In your Harness Project, click Environments.
- Click New Environment.
- Enter the following and click Save:
- Name: dev.
- Environment Type: Pre-Production.
The new environment is created.
Create variable for JSON key-value pair
Next, we'll add a variable for the JSON key-value we will be updating.
In Advanced, in Variables, click New Variable Override.
In the variable Name, enter asset_id and click Save.
The
asset_id
name is a key-value in the config.json files for both dev and prod:For variable Value, select Runtime Input:
Later, when you run the pipeline, you'll provide a new value for this variable, and that value will be used to update the config.json file.
Select GitOps clusters to add to environment
Next, we'll link the Harness GitOps clusters for dev with the dev environment. Once you link GitOps clusters to an environment, you can then select from an environment's linked GitOps clusters when you select the environment in a pipeline.
- Click GitOps Clusters.
- Click Select Cluster(s).
- Select engineering-dev.
- Click Add.
The Harness GitOps cluster is now linked to the environment.
Repeat the process for the prod environment
- Create a new environment named prod.
- Add the same
asset_id
variable to the prod environment. - Link the engineering-prod GitOps cluster to the environment.
Create the PR pipeline
To create the pipeline, we'll simply create a new service that includes the manifest you want deployed and select the dev environment you created earlier.
In your Harness project, click Pipelines.
Click Create a Pipeline.
In Create new Pipeline, enter the name PR Pipeline, and then click Start.
Click Add Stage, and select Deploy.
Enter the following and click Set Up Stage:
- Stage Name: enter PR Example.
- Deployment Type: select Kubernetes.
- Enable the GitOps option.
The stage is created and the service settings appear.
Create the service
The Harness service represents what you're deploying. In this case, we're deploying a config.json file.
- In Select Service, click New Service.
- In Name, enter PR Example.
- In Manifests, click Add Release Repo Manifest.
- In Release Repo Store, select one of the following repositories.
- GitHub
- Bitbucket
Add a Harness Github connector to configure the location for Harness to fetch the config.json file.
To add a Harness Github connector:
In Github Connector, select New Github Connector.
Enter the following Github connector settings:
Name: enter gitops-github.
URL Type: select Repository.
Connection Type: select HTTP.
GitHub Repository URL: enter the HTTP URL for repo you used for your ApplicationSet, such as
https://github.com/johnsmith/applicationset.git
.Authentication: select Username and Token. For the token, you'll need to use a Personal Access Token (PAT) from Github. If you are logged into Github, just go to https://github.com/settings/tokens.
Ensure the PAT has the repo scope selected.
You will store the PAT in a Harness Text Secret. For details on Secrets Management, go to Harness Secrets Management Overview.
Select Enable API access and use the same Harness secret.
For more information, go to the GitHub connector settings reference.
Select Continue.
In Connect to the provider, select Connect through Harness Platform., and then click Save and Continue.
When the Connection Test in complete, click Continue.
Add a Harness Bitbucket connector to configure the location from which Harness can pull the config.json file.
To add a Bitbucket connector:
In Bitbucket Connector, select New Bitbucket Connector.
Enter the following Bitbucket connector settings:
Name: enter gitops-bitbucket-cloud.
URL Type: select Repository.
Connection Type: select HTTP.
GitHub Repository URL: enter the HTTP URL for repo you used for your ApplicationSet, such as
https://bitbucket.org/johnsmith/applicationset.git
.Authentication: select Username and Token. For the token, use a Personal Access Token (PAT) from Bitbucket. If you are logged into Bitbucket, go to HTTP access tokens. You must provide an account-level app password or token. Repo-level tokens are not supported.
Store the PAT in a Harness Text Secret. For details on Secrets Management, go to Harness Secrets Management Overview.
Select Enable API access and use the same Harness secret.
For more information, go to Bitbucket connector settings reference.
Select Continue.
In Connect to the provider, select Connect through Harness Platform, and then click Save and Continue.
After the Connection Test is complete, click Continue.
Specify manifest details
Now we'll define the manifest to use for the PR pipeline. We'll use the path to the config.json files. We'll use the expression <+env.name>
in the path so that we can dynamically select the path based on the Harness environment we select: dev or prod.
In Manifest Details, enter the following settings and then click Submit.
- Manifest Name: enter config.json.
- Git Fetch Type: select Latest from Branch.
- Branch: enter the name of the main branch (master, main, etc).
- File Path: enter
examples/git-generator-files-discovery/cluster-config/engineering/<+env.name>/config.json
.
Note the use of
<+env.name>
.Back in New Service, click Save.
the service is added to the pipeline.
Click Continue to add the environment.
Add Environment Runtime Input
For the stage environment, we'll use a Harness runtime input. When you run the pipeline, Harness will prompt you for a value for the environment. You can select the environment you want to use for the PR.
Set Specify environment or environment group as a runtime input.
Click Continue.
Review execution steps
In Execution, Harness automatically adds the following steps.
You don't have to edit anything in these steps. The step are ready to run.
Update Release Repo
This step fetches your JSON files, updates them with your changes, performs a commit and push, and then creates the PR.
There is an option to provide a custom PR title. If you don't provide a PR title, Harness creates the PR with the title Harness: Updating config overrides.
This step supports hierarchical variables. If you specify a dot-separated variable in this step, it creates or updates a nested variable.
You can also enter variables in this step to update key-value pairs in the config file you are deploying.
If there is a matching variable name in the variables of the Harness service or environment used in this pipeline, the variable entered in this step will override them.
If an empty or blank value is provided for a variable, it will be disregarded, and no updates will be made to the JSON or YAML file for that specific variable.
Merge PR
This step simply merges the new PR.
Revert PR
This step reverts the commit passed and creates a new PR. Use this step if you want to run any tests or automation on the pipeline and then revert the commit done by the Update Release Repo step.
The Revert PR step uses the commitId of the Update Release Repo step as input. The commitId can be an expression, runtime input, or a static value. For example, <+pipeline.stages.deploy.spec.execution.steps.updateReleaseRepo.updateReleaseRepoOutcome.commitId>
.
The Revert PR step creates a new branch and creates a commit to revert the changes done in the Update Release Repo step commit.
You can create another Merge PR step to merge the Revert PR step.
You can create a maximum of two Merge PR steps in a stage.
Run and verify the PR pipeline
Now your PR pipeline is ready.
Click Save, and then click Run.
In Run Pipeline, in Specify Environment, select the dev Environment.
In Environment Variables, for asset_id, enter the value
12345678
.In Specify GitOps Clusters, select the engineeringdev cluster.
Click Run Pipeline.
You can review the deployment steps in real-time.
Here's an example of each step:
Service:
Starting service step...
Processing service variables...
Applying environment variables and service overrides
Processed service variables
Processed artifacts and manifests
Completed service stepGitOps Clusters:
Environment(s): {dev}
Processing clusters at scope PROJECT
Following 1 cluster(s) are present in Harness Gitops
Identifiers: {engineeringdev}
Following 1 cluster(s) are selected after filtering
Identifiers: {engineeringdev}
CompletedUpdate Release Repo:
Merge PR:
PR Link: https://github.com/wings-software/gitops-pipeline-demo/pull/155
Pull Request successfully merged
Commit Sha is bcd4f2f73a47b74dba54habbcd10a6679ed99a
Done.Revert PR:
Setting git configs
Using optimized file fetch
Created revert PR https://github.com/wings-software/gitops-pipeline-demo/pull/156
Done.Merge PR_1:
PR Link: https://github.com/wings-software/gitops-pipeline-demo/pull/156
Pull Request successfully merged
Commit Sha is da3c393560bf5e831a7b4fa123456c1eafb989ac
Done.
Check the repo to see that the config.json file for the dev environment has been updated with the new asset_id value:
Congratulations! You PR Pipeline was successful.