RHACM GitOps – Kustomize for Dev & Prod Environments
This blog demonstrates how to use RHACM GitOps to manage the MySQL application in the development and Production environment.
MySQL does not need persistent storage in development. And Persistent storage is required in production. environment. The development environment has one cluster and the production environment has the other cluster. The routes are different since applications have different domain names on 2 clusters. The rest should be the same for both environments.
We create a ‘base’ directory to store the common resource yaml files, the ‘overlays/dev’ directory has the overlay resources files for the development environment, and the ‘overlays/prod’ directory has the overlay resources files for the production environment
Create a Branch on GitHub
The first thing is to create a branch ‘MySQL’ for the application.
$ git checkout -b MySQL
Switched to a new branch 'MySQL'
Bases Resource files
Below are the resource yaml files under the directory base. All resource files are the common setting for both development and production environments.
$ tree base/
base/
├── deployment-frontend.yaml
├── deployment.yaml
├── kustomization.yml
├── service-frontend.yaml
└── service.yamlIf there is kustomization.yaml file in the Git folder, kustomize is applied. This file defined all the resource files.
$ cat base/kustomization.yml
kind: Kustomization
resources:
- deployment-frontend.yaml
- deployment.yaml
- service.yaml
- service-frontend.yamlBelow is the deployment resource file. Note that the volume resource has only the name `db-volume`. It will have a different setting in development and production environments.
cat base/deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: todonodejs
name: mysql
spec:
replicas: 1
selector:
matchLabels:
app: todonodejs
name: mysql
template:
metadata:
labels:
app: todonodejs
name: mysql
spec:
containers:
- image: registry.redhat.io/rhel8/mysql-80:1-152
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: r00tpa55
- name: MYSQL_USER
value: user1
- name: MYSQL_PASSWORD
value: mypa55
- name: MYSQL_DATABASE
value: items
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- mountPath: "/var/lib/mysql"
name: db-volume
volumes:
- name: db-volume
- name: db-init
emptyDir: {}Below is the overlay directory structure.
Overlay Resource files
$ tree overlays/
overlays/
├── dev
│ ├── dbclaim-pvc.yaml
│ ├── kustomization.yaml
│ ├── route.yaml
│ └── volume.yaml
└── prod
├── dbclaim-pvc.yaml
├── kustomization.yaml
├── route.yaml
└── volume.yamlThe directory ‘dev’ has all the overlay resource files for the development environment.
The directory ‘prod’ has all the overlay resource files for the production environment.
Development Resource Files
Below is the kustomization.yaml file for the development environment.
We can use ‘spec.packageOverrides‘ to override kustomization in bases at the subscription deployment time.
The ‘resources' adds the new resource file(s) into the kustomization in bases.
The ‘namePrefix’ add the prefix to the resource name.
$ cat overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../base
patchesStrategicMerge:
- volume.yaml
resources:
- route.yaml
namePrefix: dev-This is the ‘volume.yaml file. The lines above ’emptyDir: {}’ are used to detect the target resource to overwrite in bases. The target is the volume ‘db_volume’ inside the ‘mysql’ deployment. Please take note we cannot specify the ‘namespace’ in both bases and overlays resource files. The namespace setting will cause failure. We leave the namespace to be specified in RHACM subscription.
$ cat overlays/dev/volume.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: todonodejs
name: mysql
spec:
template:
spec:
volumes:
- name: db-volume
emptyDir: {}Since the route domain ‘apps.bn7z2-m-dev.sandbox1558.opentlc.com’ is specific to the dev cluster, it should be in the overlay instead of the base.
$ cat overlays/prod/route.yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
labels:
app: todonodejs
name: frontend
name: frontend
spec:
host: todo.apps.bn7z2-m-dev.sandbox1558.opentlc.com
path: "/todo"
to:
kind: Service
name: dev-frontend
weight: 100
wildcardPolicy: None
Use the below command to check the final settings. You can see the volume in deployment ‘mysql’ is set to ’emptyDir’, and route resource is added.
$ kubectl kustomize .
...
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: todonodejs
name: mysql
name: dev-mysql
spec:
...
template:
...
spec:
containers:
...
volumes:
- emptyDir: {}
name: db-volume
...
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
labels:
app: todonodejs
name: frontend
name: dev-frontend
spec:
host: todo.apps.bn7z2-m-dev.sandbox1558.opentlc.com
path: /todo
to:
kind: Service
name: dev-frontend
weight: 100
wildcardPolicy: None
$ git add . ; git commit -a -m "update templates" ; git push --set-upstream origin MySQLOnce the resources’ settings are right, you can push new content to the branch in GitHub.
Production Resource Files
The settings in the prod overlay are similar to the dev environment, except it has the extra file ‘dbclaim-pvc.yaml’ for persistent storage.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim-02
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Mi
storageClassName: gp2In the ‘kustomization.yaml’ file, ‘dbclaim-pvc.yaml’ is added under the ‘resources’.
$ cat overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../base
patchesStrategicMerge:
- volume.yaml
resources:
- dbclaim-pvc.yaml
- route.yaml
namePrefix: prod-The pvc name ‘mysql-pv-claim-02’ is added to the file ‘volume.yaml’ which is used to update volume ‘db-volume’ for ‘mysql’ deployment.
$ cat overlays/prod/volume.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: todonodejs
name: mysql
spec:
template:
spec:
volumes:
- name: db-volume
persistentVolumeClaim:
claimName: mysql-pv-claim-02Since the route domain ‘aaps.bn7z2-m-prod.sandbox1558.opentlc.com’ is specific to the dev cluster, it should be in the overlay instead of the base.
$ cat overlays/prod/route.yaml
apiVersion: route.openshift.io/v1
kind: Route
metadata:
labels:
app: todonodejs
name: frontend
name: frontend
spec:
host: todo.apps.bn7z2-m-prod.sandbox1558.opentlc.com
path: "/todo"
to:
kind: Service
name: dev-frontend
weight: 100
wildcardPolicy: None
RHACM
From the RHACM console, deploy the development customized MySQL application.
- On the left navigation bar, navigate to
Applications, clickCreate application, and then clickSubscription. - On the
Applicationspage, setYAML: Onto view the YAML in the console as you create the application. - In both the
NameandNamespacefields, typemysql. - Click
Repository location for resourcesand chooseGitfor the repository type for your deployable resources. - For the channel source, enter the URL
https://github.com/alpha-wolf-jin/acm-kustomizerepository in the URL field. Enter the username and token since we use the private Git repository. - Specify the Git Path ‘overlays/dev’ for the development environment.
- Scroll down, click
Select clusters to deploy to, then selectDeploy application resources only on clusters matching specified labelsconfigure a placement rule for the application. Typepurposein the Label field anddevin the Value field. - Click Create button
Note: specify the namespace here instead of resource yaml files.
Update the subscription name to ensure it is unique and meaningful for your environment.
---
apiVersion: apps.open-cluster-management.io/v1
kind: Subscription
metadata:
name: dev-mysql-subscription-1
...
spec:
...
placement:
placementRef:
name: dev-mysql-placement-1
kind: PlacementRule
...
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: dev-mysql-placement-1
...
spec:
clusterSelector:
matchLabels:
purpose: developmentRepeat the same steps for the production environment.
| Field | Value |
|---|---|
Name | mysql |
Namespace | mysql |
Repository types | Git |
URL | https://github.com/alpha-wolf-jin/acm-kustomize/ |
| username | **** |
| Access token | *************** |
| Branch | MySQL |
| Path | overlays/prod |
| Label | purpose |
| Value | prod |
| Deployment window | Always Active |
Update the subscription name to ensure it is unique and meaningful for your environment.
---
apiVersion: apps.open-cluster-management.io/v1
kind: Subscription
metadata:
name: prod-mysql-subscription-1
...
spec:
...
placement:
placementRef:
name: prod-mysql-placement-1
kind: PlacementRule
...
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: prod-mysql-placement-1
...
spec:
clusterSelector:
matchLabels:
purpose: prodOn the Topology page, navigate to Subscription →All Subscription. You can see both dev and prod subscriptions.
The above is one way to use RHACM GitOps to manage multi-clusters. Here, we use the Git branch for the application (you can have more branches for various applications), and the Git path for various environments. We use the cluster label to match the Git contents to the clusters. Of course, it is not the only way.
Of course, it is not the only way. I hope this sharing is able to give a clue on how to use RHACM GitOps to manage multi-clusters.
Disclaimer:
The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.
Tags:
Latest posts
Comments
Leave a Reply