SaFi Bank Space : Brave Downsize Plan

Created by Gnanasekaran Gajendiran, last modified on Mar 13, 2023

Component

Steps

Date

Devops Pair

Status

VPN - GKE Cluster

  1. Remove Cluster from terraform state

  2. Remove nodes pools from terraform state

  3. Rename the Cluster and run terraform cloud

  4. Ensure accordingly it’s getting updated in Argocd

  5. Update IP is the Argocd applicationset CRD

  6. Note: Need to know to import GKE cluster and node pools incase planning to revert

After 6.00 P.M

Gnanasekaran Gajendiran

BharathKumar D

DONE

VPN GKE Cluster reduced to Zonal from Regional

TMS

Since brave is pointing to Sandbox environment

  1. GKE cluster can be deleted

  2. Postgresql can be deleted

Note: Don’t remove TF workspace or comment the actual code, do terraform destroy from TF cloud

Anytime

Fol Justin Lacsina (Unlicensed)

Joven Frankie Bico (Unlicensed)

Terraform destroy.

DONE

Hcvault

Since brave is pointing to Sandbox environment

Note: Don’t remove TF workspace or comment the actual code, do terraform destroy from TF cloud

Anytime

Fol Justin Lacsina (Unlicensed)

Joven Frankie Bico (Unlicensed)

DONE

Tyk - GKE cluster

Same as VPN setup

After 6.00 P.M

Gnanasekaran Gajendiran

BharathKumar D

DONE

New Dev or Uat

Do terraform destroy from TF cloud

Anytime

Fol Justin Lacsina (Unlicensed)

Joven Frankie Bico (Unlicensed)

DONE

Applications - GKE Cluster

Same as VPN setup

After 6.00 P.M

Gnanasekaran Gajendiran

BharathKumar D

DONE

Downsize postgressql regional to zone

Note: Checked no read replicas created.

After 6.00 P.M

Gnanasekaran Gajendiran

BharathKumar D

Setup new TMS

Try this 

kubectl get all --all-namespaces -o yaml > cluster-resources.yaml
kubectl get crd -o yaml > crds.yaml

There might be some secres we need to backup as well.

(or)

we need to repeat this manual steps and migrate the exisitng TM vault secrets in hcv

Anytime

Skipped

Confluent Cloud Kafka to VM Kafka

Note: keep the existing kafka confluent cluster as it is

Regin Villamor (Unlicensed)

Monitoring setup

Same as VPN setup

Gnanasekaran Gajendiran

BharathKumar D

DONE

CICD

Cloud Composer

Application Cluster:

  1. Run the below commands 

  2. terraform state rm module.gke_shared_vpc_applications.data.kubernetes_service_v1.api_proxy[0]
terraform state rm module.gke_shared_vpc_applications.google_container_cluster.cluster
terraform state rm module.gke_shared_vpc_applications.kubernetes_cluster_role_binding_v1.argocd_manager[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_cluster_role_binding_v1.hcvault_k8s_auth_sa[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_cluster_role_binding_v1.argocd_manager[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_cluster_role_binding_v1.hcvault_k8s_auth_sa[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_cluster_role_v1.argocd_manager[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_config_map_v1.kube-api-proxy[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_deployment_v1.kube-api-proxy[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_namespace_v1.kube-api-proxy[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_secret_v1.argocd_manager[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_secret_v1.hcvault_k8s_auth_sa[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_service_account_v1.argocd_manager[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_service_account_v1.hcvault_k8s_auth_sa[0]
terraform state rm module.gke_shared_vpc_applications.kubernetes_service_v1.kube-api-proxy[0]
terraform state rm google_container_node_pool.preemptible-2vcpu-8gb_applications
terraform state rm google_container_node_pool.standard-2vcpu-8gb_runner
terraform state rm google_container_node_pool.standard-2vcpu-8gb_vpn
terraform state rm argocd_cluster.gke_applications

  3. Rename cluster and Modify the below lines

    1. https://github.com/SafiBank/SaFiMono/blob/main/devops/terraform/tf-env-applications-infra/applications_gke.tf#L16

    2. Line number 18, 75, 157

    3. Comment lines 110 to 153

  4. Rotate KMS Key

  5. Update network IPs

VPN Cluster:

  1. Run the below commands 

terraform state rm module.gke_shared_vpc_vpn.google_container_cluster.cluster
terraform state rm google_container_node_pool.standard-2vcpu-8gb_vpn

2. Rename the cluster

https://github.com/SafiBank/SaFiMono/blob/main/devops/terraform/tf-env-vpn-infra/vpn_gke.tf#L16

3. Modify below lines

  • Line number 18 and 59

4. Update the Loadbalancer IP in network.yaml for Ingress and run the terraform safionline

5. Rotate KMS Key

6. Run the terraform code

7. Ensure it’s update in the Argocd

8. Update the Loadbalancer IP in network.yaml for Ingress and run the terraform safionline

9. Get the GKE cluster IP

10. Update in the Argocd environments and refresh the app.

Rollback:

  1. Import old cluster into state

terraform import module.gke_shared_vpc_vpn.google_container_cluster.cluster 
terraform import google_container_node_pool.standard-2vcpu-8gb_vpn safi-env-brave-vpn/asia-southeast1/safi-brave-vpn/pool-2cpu8gb-20221027111930994400000001

2. Follow the above steps with old IP