Google Functions - Partially Done
Pubsub - Partially Done
Big Query - Partially Done
Vertex AI Feature store - Pending
Google Functions:
Source Code: https://github.com/SafiBank/DataTest/tree/main/services
Categorized parent folder then google functions
Google function names should be similar to subfolder/child folder name
Each function may different trigger, and needs to refer data test GCP project (https://console.cloud.google.com/home/dashboard?project=datatest-348502)
What is Done?
Sample GitHub Action Pipeline is done - https://github.com/SafiBank/DataTest/tree/main/.github/workflows
Pending:
GF should be moved under our shared VPC
Need to use our own KMS
Publish GF URL into Vault
GF mapping on the Tyk (It depends on the Tyk operator)
Stage and Production pipelines should be prepared
Dev pipeline should be created for other functions
PubSub and Big Query:
Created topics and subscriptions through TF
https://github.com/SafiBank/SaFiMono/tree/main/devops/terraform/tf-data/_files
Pending:
Sink between confluent Topics to Pubsub topic
Topics names (Note: confluent topic names and pubsub topic names are same)
customer.device-fingerprint.snapshots.temp customer.snapshots.facial_verification customer.snapshots.id_card customer.snapshots.temp transaction.interbank.temp transactions.intrabank.temp loans.overdraft.activity.temp
Reference : Event/Snapshot Stream Solution