Monorepo Structure
.github/workflows - contains the GitHub actions config
_<job-name>.yml - job components imported by the service for their workflow
tyk-brave-api-importer.yml - adds services to Tyk API Gateway
app - front-end code for mobile app
app-bo - front-end code for back-office
common - cross-cutting libraries and utilities
schema - kafka topic and schemas
schemas - contains avrò schema definitions, organized per domain
topicSchemasDefinition.json - defines which Avro schema is used for each Kafka topic
devops -
argocd/environments - contains environment specific config
<env>/apps/<service name> - contains config for setting up the services in the specific environment
<env>/apps/base - main application config for kubernetes deployment
<env>/apps/kustomization.yaml - list of files in /base folder
terraform/_files/microservices.yml - lists of services for deployment
services - contains the code for each service
template-service - template for creating new services
tm-contracts - thought machine stuff
Environments
Brave
dev environment
services are deployed upon merging to main
Tyk URL: https://api.smallog.tech
Stage
staging environment
services are deploy upon merging to epfs branch
Tyk URL: https://api.blueship.store
Environment Variables
Plain Env Vars
non-sensitive environment specific stuff
urls, service names, etc
can be put directly in the values.yml file
stored in a ConfigMap on kubernetes
Secrets
sensitive information
passwords, tokens, IDs, etc
stored in Vault: https://vault.cicd.safibank.online/ui/vault/secrets/secret/list
stored in a Secret on kubernetes
sharing secrets:
Use Vault tool to wrap and unwrap sensitive information when passing secrets via chat/email/ticket/etc
wrapped secrets are one-time use only, and have an expiration
never share secrets as plain text
Updating Environment variables
Go to devops/argocd/environtments/<env>/apps/<service name>
Open and update values.yml file
For secrets, value should be the path in vault
format: <secret:secret/data/path/to/secret~secret_key>
path/to/secret - breadcrumb in vault
# plain env: ECPAY_HOST_URL: https://myecpay.ph/ecpayoneapi # secrets secret: DB_PASSWORD: <secret:secret/data/brave/apps/ecpay-gateway/postgres~password> DB_URL: <secret:secret/data/brave/apps/ecpay-gateway/postgres~db_url> DB_USER: <secret:secret/data/brave/apps/ecpay-gateway/postgres~user>
Logging and Monitoring
URL
logs - Loki
traces - Tempo
ArgoCD
Shows services that are deployed in Kubernetes
Kafka
Main mode for async communication between
Two Kafka clusters:
Thought Machine
main mode of communication with Thought Machine
uses protobuf for messages
Service
communication between internal backend services
uses Avro for messages
schema location: common/schema/schemas/<domain>
topic definition: common/schema/topicSchemasDefinition.json
Further Reading
Dashboard for service Kafka topics (Brave): http://172.21.48.3:9021/clusters
Database
Postgres as our SQL database
Brave URL: jdbc:postgresql://10.50.0.2:5432/<db-name>
credentials can be found in Vault
Tyk API Gateway
API Gateway used to expose our services externally
Update .github/workflows/tyk-brave-api-importer.yaml to automatically include new services to tyk
Brave (dev) Environment
Stage Environment
Artifact Registry
maven repository where internal libraries/artifacts are published
URL: https://console.cloud.google.com/artifacts/maven/safi-repos/asia-southeast1/safi-maven
Further Reading
Source code organization: Coding rules
REST API versioning: REST API versioning and client lib generation
Brave Environment: https://safibank.atlassian.net/wiki/spaces/ITArch/pages/208273431/Brave+Dev+environment