SaFi Bank Space : Cloud Functions - Overview

Naming Conventions
Business Functions

This page will be used for documenting cloud functions and changes. We are using Gen 2 Cloud Functions - Cloud Functions 2nd gen is GA

Naming Conventions

The naming convention for Cloud Functions in risk will follow the naming convention of businessFunction-subcategory-task. The strings in the name will be “-” dash separated.

For example, slacker-transaction-intrabank-sync indicates a function for evaluating intrabank transactions, and this is used for the main business functionality of synchronously interacting with Slacker.

businessFunction - Refers to the main functionality that the cloud function is used for and will be grouped under
- There can only be one Business Function
- E.g. stream-xxxxx
subcategory - Within the supposed business function, there can be various subtasks that each function performs, related to the categories.
- Subcategory details are “-” underscore separated
- Can accept up to 2 subcategories, should be hierarchical in nature, where the second category is intuitively a subset of the first
- E.g. stream-logging-telestatus-xxxxxx
task - Will contain more specific descriptions of the subtask or job handled by each function. Can be anything as long as it is documented.
- Preferably indicate tasks with a verb whenever possible - E.g. write, score, ingest, etc.
- E.g. stream-logging-telestatus-fswrite

Github: Cloud Functions Reference

Naming rules for Cloud Functions - Name must start with a letter followed by up to 62 letters, numbers, hyphens or underscores and must end with a letter or a number. Name must also be lower-case.

Business Functions

SLACKER

This refers to all Cloud Functions that will be interacting with Slacker. In general, the functions here will either receive and API request from Slacker and return a decision COMMAND, or the function will listen for some signal from other areas and return an event-triggered COMMAND to slacker.

In general, there are 2 ways to monitor and trigger via Slacker

Synchronously using Slacker → Cloud Functions → Slacker command / Response
Asynchronously from Pub/Sub → Cloud Functions → Slacker command

Workflow:

Domain

Cloud Function Name

Description

Input

Output

In Brave

Ready for Integration Tests

In Stage

Cards

slacker-cards-check-sync

Used for handling REST API calls from Slacker in a synchronous workflow. Returns the approval status of cards transactions and relevant Slacker commands. This is for risk and fraud checks.

JSON

JSON <HTTP Response>

Slacker Command
TransactionStatus

Transaction

slacker-transaction-check-sync

Non-AML Transaction Checks: Used for handling REST API calls from Slacker in a synchronous workflow. Returns the approval status of the transaction, with the relevant Slacker commands. This is for risk and fraud checks.

(previously slacker-transaction-intrabank-sync)

JSON

JSON <HTTP Response>

Slacker Command
TransactionStatus

Transaction

slacker-transaction-schedule-validate

Basically, runs the same checks as slacker-transaction-check-sync and returns the approval status of the transaction, with the relevant Slacker commands. This is used for risk and fraud checks for scheduled transactions.

JSON

JSON <HTTP Response>

Slacker Command
TransactionStatus

DECISION

This refers to all Cloud Functions that will be be involved in some form of decisioning but not for interacting with Slacker. These functions can, for instance, interact with other microservices or 3rd Party Workflows such as One Stop Platform.

In general, these functions will accept some API request and return some decision response based on the received message. Also, they will most likely follow a synchronous workflows

Due to the low-latency requirements of synchronous processes, the future workflow will include AlloyDB. Meaning to say, a SQL query to extract the given features will be ran against AlloyDB, which will return the results of that query. The query results are then not only returned as a response, but also written to the Online FeatureStore for further processing.

This achieves 3 things:

Low-latency with AlloyDB’s capabilities, allowing for millisecond queries
Reduce the need to continually run SQL queries in BigQuery in order to refresh the online FeatureStore with the latest queries
Only relevant feature values and entity_id that have been triggered by decisioning workflows, will be written to the FeatureStore. This prevents a flood of continuous feature ingestions from step (2) of which most will not be used for decisioning anyway

Domain	Cloud Function Name	Description	Input	Output
Customer	`decision-dfp-history-check`	Used to check the values from `devicelabel_history_features`, which is part of the `device_fingerprint_fs`. This is largely used for anti-fraud rules. Execution logic: Pull `devicelabel_features` entity_type from `device_fingerprint_fs` online featurestore Retrieve time series calculation from `devicelabel_history_features` entity_type based on last ingestion Returns the time series features as response	JSON Message with customerId as referenceId { "referenceId":"XXXXXXXX" }	JSON <HTTP Response>
Customer	`decision-osp-devicefingerprint`	Returns the device fingerprint features of given customerId to OSP for risk decisioning. Also returns a device fingerprint score as well. Execution Logic: Pull `devicelabel_features` entity_type from `device_fingerprint_fs` online featurestore Calculate the device fingerprint score from the `devicelabel_features` and return the output	JSON Message with customerId as referenceId { "referenceId":"XXXXXXXX" }	JSON <HTTP Response>
Loans	`decision-osp-loans-approve`	Evaluates customer’s risk and returns decision whether the loan application should be approved or declined	JSON Message with customerId as referenceId { "referenceId":"XXXXXXXX" }	JSON <HTTP Response>
Loans	`decision-product-score`	This is used for evaluating a customer’s recommended and maximum loan amount, as part of the loans product table recommendation. Returns a response with the necessary parameters that can be translated downstream to retrieve a product from the recommendation table.		JSON <HTTP Response>
Customer	`decision-fraudlist-addresscheck` `decision-fraudlist-blackcheck` `decision-fraudlist-whitecheck`	Used for checking the presence or absence of a value in a specified list from a HTTP request https://safibank.atlassian.net/wiki/spaces/ITArch/pages/192806935	JSON Message with list of parameters to be queried { "dataset":"fraud_list_management", "table":"blacklist_others", "values":[] }	JSON <HTTP Response>

STREAM

This refers to all Cloud Functions related to ingesting from data streams (Kafka or Pub/Sub) and perform some task on it. This can also cover ingesting data from continuous async/sync REST API calls.

The functions here in general will operate asynchronously and are not expected to return a response as part of the data ingestion process.

Example of Stream Setup for Writing to Online FeatureStore or BigQuery

One current stream is that from Kafka Topics - Kafka Topic --> Pub/Sub --> Cloud Function
Another “stream” would be that from the Logging functionality of One-Stop-Platform - OSP --> Cloud Function
A successful ingestion will display a message similar to the one below, with status 200. This message can be viewed in the logs of the given Cloud Function

SUCCESS - TEST12-edf261c1-53f4-43fc-b88b-1573e0c82691 - Written to device_fingerprint_fs: devicelabel_features
SUCCESS - TEST12-edf261c1-53f4-43fc-b88b-1573e0c82691 - Written to device_fingerprint_fs: risklabel_features
RESPONSE LIST: [(200, 'devicelabel_features'), (200, 'risklabel_features')]

Kafka Topics → Pub/Sub Topic → Cloud Function → FeatureStore/BigQuery

Domain	Cloud Function Name	Description	Input	Output
Customer	`stream-devicefingerprint-fswrite`	Parses the Device Fingerprint response from the given Pub/Sub topic and writes the output to the Online FeatureStore Device Fingerprint Kafka Topic → Pub/Sub → Cloud Function → Writes to Online Featurestore `device_fingerprint_fs` FS: Device Fingerprint	JSON Message from Kafka Topic	HTTP Response Status Code on whether the features have been successfully written to `device_fingerprint_fs` 200 = Success
Customer	`stream-customersnapshot-fswrite`	Parses the Customer Snapshot payload from the given Pub/Sub topic and writes the output to the Online FeatureStore Device Fingerprint Kafka Topic → Pub/Sub → Cloud Function → Writes to Online Featurestore `customer_application_form_fs` FS: Customer Application Form	JSON Message from Kafka Topic	HTTP Response Status Code on whether the features have been successfully written to `customer_application_form_fs` 200 = Success
Customer	`stream-ekyc-idcardocr-fswrite`	Parses the OCR Lite Response from the given Pub/Sub topic and writes the output to the Online FeatureStore `ekyc_fs` FS: eKYC AAI	JSON Message from Kafka Topic	HTTP Response Status Code on whether the features have been successfully written to `ekyc_fs` 200 = Success
Customer	`stream-ekyc-facialverification-fswrite`	Parses the Liveness Detection and Facial Verification Response from the given Pub/Sub topic and writes the output to the Online FeatureStore `ekyc_fs` FS: eKYC AAI	JSON Message from Kafka Topic	HTTP Response Status Code on whether the features have been successfully written to `ekyc_fs` 200 = Success
Cards	`stream-cards-transaction-fswrite`	Parses the incoming Cards message payload, from the given Pub/Sub topic and writes the output to the Online FeatureStore `cards_fs` FS: Cards Transaction	JSON Message from Kafka Topic	HTTP Response Status Code on whether the features have been successfully written to `ekyc_fs` 200 = Success

OSP Logging → REST API Request → Cloud Function → FeatureStore/BigQuery

See also - One-Stop Platform Logging for FeatureStore

Domain	Cloud Function Name	Description	Input	Output
One Stop Platform	`stream-logging-telcoscore-fswrite`	Parses the OSP Logging Response from Telco Behaviour Score into `telco_score_fs` FeatureStore and BigQuery FS: Telco Behaviour Score (FinScore)	Logging JSON Message from One-Stop-Platform	HTTP Response Status Code on whether the features have been successfully written to `telco_score_fs` 200 = Success
One Stop Platform	`stream-logging-telestatus-fswrite`	Parses the OSP Logging Response from Tele Status Service into `tele_status_check_fs` FeatureStore and BigQuery FS: Tele Status Check	Logging JSON Message from One-Stop-Platform	HTTP Response Status Code on whether the features have been successfully written to `tele_status_check_fs` 200 = Success
One Stop Platform	`stream-logging-emaildetection-fswrite`	Parses the OSP Logging Response from Email Detection (Seon) into `seon_email_fs` FeatureStore and BigQuery FS: Seon (Email Detection)	Logging JSON Message from One-Stop-Platform	HTTP Response Status Code on whether the features have been successfully written to `seon_email_fs` 200 = Success
One Stop Platform	`stream-logging-socialmedia-fswrite`	Parses the OSP Logging Response from Social Media Detection (Seon) into `seon_social_media_fs` FeatureStore and BigQuery FS: Seon (Social Media Check)	Logging JSON Message from One-Stop-Platform	HTTP Response Status Code on whether the features have been successfully written to `seon_social_media_fs` 200 = Success
One Stop Platform	`stream-logging-identitycheck-fswrite`	Parses the OSP Logging Response from Identity Check into `cibi_identity_fs` FeatureStore and BigQuery FS: CIBI Identity Check	Logging JSON Message from One-Stop-Platform	HTTP Response Status Code on whether the features have been successfully written to `cibi_identity_fs` 200 = Success

INGEST

Data ingestion related, but not related to processing from data streams. Examples include nightly batch ingestion jobs. For now, this is under the purview of the data engineering team.

Domain

Cloud Function Name

Description

Input

Output

In Brave

Ready for Integration Tests

In Stage

Customer

ingest-batch-devicehistory-fswrite

Used for nightly batch jobs to the device_fingerprint_fs FeatureStore, temporarily triggered by Cloud Scheduler (CRON) job.

Writes to the entity_type devicelabel_history_features of the above featurestore.

A HTTP Request containing the source tables and source dataset in BigQuery

HTTP Response Status Code on whether the features have been successfully written to device_fingerprint_fs

DATA

This refers to all Cloud Functions that will be be deploy as part of data science or machine learning pipelines. For instance, reading a deployed model, etc. This is currently inactive.

Domain	Cloud Function Name	Description	Input	Output	In Brave	Ready for Integration Tests	In Stage
Risk	`data-model_defaultproba-call`	Example Only

SaFi Bank Space : Cloud Functions - Overview

Naming Conventions

Business Functions

SLACKER

DECISION

STREAM

Kafka Topics → Pub/Sub Topic → Cloud Function → FeatureStore/BigQuery

OSP Logging → REST API Request → Cloud Function → FeatureStore/BigQuery

INGEST

DATA

Attachments: