SaFi Bank Space : KMS encryption for GCP resources

Cloud Key Management Service allows you to create, import, and manage cryptographic keys and perform cryptographic operations in a single centralized cloud service.

Customer-managed encryption keys (CMEK)

If you need more control over the keys used to encrypt data at rest within a Google Cloud project, several Google Cloud services offer the ability to protect data related to those services using encryption keys managed by the customer within Cloud KMS. These encryption keys are called customer-managed encryption keys (CMEK). When you protect data in Google Cloud services with CMEK, the CMEK key is within your control.

Using CMEK gives you control over more aspects of the lifecycle and management of your keys, such as (but not limited to) the following abilities:

  • You can control Google's ability to decrypt data at rest by disabling the keys used to protect that data.

  • You can protect your data using a key that meets specific locality or residency requirements.

  • You can automatically or manually rotate the keys used to protect your data.

  • You can protect your data using a Cloud HSM key or a Cloud External Key Manager key, or an existing key that you import into Cloud KMS.

CMEK integrations

When a service supports CMEK, it is said to have a CMEK integration. Some services, such as GKE, have multiple CMEK integrations for protecting different types of data related to the service.

For the exact steps to enable CMEK, see the documentation for the relevant Google Cloud service. You can expect to follow steps similar to these:

  1. You create or import a Cloud KMS key, selecting a location as geographically near as possible to the location of the service's resources. The service and the key can be in the same project or different projects. This is the CMEK key.

  2. You grant the CryptoKey Encrypter/Decrypter IAM role (roles/cloudkms.cryptoKeyEncrypterDecrypter) on the CMEK key to the service account for the service.

  3. You configure the service to use the CMEK key to protect its data. For example, you can configure a GKE cluster to use CMEK to protect data at rest on the boot disks of the nodes.

As long as the service account has this role, the service can encrypt and decrypt its data. If you revoke the role, or if you disable or destroy the CMEK key, that data can't be accessed.

List of Google Cloud Services That Offer Integrations with Cloud KMS

These services generally fall under one of the following categories:

  • A Customer-managed encryption key (CMEK) integration allows you to encrypt that service's data at rest using a Cloud KMS key that you own and manage. Data protected with a CMEK key cannot be decrypted without access to that key.

  • A CMEK-compliant service either does not store data, or only stores data for a short period of time, such as during batch processing. Such data is encrypted using an ephemeral key that only exists in memory and is never written to disk. When the data is no longer needed, the ephemeral key is flushed from memory, and the data can't ever be accessed again. The output of a CMEK-compliant service might be stored in a service that is integrated with CMEK, such as Cloud Storage.

  • Your applications can use Cloud KMS in other ways. For example, you can directly encrypt application data before transmitting or storing it.

List of services with CMEK integration

The following table lists services that integrate with Cloud KMS for software and hardware (HSM) keys.

Service

Protected with CMEK

Topic

AI Platform Training

Data on VM disks

Using customer-managed encryption keys

Artifact Registry

Data in repositories

Enabling customer-managed encryption keys

BigQuery

Data in BigQuery

Protecting data with Cloud KMS keys

Cloud Bigtable

Data at rest

Customer-managed encryption keys (CMEK)

Cloud Composer

Environment data

Using customer-managed encryption keys

Cloud Data Fusion

Environment data

Using customer-managed encryption keys

Cloud Functions

Data in Cloud Functions

Using customer-managed encryption keys

Cloud Logging

Data in the Log Router

Manage the keys that protect Log Router data

Cloud Logging

Data in Logging storage

Manage the keys that protect Logging storage data

Cloud Run

Container image

Using customer-managed encryption keys with Cloud Run

Cloud Spanner

Data at rest

Customer-managed encryption keys (CMEK)

Cloud SQL

Data written to databases

Using customer-managed encryption keys

Cloud Storage

Data in storage buckets

Using customer-managed encryption keys

Compute Engine

Data on VM disks

Protecting resources with Cloud KMS keys

Database Migration Service

Data written to databases

Using customer-managed encryption keys (CMEK)

Dataflow

Pipeline state data

Using customer-managed encryption keys

Dataproc

Data on VM disks

Customer-managed encryption keys

Dataproc Metastore

Data at rest

Using customer-managed encryption keys

Datastream

Data in transit

Using customer-managed encryption keys (CMEK)

Dialogflow CX

All data-at-rest

Customer-managed encryption keys (CMEK)

Document AI

All data-at-rest and data-in-use

Customer-managed encryption keys (CMEK)

Eventarc

All data-at-rest

Use customer-managed encryption keys (CMEK)

Filestore

All data-at-rest

Encrypt data with customer-managed encryption keys

Google Distributed Cloud Edge

Data on Edge nodes

Local storage security

Google Kubernetes Engine

Data on VM disks

Using customer-managed encryption keys (CMEK)

Google Kubernetes Engine

Application-layer Secrets

Application-layer Secrets encryption

Pub/Sub

Data associated with topics

Configuring message encryption

Secret Manager

Secret payloads

Enabling Customer-Managed Encryption Keys (CMEK)

Speaker ID (Restricted GA)

Data at rest

Using customer-managed encryption keys

Vertex AI

Data associated with resources

Using customer-managed encryption keys

Vertex AI Workbench managed notebooks

User data at rest

Customer-managed encryption keys

Vertex AI Workbench user-managed notebooks

Data on VM disks

Customer-managed encryption keys

CMEK-compliant services

Cloud KMS Locations

Within a project, Cloud Key Management Service resources can be created in one of many locations. These represent the geographical regions where a Cloud KMS resource is stored and can be accessed. A key's location impacts the performance of applications using the key. Some resources, such as Cloud HSM keys, are not available in every location.

Key material for Cloud KMS and Cloud HSM keys is confined to the selected region while at rest and in use.

Types of locations for Cloud KMS

You can create Cloud KMS, Cloud HSM, and Cloud EKM resources in different types of locations in Google Cloud, depending on your availability requirements. Locations are added regularly. For specific information about each location, see Locations.

You can learn more about choosing the best type of location.

The following location types are available to Cloud KMS:

  • Regional locations: A regional location's data centers exist in a specific geographical place. For example, a resource created in the us-central1 region is located in the central United States.

  • Multi-regional locations: A multi-regional location's data centers are spread across a large geographical area. For example, a resource created in the europe multi-region persists in multiple data centers within the European Union. You can't choose which data centers within the multi-region will contain your data.

  • The global location: The global location is a special multi-region. Its datacenters are spread throughout the world. You can't choose which data centers within the global multi-region will contain your data.

Table with locations available for use in Cloud KMS for different parts of the world

Location name

Location type

Location description

Cloud HSM available

Cloud EKM available

ASIA

asia

Multi-region

Multiple regions in Asia

Yes

Via internet only

asia-east1

Region

Taiwan

Yes

Yes

asia-east2

Region

Hong Kong

Yes

Yes

asia-northeast1

Region

Tokyo

Yes

Yes

asia-northeast2

Region

Osaka

Yes

Yes

asia-northeast3

Region

Seoul

Yes

Yes

asia-south1

Region

Mumbai

Yes

Yes

asia-south2

Region

Delhi

Yes

Yes

asia-southeast1

Region

Singapore

Yes

Yes

asia-southeast2

Region

Jakarta

Yes

Yes

asia1

Multi-region

Tokyo, Osaka, and Seoul

Yes

Via internet only

australia-southeast1

Region

Sydney

Yes

Yes

australia-southeast2

Region

Melbourne

Yes

Yes

AMERICAS

nam10

Multi-region

Iowa, Salt Lake City, and Oklahoma

Yes

Via internet only

nam11

Multi-region

Iowa, South Carolina, and Oklahoma

Yes

Via internet only

nam12

Multi-region

Iowa, Northern Virginia, Oklahoma, and Oregon

Yes

Via internet only

nam3

Multi-region

Northern Virginia and South Carolina

Yes

Via internet only

nam4

Multi-region

Iowa, South Carolina, and Oklahoma

Yes

Via internet only

nam6

Multi-region

Iowa and South Carolina

Yes

Via internet only

nam7

Multi-region

Iowa, Northern Virginia, and Oklahoma

Yes

Via internet only

nam8

Multi-region

Los Angeles, Oregon, and Salt Lake City

Yes

Via internet only

nam9

Multi-region

Northern Virginia and Iowa

Yes

Via internet only

northamerica-northeast1

Region

Montréal

Yes

Yes

northamerica-northeast2

Region

Toronto

Yes

Yes

southamerica-east1

Region

São Paulo

Yes

Yes

southamerica-west1

Region

Santiago

Yes

Yes

us

Multi-region

Multiple regions in the United States

Yes

Via internet only

us-central1

Region

Iowa

Yes

Yes

us-east1

Region

South Carolina

Yes

Yes

us-east4

Region

Northern Virginia

Yes

Yes

us-east5

Region

Columbus

Yes

Yes

us-south1

Region

Dallas

Yes

Yes

us-west1

Region

Oregon

Yes

Yes

us-west2

Region

Los Angeles

Yes

Yes

us-west3

Region

Salt Lake City

Yes

Yes

us-west4

Region

Las Vegas

Yes

Yes

EUROPE

eur3

Multi-region

Belgium and Netherlands

Yes

Via internet only

eur4

Multi-region

Finland, Netherlands, and Belgium

Yes

Via internet only

eur5

Multi-region

London, Netherlands, and Belgium

Yes

Via internet only

eur6

Multi-region

Netherlands, Frankfurt, and Zürich

Yes

Via internet only

europe

Multi-region

Multiple regions in the European Union1

Yes

Via internet only

europe-central2

Region

Warsaw

Yes

Yes

europe-north1

Region

Finland

Yes

Yes

europe-southwest1

Region

Madrid

No

Yes

europe-west1

Region

Belgium

Yes

Yes

europe-west2

Region

London

Yes

Yes

europe-west3

Region

Frankfurt

Yes

Yes

europe-west4

Region

Netherlands

Yes

Yes

europe-west6

Region

Zürich

Yes

Yes

europe-west8

Region

Milan

Yes

Yes

europe-west9

Region

Paris

No

Yes

GLOBAL

global

global

Yes

No

nam-eur-asia1

Multi-region

North America, Europe, and Asia
(Iowa, Oklahoma, Belgium, and Taiwan)

No

No

Adding KMS encryption via Terraform

  • VM Instance (google_compute_instance):
    Encryption documentation
    ! Enabling encryption for existing instance causes recreation

terraform example
data "google_project" "vm_example_project" {
  provider = google
}

## KMS
## -----------------------------------------------
## Create a KMS key ring to store the key
resource "google_kms_key_ring" "vm_example_key_ring" {
  provider = google
  ## Should match VM's location
  location = var.location_zone
  name     = "vm-key-ring"
}
## Create a crypto key in the generated above key ring
resource "google_kms_crypto_key" "vm_example_crypto_key" {
  provider = google
  name            = "vm-crypto-key"
  key_ring        = google_kms_key_ring.vm_example_key_ring.id
  rotation_period = "100000s"
  lifecycle {
    prevent_destroy = true
  }
}
## Giving permissions to Service account to use the key
resource "google_kms_crypto_key_iam_binding" "crypto_key_iam_binding" {
  provider = google
  crypto_key_id = google_kms_crypto_key.vm_example_crypto_key.id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  members = [
    format("serviceAccount:service-%s@compute-system.iam.gserviceaccount.com", data.google_project.vm_example_project.number),
  ]
}
## -----------------------------------------------

resource "google_compute_instance" "vm_example" {
  provider     = google
  name         = "vm-example"
  machine_type = "e2-small"
  zone         = var.location_zone
  tags = ["vm-example"]
  boot_disk {
    initialize_params {
      image = "cos-cloud/cos-stable-89-16108-403-26"
    }

    ## Enables encryption data on VM disks
    ## `self_link` is removed: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/version_4_upgrade#self_link-is-now-removed
    ## => use `id` 
    kms_key_self_link = google_kms_crypto_key.vm_example_crypto_key.id
  }

  <...>
}
terraform example
data "google_storage_project_service_account" "bucket_account" {
  provider = google
}

## KMS
## -----------------------------------------------
## KMS key ring
resource "google_kms_key_ring" "bucket_key_ring" {
  provider = google
  name     = "bucket-key-ring"
  ## Should match bucket's location
  location = var.region
}
## Key in the key ring for a bucket
resource "google_kms_crypto_key" "bucket_crypto_key" {
  provider        = google
  name            = "bucket-crypto-key"
  key_ring        = google_kms_key_ring.bucket_key_ring.id
  rotation_period = "100000s"
  lifecycle {
    prevent_destroy = true
  }
}
## Allow google_storage_project_service_account access to the crypto key for bucket encryption
resource "google_kms_crypto_key_iam_binding" "bucket_key_iam_binding" {
  provider      = google
  crypto_key_id = google_kms_crypto_key.bucket_crypto_key.id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  members       = [
     "serviceAccount:${data.google_storage_project_service_account.bucket_account.email_address}",
  ]
}
## -----------------------------------------------

resource "google_storage_bucket" "storage_bucket" {
  provider = google
  name     = "storage-bucket-example"
  ## Check supported locations: https://cloud.google.com/storage/docs/locations
  location = var.google_region
  project  = var.google_project_id
  versioning {
    enabled = true
  }
  encryption {
    default_kms_key_name = google_kms_crypto_key.bucket_crypto_key.id
  }
  depends_on = [
    google_kms_crypto_key.bucket_crypto_key,
    google_kms_crypto_key_iam_binding.bucket_key_iam_binding
  ]
}
terraform example
data "google_project" "sm_example_project" {}

## KMS:
## for secret-manager - https://cloud.google.com/secret-manager/docs/cmek#automatic-replication
## -----------------------------------------
## Create a KMS key ring to store the key
resource "google_kms_key_ring" "secretmanager_example_key_ring" {
  provider = google
  # The provided key must be in the same region as the secret.
  location = var.google_region
  name     = "secretmanager-example-key-ring"
}
## Create a crypto key in the generated above key ring
resource "google_kms_crypto_key" "secretmanager_example_crypto_key" {
  provider = google
  name            = "secretmanager_example-crypto-key"
  key_ring        = google_kms_key_ring.secretmanager_example_key_ring.id
  rotation_period = "100000s"
  lifecycle {
    prevent_destroy = true
  }
}
## Custom SA for secretmanager api:
resource "google_project_service_identity" "secretmanager_example_service_account" {
  provider  = google-beta
  project   = data.google_project.secretmanager_example_project.name
  service   = "secretmanager.googleapis.com"
}
## Giving permissions to `secretmanager_example_service_account` to use crypto key
resource "google_kms_crypto_key_iam_binding" "secretmanager_example_crypto_key_iam_binding" {
  crypto_key_id = google_kms_crypto_key.secretmanager_example_crypto_key.id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"

  members = [
    format("serviceAccount:%s", google_project_service_identity.secretmanager_example_service_account.email),
  ]
}
## -----------------------------------------

## Secret definition
resource "google_secret_manager_secret" "secret" {
  secret_id = "secretmanager_example"
  labels = {
    env = var.env_name
  }
  replication {
    user_managed {
      replicas {
        location = var.google_region
        ## Enables encryption of Secret payloads
        customer_managed_encryption {
            kms_key_name = google_kms_crypto_key.secretmanager_example_crypto_key.id
        }
      }
    }     
  }
  depends_on = [
    google_kms_crypto_key.secretmanager_example_crypto_key,
    google_kms_crypto_key_iam_binding.secretmanager_example_crypto_key_iam_binding
  ]
}
  • Redis (google_redis_instance)
    ! Enabling encryption for existing instance causes recreation

terraform example
data "google_project" "redis_example_project" {
    provider = google
}

## KMS
## -----------------------------------------------
## Create a KMS key ring to store the key
resource "google_kms_key_ring" "redis_example_key_ring" {
  provider = google
  name     = "redis-example-key-ring"
  ## Should match location of redis instance
  location = var.google_region
}
## Create a crypto key in the generated above key ring
resource "google_kms_crypto_key" "redis_example_crypto_key" {  
  provider = google
  name            = "redis-example-crypto-key"
  key_ring        = google_kms_key_ring.redis_example_key_ring.id
  rotation_period = "100000s"
  lifecycle {
    prevent_destroy = true
  }
}
## Allow cloud-redis SA access to the crypto-key
resource "google_kms_crypto_key_iam_binding" "redis_example_key_iam_binding" {
  provider = google
  crypto_key_id = google_kms_crypto_key.redis_example_crypto_key.id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  members       = [
     format("serviceAccount:service-%s@cloud-redis.iam.gserviceaccount.com", data.google_project.redis_example_project.number),
  ]
}
## -----------------------------------------------

resource "google_redis_instance" "redis_example" {
  provider       = google-beta.tyk
  name           = "redis-example"
  region         = var.google_region
  tier           = "STANDARD_HA"
  memory_size_gb = 5
  authorized_network = google_compute_network.shared_vpc.id
  connect_mode       = "PRIVATE_SERVICE_ACCESS"
  redis_version      = "REDIS_6_X"
  display_name       = "Redis Example"
  replica_count      = 2
  read_replicas_mode = "READ_REPLICAS_ENABLED"
  auth_enabled   = true
  ## Enables encryption of Data on VM disk
  customer_managed_key = google_kms_crypto_key.redis_example_crypto_key.id
  depends_on     = [
    google_service_networking_connection.private_vpc_connection,
    google_kms_crypto_key.redis_example_crypto_key,
    google_kms_crypto_key_iam_binding.redis_example_key_iam_binding
  ]
}
  • PostgreSQL DB instance (google_sql_database_instance)
    Encryption documentation
    ! Enabling encryption for existing instance causes recreation

terraform example
## KMS: psql instance CMEK - https://cloud.google.com/sql/docs/postgres/configure-cmek
## https://github.com/terraform-google-modules/terraform-docs-samples/blob/main/sql_instance_cmek/main.tf
## -----------------------------------------------
## Create a KMS key ring to store the key
resource "google_kms_key_ring" "psql_key_ring" {
  provider = google
  ## The provided key must be in the same region as the SQL instance.
  location = var.google_region
  name     =  format("psql-%s-%s-apps-key-ring", local.prefix, var.env_name)
}
## Create a crypto key in the generated above key ring
resource "google_kms_crypto_key" "psql_crypto_key" {
  provider        = google
  name            = "psql-crypto-key"
  key_ring        = google_kms_key_ring.psql_key_ring.id
  rotation_period = "100000s"
  lifecycle {
    prevent_destroy = false
  }
}
## Custom SA for psql api:
## https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance#encryption_key_name
resource "google_project_service_identity" "gcp_sa_cloud_sql" {
  provider  = google-beta
  project   = var.google_project_id_apps
  service   = "sqladmin.googleapis.com"
}
## Giving permissions to Service account to use the key
resource "google_kms_crypto_key_iam_binding" "psql_crypto_key_iam_binding" {
  provider      = google-beta
  crypto_key_id = google_kms_crypto_key.psql_crypto_key.id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  members = [
    format("serviceAccount:%s", google_project_service_identity.gcp_sa_cloud_sql.email),
  ]
}
## -----------------------------------------------

resource "random_pet" "psql" {}

resource "google_sql_database_instance" "psql" {
  provider            = google-beta
  region              = var.google_region
  name                = "postgresql-${random_pet.psql.id}"
  database_version    = "POSTGRES_13"

  ## Enables encryption of Data written to databases
  encryption_key_name = google_kms_crypto_key.psql_crypto_key.id

  settings {
    tier              = local.sql_tier[var.env_name]
    availability_type = "REGIONAL"

    backup_configuration {
      enabled                         = true
      point_in_time_recovery_enabled  = true
      transaction_log_retention_days  = 7

      backup_retention_settings {
        retained_backups = 30
        retention_unit = "COUNT"
      }
    }

    <...>
  }

  depends_on = [
    google_service_networking_connection.private_vpc_connection,
    google_kms_crypto_key.psql_crypto_key,
    google_kms_crypto_key_iam_binding.psql_crypto_key_iam_binding
  ]
}
terraform example
data "google_project" "gke_project" {
  provider = google
}

## KMS
## + Encrypt Data at rest on node boot disks
## + Encrypt Application-layer secrets
## -----------------------
## Create a KMS key ring to store the key
resource "google_kms_key_ring" "gke_key_ring" {
  provider = google
  ## The key ring must have the same location as the GKE cluster
  location = var.google_region
  name     =  "gke-key-ring"
}

## Create a crypto key in the generated above key ring
resource "google_kms_crypto_key" "gke_crypto_key" {
  provider = google
  name            = "gke-crypto-key"
  key_ring        = google_kms_key_ring.gke_key_ring.id
  rotation_period = "100000s"

  ## GKE VM Disks can be encrypted only by a key that uses symmetric encryption: 
  ## https://cloud.google.com/kubernetes-engine/docs/how-to/using-cmek#create-key
  version_template {
    ## https://cloud.google.com/kms/docs/reference/rest/v1/CryptoKeyVersionAlgorithm
    algorithm         = "GOOGLE_SYMMETRIC_ENCRYPTION"
	## Must be SOFTWARE or HSM
    ## HSM keys enable an additional hardware layer of security but come with a price 10 times higher than Software keys
    protection_level  = "SOFTWARE"
  }
  lifecycle {
    prevent_destroy = true
  }
}

## Grant the compute engine service account permissions to use the key. 
## This is required for nodes' boot disks to access and use the CMEK.
## Nodes in a cluster use the compute engine service account, not GKE service account:
## https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster#permissions
resource "google_project_iam_member" "compute-system" {
  provider = google
  project  = data.google_project.cicd_gke_project.name
  role     = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  member   = format("serviceAccount:service-%s@compute-system.iam.gserviceaccount.com", data.google_project.gke_project.number)
}

## Grant the GKE service account permissions to use the key. 
## This is required for the cluster to access and use the CMEK
resource "google_project_iam_member" "container-engine" {
  provider = google
  project  = data.google_project.cicd_gke_project.name
  role     = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
  member   = format("serviceAccount:service-%s@container-engine-robot.iam.gserviceaccount.com", data.google_project.gke_project.number)
}
## -----------------------

resource "google_container_cluster" "app_cluster" {
  provider = google-beta
  name     = "${local.prefix}-${var.env_name}-apps"
  ## Select location according to cluster type (zonal/regional):
  ## https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#availability
  location = var.google_region

  <...>

  ## This enables application layer encryption on the gke cluster and points to
  ## the key used to encrypt secrets
  database_encryption {
    state     = "ENCRYPTED"
    key_name  = google_kms_crypto_key.cicd_gke_crypto_key.id
  }

  depends_on = [
    google_kms_crypto_key.cicd_gke_crypto_key,
    google_project_iam_member.compute-system,
    google_project_iam_member.container-engine
  ]
}

resource "google_container_node_pool" "node_pool" {
  name_prefix = "pool-2cpu8gb-"
  location    = var.google_region
  cluster     = google_container_cluster.app_cluster.name
  node_count  = 1

  node_config {
    preemptible  = true
    machine_type = "n2-standard-2"
    image_type   = "COS_CONTAINERD"
    workload_metadata_config {
      mode = "GKE_METADATA"
    }
    ## Enables encryption of data at rest on nodes boot-disks
    boot_disk_kms_key = google_kms_crypto_key.gke_crypto_key.id

    <...>
  }

  <...>
}

Important thing about Key Ring and Crypto Key

To prevent resource name collisions, key ring and key resources CANNOT be deleted. Key versions also cannot be deleted, but key version material can be destroyed so that the resources can no longer be used. For more information, see Lifetime of objects. Billing is based on the number of active key versions; if you destroy all active key version material, there is no charge for the key rings, keys, and key versions which remain.