BigQuery is a fully managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence. BigQuery's serverless architecture lets you use SQL queries to answer your organization's biggest questions with zero infrastructure management. BigQuery's scalable, distributed analysis engine lets you query terabytes in seconds and petabytes in minutes.
Provisioning in Terraform
Yaml definitions this requires information on how many dataset and tables to provision the bigquery resources.
location ./devops/terraform/tf-environments/_files/datateam_google_bigquery.yaml
Eg.
bigquery: - dataset_id: device_fingerprint_data friendly_name: device finger print description: device finger print default_table_expiration_ms: 3600000 table: - table_id: device_labels dataset_id: 0 time_partitioning_type: DAY schema: deviceLabel.json - table_id: risk_label dataset_id: 0 time_partitioning_type: DAY schema: riskLabel.json
Adding addtional files
schema should be put in /devops/terraform/tf-environments/_files/data/bigquery