Add Cloud Database Panel
Overview
The Add Cloud Database panel enables administrators and data engineers to securely integrate cloud data warehouse and database solutions (such as Google BigQuery, Snowflake, etc.) using Service Account / IAM authentication methods into the DataFloem platform.
Connections created via this interface act as high-performance endpoints within data pipelines, executing analytical queries (SQL), extracting massive datasets, or streaming transformed payloads into cloud target tables.
Access Requirements
To access this page, the user must either have Administrator privileges or the CREATE permission for the Target module.
The CREATE permission allows users to create new groups. Users without administrator privileges or the required permission cannot access this page.
If you do not have the required permission, ask your administrator to grant it from one of the authorization pages below.
Authorization
Users without the required authorization will receive the following error:
Access Denied
You don't have access to this resource.
Prerequisites
Before establishing a Cloud Database connection, ensure that:
- You have provisioned a dedicated Service Account on your cloud provider console (e.g., GCP Console) with appropriate database/warehouse access privileges.
-
You have retrieved the required credential parameters (
Project Id,Client Id,Service Account Email,Private Id, andPrivate Key). - Necessary IAM roles (e.g., BigQuery Data Editor/Viewer, Job User) are explicitly assigned to the service account for target datasets.
Form Elements
The fields that need to be configured within the panel are as follows:
| Field | Required? | Description |
|---|---|---|
| Connection Name | Yes | A unique alias identifying this cloud database connection across the platform. |
| Database Type | Yes (Dropdown) | Selects the target cloud service provider engine (e.g., BigQuery, Snowflake). |
| Database Version | Yes (Dropdown) | Selects the precise version mapping to ensure client driver compatibility. |
| Is Active | Yes (Toggle) |
Defines whether the connection is active immediately upon
deployment. Enabled (on) by default.
|
| Project Id | Yes | The master project identifier hosted on the cloud platform (e.g., GCP Project ID). |
| Service Account Email | Yes | The authorized service account email address used for IAM authentication. |
| Service Account Private Id | Yes | The unique Key ID corresponding to the service account private key. |
| Service Account Private Key | Yes | The RSA/PEM formatted private key string (Visibility can be toggled using the eye icon). |
| Client Id | Yes | The unique OAuth Client ID string associated with the service account. |
| Test Connection | Button | Simulates an immediate authentication handshake with the cloud IAM service. |
| Optional Parameters (Name / Value) | No |
Dynamic entries used to pass fine-grained parameters (e.g.,
location,
dataset,
quotaProjectId).
|
| Is Secret (Optional Parameters) | Yes (Toggle) | Encrypts and masks the specific parameter value within internal stores. |
Test Connection Behavior
-
Clicking the Test Connection button initiates a
real-time verification request with the cloud IAM authentication service
using the provided
Project Id,Service Account Email, andPrivate Key. - If authentication and project scope authorization succeed, a green success banner is displayed confirming connectivity.
- If authentication fails due to invalid keys, revoked service accounts, or missing permissions, a red error prompt appears. Validating the connection prior to saving is highly recommended.
Cloud Database Connection Deployment Process
- Click the Add Cloud Database action button on the workspace dashboard to trigger the side panel.
- Enter a unique Connection Name, then designate the Database Type and Database Version.
- Fill in the IAM credential parameters including Project Id, Service Account Email, Service Account Private Id, Service Account Private Key, and Client Id.
- If required, add custom key-value arguments under Optional Parameters and toggle Is Secret appropriately.
- Click Test Connection to verify access with the cloud database provider.
- Once validated, click Save to commit the asset to the platform, or click Cancel to abort the operation.
Validation Rules
- Connection Name, Project Id, Service Account Email, Service Account Private Id, Service Account Private Key, and Client Id fields cannot be left empty.
- Service Account Email must strictly adhere to valid email address syntax.
- Service Account Private Key must contain a valid PEM/RSA key structure.