SQL Queries

Overview

The SQL Queries module in DataFloem is an interactive, web-based SQL editor designed for ad-hoc data analysis, query execution, and data exploration across registered catalogs, databases, schemas (namespaces), and tables.

Through this module, users can:

• Execute ad-hoc SQL queries against registered data catalogs and data sources.

• Save frequently used queries (Saved Queries) and work with multiple tabs concurrently.

• Select the appropriate Execution Engine / Cluster for processing queries.

• Inspect execution history logs (Recent) and review dataset outputs or detailed error messages (Query Results).

Access and Interface Components

The SQL Queries interface consists of 4 main sections:

1. Left Navigation Panel (Saved Queries & Catalogs)

2. Top Tabs & Toolbar

3. SQL Code Editor

4. Bottom Panel (Recent & Query Results)

1. Left Navigation Panel

Saved Queries

Lists all previously created and saved SQL queries.

• Click any query from the list to open it in a new tab within the editor.

• Use the action menu (...) next to any saved query to rename or delete it.

Catalogs (Catalog & Data Hierarchy Tree)

Displays all system-defined data sources, schemas/namespaces, and tables in a hierarchical tree view.

• Search catalogs...: Allows quick filtering across catalogs, schemas, or table names.

• Hierarchy Structure: Catalog (e.g., Catalog1) ➔ Schema/Namespace (e.g. Namespace1) ➔ Table (e.g. Table1).

• Open in Editor: Clicking the row action (...) on a table provides the "Open in editor" option, which automatically generates a SELECT * FROM catalog.schema.table query in a new editor tab.

2. Top Tabs & Toolbar

Located at the top right of the workspace, the toolbar manages query tabs and execution environments:

Keyboard Shortcuts

The following keyboard shortcuts can be used to manage query saving directly from the editor:

3. SQL Code Editor

The primary workspace featuring a dark-themed code editor with SQL syntax highlighting, auto-completion, and multi-line editing support.

Queries require Fully Qualified Names (FQN) referencing the catalog, schema, and table:

SELECT * FROM dfl.new_namespace.z_postgre_struct LIMIT 100;

4. Bottom Panel (Recent & Query Results)

Located below the code editor, this panel displays execution outputs and history in two primary tabs:

Recent (Execution History)

Lists recently executed queries on the left:

• Displays a brief preview of the query, execution timestamp, and status.

• Status Indicators:

  • Failed (Red): Indicates that the query encountered an error during execution.

  • Success (Green): Indicates successful query completion.

Query Results

Displays execution output for the active query on the right:

• Successful Execution: Returned data is displayed in an interactive data grid that can be filtered or exported.

• Error Handling: If a query or catalog connection error occurs (e.g., HTTP 406, Catalog request error, Syntax error), gRPC/HTTP status codes and detailed payload diagnostic messages are reported here.

Basic Usage Steps

1. Start a New Query: Click the + icon in the toolbar to open a new tab, or select Open in editor from a table row in the Catalogs tree.

2. Select Engine: Choose an active computing cluster/engine (Engine/Cluster) from the dropdown menu adjacent to the Run button.

3. Write and Run: Type your SQL statements and click the green Run (▶) button.

4. Inspect Results: Upon completion, review the data grid or error diagnostic details under the Query Results tab in the bottom panel.

5. Save Query: Use Cmd + S (Ctrl + S) or click the Save (disk) icon to save your query for future reuse. Use Cmd + Shift + S (Ctrl + Shift + S) to save all open tabs.

Best Practices & Tips

• Use Cmd + S / Ctrl + S to quickly save active query tabs, and Cmd + Shift + S / Ctrl + Shift + S to save all open tabs without leaving the editor.

• Use LIMIT clauses when querying large tables to optimize performance and resource utilization.

• An engine status of Idle indicates readiness to receive and process query workloads.

• If you encounter catalog connection errors (Status(StatusCode="Internal")...), verify the operational status of the underlying Source/Target connection or catalog service.