Jobs
Jobs are workspaces created under a project where data processing workflows are defined.
Within a Job, operations such as reading data from data sources, transforming data, combining different datasets, and writing the results to target systems can be designed as a visual flow.
Each Job belongs to its related Project.
Jobs List
The Jobs page is used to view and manage the Jobs available under the selected project.
From this page, existing Jobs can be searched, filtered, sorted, and displayed using different view options.
To create a new Job, use the Add Job button in the upper-right corner.
List Columns
The following information is displayed in the list view:
| Column | Description |
|---|---|
| Name | Displays the name of the Job. |
| Description | Displays the description defined for the Job. |
| Create Date | Displays the date and time when the Job was created. |
| Update Date | Displays the date and time when the Job was last updated. |
| Status | Displays the current status of the Job. |
If a Job is locked for editing by another user, a Locked label is displayed next to the Job name.
Sorting
Supported columns in the list view can be sorted.
The sorting control in the column header can be used to sort records in ascending or descending order.
The following columns support sorting:
- Name
- Create Date
- Update Date
- Status
The sorting icon in a column header indicates that the column can be sorted.
Quick Search
The Filter... field in the upper-right section of the page can be used to quickly search through Jobs.
When a value is entered into the search field, the list is narrowed down to display records matching the entered value.
This feature makes it easier to quickly find a specific Job, especially in projects containing a large number of Jobs.
Advanced Filtering
The Filter button next to the quick search field can be used for more detailed filtering.
The advanced filtering screen allows filter conditions to be created based on Job fields.
Multiple conditions can be used to narrow down the list in greater detail. This is particularly useful for finding Jobs that match specific criteria in projects containing many records.
Active filters can be cleared when necessary to display all Jobs again.
View Options
The Jobs page supports two different view types:
List View
Jobs are displayed in a table.
This view is useful for comparing Job information, sorting by columns, and viewing a large number of records on the same screen.
Card View
Jobs are displayed as cards.
Card View provides a more visual and compact layout for displaying Job information.
The view buttons in the upper-right section of the page can be used to switch between List View and Card View.
Job Actions
The three-dot menu on the right side of each Job provides access to various actions for that Job.
Available actions may vary depending on the permissions assigned to the user.
The following actions can be performed from the Job actions menu:
| Action | Description |
|---|---|
| Edit | Used to edit Job information. |
| Copy Job | Creates a new Job by copying the existing Job. |
| Export | Exports the Job definition. |
| Delete | Deletes the Job. |
| View Details | Displays detailed information about the Job. |
| Logs | Displays log records associated with the Job. |
| Users | Allows users with permissions on the Job to be managed. |
| Groups | Allows groups with permissions on the Job to be managed. |
Pagination
Pagination controls are available at the bottom of the Jobs list.
The total number of records is displayed in the lower-left section.
For example:
Total 58 records
The pagination controls in the center can be used to:
- Go to the first page.
- Go to the previous page.
- Go to the next page.
- Go to the last page.
- View the current page and the total number of pages.
The Rows per page option in the lower-right section can be used to change the number of Jobs displayed on each page.
What is a Job?
A Job represents a data processing workflow in DataFloem.
Within a Job, the processing steps that take place from reading data from a source to writing the processed data to a target system can be visually designed.
Jobs can generally be used to:
- Read data from source systems
- Filter data
- Select or transform columns
- Combine different datasets
- Perform SQL- or Python-based data processing
- Write processed data to target systems
- Execute the designed data flow
- Monitor execution results and logs
These operations are created using nodes within the Job.
Job Workspace
When a Job is opened from the list, the Job workspace is displayed.
At the center of the workspace is a flow canvas, where the data flow is visually designed.
Nodes that perform different tasks can be added to this canvas. The nodes are connected to define how data moves through the processing steps.
A simple data flow can have the following structure:
Source → Transformation → Target
More complex Jobs can contain multiple sources, transformations, and targets.
For example:
Database Source → Select → Filter → Join → Target
Connections between nodes indicate the direction in which data flows through the Job.
Job Nodes
Different types of nodes can be used within a Job to perform different tasks.
Nodes can generally be grouped into three main categories.
Source Nodes
Source nodes are used to bring data into the Job.
Data can be read from databases, object storage systems, or other data sources supported by DataFloem.
Transformation Nodes
Transformation nodes are used to process data received from source nodes.
Operations such as filtering, column selection, combining datasets, and other data transformations can be performed using these nodes.
Target Nodes
Target nodes are used to write processed data to a target system.
The results produced by a data flow can be transferred to supported target systems.
Job Execution
A Job can be executed to run the defined data flow.
During execution, data processing steps are performed according to the connections and dependencies between nodes.
The execution status and related log records can be monitored through the system.
Job Locking
A Job can be locked by the system while it is being edited by another user.
Locked Jobs are identified by a Locked label displayed next to the Job name in the Jobs list.
The locking mechanism helps prevent conflicts that could occur when multiple users attempt to modify the same Job at the same time.
Users with the required permissions can view and manage locked Jobs from the system's lock management page.