7.3 Model

The Model is the combination of Target Entities, Relationships, Data Sources, and Data Import Configuration.

Model Tabs

The search functionality enables searching through all three tabs. Users can drag the table to view more fields as a list, as shown in the image below.

Extend Table View

The right side of the table displays all items of each component (Target Entity, Relationships, Data Sources, and Data Import Configurations) as nodes and edges.

Canvas Representation

• 🟪 Purple node: Data Sources—either containers or workers
• 🔵 Blue node: Target Entities
• 🟩 Green node: Relationships
• Between the Data Source and a Target Entity, you can find the Data Import Configuration

Target Entities

In this section, we will focus more on the functionality available while creating or editing a Target Entity.

Creation of a Target Entity

A new Target Entity can be created by clicking or dragging in the button on the canvas, as shown in the image below.

A new contextual menu for creating a Target Entity appears as shown below:

Mandatory Fields for Creating a Target Entity:

• Title
• At least one property
• One Business Key

Actions: All major actions associated with a Target Entity are listed here:

• Bulk Add Properties: Add multiple properties at the same time
• Build Business Key: Build a custom business key
• Preview Data: View the data present in the staging layer of the Data Context Hub
• Duplicate Target Entity: (Feature coming soon)
• Delete Target Entity: Delete the current Target Entity (not available during creation)

Import Configuration Listing

After creation, Data Import Configurations will be listed as shown in the image below:

Relationships

Creation of a Relationship

A new Relationship can be created by clicking or dragging in the button on the canvas, as shown in the image below.

A new contextual menu for creating a Relationship appears as shown below:

Users should select both entities and the properties to be connected. The Actions section lists the actions associated, which will be enabled once the Relationship is established. Reverse relationship: Exchanges the current From Entity and Property with the Current To Entity and Property.

Separator: The separator serves as the split character in the many to many resolution in the Data Context Hub. Alias: This is the Label of the relationship in the graph. The name must not contain whitespaces or special characters. (Sample: Defined_By)

Data Sources

Data Sources are classified as either containers or workers, separated by tabs as shown in the image below:

Containers

New Container Creation

Mandatory Fields Required for Creation:

• Title
• The Connector to be used: Based on the connector selected, the user needs to enter the credentials and details to establish a connection
• The Actions section will be disabled until the user creates the container

Container Contextual Menu After Creation

The image below shows the contextual menu of an already created container:

The user will have the option to initialize the container to establish the connection.

Actions Section Includes:

• Initialize: Retrieve a small amount of data from the source and store it in the database. This step must be completed before you can work with the data
• Create New Entities: Once initialized, you can create Target Entities from the container

The above image shows the modal once “Create new entities” is selected. It displays a list of all source entities. Upon selecting one, the user can define a custom name and property to be used as a Business Key. Users can toggle Advanced settings to select the necessary properties.

• Preview Data: Displays stored data from the Initialize operation, will be available after initialization
• Delete Container: Deletes the current container

Data Provider Workers

Canvas enables the management of Data Provider Workers. The user can register Workers using a simple form by providing the required information, as shown in the image below:

Data Import Configuration

Users can access the Data Import Configuration from the canvas or the contextual menu of a Target Entity.

Creation of a New Data Import Configuration

To create a new Data Import Configuration, the user must select a Target Entity and then click the “Data Source” option.

Data Import Configuration Modal

Once the user selects the “Data Source” option, the modal appears as shown below:

The modal is pre-filled with the selected Target Entity. The user can select the Data Source to connect to. Available configurations and the primary source entity are listed, followed by a table displaying available mappings.

Load Type:

• Insert: Inserts new rows. Existing rows with matching Business Keys are skipped. Duplicate Business Keys cause an error
• Preserving Merge: Updates existing rows and inserts new ones. Unmatched rows in the Target Entity remain untouched
• Update: Only updates existing rows with matching Business Keys. No new rows are inserted. If no matches exist, nothing is imported

Distinct: When marked, transferred rows are filtered for duplicates using all columns. Clear: When marked, transferred rows are cleared.

Mapping:

• Select one Target property and one Source property
• A “Map properties” button appears upon selection
• Mappings appear in the Mappings table
• Individual mappings can be edited or deleted
• “Delete all mappings” removes all mappings at once

Additional Options:

• Preview Data: Show preview of the selected Data Pump Source entity
• Auto Map: Automatically map matching properties by name

Expressions can be defined using properties from the source entities. These allow dynamic combinations with other properties or static strings. Once saved, a new source property becomes available to be mapped. An Expression-supporting Worker must be installed in the GBS to use this feature.

String Expressions: Use + to concatenate strings.

Examples:

• "hello" + propertyA
• propertyA + propertyB
• str1 + str2 + str3
• str1 + "abc" + str3
• "abc" + "def"
• str4.str.split('-') → ["a", "b"]
• str1.upper() / str1.lower()
• str1.str.replace('t', 'a')
• date1.astype('datetime64[ns]').dt.strftime('%Y/%d/%m')

Limitations:

• No support for special characters outside Unicode U+0001..U+007F
• Cannot extract substrings like property1[:2]
• Paragraph character § is not allowed

Joins

Joins enable integration of multiple source entities from a Data Pump. One Target Entity can receive data from multiple sources. To create a Join, select the entities and their connecting properties in the Join rule.