📄️ 2.1 Data Engineering & Data Integration
The basic concept regarding Data Engineering and Data Integration in is transforming incoming data, which is delivered via Data Pumps, from different sources into a common Model based on Target Entities and Relationships. These components represent the Staging Layer. Based on the Model and the data present in the Staging Layer the knowledge graph can be generated via the Processing step.
📄️ 2.2 Processing
As in chapter Data Engineering & Data Processing described the generation of the knowledge graph is called Processing. In this process, data is stored in the Staging Layer and transformed into nodes and edges, then saved in the graph database Neo4j.
📄️ 2.3 Dashboard
GBS Dashboard
📄️ 2.4 Data Pumps & Containers
Data Pump Images
📄️ 2.5 Target Entities
Target Entities
📄️ 2.6 Relationships
Create and edit Relationships
📄️ 2.7 Graph Building - Data Importer
The DataImporter is the interface where the user can define the mapping between the extracted source objects the the Target Entities.
📄️ 2.8 File Storage
The File Storage enables storage of files in the system. These files can be accessed via an URL and can be utilized by Data Pump developers to implement File Artifacts which are attached to nodes. These can then be accessed via the Frontend when exploring the data.
📄️ 2.9 Load Plans
To populate the knowledge graph in the with data we have to perform two main tasks:
📄️ 2.10 Integrations
The Integration enables the user to define custom JavaScript being executed in the context of the particular pattern of nodes and edges defined in the Template.
📄️ 2.11 Workers
GBS enables the management of Workers. The user can register Workers in this simple form by providing the required information.
📄️ 2.12 Graph Access Rules
The Graph Access Rules define rules for the Graph Security Layer. Through these definitions
📄️ 2.13 Settings
The Settings affect the representation of information in the Frontend.
📄️ 2.14 Workspaces
Workspaces assign Target Entities to users. Therefore the creator of a workspace has to assign users to a workspace and select the Target Entities that the users are allowed to see. Users that are not included into the workspace cannot see the Target Entities in the GBS. Workspaces are used to exclude unpermitted users from accessing Target Entities and can also be used to structure Target Entities in groups and only show them when a workspace is selected.
📄️ 2.15 System
Configuration
📄️ 2.16 API Push Service
The API Push Service is a generic method for importing data into the Graph Builder Service (GBS) system, in addition to the Data Pumps.
📄️ 2.5 Import Export Feature
The Import Export Feature in the Target Enities is used to export Target Entities with their attributes like properties,