DCH 4.0
Version 4.0 of the DCH stack introduces the second generation of Memory for AI and brings AI assistance to modelling and querying. It also removes several legacy features and APIs, so read the removed features, API changes and upgrade notes before upgrading.
Highlights
- Memory for AI v2: agents with versioned capabilities, built with AI or step by step, plus skills, Analysis Engines and a new Sessions workspace.
- AI in the Model: suggestions when creating Target Entities, AI-written descriptions and analyses, and help with expressions.
- Ask AI in Projections: write and fix Cypher with an AI assistant, in a new editor with query timeouts.
- Transparent jobs: warnings, logs, metrics and a timeline for every Queue Job.
Memory for AI v2
M4AI has been rebuilt. Everything an agent uses is versioned, and a published agent pins the exact versions of its memories, skills, child agents and model.
- Agents are configured through six capabilities: instructions, knowledge, skills, tools, child agents, and model and runtime. Create them with Build with AI, which proposes a complete setup from a description of the task, or step by step in a guided setup. Publishing compiles an agent into a fixed artifact that is what actually runs.
- Library: the building blocks agents use.
- Model & runtime configs define the model, its provider and the agent's budgets. 18 providers are supported, and every save creates a new revision.
- Memories give agents access to a part of the knowledge graph, either Target Entities or Views and Functions.
- Skills are reusable instructions an agent loads only when it needs them. Optimize for memory lets AI write a skill for a memory.
- Analysis Engines analyse files you upload in a sandbox and publish their results as files.
- Sessions collects all conversations with agents and Analysis Engines. You can follow every step of an agent, answer its questions and inspect each model call.
The M4AI menu is now locked when the license does not include M4AI.
The first generation of M4AI (Agent Systems, Agents, Memories, Provider and Embedding Configurations) is still available as Legacy: M4AI v1, but hidden by default. It will be removed in 4.1. Move your agents to M4AI v2 before upgrading to 4.1.
Data Context Hub
Model
- AI suggestions for new Target Entities: when creating Target Entities from a Data Source, AI proposes names, descriptions, property names, business keys and which properties to include. Each suggested field can be refined with AI, and nothing is saved until you create the entities.
- Suggest descriptions and Analyze: AI writes descriptions for selected Target Entities, or a read-only analysis of them and their connections.
- Expression editor: expressions are written in a new editor with a property picker, Build with AI and Fix with AI, and can be tested against sample values.
- Data Sources now list the Intake Agents directly. Intake Agents are edited in a new details panel.
Build and Queue Jobs
- Warnings instead of failures: rows with an empty business key or a failing expression are skipped and reported as warnings, and the job finishes with warnings. A job still fails if many rows are skipped, or at the first warning if you turn on Fail on warnings.
- Logs, metrics and timeline: job details have new Logs and Metrics tabs, running jobs show their elapsed time, and a timeline shows when jobs ran.
- Date expressions in Imported since: relative dates such as
${yesterday:yyyy-MM-dd}are resolved when the job runs, so scheduled builds always import the latest data. - Dashboard: select several Target Entities at once and see their staging and graph statistics, relationships and recent jobs. The same statistics are available for a selection in the Model.
Projections
- Ask AI: describe what you need, and the AI assistant writes or fixes the Cypher query. It can explore the graph for context. Accepted suggestions are saved as a new draft.
- New editor with Cypher syntax support and formatting, and results as a graph or JSON.
- Query timeouts: every query now has a time limit, 10 seconds by default. A version can set its own, up to a maximum set by the administrator.
- Run the draft or any version directly, with typed parameters. A View or Function has at most one draft.
- Functions are now managed by the Linked Data API, like Views (see API changes).
Workflows and Exploration
- The Workflow editor was reworked: schedule, time zone, active state and email notifications are set in one Setup section.
- Exploration searches labels on the server and draws several relationships between the same two nodes side by side.
User interface
The whole user interface has been reworked for a consistent look, with consolidated colours, tables, buttons and inputs. AI features are marked in purple throughout.
Removed features
- Data pumps, Containers and Connectors have been removed, including the upload of connector files. Intake Agents are now the only way to connect data sources.
- Graph Security Layer: graph access rules and the filtering of graph data per user have been removed. All users can see all graph data their role allows.
- Rule transformation and its Workflow step have been removed.
- File storage in GBS and the Linked Data API has been removed.
- Aliases have been removed everywhere, as announced in 3.2. Use slugs instead.
API changes
The API documentation of GBS, the Linked Data API and M4AI moved from Swagger to Scalar. GBS and the Linked Data API serve it at /scalar/, and their OpenAPI documents at /openapi/.
Clients that manage or run Functions must be updated:
- Functions are managed at
/api/v2/functionsof the Linked Data API. The/api/v2/functionsendpoints of GBS no longer exist. - Function ids are GUIDs instead of numbers. Old numeric ids are still accepted where a slug is accepted.
- The audit fields
sysInserted,sysUpdated,sysInsertedUserandsysUpdatedUserare nowcreatedAt,updatedAt,createdByandupdatedBy, andaliashas been removed. - Run Functions with
POST /api/v2/functions/{idOrSlug}/execute.POST /api/graph/functions/{idOrAlias}still works, but is obsolete.
GBS
- Problem Details: the legacy
messageandstatusCodefields are no longer included, as announced in 3.2. All routes return the standard Problem Details format. - Removed: all data pump endpoints (
/api/datapump/*), the graph access rule endpoints (/api/graph-access-rule/*) andGET /api/system/licenses. - Functions moved to the Linked Data API (see above).
- Aliases removed from Target Entities, Relationships and Workflows. A Relationship's
typeis now required. - New:
- Job metrics:
GET /api/v2/queue-jobs/{jobId}/metrics,GET /api/v2/queue-jobs/metrics/catalogandGET /api/v2/target-entities/{idOrSlug}/metrics. - Jobs report
hasWarningsandlogs. Intake job requests acceptfailOnWarnings, and date expressions inprocessDataImportedSince. - Expression testing:
POST /api/v2/expressions/test. - Optimistic locking: Target Entities and Relationships return a
version. If you send it back on update, the update fails with409 Conflictwhen the object changed in the meantime. It will become required in a future release.
- Job metrics:
- Property names containing quotes are rejected.
Linked Data API
- Removed: all deprecated
/api/graph/*endpoints, exceptPOST /api/graph/functions/{idOrAlias}, which is now obsolete. Use the/api/v2endpoints instead, for examplePOST /api/v2/views/{idOrSlug}/execute. The file endpoints (/api/files/*) andGET /api/system/licenseshave also been removed. - Functions are now managed here (see above).
- Query timeouts: Views and Functions accept
timeoutSeconds. A query that runs longer is cancelled with504 Gateway Timeout. - Single draft: adding a version to a View or Function updates the existing draft instead of creating another one.
- Aliases removed from Views and Functions.
M4AI
- New v2 API for agents, executions, memories, skills, model and runtime configurations, Analysis Engines and sessions, under
/v2/. - v1 Agent Systems:
aliashas been removed andslugis required, as announced in 3.2. Thealiasvalidation and generation endpoints have been removed. - v1 Memories: View and Function ids are validated against the Linked Data API, and Function ids are now GUIDs.
GET /system/licenseshas been removed.
Intake Agents
GBS now tells Intake Agents when an intake job has finished, so they can release state they kept for it. Intake Agents must serve GET /api/v1/core/info; the fallback to the root URL has been removed. See Develop an Intake Agent in the Intake Agents documentation.
Upgrade notes
- Data Import Configurations without an Intake Agent are deleted during the upgrade, together with their mappings and joins. Move any that still use a Container to an Intake Agent before upgrading.
- Heavy Views and Functions may time out. Queries were previously allowed to run for up to 30 minutes; now the default is 10 seconds. Raise the timeout of slow Views and Functions, or the defaults in the Helm chart (
linked_data_api.*_timeout_seconds). - Graph Security Layer: if you had it enabled, all users now see all graph data their role allows.
- Functions are moved from GBS to the Linked Data API automatically on the first start.
- AI features in the Model and Projections use the M4AI v1 Agent Systems with the slugs
target-entity-assistantandcypher-query-assistant. Since M4AI v1 is hidden by default, show it withm4ai.legacy: "true"to set them up. - Build with AI needs a model configured in the deployment (
m4ai.system_llm_agent_builder_v2_providerand…_provider_config).
Helm chart
-
Kubernetes 1.30 or newer is required. Helm refuses to install or upgrade the chart on an older cluster.
-
Required values are now checked. The chart no longer installs or upgrades when one of these values is missing, and names the missing one:
fqdnpullSecret, unless you reference your own pull secrets inimagePullSecrets- The passwords of the services the chart runs:
postgres.password,keycloak.password,keycloak.default_password,rabbitmq.password,airflow.password,neo4j.passwordandopensearch_node.password
keycloak.default_passwordandopensearch_node.passwordno longer have defaults. -
The default ingress class is now
nginx-f5. SetingressClassNameto keep another one. -
The Airflow Fernet key is now generated for each installation. If you have stored connections or variables in Airflow, follow Airflow Fernet key before upgrading.
Airflow Fernet key
Airflow encrypts the connections and variables you store in it with a Fernet key. Up to 3.2, the key was part of the chart. From 4.0 on, the chart generates a key for each installation, stores it in the secret airflow-generated-secrets and keeps it across upgrades. Alternatively, you set your own key in airflow.fernetKey.
When you upgrade from 3.2 without further steps, Airflow starts with a new key and can no longer read what it stored before, for example the smtp_default connection. If you have not stored any connections or variables, you can skip this section. Otherwise, move them to a new key during the upgrade:
-
Before the upgrade, read the key of your installation:
kubectl get configmap airflow-config -n <namespace> -o jsonpath='{.data.AIRFLOW__CORE__FERNET_KEY}' -
Create a new key:
head -c 32 /dev/urandom | base64 | tr '+/' '-_' -
Set both keys in your
values.yaml, the new one first, and upgrade to 4.0:airflow:fernetKey: "<new-key>,<old-key>" -
Encrypt the stored connections and variables with the new key:
kubectl exec -n <namespace> deploy/airflow -c airflow-scheduler -- airflow rotate-fernet-key -
Remove the old key from
airflow.fernetKey, so that only the new key remains, and upgrade again.
Change the Fernet key only this way. Airflow cannot decrypt connections and variables that were stored with a key it no longer has.