Abstract Data Background

// DataSpace

One platform to Transform, Analyze and Manage your data.

DataSpace is an all-in-one, AI-assisted data platform: ingest data, create pipelines, write transformations, preview datasets, monitor workloads, and seamlessly generate charts and dashboards.

//Transformation

Transformation Engine

Write arbitrary Python code in an integrated IDE. Transformations run in secure Docker containers, leveraging the speed of Polars and the efficiency of Parquet files.

Git-based Workflow & Branching

Develop in isolated branches and merge logic seamlessly.

Secure Docker Containers

Transformations run securely in isolated environments.

Optimized Parquet Storage

Lightning fast analytical queries backed by Parquet.

Read Documentation
DataSpace code editor
//Analysis

Dashboards & Insights

Generate and publish dashboards and charts. Handle unstructured data like documents and images with artifact storage. Time-travel through dataset versions by time or branch.

// 01

Dashboards

Visualise data & plots

// 02

Artifacts

Unstructured Data

//Lineage

Lineage & Build Stats

DataSpace analyses source code to generate column-level lineage in real-time. Build visibility allows you to track the evolution of row counts, file sizes, and duration over time.

Dependency Tracking

Resolve upstream and downstream dependencies instantly.

Build Statistics

Monitor dataset evolution (rows, size, duration) per build.

View Lineage Docs
main @ 8a2b3cIngestIngestHistorizeHistorizeJoinJoinPlotPlotDataDataReportReportNotifyNotifyfeat/sanitization @ f4e1d2main @ b105f2
//Lineage

Pipeline History

Click to explore the pipeline lineage and execution states.

1

Initial Pipeline Setup

8a2b3c

Pipeline structure defined on main branch. All nodes initialized in neutral state.

Branch:main
2

Daily Build

BUILD

Scheduled execution triggered. Ingest and Join processing completed successfully.

Branch:main
3

Build Failed

BUILD

A failure occured during a build

Branch:main
4

Branching Out

f4e1d2

Introduced a "Sanitize" node to filter bad data before validation to fix the build failure.

Branch:feat/sanitization
5

Running Tests

BUILD

Running sanitation checks on the new node.

Branch:feat/sanitization
6

Build #1025 Success

BUILD

Feature branch build passed all checks. Sanitize node successfully filtered outliers.

Branch:feat/sanitization
7

Merge into Main

b105f2

Merged feat/sanitization into main. Production pipeline updated with robustness fixes.

Branch:main
//Assistant

AI Data Assistant

Explore, analyze, and visualize your data using natural language directly on top of your existing workspaces. No data leaves your infrastructure.

Context Discovery

Iterative data exploration. Automatically reads workspace README.mds to learn your business logic.

Visualizations

Generates charts and data tables instantly.

On-Premise

Fully offline operation. 100% data sovereignty.

//Security

Secure by Design

DataSpace allows for complete self-hosting. Segregate projects into workspaces and control access with resource-based permissions.

// 01

Self-Hosted

Deploy on-premise or in your private cloud. Keep total control over your data.

// 02

RBAC

Resource-based access control for granular permissions.

// 03

Workspaces

Strict segregation of projects, data, and pipelines into workspaces.

//Capabilities

Platform Features

// 01

Workspaces

Strict segregation of projects. Manage resources, access control, and configurations independently for each workspace.

// 02

Transformation Engine

Blazing fast transformation engine using Polars and efficient Parquet file storage for high-performance ETL.

// 03

Git Workflow

Full versioning support. Branching, commits, and time-travel for datasets ensuring total reproducibility.

// 04

Column Lineage

Code-based lineage analysis. Visualize dependencies at the column level to understand data flow and impact.

// 05

Artifact Storage

Manage unstructured data. Store and analyze documents, images, and arbitrary binaries within your data pipeline.

// 06

Build Scheduler

Automate pipelines. Schedule builds based on time or events, with full visibility into duration and statistics.

// 07

On-prem Hosting

Deploy DataSpace natively within your own infrastructure. Total control over your network and hardware.

// 08

Data Health Checks

Declarative health checks and data quality assertions that run concurrently with your pipelines.

// 09

Data Assistant

Chat with your data using natural language. Fast, secure, and operates entirely on your infrastructure.