Overview

Streamdal is an open-source project that provides real-time data observability, data governance, data quality monitoring, and data transformation capabilities. It offers a combination of services and software development kits (SDKs) to enable users to monitor and manage their data. With Streamdal, users can observe data in real-time, enforce data governance rules, monitor data quality, and perform data transformation operations such as obfuscation, masking, and stripping.

Features

• Data observability: Users can monitor their data in real-time using the tail -f functionality.
• Data governance & enforcement: Streamdal allows users to enforce rules and policies on their data, ensuring compliance and data integrity.
• Data quality monitoring: Users can monitor the quality of their data and identify issues or anomalies.
• Data transformation: Streamdal provides capabilities for data transformation, including obfuscation, masking, and stripping.

Installation

To install Streamdal, follow these steps:

1. Clone the Streamdal repository:

   curl | bash
   

2. Install the necessary dependencies, including git, docker, and docker-compose.

3. Bring up all the components using docker-compose.

4. Open http://localhost:8080 in your browser to access the Streamdal UI.

For alternative installation methods, refer to the install directory.

To instrument your code with Streamdal, follow these steps:

1. Install the Streamdal server and console.

2. Use one of the provided SDKs (Go, Python, Node.js) to instrument your code. Examples of instrumentation can be found in the documentation. Note that the provided examples are basic and should not be used in production code.

For more detailed instructions, refer to the Streamdal documentation.

Summary

Streamdal is an open-source project that enables real-time data observability, governance, quality monitoring, and transformation. It provides a combination of services and SDKs for users to monitor, enforce rules, and transform their data. With Streamdal, users can easily observe their data in real-time, enforce data governance policies, monitor data quality, and perform data transformations. The installation process is straightforward, and users can instrument their code using one of the provided SDKs. Streamdal offers a comprehensive solution for managing and monitoring data in real-time.