Sifflet’s core mission is to make data observability simpler by providing data stakeholders with the most comprehensive overview of their data pipelines. We are thrilled to announce that declarative assets and lineage are officially live, going one step further in ensuring end-to-end observability for your teams.
Leveraging our declarative framework, you can now programmatically push assets and lineage onto Sifflet to catalog and visualize lineage for your entire data stack. This allows teams to have a more comprehensive data documentation and governance while simplifying the troubleshooting experience thanks to a more complete impact assessment and root cause analysis.
Sifflet comes with a large number of built-in integrations spanning your entire data pipelines’ stack. Built-in integrations automatically collect metadata and lineage information and make it available in the Data Catalog.
In some cases though, you might need to programmatically declare some of your data pipeline assets to ensure a truly end-to-end observability experience.
Declared assets show up on your Data Catalog, just like any asset pulled from Sifflet built-in sources. Users can browse them as well as filter and search on them using declared metadata. Ensuring data stakeholders are able to locate the asset they might be interested in, regardless of the asset type and technology is key to avoid asset duplication and improve data self service.
Declared asset pages also surface the same amount of metadata as regular assets: you can include descriptions and tie business terms to the asset to ensure all data stakeholders have the context they need to properly understand the asset. You can also include external links to make sure catalog users can swiftly access the asset itself.
As your number of data use cases and assets grows, managing your data becomes increasingly challenging. Declarative assets allow you to ensure your entire data fleet is cataloged, simplifying data governance in turn.
You can assign tags to your declared assets to classify them and ensure any PII data is handled appropriately and in a way that is compliant with regulation. Similarly, you can assign owners to your declared assets in order to drive accountability and simplify collaboration.
This feature would not be complete without a lineage integration. The great news is that you can connect your declared assets to any other assets, whether they are declared or not, within your data pipelines for true end-to-end lineage.
This end-to-end lineage comes with countless benefits for your team. By reflecting all downstream assets, it ensures accurate and comprehensive impact assessments, preventing any overlooked parties before implementing changes. Similarly, in the event of a data quality issue, having all sources visible upstream of the data warehouses simplifies root cause analysis and avoids the time consuming hustle of piecing together pipeline assets to identify the problematic one.
Want to start declaring assets? Check out the declarative assets documentation page to get started!
If you want to learn more about how Sifflet can help you with data observability, you can check out our documentation or reach out for a demo.
.png){kind=logo}
