Where metadata management is heading in the age of GenAI
Julian Thomas, Principal Consultant at PBT Group
In the first blog of this series, I looked at how metadata management frameworks have evolved over the past 30 years, from Word documents and Visio diagrams through to specialist tools, cloud platforms, and open-source projects. The thread running through it all was simple: organisations need a reliable way to describe and govern their data if they want to use it with confidence.
Today, that requirement has not changed. What has changed is the scale, the speed, and now the arrival of generative artificial intelligence (GenAI). The question is no longer whether you need a metadata framework, but whether the one you have can cope with a world of cloud platforms, streaming data, and AI models that depend heavily on context.
Where we are now
Most organisations have moved beyond pure documentation. They are working with catalogues, business glossaries, and lineage tools that sit closer to the flow of data. Some are using commercial platforms, while others are choosing open-source options or building their own components. In many places, these tools are stitched into the broader data ecosystem through ingestion, transformation, quality checks, and consumption.
The mindset has shifted from “capture a description once” to “keep metadata connected and current”. Lineage should be updated when pipelines change. Definitions should be embedded where business users actually work. Governance rules should be enforced, not just filed away in a policy document.
We also see more distributed ownership. Instead of one central team trying to define everything, domain-aligned teams take responsibility for the meaning and quality of their own data. In contrast, a central function sets standards and provides shared tooling. That balance is important. Too much centralisation slows everything down. Too little, and you lose coherence.
Even with these advances, there are still common gaps:
- Metadata that is technically complete but unreadable for business users.
- Lineage that only covers part of the path from source to consumption.
- Frameworks that exist on paper but are not reflected in delivery practices.
This is the environment into which GenAI is arriving.
What GenAI really changes
There is a lot of marketing noise around GenAI and metadata. Some of it is justified, while some aspects gloss over complex problems. I see GenAI adding value in a few specific ways.
First, it can help with the heavy lifting of capture and classification. Instead of asking humans to tag every column manually, you can use models to propose business-friendly names, group related fields, and identify possible relationships. Given a set of tables, for instance, a model can often make a reasonable first attempt at describing their purpose and suggesting how they fit into an existing glossary.
Second, GenAI can assist in stitching metadata together. It can look across logs, schemas, and documentation to infer how data flows through the environment, making lineage more complete and easier to explore.
Third, it can improve discoverability. Natural language interfaces allow users to ask questions like “Where does our churn metric come from?” or “Which datasets contain customer consent information?” and get guided responses that draw on catalogues, glossaries, and lineage data behind the scenes.
What GenAI does not do is remove the need for a framework. Without agreed-upon standards, good source data, and clear ownership, you only succeed in automating confusion faster. So, while models can suggest, people must still make the decisions.
Designing frameworks for the next decade
If you are thinking about where to take your metadata framework next, it helps to separate the three layers:
- Foundations: Clear definitions, ownership, quality expectations, and governance rules. This has not changed and remains non-negotiable.
- Plumbing: The technical integration work that collects, stores, and synchronises metadata from multiple systems. Here, cloud services, APIs, and open-source components give you more flexibility.
- Intelligence: The analysis and assistance layer where GenAI can play. This is where you use models to propose new links, fill gaps, and make the framework more usable.
The sequence is important. If you jump straight to the intelligence layer without solid foundations and plumbing, the results will be unreliable. GenAI is best used as an accelerator for work you already know you need to do, not as a substitute for thinking.
Practical way forward
For most organisations, the way forward is about evolving.
Start by being honest about where you are. Do you have a shared business glossary that people actually use? Is the lineage sufficient to trace key metrics in the report back to the source? Do domains know what they own? If not, those basics are still the first steps.
From there, you can identify the points where automation and GenAI will make a real difference. For example:
- Proposing business terms and descriptions for new datasets.
- Filling in missing lineage where technical logs already exist.
- Powering a search experience that lets users find and understand data faster.
You do not have to switch everything on at once. Choose a high-value domain, apply your framework there, and let it serve as the reference example.
Finally, keep people in the loop. Use GenAI to draft, while giving stewards the ability to approve, correct, or reject suggestions. Make sure the framework reflects how teams actually work, rather than how a tool vendor thinks they should work.
The way ahead
Looking back over the past 30 years, the story of metadata management has always been about bringing structure to complexity so that people can use data with confidence. GenAI does not change that story. However, it gives us new tools to move faster and go deeper.
The organisations that will benefit most are not the ones that chase the most impressive demo. They are the ones that take the frameworks they already use and apply GenAI thoughtfully to make them more scalable, more usable, and more closely aligned to how the business actually runs.
In other words, the way forward is not “GenAI instead of metadata frameworks”. It is “better metadata frameworks, amplified by GenAI”.
