The Semantic Layer Chaos: Why Your Enterprise AI Will Fail Without Knowledge Graphs

The Semantic Layer Chaos: Why Your Enterprise AI Will Fail Without Knowledge Graphs

Issue #8 | March 12, 2026

Everyone’s building a semantic layer now. Every vendor, every platform, every team. But ask them what “semantics” actually means and you get hand-waving about business intelligence and dashboards. Meanwhile, your AI agents are hallucinating, your data teams are rebuilding the same logic across 15 different tools, and nobody can explain how revenue is actually calculated.

Juan Sequeda, Principal Scientist at data.world (acquired by ServiceNow), has spent 20 years in the semantic web and knowledge graph space. He also hosts Catalog & Cocktails: The Honest, No-BS Data Podcast - YouTube a weekly, non-sales-y virtual hangout where we have a tasty beverage in hand and good conversations on data. At last year’s MLOps World in Austin, he delivered a no-BS definition of what semantics actually means and why knowledge graphs are an essential part of the path to enterprise-scale AI that doesn’t collapse under its own weight.

📊 By The Numbers

3x accuracy improvement - Knowledge graphs increased LLM accuracy for database question-answering compared to zero-shot prompting

4x improvement - Further research validated up to 4x better performance with semantic approaches

November 2023 - Benchmark paper published showing knowledge graph advantage; replicated independently by DBT within weeks

50+ years - History of semantic technology foundations; this isn’t new, we’re just finally paying attention

February 2024 - GraphRAG manifesto published citing knowledge graph research as first reason for using graphs

1 definition, infinite deployments - Define semantics once, reuse across BI layers, knowledge graphs, AI agents, and MCP servers

🔥 The Core Problem

Everyone has a “semantic layer”, but we can’t even define what semantics means anymore.

Teams think the semantic layer equals business intelligence. Mapping your fact tables to dimensions and measures for dashboards. That’s fine for BI, but that’s not enough for AI agents.

"Everybody has a freaking semantic layer. What the [hell] does it all mean? The irony is that we can’t even define what semantics is anymore."

When your AI agent asks “what is net revenue?” You need to provide the English definition approved by finance, the SQL formula that calculates it, which specific tables and columns it uses, who approved it and when, why it’s different from the sales team’s net revenue definition, and the complete lineage showing how that answer was derived.

Without this, you’ll be at risk of simply building a POCs that looks fancy but won’t scale in production. The funding will dry up when you can’t explain why your AI gave that answer or when it produces different results than last week.

🧠 The Ontology Progression Framework

Knowledge graphs aren’t new. The concepts behind them go back 50+ years in computer science. But most teams jump straight to “let’s build a knowledge graph” without the foundation. That’s why they fail.

Here’s the actual progression you need:

Level 1: Business Glossary

What it does: Define your business terms with English definitions.

Start here. What is a customer? What is a client? What’s an account? What’s ACV vs ARR? You’re building dictionaries. Getting alignment across teams on what words actually mean.

Example: “Customer” has a term definition that everyone agrees on. “Order line item” is defined as “a line item that is part of an order including a part number and quantity.”

Why this matters: This is your controlled vocabulary. You can’t build meaning on top of undefined terms. Skip this and you’re building on sand.

Level 2: Taxonomy

What it does: Take your glossary and add hierarchical parent-child relationships.

Taxonomies aren’t new - the Greeks and ancient philosophers used them. You’re organizing your terms into structures that show how concepts relate hierarchically.

Example: Customer → Enterprise Customer → Strategic Account. Product → Physical Product → Digital Product.

Why this works: You’re adding your first layer of relationships. Still simple, still manageable, but now you have structure.

Level 3: Ontology

What it does: Take your taxonomy and add rich relationships, cardinality constraints, and rules.

"An ontology - you take your taxonomy and you start adding more relationships around these things. This is where you start talking about your conceptual model."

Now you’re modeling the domain. Customer purchases Order, Order is sent to Shipping Address, Order contains Order Line Items, Order Line Item references Product. You add constraints: one customer can have many orders, one order must have at least one line item.

Common patterns:

Example: You define that an “order line item” has properties: quantity (integer), extended price (decimal), discount (decimal). Each property has a domain (belongs to order line item) and range (expected data type).

Why this requires discipline: You’re codifying business logic. AI needs this context to understand what the data means. This isn’t optional for enterprise AI.

Critical requirement: Human governance. This doesn’t get auto-generated. Business stakeholders need to approve definitions, stewards need to maintain them.

Level 4: Knowledge Graph

What it does: Ontology + your actual data.

The equation is simple: Knowledge Graph = Ontology + Data

Example: You have the ontology defining what an order line item is. Now you populate it with actual 10 million orders from last quarter

Why this enables scale: Now AI can reason over both the structure (ontology) and the instances (data). It knows what net revenue means AND can calculate it from actual transactions.

✅ When Knowledge Graphs Actually Work

1. You need explainability at enterprise scale

Your CFO asks “how was this revenue number calculated?” You need to trace back through the complete lineage: which tables, which columns, which transformation logic, who approved the definition.

Example: AI agent answers a revenue question. You can show: “Net revenue definition approved by Bob in Finance on Jan 15, 2026. Calculated from ORDERS.extended_price and ORDERS.discount columns using this formula. Last updated March 1, 2026.”

Counter-example: POC demos where “it just works” but nobody can explain why.

2. You have multiple conflicting definitions

Different departments define the same term differently. Sales net revenue ≠ Finance net revenue. You need to make this explicit.

"There could be another definition of net revenue. Fine. Let’s create another definition. This is going to be the net revenue definition for sales and there’s going to be a net revenue definition for finance. We’re making these things explicit."

Good examples: Customer (sales view vs support view), Revenue (gross vs net vs recognized), Active user (daily vs monthly vs engaged)

This enables your AI to ask: “Which net revenue definition should I use? You’re in finance, so I’ll use the finance-approved version.”

3. You need to reuse logic across multiple systems

The same definition needs to be deployed to your BI semantic layer (ie .Snowflake Cortex), your knowledge graph database (Neo4j, RDF), your AI agents, your MCP servers, and your data catalog.

Example: Define net revenue once with complete semantics. Auto-generate YAML for the semantic layer, R2RML mappings for RDF knowledge graph, metadata for catalog, context for AI agents.

The test: Are you manually writing YAML files for semantic layers? If you are, you’re doing it wrong. That should be auto-generated from governed metadata.

❌ When Knowledge Graphs Are a Mistake

1. You skip the foundation and jump to the graph

You want to “build a knowledge graph” without having your business glossary, taxonomy, or ontology defined first.

"If you just start jumping in here, you’re going to have great POCs, but if you want to have something that’s scalable and survive the test of time for your organization, there is a progression. There’s a crawl, walk, and run."

The test: Do you have business terms with approved definitions? Do you have a taxonomy? Can you show your ontology? If no, you’re not ready for a knowledge graph.

2. You’re not incentivized for reusability

Your organization rewards “1 + 1 = 2” work. One team solves problem A, another team solves problem B, everyone rebuilds the same logic.

You’re incentivized to ship fast this quarter, not invest in medium-term reusability. Knowledge graphs require investment upfront to pay dividends later.

The test: Is your team measured on quarterly velocity or long-term leverage? Knowledge graphs need the latter.

3. You think you can skip the governance

You want the AI to auto-generate everything. No human review, no stewardship, no approval processes.

Knowledge graphs require human governance. Humans define what terms mean, approve definitions, maintain mappings. AI can help (knowledge engineering agents), but humans still govern.

The test: Who owns the definition of “customer” in your org? If the answer is “nobody” or “the AI,” you’re not ready.

🏗️ The Three Types of Metadata (The Foundation)

Technical Metadata

What it is: The metadata you automatically observe from your systems.

Database schemas, table structures, column names, primary keys, foreign keys. This gets extracted from source systems: Snowflake, Postgres.

Example from production: You have a TPCH database with a LINEITEM table containing columns: L_QUANTITY, L_EXTENDEDPRICE, L_DISCOUNT. This metadata exists in the system already.

Critical components:

Business Metadata

What it is: Metadata that comes from humans. Business terms, definitions, approved concepts.

This is your glossary, your ontology. What does “order line item” mean in English? What properties does it have? What are the cardinality constraints?

Example from production: “Net revenue” is defined as “revenue after discounts and returns, calculated as extended_price * (1 - discount).” Approved by CFO Bob on Jan 15, 2026. Used by the Finance team.

Critical components:

Mapping Metadata

What it is: The connective tissue. How technical metadata maps to business metadata.

You’re saying “this database column actually represents this business concept.” Sometimes it’s one-to-one (LINEITEM.L_QUANTITY → order line item quantity). Sometimes it’s calculated (net revenue doesn’t exist as a column, but is calculated from multiple columns).

Example from production:

Critical components:

💡 The Core Principle: Semantics = Business + Mapping + Technical Metadata

"Here is my honest no BS definition of semantics: It’s the business metadata plus the mapping metadata plus the technical metadata. When everybody comes up and says ‘oh semantics’ the irony is that we can’t even define what semantics is anymore. That’s the definition. If you don’t agree with it then give me one. But I’m giving you a very specific definition."

Translation:

When someone asks “what does net revenue mean?” you need to answer with all three types of metadata:

This is how you make AI systems scale at enterprise level. This is how you provide explainability. This is how you avoid the POC graveyard.

⚙️ Best Practices for Production

DO: Start with your metadata knowledge graph

Your first knowledge graph should be about the metadata of your systems. Not your domain data → your metadata about your data.

Model: databases → schemas → tables → columns. Business concepts → properties → relationships. Mappings connecting the two.

This gives AI the context it needs to understand what your data means.

DO: Treat semantics as a deployment function

Define semantics once (business + mapping + technical metadata). Deploy it everywhere: BI semantic layers, knowledge graphs, AI agent context, MCP servers, data catalogs.

DO: Govern metadata like code

Business metadata needs approval workflows. Who approved this definition? When? What’s the version history? Who’s the steward?

Mapping metadata needs validation. Does this SQL expression actually work? Does it reference valid columns?

Technical metadata needs automatic sync. When schemas change, metadata updates.

DON’T: Skip the progression (glossary → taxonomy → ontology → graph)

You can’t jump straight to knowledge graphs without the foundation. You’ll build POCs that don’t scale.

"If you want to have something that’s scalable and survive the test of time for your organization, there is a progression."

DON’T: Treat semantic layers as just BI tools

The vendor hype around “semantic layers” is almost entirely about business intelligence. Dashboards and reports.

AI needs semantics way beyond BI. It needs the complete metadata knowledge graph with business definitions, mappings, lineage, approval workflows, and governance.

DON’T: Expect AI to build this without human governance

AI can help. Knowledge engineering agents can automate discovery, identify inconsistencies, propose definitions. But humans still govern.