A project engineer types a question into your shiny new document intelligence assistant: “What changed about the curtain wall anchorage on the east elevation?” The system does exactly what it was built to do. It embeds the question, finds the five chunks that are semantically closest, and writes a fluent paragraph. The paragraph is wrong. It quotes the original specification section on anchorage, misses the RFI that questioned it, never sees the submittal that proposed an alternate detail, and has no idea that a change order three weeks later superseded all of it.
Nobody on the floor is surprised. They have lived this story before, just without the chatbot. The answer to “what changed” was never in one chunk of text. It lived in the relationships between four documents that referenced each other by number, each authored weeks apart, each correct in its moment and stale the next.
This is the part of retrieval-augmented generation that the demos skip. Vector search is genuinely good at finding text that means roughly the same thing as your question. It is structurally blind to the fact that a submittal exists because of a spec section, that an RFI modifies an assumption, that a change order supersedes a prior instruction. Engineering, AEC, and manufacturing document sets are built almost entirely out of those “because,” “modifies,” and “supersedes” links. When the links carry the meaning, similarity search alone is the wrong tool.
So the real question is not “vector or graph.” It is: how much of the answer your users need lives in the content of documents versus the connections between them? That ratio should decide your architecture. Here is how to think about it without buying into anyone’s hype.
What vector RAG actually does well
Strip away the marketing and vector RAG is a clean idea. You split documents into chunks, run each chunk through an embedding model that maps text to a point in high-dimensional space, and store those points. At query time you embed the question and pull back the nearest neighbors. “Nearest” means semantically similar, so a question about “water intrusion at the parapet” can find a passage about “moisture penetration along the roof edge” even with no shared keywords.
For a large class of problems this is exactly right:
- Definitional and descriptive lookups. “What fire rating does this assembly require?” The answer sits inside one passage. Find the passage, you have the answer.
- Unstructured or loosely structured corpora. Knowledge bases, meeting notes, manuals, support tickets. There is no strong relational skeleton to exploit, so you lose little by ignoring one.
- Fuzzy recall across vocabulary drift. Different authors describe the same thing differently. Embeddings smooth over that better than keyword search ever did.
If most of your users’ questions can be answered by a competent person reading one well-chosen section, vector RAG is not just adequate. It is the correct, cheaper, simpler choice, and you should resist the urge to over-build. Reaching for a graph here is engineering vanity.
The trouble starts when the honest answer to a question is “it depends on three other documents.”
Why interlinked documents break naive vector search
Construction and engineering documents are a citation network wearing a PDF costume. A submittal register points at spec sections. RFIs cite drawings and details. Change orders reference RFIs and prior change orders. Transmittals bundle revisions. The number — RFI-114, CO-07, Section 08 44 13 — is the load-bearing element, and embeddings treat it as a low-signal token.
Three failure modes show up again and again.
Relationships are invisible to similarity
“RFI-114” and “the change order that resolved it” may be only loosely similar as text. One is a question about a dimension; the other is a contractual instruction with cost and schedule language. Their meaning to the reader comes from the link between them, and that link is precisely what an embedding does not encode. Vector search retrieves things that look alike. The documents you need often do not look alike at all.
Chunking severs the connective tissue
To embed documents you have to cut them up. The cut that makes a paragraph a tidy chunk is often the same cut that separates a clause from the table it refers to, or a revision note from the detail it revises. After chunking, “see revised detail 5/A-501” and the actual revised detail live in different vectors with no edge between them. The model that answers the question never gets to follow the pointer, because there is no pointer left to follow.
Recency and supersession are unrepresented
A pure vector index has no opinion about time or authority. The superseded original and the current change order are equally retrievable, and the superseded one is often more similar to a naive question because it uses the original vocabulary. So the system confidently surfaces the stale answer. In a knowledge base that is an annoyance. On a job site it is a field error, a rework ticket, and a margin you will not get back.
None of this means embeddings are bad. It means a flat list of similar chunks is the wrong data model for documents whose meaning is relational.
What graph-aware retrieval adds
Graph RAG keeps the embeddings and adds a second structure: an explicit graph of entities and relationships. Nodes are the things your domain actually talks about — spec sections, RFIs, submittals, change orders, drawings, assemblies, parties. Edges are the verbs — references, supersedes, responds-to, belongs-to, revises.
Retrieval changes shape. Instead of “find the five most similar chunks,” you can do something closer to how an experienced reviewer works:
- Find the entry point by similarity or by exact identifier (“start at RFI-114”).
- Traverse the edges to gather what is connected — the spec it questions, the change order that answers it, the drawing it cites.
- Rank and assemble that connected neighborhood into context for the model.
This buys you things vectors cannot give you on their own:
- Multi-hop answers. “What changed and why” becomes a walk from spec to RFI to change order, returning the whole chain rather than one node of it.
- Authority and recency as first-class facts. A supersedes edge lets you prefer the current instruction and, when useful, show what it replaced.
- Auditable provenance. The graph path is the citation trail. For regulated or contractual work, being able to show why the system surfaced something matters as much as the answer.
The cost is real and you should name it honestly. Someone has to define the entity and edge types, extract them reliably, and keep the graph fresh as documents flow in. Identifier parsing, entity resolution (“Section 08 44 13” and “the glazing spec” are the same node), and relationship extraction are ongoing engineering, not a one-time import. A graph that is 80 percent populated can be worse than no graph, because people trust it and the missing edges fail silently. This is exactly the kind of tradeoff we pressure-test in Ryshe Labs before anyone commits to a production build.
Hybrid is usually the real answer
In practice the interesting systems are not graph or vector. They are both, with each doing the job it is good at.
A pattern that holds up:
Vectors for entry, graph for expansion
Use embeddings to find a strong starting point from a natural-language question, then use the graph to expand outward along meaningful edges. Vectors handle the fuzzy “what is the user even asking about.” The graph handles the precise “what is this connected to that they will need.” You get the recall of similarity search and the precision of explicit relationships without forcing either to do the other’s job.
Graph as a re-ranker and filter
Even where a full traversal is overkill, the graph can quietly improve a vector pipeline. Retrieve candidates by similarity, then let relationship metadata reorder them: down-rank a chunk that supersedes points away from, promote one that sits on the path between two entities the question named. The user still asks in plain language; the relationships shape the result behind the scenes.
Metadata-light middle ground
You do not always need a graph database to get graph-ish benefits. For many engineering corpora, extracting identifiers and a handful of relationship fields into structured metadata on each chunk — this chunk references CO-07, is superseded-by REV-3 — gets you a large share of the value with a fraction of the operational weight. Start here. Graduate to a real graph only when multi-hop traversal is genuinely load-bearing for your users, not because graph databases are interesting.
The instinct to reach for the most sophisticated architecture is the instinct to manage. The right system is the simplest one that answers your users’ actual questions correctly, and most teams overshoot.
A practical decision guide
Before choosing, look hard at the questions your users actually ask. Pull a few hundred real ones if you have them. The distribution tells you more than any benchmark.
Lean toward plain vector RAG when:
- Most answers live inside a single passage.
- Your corpus is loosely structured — notes, manuals, articles, tickets.
- Cross-references are rare or not central to what people need.
- You want to ship something useful in weeks, not quarters.
Invest in graph or hybrid retrieval when:
- The frequent, painful questions are multi-hop: “what changed,” “why,” “what does this depend on,” “what’s the current version.”
- Documents cite each other by identifier and those identifiers carry the meaning.
- Supersession, revision, and authority decide whether an answer is right or dangerously stale.
- Wrong answers have real downstream cost — rework, claims, field errors, compliance exposure.
Two questions that cut through most of the debate:
- Can a knowledgeable human answer the typical question by reading one section? If yes, vectors. If they would flip between four cross-referenced documents, you need relationships.
- Does being wrong cost a sentence or a site visit? The higher that cost, the more the provenance and recency a graph gives you is worth its overhead.
Be wary of the two failure modes at the extremes. Pure vectors on a deeply interlinked corpus produce confident, stale, partial answers that erode trust fast. A premature graph on a corpus that does not need one produces an expensive pipeline that breaks every time the document format shifts. Both are common. Both are avoidable by looking at the question distribution first.
The takeaway
Retrieval architecture is a modeling decision, not a tooling decision. The question is whether the meaning your users need lives inside documents or between them. For a lot of engineering and AEC work the honest answer is “between,” and a flat vector index quietly drops exactly the connections that make the answer correct. That is not a reason to graph everything. It is a reason to match the structure of your retrieval to the structure of your documents, and to start as simple as the questions allow.
If you are weighing this for a real corpus, the most useful first step is rarely picking a database. It is reading a sample of the questions your people actually ask and being honest about how many are single-hop. If it would help to think through that tradeoff against your own documents, that is a good use of a 30-minute call — scope and approach are easier to judge once we have seen what your documents actually look like.
