LlamaIndex ‘legal-kb’: Agentic recovery over Index v2 with find, find, read, and grep tools

LlamaIndex published legal-kbpublic reference request on GitHub. It is defined as a database of legal documents, powered by LlamaIndex Index v2 (LlamaParse Platform). The project demonstrates a pattern the team calls the Retrieval Harness for agent retrieval.

The method is different from single shot recovery. Instead of embedding searches per query, the agent is provided with system-style tools. It can then tap into a large, dynamic knowledge base to solve a task. Developers of mirror operations already know the tools: semantic and keyword search, regex grep, file search, and read.

What is legal-kb?

legal-kb is a TanStack Start web application, not a library. You sign in, create a project, upload files, and chat with an agent. Each project is displayed as a managed LlamaCloud Index v2. Uploaded files are processed and indexed automatically in the background. The chat agent then queries that reference live each time.

Retrieval Harness, in plain terms

The harness provides a continuous data pipeline over your documents. It connects to the data source, indexes it, and keeps it updated. On top of that pipeline, it exposes a set of tools to the agent.

Those tools are intentionally close to the functionality of the file system. An agent can list files, read a file, insert within a file, or perform a hybrid search. Because the tools are standard, you can connect the harness to your agents.

Agent in src/lib/agent.ts you are given four tools. Each one maps to the Index v2 retrieval API. The table below lists them as initialized.

A system prompt enforces the command. The agent should make the call findFiles first to establish a list of documents. It then decreases with retrieveand confirms the exact words with readFile or grepFile before saying.

How it works under the hood

Loads follow a clear pipe in between src/lib/files.ts. Bytes are pushed to the project’s LlamaCloud source directory. A File again ProjectFile row is written in PostgreSQL with Prisma. Index synchronization is triggered but not pending; the polling state of the UI is up to date.

Versioning is performed in the (project, filename) pair. Reloading nda.pdf in the same project produces v1, v2, v3 side by side. The retrieval layer filters to version metadata field. This gives version control over the knowledge base itself.

The agent uses the ToolLoopAgent from Vercel AI SDK 6. You choose OpenAI or Anthropic each time and bring your keys. Reasoning is distributed: Claude’s models use extended thinking; OpenAI’s reasoning models use moderate reasoning effort.

Here is a brief but honest view of retrieve instrument and agent.

import { LlamaCloud } from '@llamaindex/llama-cloud'
import { tool, ToolLoopAgent } from 'ai'
import { z } from 'zod'
import { makeCitationId } from './citations'

// One tool closure per index. Wraps Index v2 retrieval APIs.
function createLlamaParseTools(apiKey: string, projectId: string, indexId: string) {
  const client = new LlamaCloud({ apiKey })

  const retrieve = tool({
    description: 'Run a semantic retrieval query against an index.',
    inputSchema: z.object({
      query: z.string(),
      top_k: z.number().nullable(),
      score_threshold: z.number().nullable(),
      rerank_top_n: z.number().nullable(),   // set to enable reranking
      file_name: z.string().nullable(),      // metadata filter
      file_version: z.number().nullable(),
    }),
    execute: async ({ query, top_k, score_threshold, rerank_top_n, file_name }) => {
      const custom_filters = file_name
        ? { file_name: { operator: 'eq' as const, value: file_name } }
        : undefined

      const response = await client.beta.retrieval.retrieve({
        index_id: indexId,
        project_id: projectId,
        query,
        top_k,
        score_threshold,
        rerank: rerank_top_n != null ? { enabled: true, top_n: rerank_top_n } : undefined,
        custom_filters,
      })

      // Return a model-readable list plus citations that drive the UI chips.
      const citations = response.results.map((r) => ({
        id: makeCitationId(),                    // e.g. "c7f2qa"
        fileName: r.metadata?.file_name,
        score: r.rerank_score ?? r.score ?? null,
        preview: r.content.slice(0, 500),
      }))
      const formatted = response.results
        .map((r, i) => `### Result #${i + 1}nn${r.content.slice(0, 600)}`)
        .join('nn---nn')
      return { formatted, citations }
    },
  })

  // findFiles / readFile / grepFile follow the same shape, backed by
  // client.beta.retrieval.find / .read / .grep
  return { retrieve /* , findFiles, readFile, grepFile */ }
}

export function buildAgent(model, apiKey: string, projectId: string, indexId: string) {
  return new ToolLoopAgent({
    model,
    tools: createLlamaParseTools(apiKey, projectId, indexId),
    instructions:
      'Always call findFiles first, ground every answer in the documents, ' +
      'and cite ids inline as `cite:`.',
  })
}

Answers contain visual excerpts. Each detected episode receives a short ID, such as cite:c7f2qa. The agent refers to the id on the line, and the UI provides a clickable quoting chip. Clicking it opens a screenshot of the source page with a bounding box rectangle over the highlighted text.

Naive RAG versus agent Retrieval Harness

The harness is a different use model than the single shot RAG. The comparison below focuses on behavior.

Use situations, and examples

The design targets domains where agents navigate large sets of documents. Legal and fintech are examples mentioned.

• Consider the contractual question: ‘What notice is required to terminate an MSA?’ The agent lists the files, it works retrievethen greps the specific clause. Responds with a quote from a specific page.
• Consider due diligence throughout the data room: An agent can findFiles by name then readFile each candidate. Checks paragraphs without opening all PDFs.
• Consider a policy base with version: Because retrieve he accepted a file_version filter, the agent can query a specific version. This supports tracking of change over time.

Reference implementation