r/Rag • • Sep 02 '25

Showcase 🚀 Weekly /RAG Launch Showcase

34 Upvotes

Share anything you launched this week related to RAG—projects, repos, demos, blog posts, or products 👇

Big or small, all launches are welcome.


r/Rag • • 2h ago

Discussion Fiabilité de l'utilisation de l'outil Magistral 24B dans le cadre d'une analyse RAG basée sur des documents (Ollama vs vLLM)

1 Upvotes

Titre : Fiabilité de l’utilisation de l’outil Magistral 24B dans un système RAG ancré dans les documents (Ollama vs vLLM)

Bonjour à tous,

Je développe un assistant IA entièrement local, en français, pour un domaine de connaissances spécialisé exigeant une attribution rigoureuse des sources et un raisonnement complexe.

Notre architecture actuelle utilise :

Magistral 24B pour le raisonnement et la génération de réponses.

Python pour l’orchestration, la récupération de documents, l’exécution des outils, la gestion des permissions et la validation des sources.

Un pipeline RAG ancré dans les documents avec un corpus de référence conséquent.

Ollama, avec des expérimentations complémentaires utilisant vLLM et l’appel d’outils natifs.

Une carte graphique RTX 5070, 12 Go de RAM et 64 Go de RAM système, avec déchargement partiel du CPU/GPU.

Après de nombreux tests, nous avons rencontré plusieurs problèmes récurrents :

1. Fiabilité de l’utilisation des outils

Le modèle répond parfois sans demander de documents supplémentaires, même lorsque les preuves disponibles sont insuffisantes. Dans d'autres cas, les appels d'outils peuvent être mal gérés par le moteur d'inférence ou ses modèles.

2. Recherche versus raisonnement Lorsque les documents nécessaires sont correctement fournis, Magistral peut produire un excellent raisonnement spécifique au domaine. Cependant, il arrive que des passages critiques des sources ne parviennent pas au modèle. Nous étudions le classement des résultats de recherche, la sélection du contexte et le transport des documents indépendamment du raisonnement du modèle.

3. Sur-contrainte du modèle Nous avons expérimenté des invites, des règles de validation et des contrôles de flux de travail de plus en plus restrictifs. Ces tests n'ont pas amélioré la qualité des réponses de manière constante et ont parfois complexifié le système.

Nous nous orientons désormais vers une architecture plus simple :

Python garantit une recherche documentaire initiale avant les réponses de fond.

Magistral gère l'analyse intellectuelle, le raisonnement, les demandes de recherche complémentaires et la rédaction.

Python exécute les outils, gère les permissions, valide les références aux sources et contrôle la boucle d'exécution.

Nous évitons les instructions inutiles qui dictent la manière dont le modèle doit raisonner. Mes questions aux développeurs ayant conçu des systèmes similaires :

Avez-vous réussi à obtenir une utilisation fiable d'outils multi-étapes avec Magistral 24B ou Magistral 3 14B Reasoning ?

Quel moteur d'inférence, quelles versions, quel analyseur syntaxique d'appels d'outils et quels modèles de conversation ont fonctionné de manière fiable ?

Le passage à Python pour l'orchestration et la validation a-t-il amélioré la fiabilité tout en offrant au modèle une plus grande liberté de raisonnement ?

Comment assurez-vous la continuité du raisonnement entre les appels d'outils, gérez-vous les appels manquants et évitez-vous les boucles de raisonnement excessives ?

Avez-vous mesuré la justesse et l'exhaustivité réelles des réponses, et pas seulement la réussite de l'exécution des outils ?

Je suis particulièrement intéressé par les solutions ayant fait leurs preuves dans des applications réelles, avec des configurations reproductibles et des limitations connues.

Je peux partager des traces d'exécution anonymisées et des cas de reproduction minimaux si nécessaire.

Merci d'avance pour vos retours techniques.

Texte rédigé avec l'aide d'une IA ; le projet, les tests et les observations sont de moi.


r/Rag • • 21h ago

Tools & Resources How to find one memory in 60 million tokens - lessons learnt from building AI memory at scale

2 Upvotes

I've been building RAG systems for 3+ years now. The largest scale system I built powers my AI memory platform MemoryPlugin. We have a multi-stage RAG pipeline, and I wrote a blog post to share my experience, the mistakes I made, and share details about our pipeline with a lot of transparency.

Happy to answer questions, get your feedback, and learn about your experiences!

https://blog.memoryplugin.com/labs-002-large-scale-memory-recall/


r/Rag • • 20h ago

Discussion Magistral 24B tool-use reliability in document-grounded RAG (Ollama vs vLLM)

1 Upvotes

Title: Magistral 24B tool-use reliability in document-grounded RAG (Ollama vs vLLM)
Hi everyone,
I’m developing a fully local, French-language AI assistant for a specialized knowledge domain requiring rigorous source attribution and complex reasoning.
Our current architecture uses:
Magistral 24B for reasoning and answer generation.
Python for orchestration, document retrieval, tool execution, permissions and source validation.
A document-grounded RAG pipeline with a substantial authoritative corpus.
Ollama, with additional experiments using vLLM and native tool calling.
An RTX 5070 12 GB and 64 GB system RAM, with partial CPU/GPU offloading.
After extensive testing, we’ve encountered several recurring problems:
1. Tool-use reliability
The model sometimes answers without requesting additional documents, even when the available evidence is insufficient. In other cases, tool calls may be incorrectly handled by the inference engine or its templates.
2. Retrieval versus reasoning
When the necessary documents are correctly supplied, Magistral can produce excellent domain-specific reasoning. However, critical source passages sometimes fail to reach the model. We are investigating retrieval ranking, context selection and document transport separately from model reasoning.
3. Over-constraining the model
We experimented with increasingly restrictive prompts, validation rules and workflow controls. These did not consistently improve answer quality and sometimes made the system more complicated.
We are now moving toward a simpler architecture:
Python guarantees an initial documentary search before substantive answers.
Magistral handles intellectual analysis, reasoning, follow-up research requests and writing.
Python executes tools, manages permissions, validates source references and controls the execution loop.
We avoid unnecessary instructions that dictate how the model must reason.
My questions to developers who have built similar systems:
Have you achieved reliable multi-step tool use with Magistral 24B or Ministral 3 14B Reasoning?
Which inference engine, versions, tool-call parser and chat templates have worked reliably?
Has moving orchestration and validation into Python improved reliability while allowing the model more freedom to reason?
How do you preserve reasoning continuity across tool calls, handle missing tool calls and prevent excessive reasoning loops?
Have you measured actual answer correctness and completeness, rather than just successful tool execution?
I’m particularly interested in solutions that have been tested successfully in real applications, including reproducible configurations and known limitations.
I can share anonymized execution traces and minimal reproduction cases if helpful.
Thanks in advance for any technical feedback.
English wording prepared with AI assistance; the project, tests and observations are my own.


r/Rag • • 22h ago

Discussion SharePoint mcp vs RAG vs notebook llm

1 Upvotes

Hi folks, has anyone found any issues with SharePoint mcp or notebook llm and why did you choose Rag over these available tools?


r/Rag • • 1d ago

Discussion What I did in the last 4 months on my retirement project RAG-LCC

7 Upvotes

This is a follow-up to my earlier post about the first 15 months of building RAG-LCC in retirement:

https://www.reddit.com/r/Rag/comments/1valvk6/15_months_building_a_rag_system_in_retirement/

The project kept evolving over the last 4 months, so I thought it might be interesting to share what worked, what failed, and what I learnt along the way. Since the last writeup I had success and failure moments. Of course, I learnt more from the failures about RAG.

Orchestrators

Because RAG-LCC provides a lab setup for RAG experimentation, I found it useful to introduce an Orchestrator component. The orchestrator defines RAG flows. Users can create new flows and then activate them during a RAGChat session.

Confidence scoring

With orchestrators, confidence became an issue. So static and dynamic confidence evaluation was added. The latter uses a LLM judge call. There is currently no validation against "golden answers" implemented. The LLM judge rates internal consistency and explanation clarity but not factual correctness.

TableChunker

With these new features, it was time to look at answer quality. There was a query asking for the details of PCIe card slots on my workstation's manual. The information is stored in a table. The goal was that the information is found and the grounder marks it.

The first step was to introduce a specialized TableChunker class and wire it into the already existing content specific chunkers. These chunkers pass table blocks to the TableChunker.

This showed partial success: The table was now consistently in the chunks. But it did not reliably show up in the query answers.
BTW, while looking for specific content in the chunk, the debug levels of RAG-LCC proved helpful. The levels can be changed mid-session so users get the information at the detail level they need currently.

Insight: If retrieval signals are weak, think about adding the best matching table result. Do this only if there are chunks found. Adding the table result to an empty answer set could confuse the user. This rescue may help in a large but sparse candidate pool.

Answer rendering

RAGChatServices delivers answers to Open WebUI. In the browser, .md and even Mermaid diagrams are nicely rendered. However, RAGChat showed tables with unaligned content. It was not a nice sight. So using the Rich library, answers with Markdown can be displayed in the RAGChat CLI also. But the rendering broke grounding, so additional work was required to separate the raw and display ready answer.

Raw answers gave the grounding signals and text to be grounded. The raw answer text is marked for grounding before Rich renders the resulting CLI text.

Insight: Keep the original, unrendered LLM answer for grounding evaluation. Once the grounded text is available, apply the visual grounding.

Document summarization

This was a "you wake up in the night" and have an insight moment. RAGChat has a document pin option. So once you found the right document about hedgehogs, you pin it and ask to summarize it. Hmm... the query is "create a summary" and this will not retrieve the content you are looking for.

So I introduced a summarize: option, using the pinned document or allowing the user to enter a path to retrieve the specific document by its metadata. Very large documents required batches of n chunks to be summarized and then summarize the batches in further passes until the final summary fits. This works, but it is admittedly slow.

Insight: If you want a specific action to be performed on the retrieved context, the action should not be part of the query. This leads to the next topic:

Query and query action splitting

With table, bullet point list and Mermaid rendering I wrote often queries like "Look for all animals contained in the documents and create a table containing animal, mammal yes, no, don't know".

Thought by the document summarization experience, I wondered whether it would make sense to ask a LLM to determine the effective query, the required action on the query results and the query keywords. Yes, you assume it: Each decomposed retrieval query goes now to enabled local retrievers, including vector search. In addition, the keywords go in an extra BM25-only pass. The QuerySplitter class was born.

Translation

Living in a multilingual country where you have 4 official languages and the unofficial English one, you have to bother about document languages. Luckily, during RAGLoad, the document language is added to metadata.

The old flow translated the question into English. It was also rewritten using the conversation history. Vector and indexed retrieval used the rewritten English query. The resulting candidates were reranked against the rewritten query.

The new flow rewrites the query with the conversation history in the user's language. The vector search runs the user language query against the ChromaDB collection. For reranking the rewritten original query is used. Indexed retrieval is for every active and present document-language bucket. When required, the query is translated.

vLLM integration

While inference calls were already supported using Ollama or vLLM backends, the embedder and reranker ran as local Hugging Face models previously.

What still works well

The configuration first approach is an extensible pattern. A new class gets its own config file when required. Abstracting functionality into classes helps understanding and maintaining the code.

Parallel to my work on RAG-LCC I worked on harinezumigel-llm-stack. I introduced groups such as rag and rag_old, which start up the required models after doing an available memory check.

So this is at a high level what happened in the last 4 months. The journey is going on.

If you want to have a look:

As always, feedback, ideas and bug reports are welcome.


r/Rag • • 1d ago

Tools & Resources Benchmarking PDF Extractors: Docling, Reducto, PDF Miner, etc

2 Upvotes

https://dev.to/vtempest/pdf-gec

Choosing a PDF extractor isn’t just about speed—it’s about getting the text, tables, reading order, and output format your application needs. My latest article compares PyMuPDF, Docling, Reducto, extract-pdf, and more, and outlines how to benchmark them fairly across PDF and webpage extraction tasks.


r/Rag • • 2d ago

Discussion Best Practices for Implementing and Troubleshooting a RAG system?

15 Upvotes

Wanted to ask the experts here from a newbie standpoint. For context, my background is 15+ years in finance (and then some data science as that was my MBA focus). The job market's been tough so I've been working on building some finance software that makes use of an AI Agent, which will include RAG (which is the part I'm working on right now - hence this post).

I'm trying to learn as best I can but the resources I have so far is LLM itself, Chip Huyen's AI Engineering Book, and random posts from reddit. Is there like a single book or reference manual that's focused and pretty good at best practices regarding building a high quality production level RAG system?

Also am doing this because I'm pursuing a Masters in AI and targeting Finance FDE roles (unless the business becomes really successful) so would like to be able to build something with all the best practices to demonstrate experience in the area.

Edit: Also wouldn't mind materials on best practices relating to number gating and prompt engineering. So I can cross-reference to see if mine are set up well or not.


r/Rag • • 1d ago

Discussion What is the difference between cosine, euclidean, and HNSW?

2 Upvotes

What is the difference between cosine, euclidean, and HNSW?


r/Rag • • 1d ago

Discussion Three measurements that changed how I trust an eval set

2 Upvotes

I run a local index over a codebase and a harness around it: fixed operations, a gate, a trace. The index answers with file:line facts instead of dumping files into the prompt. Numbers below come from one repository and one machine, static analysis only, over roughly four weeks.

Evals are the part of this project I got wrong the longest. Not because the tasks were bad, but because the numbers I looked at could not fail the way the system actually failed. Three measurements fixed that.

1. A model call counter, not a score.

My worst regression looked like a model downgrade: 90% to 60% on a fixed task set. It was not the model. The local build was answering nothing - every task returned an empty result, and the harness scored the empty result as a normal answer. The score with no model call at all was exactly the 60%.

Now every task asserts two things before the score counts: at least one model call happened, and the call returned non-empty output. Two lines per task. It has caught more real breakage than any quality metric I have, because silence in an agent stack is usually indistinguishable from a confident wrong answer.

2. Per-case results, and a rule that beats the average.

With 14 fixed navigation tasks, one flaky case moves the average by 7 points. My first two "improvements" were noise: the average went up and nothing got better. Worse, one of them hid a real regression - the answer got shorter and more readable but stopped citing file and line, so the number improved while the thing I needed disappeared.

The replacement is a rule: no previously passing case may fail. Not a threshold, a prohibition. It caught two regressions the average had hidden, both in tasks I was not looking at.

3. Missed answers counted separately from wrong answers.

The index returns three kinds of "no": absent (not in the corpus), stale (the index knows the file changed and will not answer until refreshed), and out-of-scope (the corpus was never meant to cover it). Only the first is a real "not found". Before I split them, all three looked like an empty list.

That split also exposed why a single accuracy number is misleading: a system that abstains on everything has perfect honesty and zero utility. I now track wrong answers and missed answers as separate numbers, because they fail differently in production - one invents content, the other sends every question to a human.

Where it broke, and the uncomfortable part.

The gate. I made it refuse raw file reads until the index was used, and the refusal said "raw search is blocked". The agent retried twice, then asked me to run the command for it. Three turns lost on the first session. The fix was one sentence: name the next call - "run ensure, then search_multi". Retries went to zero, same model, same weights.

The uncomfortable part is that all three measurements are about the system, not the model. They would still apply with a much better model, and they are where I would spend the next week: a counter for silence, a rule for regressions, and a taxonomy for absence.

If your eval cannot distinguish "did nothing" from "answered wrong", it is measuring the wrong thing.


r/Rag • • 2d ago

Discussion do you cap how many chunks a rag system pulls from one doc?

3 Upvotes

say you retrieve 5 chunks and 4 come from the same guide. two of them basically say the same thing and the question still needs one detail from a different doc

would you cap it at 2 chunks per doc or pull more candidates and rerank them? curious if anyone tried a cap and found it dropped useful context on questions where most of the answer really was in one doc


r/Rag • • 2d ago

Discussion Comparison and "alternatives" pages get cited the most. How are you keeping the competitor data on them accurate?

3 Upvotes

Seems settled that comparison style pages (X vs Y, alternatives to X, best X for Y) are some of the easiest pages to get cited in AI answers

The catch is they're full of competitor data: prices, plans, features, specs. That goes stale fast, and then the engines keep quoting the stale version with your page as the source.

For people running these pages at any scale:

- how often do you refresh competitor prices and specs?

- manual, page monitors, or a data feed?

- what do you do when you can't verify a number anymore? Remove it, or keep it with a date?

Has anyone seen a citation drop because the data on the page went out of date?


r/Rag • • 2d ago

Discussion Should users be able to see what a RAG system ignored?

6 Upvotes

Most RAG systems show users the sources they used to generate an answer... What about the documents that RAG systems retrieved and then discarded?

Imagine asking a RAG system, about a company policy. The RAG system finds five documents uses two and ignores three. One of those ignored documents might contain information or an important exception.

Would showing users, the rejected sources make RAG more transparent or just create more confusion? I have rarely seen this topic discussed. I wonder what RAG engineers think.


r/Rag • • 2d ago

Discussion Half our RAG eval swings came from the ingestion pipeline, not the retriever

2 Upvotes

We build a document assistant for accounting firms. Customers dump PDFs, scans, Excel exports and the odd .msg file, we parse it, chunk it, embed it into pgvector and answer questions on top. Our eval is about 400 questions with known answers, scored with Ragas every night.

For weeks recall moved by 6 or 7 points between runs with no change to chunking, embeddings or prompts. We kept tuning the retriever. The retriever was fine.

The problem was ingestion. Three things kept changing under us.

Parser versions. Docling and LibreOffice got updated with the base image, and a minor release changed how tables were split. Same PDF, different chunks.

OCR. Scanned docs went through Tesseract on whatever node picked up the job, and two nodes had different language packs installed. Some scans came out as garbage on one and fine on the other.

Shared workers. One customer's 900 page scan could hog a worker for an hour, and the nightly reindex for everyone else timed out halfway. So the eval ran against a half built index.

What we do now. Ingestion runs in its own Linux VM per customer, started from one frozen snapshot with pinned versions of Docling, LibreOffice and Tesseract. The snapshot id is stored with every chunk, so when a score moves we know right away if the parser changed. When we upgrade the parser we build a new snapshot, reindex the eval set with both and compare before rolling it out. And a nasty file only takes down that one customer's run.

Side benefit we didn't plan for. Customer documents never sit on a shared machine anymore, which made our last security questionnaire a lot shorter.

The VMs are on boat.dev. We looked at Cloud Run jobs and at Kubernetes jobs with stricter images. boat won because a full VM runs LibreOffice and a headless Chrome for HTML exports without fighting the sandbox. No GPU there, so embeddings still run elsewhere, and a VM takes a few seconds to boot instead of being instant, which is fine for batch jobs.

Since then recall moves by about 1 point between runs. When it moves more, it's because we changed something on purpose.


r/Rag • • 2d ago

Showcase I let an agent file and link my notes instead of building a retrieval pipeline for them

1 Upvotes

Most of my thinking happens out loud, so I stopped treating my notes as a pile to retrieve from.

I'm the developer of Zenod, so weigh this. It's built around one idea: an agent keeps the notes for me.

How it works today:

  • Each voice note or message is kept as I said it, in a dated log that is never edited
  • An agent files it under the projects and topics it touches and links it to what I said before
  • Any MCP client (Claude, ChatGPT, Cursor) can search it, and every answer points back to the note
  • The library is plain Markdown in my own Google Drive or GitHub

What I haven't solved: agents don't always save what matters unless I ask them to.

If you keep a personal knowledge base for agents, how do you catch it when the filing is wrong?

The first ten testers get 30 memories free, no card: https://zenod.dev/testers?s=P-0448


r/Rag • • 3d ago

Tools & Resources I built an open-source viewer for ParseBench 2 so you can find the exact PDF pages where 56 parsers fail and see why

2 Upvotes

I've been digging into ParseBench 2 (currently the most active document-parsing benchmark, covering 56 parsers across Text, Formatting, Tables, Charts, Layout, and Forms) to figure out which parsers actually hold up for RAG instead of just trusting summary scores.

Since LlamaIndex open-sourced the ParseBench 2 dataset, scorer, and raw parser outputs, I threw together a free, static viewer so it's easier to audit this stuff directly in the browser (no login, no backend, not affiliated with LlamaIndex or any parser vendor). You can inspect cost/latency vs. accuracy tradeoffs, drill into any page to see the exact scored errors, and compare any two parsers head-to-head on any dimension to find the exact pages where one outperforms the other.

Built it quickly in a day, so expect rough edges. Please report any bugs you hit (I already know of a few and will keep fixing them), and additional suggestions are very welcome.


r/Rag • • 3d ago

Discussion Benchmarked 9 models on extracting data out of documents, scoring wrong values as worse than missing ones. Results and open bench+dataset

5 Upvotes

Disclosure first: I run a small document extraction API (Refractr), and its model is row 4 below. I built the benchmark, so don't take my word for anything: the documents, answer key and scorer are all open. None of the benchmark data appears in my training dataset.

Why I made it: When you extract fields or metadata from documents at ingestion, a missing value is annoying but visible. A wrong value that looks plausible goes into your store and every query downstream trusts it. Most extraction benchmarks only count right answers, so a model that guesses confidently looks the same as one that leaves a blank. I wanted a score that separates them. A correctly extracted field gives one point, a missed extraction is 0 points, a confidently wrong extraction gives -1.

About one field in seven is deliberately not in its document. The right answer there is null, which is how made-up values get caught.

Setup: 260 documents (invoices, receipts, payslips, bank statements, court opinions, SEC filings, Federal Register rules, US government forms), each with a JSON schema. Models get the original PDF, image or HTML, so parsing is part of the test. Same published prompt for every model, one request, first reply scored. Scoring is deterministic, no LLM judge.

SystemScore95% intervalMedian time per doc
Gemini 3.8 Flash94.692.9–96.17.2 s
DeepSeek v4.1 Flash93.291.3–94.93.0 s
Mistral Large 491.489.4–93.227.4 s
Refractr (mine)89.487.2–91.40.7 s
GPT-5.4 mini86.283.9–88.52.2 s
Ministral 8B85.583.0–87.73.3 s
Llama 3.1 8B73.970.5–77.20.5 s
NuExtract 2.0 (2B)58.654.7–62.41.4 s
GLiNER2 (base)24.220.8–27.60.5 s

Two things I'd like to hear from people here:

  1. How do you catch wrong extracted values in your own pipeline today? Confidence scores, a second model, sampling by hand?
  2. If you run it on your own stack (parser plus model), I'll add the result. It takes any OpenAI-style endpoint or a Python function, so a full pipeline counts as a system.

Repo: https://github.com/Refractr-io/DocFieldBench

Write-up: https://refractr.io/docfieldbench


r/Rag • • 3d ago

Discussion My RAG pipeline was working fine until I realised its knowledge was six months old

17 Upvotes

I have been working on a RAG pipeline that pulls information from documentation sites, product pages and company websites.

The initial setup was pretty straightforward. Scrape the pages, convert everything to Markdown, chunk the content, generate embeddings and push it into the vector database.
The problem showed up later.

A product changed its pricing, documentation got updated and a few pages were removed. But the vector database was still serving the old information.
I initially thought about running a complete re-index every week, but that feels wasteful when only a small percentage of the content actually changes.
I'm now looking at a setup where I can monitor the original pages, identify what changed and only reprocess the relevant content.

I have been looking at Firecrawl, Apify and context.dev for different parts of this workflow.

Context.dev caught my attention because it has scraping and website monitoring under the same API, so I wouldn't need to build another integration just for change detection.

Still figuring out the best architecture here.

For people running production RAG systems, how are you keeping your external knowledge base fresh without rebuilding the entire index every time?


r/Rag • • 3d ago

Discussion Does agentic retrieval actually beat a simple reranker?

6 Upvotes

Abdullahi Dattijo tested four retrieval approaches on 298 questions over 2,000 arXiv papers. The metric was whether the target paper appeared in the top five results, not whether the system generated a correct answer.

Dense retrieval reached 95.6% Recall@5. Reranking the top 50 candidates with a small cross-encoder reached 97.0%. Multiquery rewriting scored 93.6%, while an agent allowed up to three searches reached 96.9%.

The agent gained about 1.2 percentage points over dense retrieval, with a 95% bootstrap interval from −0.8 to +3.5. Its benefit remained uncertain. The reranker’s recall gain was also uncertain, but its ranking improvement was clearer. MRR@10 rose from 0.895 to 0.959, the only improvement whose interval excluded zero.

The useful detail is what extra searches gained and lost. Per run, the agent recovered 7–8 papers dense retrieval missed, but lost 3–5 papers it had already found. Some rewrites introduced unsupported terminology that pulled retrieval toward distractors. Extra searches could undo a good first result.

Cost wasn’t the main problem. The agent averaged roughly $0.00009 per query. Both agentic retrieval and reranking took around 1.4–1.5 seconds on the test setup. The local reranker incurred no LLM API fees, though compute still costs something.

The limitation matters. Questions were generated from abstracts, and retrieval used titles and abstracts rather than full papers. Dense retrieval already missed only 13 questions, leaving little headroom.

For anyone testing this in production, count both recovered hits and previously correct hits that get displaced. Inspect the rewritten queries on those regressions, then compare the net improvement against a reranker baseline. That gives you a concrete reason to decide whether the agent loop is worth keeping.


r/Rag • • 3d ago

Discussion Unsiloed's $7.5M seed round, and why the document layer comes before retrieval

0 Upvotes

We're Unsiloed, and we've just announced a $7.5M seed round led by Felicis to make documents as computable as databases.

A lot of RAG problems we see start before retrieval. A table gets chunked as flat text and loses which number belongs to which row and column, or a two-column page gets read in the wrong order, and no amount of reranking fixes that. We focus on that step. Documents come back as clean markdown or as structured JSON against your schema, with the page location for each value, so an answer can point at the exact place in the source.

If you're dealing with scanned tables, filings or messy multi-column PDFs in your pipeline, we're happy to talk through what works.

Announcement and launch video · TechCrunch coverage


r/Rag • • 3d ago

Tools & Resources Looking for testers: run and manage local AI agents, models and GPU servers with Yorath

1 Upvotes

Hey everyone! 👋

We’ve been building Yorath, an open-source operating environment for running AI models and agents on your own infrastructure, and we’re looking for early users willing to test it and give honest feedback.

What it does:

Yorath lets you run local AI models, connect agents such as Codex, OpenClaw and Hermes, manage multiple machines with GPUs, and control access to models and sensitive data from one place.

Think of it as an operating layer for your AI infrastructure: models, agents, compute, permissions and audit logs under one roof.

Why we built it:

Running one local model is relatively easy. Managing several models, agents and GPU servers securely is not.

You quickly end up with separate dashboards, API keys, containers, network rules and logs. More importantly, access restrictions are often implemented inside prompts — where they can be ignored or bypassed.

We wanted a system that installs quickly, keeps the infrastructure understandable, and enforces data-access rules outside the model.

Tech stack and architecture:

  • Containerized deployment with Docker
  • Local and OpenAI-compatible model endpoints
  • OpenAI-compatible API for applications and agents
  • Isolated containers for connected agents
  • gVisor sandbox for the built-in Developer Agent
  • Gateway-level access policies and data classification
  • Multi-node GPU cluster support
  • Web interface for models, agents, nodes and policies
  • Central journal for model and agent calls

Key features:

  • One-command installation: curl -fsSL https://get.yorath.io | sh
  • Automatically detects the server’s hardware and selects an appropriate local model
  • Installs, runs and monitors local models
  • Works with Codex, OpenClaw, Hermes, Ollama and OpenRouter
  • Provides a single endpoint regardless of how many GPU servers are behind it
  • Adds new servers to a cluster using a one-time join token
  • Supports mixed hardware across cluster nodes
  • Enforces access to data, tools and models before a request reaches the model
  • Blocks calls that violate data-classification policies
  • Runs connected agents in isolated, non-root containers
  • Lets you disable internet access for connected agents
  • Records calls in a central audit journal
  • Sends no telemetry

Current requirements:

  • Ubuntu 24.04 on x86-64
  • Minimum 4 vCPU and 8 GB RAM
  • A GPU for practical use with Codex and connected agents
  • Docker 29+ for building from source

What we’re looking for:

  • Feedback on the installation and setup experience
  • Testing on different server and GPU configurations
  • Real-world testing with Codex, OpenClaw, Hermes and custom agents
  • Multi-node cluster testing
  • Security and policy edge cases
  • Bug reports accompanied by yorath diag
  • Feedback on what’s confusing, missing or unnecessarily complicated

If something breaks, yorath doctor runs the initial checks, while yorath diag creates an archive that can be attached to a bug report.

Repository: https://github.com/yorath-os/yorath

Website and documentation: https://yorath.io

Yorath is still early, so we’re particularly interested in discovering the rough edges that only appear outside our own environment.

Happy to answer any questions!

Yorath Community is open-source under the Apache License 2.0. Core agent registry, data classification, access policies and the audit journal are included in the Community edition. No telemetry.


r/Rag • • 4d ago

Tools & Resources In RAG, what goes in determines what comes out. Prajnya: a local ingestion engine for scientific PDFs

8 Upvotes

I'm a researcher with background in Developmental biology, biochemistry and biophysics. I kept finding that retrieval failures started at ingestion: figures dissolved into prose, tables flattened, captions merged with body text, reference lists polluting search. So I built the ingestion layer first. I am now building the RAG part and after having dealt with many issues, I feel a bit confident in sharing this project with you all. Prajnya is open source (Apache-2.0) and fully offline by default.

How it works

  1. A CPU pre-pass classifies each document (paper, book or chapter) and every page.
  2. Plain prose comes straight from the PDF's text layer. Scans and visually complex pages go to a local vision model.
  3. A layout model finds figures and tables, and each is cut out and handled as its own object.
  4. Everything lands in one JSON per document, with a documented schema.

What you get, and why it matters for retrieval

  • Evidence objects, not chunks. Figures are cropped and captioned with stable IDs, and panels are grouped under the label a reader cites ("Fig. 3"). Tables carry a reconstructed grid, so numbers are searchable while the image stays available to a multimodal model. Tables that run onto the next page keep their headers.
  • Provenance. Every item records its page and how it was read (text layer, model read or repaired read), because those differ in reliability.
  • Citations linked conservatively. In-text citations, numbered and author-year, link to the reference list. A wrong link is worse than none, so ambiguous ones stay unlinked.
  • Metadata verified, not believed. The model's reading of the title page, the PDF text and an optional authority (Zotero, or Crossref/OpenAlex sent only the DOI or title) are reconciled, with the source of each value recorded.

What it doesn't do (yet)

  • No chunking, embedding or retrieval. That’s the next layer, Shastrasaara (in development), which builds on these evidence objects.
  • PDFs only. No DOCX, HTML or slides.
  • It isn’t one-click. Local mode needs llama.cpp and the model weights. Cloud models are optional, per role.
  • It doesn’t race to be the fastest. It aims for the most faithful data, and would rather take longer than hand you a wrong table, caption or citation.

How it compares

  • NotebookLM: for public papers and quick cited Q&A, use it. But your PDFs leave your machine, and you get a chat, not structured data you can index your own way.
  • GROBID: the standard for metadata and references, fast on CPU. Its weak spot is pages where figures and tables break the reading order. Prajnya is built around those pages.
  • Docling, Marker and MinerU: solid general converters. Prajnya is narrower and specific to scientific documents, aimed at what a literature RAG needs to cite and trace.

Repo: Prajnya-Academic-PDF-Processor

Question for this sub: how do you index figures and tables in scientific RAG? Captions, images, both, or one-hop expansion from the text that cites them?

Edit (21 Nov 2026): benchmark results

I benchmarked Prajnya v6.7.0 against leading open-source PDF tools on 40 open biology papers. The papers were drawn at random after the protocol and scoring code were fixed. Each tool was scored against the publisher's own structured version of the article (the JATS XML in PubMed Central), so nothing was labelled by hand.

In short: Prajnya is strongest at finding figures and tables under their printed labels, reading table structure, and linking in-text mentions ("Fig. 3B") to the right figure or table. It trails on body-text cleanliness, author order and citation recall for author-year styles.

It is also the slowest of the tools measured, at about 3 minutes per paper on a single consumer GPU. That time goes into reading each table with a vision model, checking every figure against its printed label, and reconciling metadata from several sources, all on your own machine. A paper is ingested once and then queried many times, so I think a few minutes up front for evidence you can cite and trace is worth it.

Every result, losses included, plus the protocol and the code to rerun it: BENCHMARK.md

Also new in v6.7.0: figure and table mentions in the text now link to the figure or table they cite.


r/Rag • • 3d ago

Showcase Looking for feedback on my RAG "longevity assistant" app

1 Upvotes

Hi all,

I built a longevity assistant using Google's Embeddings model. It is really cool and I was able to easily get a RAG system going with thousands of published medical journals. I do have a software dev background, so it there was not much friction in getting this built.

Would love to get some feedback on your thoughts? I think I've got hallucinations down to near non-zero. Thank you!

URL: https://www.modernmedlife.com/longevity-assistant


r/Rag • • 4d ago

Discussion RAG over a product catalog: how do you stop it quoting yesterday's prices?

7 Upvotes

Shop assistants and product Q&A bots are a common RAG use case, but prices and stock change daily, sometimes hourly. If the price sits inside an embedded chunk, the bot can confidently quote an old one.

How are people handling the volatile fields:

- keep price and stock out of the chunks and fetch them at answer time?

- re-embed changed products nightly?

- something else, like a tool call to the store API?