Skip to content
Harmonize IT

Case study · Generative AI

Case study: RAG-based internal assistant

We designed and built a RAG-based internal assistant that answers questions from a company's own documentation, with a link to every source. Built with Python, LangChain and a large language model, it respects document access rights and is evaluated on real user questions.

Last updated:

Context

A European technology company had years of knowledge spread across technical documentation, internal wikis, procedures and project archives. Finding the right answer meant knowing where to look, or whom to ask. New joiners depended heavily on a few experienced colleagues.

The company wanted an internal assistant: ask a question in plain language, get a reliable answer with the documents that support it.

Challenge

  • Trust. A general-purpose chatbot answers confidently even when it is wrong. Here, an answer without a verifiable source was not acceptable.
  • Heterogeneous content. PDFs, wiki pages, office documents and tickets, with duplicates and outdated versions.
  • Confidentiality. Not every employee may read every document. The assistant had to respect existing access rights.
  • From prototype to tool. An earlier demo had impressed, but nobody could say how often it was right, and it was not connected to the company’s systems.

Solution

Retrieval-augmented generation (RAG). Rather than retraining a model, we give it the relevant passages at question time. Documents are ingested, cleaned, split into coherent chunks and indexed as embeddings in PostgreSQL with pgvector, alongside a keyword index.

Hybrid search and re-ranking. Each question triggers a semantic search and a keyword search. Results are merged and re-ranked, which noticeably improves retrieval for internal jargon, product names and reference numbers.

Answers with sources, or no answer. The model is instructed to answer only from the retrieved passages and to cite them. When the documentation does not contain the answer, the assistant says so and suggests where to ask, instead of inventing one.

Access control at retrieval. Each chunk carries the permissions of its source document. The search is filtered by the user’s rights before anything reaches the model, so the assistant cannot reveal content the user could not open directly.

Evaluation from day one. With the client’s experts, we built a test set of real questions and expected answers. Every change to chunking, prompts or models is scored on accuracy, source grounding, latency and cost. Decisions are made on numbers.

Engineered for production. The pipeline is orchestrated with LangChain behind a FastAPI service, with a web interface integrated into the company’s intranet. Logging, usage monitoring and cost limits are built in. The LLM sits behind an abstraction, so the company can use OpenAI or Claude models, or move to a self-hosted model later.

Outcome

  • Employees get answers grounded in internal documentation, each with links to its sources.
  • The assistant respects the access rights of the original documents.
  • Answer quality is measured continuously on a test set of real questions, which made the go-live decision an informed one.
  • The design is independent of a single LLM provider.

This project combines our Generative AI & LLM Engineering and On-Demand R&D Expertise services. To understand the main design choice behind it, read RAG vs fine-tuning for enterprise.

Expertise behind this project

More case studies

EU public institutions

SAP integration for an EU institution

How we connected a Java and Angular business application to SAP and an e-signature service for a European public institution.

  • Java
  • Spring Boot
  • Angular
  • SAP

KYC and compliance

KYC platform for a global investment bank

Building the Java Spring Boot microservices and Angular front end of a know-your-customer platform for a global corporate and investment bank.

  • Java
  • Spring Boot
  • Angular
  • REST microservices

Have a similar challenge?

Tell us about your project, your contract need or the team you want to extend. We reply within two working days.