Welcome! If you’ve got a collection of documents—like PDFs, texts, or handbooks—and you want to build your own AI Adviser, Open WebUI (OWUI) with Retrieval Augmented Generation (RAG) is a simple, powerful tool to help you get started.

With OWUI, you don’t need to be a coding expert to start. You can upload your information documents right through the web interface. When someone asks your AI Adviser a question, OWUI will search your documents for the most relevant information and then use AI to generate a clear, easy-to-understand answer that combines both your files and the AI's own knowledge.

Here’s a case study to turn the OWUI into a CoLab Adviser.

To start, we need the following:

1. Set up an OWUI instance using Duke’s new GPTBuilder service

2. Documents in the form of text and markdown(PDFs work but require OCR and often need manual cleanup)

3. (in part 2) Set up a MCP server with data in a database

Set Up OWUI Instance

1. Go to Duke’s new GPTBuilder portal and log in with your Duke NetID and password.

2. Create either a personal workspace. Wait until its status turns “active.” After the status turn to active, click on the “alias” of the workspace to access your OWUI instance.

3. You’ll see a chat window and model selector in your workspace. You can select a model to start chatting.

Configure Document Setting

Important! If you’re using an external embedding model, you must complete this setting before uploading the knowledge files.

1. Click your profile icon at the bottom left and open the Admin panel.

2. Go to the settings tab and select “documents”. On this page, you’ll be able to modify the settings for RAG(what is RAG?).

PDF Extract Images (OCR): Enable if you’ll upload PDFs.
Text Splitter: character splitter (by words) or token splitter (1 token ≈ 0.75 characters).
- Chunk size: how large each text chunk is
- Chunk overlap: how much each chunk overlaps the next

Pro Tip: Smaller chunks (<300 tokens) improve precision; larger chunks (>800 tokens) widens retrieval search.

Embedding Model Engine: If you want Duke’s OpenAI embeddings, obtain an API key from AI Gateway.
Top K: Number of document chunks to retrieve. Higher K yields more creative but possibly less precise answers. Lower K gives tighter, more relevant matches.
RAG Template: The system instructions that guide how RAG combines retrieved content into an answer.

Create your adviser

Upload knowledges

1. In the left panel, click “Workspace,” then choose the “Knowledge” tab.

2. Click the “+” icon to add a new knowledge base. Give it a name and description.

3. Upload your .txt or .md files (cleaned of metadata, menus, broken links, etc.). If you need to turn your website or PDF into a .txt or .md file, try CoLab’s Raven Read Service.

Pro Tip: Clean and accurate data is key to a successful RAG session. Remove any metadata, menu bar content, broken images, and links.

For example, using Raven Read, we turned the location and equipment page into markdown file below, but there is some metadata information from the website. We need to do a clean up.

After removing the metadata from the website, we have a brief overview of the locations and equipment at the CoLab. This data-cleaning process will also help you identify what data is missing from the documents.

Pro Tip: Break large documents into smaller files by topic. This helps the LLM find precise answers.

Create Custom Models

1. Go to the “Models” tab and click the “+” icon.

2. Fill in these fields:

Model size (smaller = faster, less accurate; larger = slower, more accurate, GPT 4.1-mini is a good starting point)
System prompt (instructions, response steps, format)
Knowledge base (select the one you just created)
Capabilities (e.g., citation formatting)

Example System Prompt:

You are an AI Adviser. Your primary goal is to help users by answering questions strictly based on the context they provide and using the native tool that is available to you. Follow these rules:

Context first
• Always read and internalize the user’s context before answering.
• Quote or reference the relevant snippet of the user’s context when it informs your answer.
Clarify when needed
• If the context is incomplete, ask one clear follow-up question.
• Do not guess missing details.
Structure your advice
• Start with a brief summary of your understanding.
• Provide 2–5 actionable recommendations or explanations.
• Conclude with “Next steps” or “Further questions.”
Tone & style
• Professional, friendly, and concise.
• Use bullet points or numbered lists—no long paragraphs.
Handle edge cases
• If you cannot answer from the given context, say:
“I’m sorry, but I need more information to help with that.”
• Never invent facts or quote sources you haven’t actually seen.
Now await the user’s context. Once they’ve provided it, apply these guidelines to deliver clear, context-driven advice.”

You can adjust or expand any of the numbered rules to better fit your domain (legal, technical, medical, etc.).