Creating Custom GPTs with Open Source Tools

Bypassing Proprietary Custom GPTs

OpenAI's launch of 'Custom GPTs' allowed users to create specialized AI assistants containing customized instructions, uploaded documents, and API capabilities. However, these custom assistants are locked inside paid subscription layers, and any uploaded data is stored on external cloud infrastructure.

For developers wanting absolute control or hosting sensitive company assets, building **open-source Custom GPTs** is the ideal solution. By combining open-source chat frameworks with local model runners, you can deploy custom, document-aware AI agents for free. This guide introduces the best self-hosted frontends and walks you through setting up custom system prompts and document RAG pipelines locally.

Key Takeaway: Open-source Custom GPTs give you a private, subscription-free alternative to ChatGPT Plus, allowing you to run custom AI tools offline on your own hardware.

The Best Open-Source Chat Frontends

To replicate the ChatGPT interface, you need a high-quality frontend client that can talk to your LLM runners. Here are the top open-source options:

• Open WebUI: The gold standard. It clones the OpenAI dashboard design and features native user accounts, chat sharing, model pulling, and built-in vector search for local documents.
• LibreChat: A highly secure, enterprise-grade frontend. LibreChat connects to multiple providers (OpenAI, Anthropic, Ollama, Bedrock) and supports plugins, custom presets, and multi-user authentication.
• LobeChat: A modern, glassmorphic UI that focuses on agent marketplaces. LobeChat features text-to-speech integrations and a modular plugin engine.

Core Architectural Components

A self-hosted Custom GPT architecture requires three decoupled modules:

Configuring System Prompts & Behaviors

The core of a Custom GPT is its **System Prompt**—the set of permanent instructions that dictate how the model behaves. Unlike conversational prompts, system prompts define formatting styles, role models, and boundaries.

To create a custom software engineering assistant, load Llama-3 in Open WebUI and apply this system instruction:

You are an expert React and TypeScript architect.
Guidelines:
1. Only return clean, functional code blocks without verbose explanations.
2. Prefer React Functional Components using Tailwind CSS.
3. Add type definitions to all function parameters.
4. If a query is ambiguous, ask for clarification before writing code.

Open WebUI Pipelines: Dynamic Python Middleware

One of the most powerful features of Open WebUI is the **Pipelines** framework. Pipelines allow you to inject custom Python scripts directly into the message routing path. You can use this to execute custom calculations, filter outputs, or route queries to different models based on intent:

# Example of a simple Open WebUI filter pipeline
class Pipeline:
    def __init__(self):
        pass

    async def inlet(self, body: dict, user: dict) -> dict:
        # Clean user prompt input before sending to LLM
        body["messages"][-1]["content"] = body["messages"][-1]["content"].strip()
        return body

    async def outlet(self, body: dict, user: dict) -> dict:
        # Intercept output, append local warning flags
        body["messages"][-1]["content"] += "\n\n[Local Audit Approved]"
        return body

Multi-User Security & LDAP Integration

For office team configurations, Open WebUI supports OAuth and LDAP/Active Directory authentication, ensuring only authorized personnel access internal models. You can restrict model visibility, define role permissions (Admin, User, Pending), and track user chat histories securely in a PostgreSQL backend database.

Scale Deployments: Kubernetes Architecture

If you serve a large engineering team, you can deploy Open WebUI on a Kubernetes cluster, mounting persistent volumes (PV) for databases and utilizing Traefik or Nginx load balancers to route websocket traffic cleanly:

# Kubernetes persistent volume claim snippet
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: open-webui-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Step-by-Step Local Deployment: Open WebUI

The easiest way to host Open WebUI alongside Ollama is utilizing Docker. Run this simple container command to deploy the unified stack:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Open your browser and navigate to http://localhost:3000. Create a local admin account, select your loaded Ollama model, and start chatting. To add custom files, click the **Documents** tab and drag and drop your folders. The system will automatically chunk, embed, and index them for conversational retrieval.

Building a Custom Chatbot in Python using LangChain

If you need to build custom GPT logic inside a custom application instead of a pre-made frontend, you can orchestrate the loop in Python using LangChain and Ollama:

from langchain_community.llms import Ollama
from langchain_core.prompts import ChatPromptTemplate

# Initialize local model
llm = Ollama(model="llama3")

# Define system prompt boundaries
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful customer support agent. Answer questions concisely."),
    ("user", "{input}")
])

# Chain and invoke
chain = prompt | llm
response = chain.invoke({"input": "How do I reset my local database connection?"})
print(response)

Conclusion and Next Steps

Building Custom GPTs with open-source tools is a cost-effective, secure strategy for modern AI development. By combining Open WebUI for the frontend interface and Ollama for local model execution, you replicate premium ChatGPT features privately. Deploy the Docker stack locally, set up custom system prompts, and begin indexing documents to create specialized virtual workspaces.