Private & Self-Hosted AI Web Search: Integrating SearXNG with Open WebUI for Local RAG

Wed, 30 Sep 2026 10:00:00 +0200

Enabling real-time web browsing in local AI chat interfaces transforms large language models from static knowledge bases into dynamic research assistants. However, relying on default search providers like DuckDuckGo frequently breaks when running in containerized homelabs or cloud datacenters.

In this guide, I explain why datacenter search gets blocked and how to deploy a private SearXNG instance to provide reliable, rate-limit-free search grounding for Open WebUI.

The Problem with Datacenter Web Search

When Open WebUI queries search engines via DuckDuckGo:

DuckDuckGo aggressively rate-limits or outright blacklists IP ranges belonging to hosting providers, VPSs, and residential VPN endpoints (e.g., Cloudflare tunnels or VPS proxies).
The user sees generic RAG search error: 403 Forbidden or empty context cards.

The Solution: Self-Hosted SearXNG Meta-Search Engine

SearXNG is a privacy-respecting, hackable metasearch engine that aggregates results from more than 70 search services (Google, Bing, Wikipedia, DuckDuckGo, Reddit, GitHub, etc.) while obfuscating client requests.

1. Docker Compose Deployment

services:
  searxng:
    image: searxng/searxng:latest
    container_name: llm-searxng
    restart: unless-stopped
    environment:
      - SEARXNG_BASE_URL=http://llm-searxng:8080/
      - UWSGI_WORKERS=4
      - UWSGI_THREADS=4
    volumes:
      - /mnt/appdata/searxng:/etc/searxng
    networks:
      - frontend

Pro-Tip on Port Management: Avoid exposing SearXNG directly on host ports like 8080:8080 without checking for port collisions (ss -tulpn | grep 8080). In our homelab architecture, we keep SearXNG inside the internal Docker frontend bridge network, accessible only to Open WebUI!

2. Configuring Open WebUI

In Open WebUI:

Navigate to Admin Panel → Settings → Web Search.
Set Search Engine to searxng.

Set Searxng Query URL to:

`1`	`http://llm-searxng:8080/search?q=<query>`

Configure RAG extraction:
- RAG Top K: 5
- RAG Template: Include current timestamp and domain source citations.

Now, when you ask your model: “What happened in tech today?”, Open WebUI queries SearXNG across multiple search engines in parallel, extracts the raw HTML text, and feeds it directly into your model’s context window!

Solving LLM Tool Calling Incompatibilities: A Python Middleware Auth Proxy for LiteLLM & Cloud Run

Wed, 23 Sep 2026 10:00:00 +0200

When connecting modern web interfaces like Open WebUI or LiteLLM to remote inference engines on Google Cloud Run, you quickly run into two major technical hurdles:

Tool-choice Schema Conflicts: Open WebUI automatically injects tool_choice: "auto" and tools: [...] into every OpenAI-compatible request. Some specialized vLLM builds and distilled models reject these parameters with 400 Bad Request.
Dynamic OIDC Authentication: Securing Cloud Run behind IAM requires generating fresh Google OIDC identity tokens, which standard OpenAI client libraries do not natively handle.

Here is how I solved both problems by building a lightweight Python Auth & Filtering Proxy.

The Problem Breakdown

When an end-user chats in Open WebUI:

{
  "model": "qwen3.8-27b",
  "messages": [{"role": "user", "content": "Hello!"}],
  "tool_choice": "auto",
  "tools": []
}

If the backend vLLM instance does not have auto-tool-choice enabled for that specific model branch, vLLM returns: {"error": "tool_choice is not supported with the current parser"}.

Furthermore, hitting a private Cloud Run endpoint requires an Authorization: Bearer <GCP_OIDC_TOKEN> header that expires every 60 minutes.

The Solution: `llm-auth-proxy`

We placed a lightweight Python service between LiteLLM and Cloud Run:

`1`	`[Open WebUI] ──► [LiteLLM] ──► [llm-auth-proxy (Port 8000)] ──► [Google Cloud Run (vLLM)]`

The Proxy Implementation (`proxy.py`)

import os
import requests
from flask import Flask, request, Response
from google.auth.transport.requests import Request as GoogleRequest
from google.oauth2 import id_token

app = Flask(__name__)
CLOUD_RUN_URL = os.environ.get("CLOUD_RUN_URL")

def get_oidc_token(audience):
    auth_req = GoogleRequest()
    return id_token.fetch_id_token(auth_req, audience)

@app.route("/v1/chat/completions", methods=["POST"])
def proxy_completions():
    req_json = request.get_json(silent=True)
    
    # 1. Sanitize JSON payload
    if req_json and isinstance(req_json, dict):
        req_json.pop("tool_choice", None)
        req_json.pop("tools", None)
        
    # 2. Fetch fresh Google OIDC token
    token = get_oidc_token(CLOUD_RUN_URL)
    headers = {
        "Authorization": f"Bearer {token}",
        "Content-Type": "application/json"
    }
    
    # 3. Forward stream to Cloud Run
    resp = requests.post(
        f"{CLOUD_RUN_URL}/v1/chat/completions",
        json=req_json,
        headers=headers,
        stream=True
    )
    
    return Response(
        resp.iter_content(chunk_size=1024),
        status=resp.status_code,
        content_type=resp.headers.get("Content-Type", "application/json")
    )

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

Configuring LiteLLM

In litellm_config.yaml, LiteLLM routes queries to the local proxy instead of directly to GCP:

model_list:
  - model_name: Qwen-3.8-27B-BF16
    litellm_params:
      model: openai/qwen3.8-27b
      api_base: http://llm-auth-proxy:8000/v1
      api_key: proxy-key

This decoupling ensures that any quirks in frontend payloads are cleaned up before reaching production inference endpoints!

Homelab & Containers on Wojciech Marusiak IT Blog