xiry.

Development · GitHub repo

NLWeb

Open protocols and tools for conversational website interfaces via MCP

From the Xiry library · Updated 10 October 2026

Visit github.com ↗

NLWeb is an open source reference implementation by Microsoft for adding natural language querying to websites using Schema.org metadata. It natively acts as a Model Context Protocol (MCP) server, allowing both human users and AI agents to query structured site data through standardized natural language endpoints.

What it's for

  • Add a conversational search interface to a content or e-commerce website
  • Expose website content directly to AI agents via Model Context Protocol (MCP)
  • Convert natural language queries to SQL against enterprise data stores using Schema.org ontologies
  • Route user queries to appropriate LLMs based on cost and quality thresholds

Features

  • Native Model Context Protocol (MCP) server integration using standard 'ask' endpoints
  • JSON responses formatted using standardized Schema.org vocabularies
  • Pre-built AskAgent query agent with sample web UI and data ingestion tools
  • Multi-vector database support including Qdrant, Milvus, Postgres, and Azure AI Search
  • LLM provider support for OpenAI, DeepSeek, Gemini, Anthropic, and Hugging Face
  • DataFinder module for natural-language-to-SQL translation across enterprise schemas
  • ModelRouter and NLWebScorer for dynamic model selection and search result ranking

Pricing

Open source

Details

Type
GitHub repo
Category
Development
Topic
AI Protocols
Open source
Yes
Platforms
Linux, macOS, Windows, Self-hosted, API
For
Web developers and AI engineers building agent-ready websites and conversational search interfaces
Tags
mcp, python, schema org, ai agents, search, conversational ai

Why it was saved

  • @omarsar0, 23 May 2025: Lets developers turn any website into an AI app using standardized natural language APIs; NLWeb natively speaks MCP, so the same APIs serve both humans and agents. Positions itself as a foundational protocol layer for the AI web ('MCP is to NLWeb what HTTP is to HTML'), leveraging schema.org and RSS as the semantic layer.

Related

  • Composio

    Tooling and authentication layer connecting AI agents to external applications

  • RAGLite

    Lightweight Python toolkit for RAG using DuckDB or PostgreSQL

  • Scrapling

    Adaptive Python web scraping framework with anti-bot bypass and element relocation.

  • Firecrawl

    Turn websites and documents into clean, LLM-ready markdown or structured data.

  • awesome-llm-apps

    Open-source collection of runnable AI agent, skill, and RAG templates

  • Parse

    Turn any website into a structured REST API using natural language

Updated