xiry.

Development · Framework

"Easy Mode" Open Source AI Stack

An opinionated reference architecture for building full-stack open-source AI applications

From the Xiry library · Updated 11 October 2026

This reference framework outlines a battle-tested, developer-friendly tech stack for building self-hosted AI apps without vendor lock-in. It recommends specific open-source tools across every layer, from local LLMs and embeddings to vector databases and web frameworks. It is designed to help teams run AI workloads with full data privacy and minimal API costs.

The framework

Full stack, layer by layer:

  • LLMs: Llama 3.3, Mistral, Qwen (Qwen2.5), Phi 3, Gemma 2 — matching Claude/GPT on many benchmarks with better data privacy.
  • Embedding models: Nomic, BGE (BAAI), SBERT.net, Jina — accurate search and RAG without per-token payments or third-party API dependence.
  • Model access/deployment: Ollama — run dozens of state-of-the-art open models with one command.
  • Data and retrieval: PostgreSQL + pgvector + pgai — handles AI workloads better than specialized vector DBs.
  • Backend: FastAPI — fastest way to production-ready, scalable AI backends.
  • Frontend: Next.js — streaming, caching, real-time updates out of the box.
  • Evaluation: ??? (the author asks what's missing).

What it's for

  • Select an open-source toolchain for building a privacy-first AI application
  • Replace proprietary AI APIs like OpenAI with self-hosted models and embeddings
  • Architect a retrieval-augmented generation (RAG) backend using standard PostgreSQL

Key points

  • Open LLM tier recommendations including Llama 3.3, Mistral, Qwen 2.5, Phi 3, and Gemma 2
  • Open-source embedding model suggestions including Nomic, BGE, SBERT, and Jina
  • Local model serving and deployment via Ollama
  • Data storage and vector search handled natively via PostgreSQL, pgvector, and pgai
  • Backend API service layer powered by FastAPI
  • Frontend user interface layer powered by Next.js for streaming and caching

Pricing

Open source. Free and open-source components

Details

Type
Framework
Category
Development
Topic
Architecture Frameworks
Open source
Yes
Platforms
Self-hosted, Linux, macOS, Windows
For
Software engineers and indie developers building full-stack AI applications
Alternative to
OpenAI API, Pinecone, LangChain
Tags
ai, open source, llm, rag, pgvector, fastapi, nextjs, ollama

Why it was saved

  • @avthar, 16 December 2024: A one-glance, opinionated recipe for building AI apps without the OpenAI tax — free tools, full data control.

Compare

Related

  • awesome-llm-apps

    Open-source collection of runnable AI agent, skill, and RAG templates

  • Firecrawl

    Turn websites and documents into clean, LLM-ready markdown or structured data.

  • Hyperbrowser App Examples

    Open-source web apps and AI browser tools built with Hyperbrowser

  • ScrapeGraphAI

    Open-source Python library and API for prompt-based web scraping using LLMs

  • AgentUI

    Open-source, self-hosted chat interface for Agno agents and teams

  • Bytez

    Unified serverless API to discover and run open and closed AI models

Updated