xiry.

Development · GitHub repo

gpt-prompt-engineer

Automate prompt creation, testing, and distillation between large and small models.

From the Xiry library · Updated 10 October 2026

Visit github.com ↗

A collection of Jupyter and Google Colab notebooks created by Matt Shumer for prompt engineering and model distillation. It automatically generates candidate system prompts, benchmarks them using an ELO rating system or task accuracy, and distills large models like Llama 3.1 405B or Claude Opus down to smaller models like Llama 8B or Claude Haiku.

What it's for

  • Distill large-model performance into a smaller 8B model using few-shot prompt generation
  • Generate and rank candidate system prompts using an automated ELO competition
  • Optimize prompts for classification tasks against defined true/false benchmarks
  • Transfer Opus-level reasoning into lightweight Haiku prompts to reduce API costs

Features

  • Automated prompt generation using OpenAI GPT or Anthropic Claude models
  • Tournament-style ELO rating system that evaluates and ranks prompt effectiveness
  • Model distillation workflows (Opus to Haiku, Llama 3.1 405B to 8B)
  • Synthetic test case and few-shot example generation
  • Exportable prompts pre-formatted directly into Python code
  • Optional logging and tracing integrations with Weights & Biases and Portkey

Pricing

Open source. Free and open source; requires your own LLM API keys for generation

Details

Type
GitHub repo
Category
Development
Topic
Prompt Engineering
Open source
Yes
Platforms
Web
For
AI developers, prompt engineers, and builders looking to lower LLM latency and API costs.
Alternative to
DSPy, Promptfoo
Tags
prompt engineering, distillation, llama, claude, gpt 4, colab notebook

Why it was saved

  • @mattshumer_, 26 July 2024: Lets anyone achieve 405B-level output quality in production at 8B prices by generating training-style examples and a system prompt from a single example.

Related

All 18 in Prompt Engineering →

Updated