# gpt-prompt-engineer

> Automate prompt creation, testing, and distillation between large and small models.

A collection of Jupyter and Google Colab notebooks created by Matt Shumer for prompt engineering and model distillation. It automatically generates candidate system prompts, benchmarks them using an ELO rating system or task accuracy, and distills large models like Llama 3.1 405B or Claude Opus down to smaller models like Llama 8B or Claude Haiku.

Website: https://github.com/mshumer/gpt-prompt-engineer

## What it is for

- Distill large-model performance into a smaller 8B model using few-shot prompt generation
- Generate and rank candidate system prompts using an automated ELO competition
- Optimize prompts for classification tasks against defined true/false benchmarks
- Transfer Opus-level reasoning into lightweight Haiku prompts to reduce API costs

## Features

- Automated prompt generation using OpenAI GPT or Anthropic Claude models
- Tournament-style ELO rating system that evaluates and ranks prompt effectiveness
- Model distillation workflows (Opus to Haiku, Llama 3.1 405B to 8B)
- Synthetic test case and few-shot example generation
- Exportable prompts pre-formatted directly into Python code
- Optional logging and tracing integrations with Weights & Biases and Portkey

## Pricing

Open source. Free and open source; requires your own LLM API keys for generation

## Details

- Type: GitHub repo
- Category: [Development](https://xiry.app/topics/development.md)
- Topic: [Prompt Engineering](https://xiry.app/topics/ai/prompt-engineering.md)
- Open source: yes
- Platforms: Web
- For: AI developers, prompt engineers, and builders looking to lower LLM latency and API costs.
- Alternative to: DSPy, Promptfoo
- Tags: prompt engineering, distillation, llama, claude, gpt 4, colab notebook

## Why it was saved

- [@mattshumer_, 26 July 2024](https://x.com/mattshumer_/status/1816891950903279827): Lets anyone achieve 405B-level output quality in production at 8B prices by generating training-style examples and a system prompt from a single example.

## Related

- [Prompt Engineering (Google Whitepaper)](https://xiry.app/resources/prompt-engineering-google-whitepaper-2086): A comprehensive guide to prompt engineering techniques and LLM parameters by Google. (Article or course, Free)
- [3 Rules for Realistic AI Video Generation (Kling)](https://xiry.app/resources/3-rules-for-realistic-ai-video-generation-kling): Prompt formula and best practices for realistic AI video generation (Tip, Free)
- [50+ Nano Banana Pro Prompt Library](https://xiry.app/resources/50-nano-banana-pro-prompt-library): Curated collection of 50+ tested prompts for Nano Banana Pro image generation (Prompt, Free)
- [AI Voice Receptionist Prompt Template](https://xiry.app/resources/ai-voice-receptionist-prompt-template): Prompt framework to build structured conversational scripts for voice agents. (Prompt, Free)
- [Anthropic Console Prompt Generator](https://xiry.app/resources/anthropic-console-prompt-generator): Turn rough task descriptions into structured, production-ready AI prompts automatically (Web app, Freemium)
- [awesome-nanobanana-pro](https://xiry.app/resources/awesome-nanobanana-pro): Curated repository of prompts and styles for the Gemini Nano Banana model (GitHub repo, Open source)

Updated 10 October 2026. Page: https://xiry.app/resources/gpt-prompt-engineer

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.