# Multimodal AI Agent with Gemini 2.0 and Phidata

> Python recipe for an agent handling image, audio, video, and web search

A concise Python code snippet demonstrating how to build a multimodal agent using Google's Gemini 2.0 Flash model and the Phidata (phi) framework. It combines image, audio, and video inputs with live DuckDuckGo web search into a single unified query. It serves as a practical starter template for complex multimodal reasoning workflows.

## How it works

Full code (multimodal\_ai\_agent.py):

from phi.agent import Agent

from phi.model.google import Gemini

from phi.tools.duckduckgo import DuckDuckGo

from google.generativeai import upload\_file, get\_file

import time

\# 1. Initialize the Multimodal Agent

agent = Agent(model=Gemini(id="gemini-2.0-flash-exp"), tools=\[DuckDuckGo()\], markdown=True)

\# 2. Image Input

image\_url = "https://example.com/sample\_image.jpg"

\# 3. Audio Input

audio\_file = "sample\_audio.mp3"

\# 4. Video Input

video\_file = upload\_file("sample\_video.mp4")

while video\_file.state.name == "PROCESSING":

time.sleep(2)

video\_file = get\_file(video\_file.name)

\# 5. Multimodal Query

query = """

Combine insights from the inputs:

1. \*\*Image\*\*: Describe the scene and its significance.

2. \*\*Audio\*\*: Extract key messages that relate to the visual.

3. \*\*Video\*\*: Provide insights that connect with the image and audio context.

4. \*\*Web Search\*\*: Find the latest updates or events linking all these topics.

Summarize the overall theme or story these inputs convey.

"""

\# 6. Multimodal Agent generates unified response

agent.print\_response(query, images=\[image\_url\], audio=audio\_file, videos=\[video\_file\], stream=True)

## What it is for

- Analyze multimedia files and ground the analysis with live web search results
- Synthesize narratives across mixed media such as images, spoken audio, and recorded video
- Bootstrap a Python agent using Phidata and the Google Gemini API

## Key points

- Uses Google Gemini 2.0 Flash experimental model via Phidata
- Integrates DuckDuckGo search tool for real-time web retrieval
- Demonstrates video upload and processing status polling via Google's File API
- Executes a unified prompt accepting simultaneous image, audio, and video inputs
- Streams structured markdown responses directly to the terminal

## Pricing

Open source. Code template is free; relies on Gemini API which has free and paid tiers

## Details

- Type: Tip
- Category: [Development](https://xiry.app/topics/development.md)
- Topic: [AI Agents](https://xiry.app/topics/ai/ai-agents.md)
- Open source: yes
- Platforms: Code library
- For: Python developers and AI engineers building multimodal applications
- Alternative to: LangChain, LlamaIndex, CrewAI
- Tags: python, gemini, ai agents, phidata, multimodal, llm

## Why it was saved

- [@Saboo_Shubham_, 16 December 2024](https://x.com/Saboo_Shubham_/status/1868498039578747241): A working, copy-ready pattern for multimodal agents, kept because the code is complete and readable in the screenshot.

## Compare

- [Alternatives to LangChain](https://xiry.app/alternatives/langchain.md)
- [Alternatives to CrewAI](https://xiry.app/alternatives/crewai.md)
- [Alternatives to LlamaIndex](https://xiry.app/alternatives/llamaindex.md)

## Related

- [Skills.sh](https://xiry.app/resources/skills-sh): Open directory and CLI for discovering and installing AI agent skills (Web app, Free)
- [Odin AI](https://xiry.app/resources/odin-ai): Enterprise platform to build autonomous AI agents and automated business workflows. (Web app, Paid)
- [ai-agent-cognitivedriftt](https://xiry.app/resources/ai-agent-cognitivedriftt): Starter repository for building an autonomous social AI agent with Eliza (GitHub repo, Open source)
- [Botpress](https://xiry.app/resources/botpress): AI agent and chatbot platform for automated customer support and workflows. (Web app, Freemium)
- [Browser Use Box](https://xiry.app/resources/browser-use-box): Always-on cloud agent box with preinstalled browser and Telegram interface (Web app, Paid)
- [Browser Use WebUI](https://xiry.app/resources/browser-use-webui): Open-source graphical interface for running autonomous AI browser agents locally. (Web app, Open source)

Updated 11 October 2026. Page: https://xiry.app/resources/multimodal-ai-agent-with-gemini-2-0-and-phidata

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.