Unifero AI Open Deep Search

Unifero AI Open Deep Search: A Powerful Tool for AI-Powered Searches

Unifero AI Open Deep Search is a lightweight yet powerful search tool designed to work seamlessly with AI agents. It enables deep web search and retrieval, optimized for use with Hugging Face's ecosystem. This tool performs on par with closed-source search alternatives on single-hop queries and excels on multi-hop queries, making it a versatile choice for various search needs.

Benefits

Unifero AI Open Deep Search offers several key advantages:

• Semantic Search: Uses advanced techniques to provide accurate and relevant results.
• Two Modes of Operation: Choose between Default Mode for quick searches and Pro Mode for more in-depth and accurate results.
• Optimized for AI Agents: Works seamlessly with SmolAgents like CodeAgent.
• Fast and Lightweight: Designed for speed and efficiency with minimal setup.
• Extensible: Easily configurable to work with different models and APIs.

Use Cases

Unifero AI Open Deep Search can be used in various scenarios:

• Quick Information Retrieval: Use Default Mode for fast, straightforward searches.
• Complex Research: Utilize Pro Mode for detailed, multi-hop queries that require comprehensive data gathering and cross-reference verification.
• Integration with AI Agents: Enhance the capabilities of AI agents like CodeAgent by integrating Unifero AI Open Deep Search for advanced search functionalities.

Installation

To install Unifero AI Open Deep Search, run the following commands:

pip install -e . # or use: uv pip install -e .pip install -r requirements.txt # or use: uv pip install -r requirements.txt

Note: You must havetorchinstalled. Usinguvinstead of regularpipcan make the installation process smoother.

Using PDM (Alternative Package Manager)

You can also use PDM as an alternative package manager for Unifero AI Open Deep Search. PDM is a modern Python package and dependency manager that supports the latest PEP standards. To install PDM and set up Unifero AI Open Deep Search, follow these steps:

# Install PDM if you haven't alreadycurl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 -# Initialize a new PDM projectpdm init# Install Unifero AI Open Deep Search and its dependenciespdm install# Activate the virtual environmenteval "$(pdm venv activate)"

PDM offers several advantages, including lockfile support for reproducible installations, PEP 582 support (no virtual environment needed), fast dependency resolution, and built-in virtual environment management.

Setup

To set up Unifero AI Open Deep Search, follow these steps:

1. Choose a Search Provider:

• Option 1: Serper.dev: Get free 2500 credits and add your API key.
• Visit https://serper.dev to retrieve your API key and store it as an environment variable:

export SERPER_API_KEY='your-api-key-here'

• Option 2: SearXNG: Use a self-hosted or public SearXNG instance.
• Specify the SearXNG instance URL when initializing Unifero AI Open Deep Search.
• Optionally provide an API key if your instance requires authentication:

export SEARXNG_INSTANCE_URL='https://your-searxng-instance.com'export SEARXNG_API_KEY='your-api-key-here' # Optional

2. Choose a Reranking Solution:

• Quick Start with Jina: Sign up at https://jina.ai.
• Self-hosted Option: Set up your own reranking solution.

3. Set up LiteLLM Provider:

• Choose a provider from the list including OpenAI, Anthropic, Google (Gemini), OpenRouter, HuggingFace, Fireworks, and many more!
• Set your chosen provider's API key as an environment variable:

export <PROVIDER>_API_KEY='your-api-key-here' # e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY

• For OpenAI, you can also set a custom base URL (useful for self-hosted endpoints or proxies):

export OPENAI_BASE_URL='https://your-custom-openai-endpoint.com'

• You can set default LiteLLM model IDs for different tasks:

# General default model (fallback for all tasks)export LITELLM_MODEL_ID='openrouter/google/gemini-2.0-flash-001'# Task-specific modelsexport LITELLM_SEARCH_MODEL_ID='openrouter/google/gemini-2.0-flash-001' # For search tasksexport LITELLM_ORCHESTRATOR_MODEL_ID='openrouter/google/gemini-2.0-flash-001' # For agent orchestrationexport LITELLM_EVAL_MODEL_ID='gpt-4o-mini' # For evaluation tasks

• When initializing Unifero AI Open Deep Search, you can specify your chosen model using the provider's format (this will override the environment variables):

search_agent = OpenDeepSearchTool(model_name="provider/model-name") # e.g., "anthropic/claude-3-opus-20240229", 'huggingface/microsoft/codebert-base', 'openrouter/google/gemini-2.0-flash-001'

Usage

You can use Unifero AI Open Deep Search independently or integrate it with SmolAgents for enhanced reasoning and code generation capabilities.

Using Unifero AI Open Deep Search Standalone

from opendeepsearch import OpenDeepSearchToolimport os# Set environment variables for API keysos.environ["SERPER_API_KEY"] = "your-serper-api-key-here" # If using Serper# Or for SearXNG# os.environ["SEARXNG_INSTANCE_URL"] = "https://your-searxng-instance.com"# os.environ["SEARXNG_API_KEY"] = "your-api-key-here" # Optionalos.environ["OPENROUTER_API_KEY"] = "your-openrouter-api-key-here"os.environ["JINA_API_KEY"] = "your-jina-api-key-here"# Using Serper (default)search_agent = OpenDeepSearchTool(model_name="openrouter/google/gemini-2.0-flash-001",reranker="jina")# Or using SearXNG# search_agent = OpenDeepSearchTool(# model_name="openrouter/google/gemini-2.0-flash-001",# reranker="jina",# search_provider="searxng",# searxng_instance_url="https://your-searxng-instance.com",# searxng_api_key="your-api-key-here" # Optional# )if not search_agent.is_initialized:search_agent.setup()query = "Fastest land animal?"result = search_agent.forward(query)print(result)

Running the Gradio Demo

To try out Unifero AI Open Deep Search with a user-friendly interface, simply run:

python gradio_demo.py

This will launch a local web interface where you can test different search queries and modes interactively. You can customize the demo with command-line arguments:

# Using Serper (default)python gradio_demo.py --model-name "openrouter/google/gemini-2.0-flash-001" --reranker "jina"# Using SearXNGpython gradio_demo.py --model-name "openrouter/google/gemini-2.0-flash-001" --reranker "jina" \--search-provider "searxng" --searxng-instance "https://your-searxng-instance.com" \--searxng-api-key "your-api-key-here" # Optional

Available options:*--model-name: LLM model to use for search*--orchestrator-model: LLM model for the agent orchestrator*--reranker: Reranker to use (jinaorinfinity)*--search-provider: Search provider to use (serperorsearxng)*--searxng-instance: SearXNG instance URL (required if usingsearxng)*--searxng-api-key: SearXNG API key (optional)*--serper-api-key: Serper API key (optional, will use environment variable if not provided)*--openai-base-url: OpenAI API base URL (optional, will use OPENAIBASEURL env var if not provided)

Integrating with SmolAgents & LiteLLM

from opendeepsearch import OpenDeepSearchToolfrom smolagents import CodeAgent, LiteLLMModelimport os# Set environment variables for API keysos.environ["SERPER_API_KEY"] = "your-serper-api-key-here" # If using Serper# Or for SearXNG# os.environ["SEARXNG_INSTANCE_URL"] = "https://your-searxng-instance.com"# os.environ["SEARXNG_API_KEY"] = "your-api-key-here" # Optionalos.environ["OPENROUTER_API_KEY"] = "your-openrouter-api-key-here"os.environ["JINA_API_KEY"] = "your-jina-api-key-here"# Using Serper (default)search_agent = OpenDeepSearchTool(model_name="openrouter/google/gemini-2.0-flash-001",reranker="jina")# Or using SearXNG# search_agent = OpenDeepSearchTool(# model_name="openrouter/google/gemini-2.0-flash-001",# reranker="jina",# search_provider="searxng",# searxng_instance_url="https://your-searxng-instance.com",# searxng_api_key="your-api-key-here" # Optional# )model = LiteLLMModel("openrouter/google/gemini-2.0-flash-001",temperature=0.2)code_agent = CodeAgent(tools=[search_agent], model=model)query = "How long would a cheetah at full speed take to run the length of Pont Alexandre III?"result = code_agent.run(query)print(result)

ReAct agent with math and search tools

from opendeepsearch import OpenDeepSearchToolfrom opendeepsearch.wolfram_tool import WolframAlphaToolfrom opendeepsearch.prompts import REACT_PROMPTfrom smolagents import LiteLLMModel, ToolCallingAgent, Toolimport os# Set environment variables for API keysos.environ["SERPER_API_KEY"] = "your-serper-api-key-here"os.environ["JINA_API_KEY"] = "your-jina-api-key-here"os.environ["WOLFRAM_ALPHA_APP_ID"] = "your-wolfram-alpha-app-id-here"os.environ["FIREWORKS_API_KEY"] = "your-fireworks-api-key-here"model = LiteLLMModel("fireworks_ai/llama-v3p1-70b-instruct", # Your Fireworks Deepseek modeltemperature=0.7)search_agent = OpenDeepSearchTool(model_name="fireworks_ai/llama-v3p1-70b-instruct", reranker="jina") # Set reranker to "jina" or "infinity"# Initialize the Wolfram Alpha toolwolfram_tool = WolframAlphaTool(app_id=os.environ["WOLFRAM_ALPHA_APP_ID"])# Initialize the React Agent with search and wolfram toolsreact_agent = ToolCallingAgent(tools=[search_agent, wolfram_tool],model=model,prompt_templates=REACT_PROMPT # Using REACT_PROMPT as system prompt)# Example query for the React Agentquery = "What is the distance, in metres, between the Colosseum in Rome and the Rialto bridge in Venice"result = react_agent.run(query)print(result)

Search Modes

Unifero AI Open Deep Search offers two distinct search modes to balance between speed and depth:

Default Mode

• Uses SERP-based interaction for quick results
• Minimal processing overhead
• Ideal for single-hop, straightforward queries
• Fast response times
• Perfect for basic information retrieval

Pro Mode

• Involves comprehensive web scraping
• Implements semantic reranking of results
• Includes advanced post-processing of data
• Slightly longer processing time
• Excels at:
• Multi-hop queries
• Complex search requirements
• Detailed information gathering
• Questions requiring cross-reference verification

Acknowledgments

Unifero AI Open Deep Search is built on the shoulders of great open-source projects:*Various Open-Source Libraries: Enhancing search and retrieval functionalities.

Citation

If you useUnifero AI Open Deep Searchin your works, please cite it using the following BibTex entry:

@misc{alzubi2025opendeepsearchdemocratizing,title={Open Deep Search: Democratizing Search with Open-source Reasoning Agents},author={Salaheddin Alzubi and Creston Brooks and Purva Chiniya and Edoardo Contente and Chiara von Gerlach and Lucas Irwin and Yihan Jiang and Arda Kaz and Windsor Nguyen and Sewoong Oh and Himanshu Tyagi and Pramod Viswanath},year={2025},eprint={2503.20201},archivePrefix={arXiv},primaryClass={cs.LG},url={https://arxiv.org/abs/2503.20201},}

Contact

For questions or collaborations, open an issue or reach out to the maintainers.