feat: add Tavily search engine option to deep_research configuration#892
feat: add Tavily search engine option to deep_research configuration#892Tavily-FDE-Bot wants to merge 3 commits intomodelscope:mainfrom
Conversation
…: Add tavily as selectable engine
…: Add tavily as selectable engine
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request integrates Tavily as a new search engine into the deep_research project. This additive change provides users with an additional powerful web search option, complementing the existing engines without altering their functionality. The integration involved updating core search components, adding Tavily-specific logic, and modifying configuration files to enable its use. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for the Tavily search engine, integrating it into the existing search framework. This includes defining new data structures for Tavily requests and results, implementing the Tavily search logic, and updating various modules to recognize and utilize Tavily. Review feedback highlights a critical bug where the 'research_goal' field, though required by the schema, is incorrectly filtered out during request generation, and suggests adding it to the 'TavilySearchRequest' dataclass. Additionally, several comments recommend replacing 'print()' statements with proper logging calls ('logger.warning', 'logger.info') for improved log management and consistency within the new Tavily-related files.
| # Filter out keys not in TavilySearchRequest fields | ||
| valid_keys = {'query', 'num_results', 'search_depth', 'topic', | ||
| 'include_domains', 'exclude_domains'} | ||
| filtered = {k: v for k, v in search_request_d.items() if k in valid_keys} | ||
| return TavilySearchRequest(**filtered) |
There was a problem hiding this comment.
The current implementation filters out the research_goal field, which is defined as a required field in get_json_schema. This is inconsistent with other request generators and likely causes a bug, as research_goal is probably used by downstream components.
To fix this, you can remove this filtering logic. This will also require adding research_goal: Optional[str] = None to the TavilySearchRequest dataclass in ms_agent/tools/search/tavily/schema.py to avoid an error when instantiating the dataclass.
| # Filter out keys not in TavilySearchRequest fields | |
| valid_keys = {'query', 'num_results', 'search_depth', 'topic', | |
| 'include_domains', 'exclude_domains'} | |
| filtered = {k: v for k, v in search_request_d.items() if k in valid_keys} | |
| return TavilySearchRequest(**filtered) | |
| return TavilySearchRequest(**search_request_d) |
| def to_list(self) -> List[Dict[str, Any]]: | ||
| """Convert the search results to a list of dictionaries.""" | ||
| if not self.response or not self.response.get('results'): | ||
| print('***Warning: No search results found.') |
There was a problem hiding this comment.
Using print() for warnings in library code is not ideal as it writes to standard output and can't be easily controlled (e.g., silenced, redirected to a file, or formatted). It's better to use the application's logger.
You can add from ms_agent.utils.logger import get_logger and logger = get_logger() at the top of the file, then replace this print call with logger.warning().
| print('***Warning: No search results found.') | |
| logger.warning('No search results found.') |
| return [] | ||
|
|
||
| if not self.query: | ||
| print('***Warning: No query provided for search results.') |
There was a problem hiding this comment.
Similar to the comment above, it's better to use a logger for warnings instead of print().
| print('***Warning: No query provided for search results.') | |
| logger.warning('No query provided for search results.') |
|
|
||
| with open(file_path, 'r', encoding='utf-8') as f: | ||
| data = json.load(f) | ||
| print(f'Search results loaded from {file_path}') |
There was a problem hiding this comment.
For consistency and better log management, please use the application logger (logger.info) instead of print() for this informational message.
| print(f'Search results loaded from {file_path}') | |
| logger.info(f'Search results loaded from {file_path}') |
1 participant
Summary
Adds Tavily as a configurable search engine option in the
deep_researchproject, alongside the existing EXA, SERPAPI, and ARXIV engines. This is an additive (parallel) change — no existing functionality is modified.Changes
projects/deep_research/conf.yamlSEARCH_ENGINEblock fortavilyengine withtavily_api_key: $TAVILY_API_KEY, matching the style of existing exa/serpapi blocksprojects/deep_research/.env.exampleTAVILY_API_KEY=xxxplaceholder alongside existingEXA_API_KEYandSERPAPI_API_KEYentriesEnvironment variable changes
TAVILY_API_KEYreference in.env.exampleNotes for reviewers
core-search-frameworkmigration unit to add runtime support for thetavilyengine type inget_web_search_tool().SEARCH_ENGINEblock and commenting out the activearxivblock.🤖 Generated with Claude Code
Automated Review