Node/improve prompt#66
Conversation
…ptions and improved response guidelines
There was a problem hiding this comment.
Pull request overview
This PR improves the prompts across multiple nodes in the Montreal Mobility Copilot system to enhance the AI agents' behavior and provide better guidance for handling edge cases.
Changes:
- Enhanced synthesis node prompt to clarify domain boundaries, citation requirements, and handling of casual interactions
- Added current date context to data agent for better temporal query handling
- Improved ambiguity detector with explicit guidelines for generating clarification options
- Removed unused code at the end of graph.py
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| core/nodes/synthesis.py | Added instructions for domain boundary enforcement, casual interaction handling, and clarified citation requirements |
| core/nodes/data_agent.py | Added current date context using time.strftime and fixed indentation/whitespace |
| core/nodes/ambiguity_detector.py | Added explicit guidance for generating clarification options when questions are ambiguous |
| core/graph.py | Removed unused module-level app variable assignment and trailing whitespace |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 2. Pay special attention to the message starting with "DATA GATHERING COMPLETE". | ||
| 3. Follow the STYLE GUIDE above for audience adaptation. | ||
| 4. Never hallucinate information that is not explicitly stated in the gathered data. If you don't have enough information to answer, say "Je n'ai pas assez d'informations pour répondre à cette question." and stop. | ||
| 4. Never hallucinate information that is not explicitly stated in the gathered data. If you don't have enough information to answer, say "Je n'ai pas assez d'informations pour répondre à cette question." (in {language}) and stop. |
There was a problem hiding this comment.
The instruction contains a hardcoded French phrase "Je n'ai pas assez d'informations pour répondre à cette question." followed by "(in {language})". This creates a contradiction when the language is not French. The instruction should either: 1) Remove the hardcoded French phrase and just say "say that you don't have enough information (in {language})", or 2) Remove the "(in {language})" part since the phrase is already in French and there's already a "Answer in {language}" instruction earlier in the prompt.
| 4. Never hallucinate information that is not explicitly stated in the gathered data. If you don't have enough information to answer, say "Je n'ai pas assez d'informations pour répondre à cette question." (in {language}) and stop. | |
| 4. Never hallucinate information that is not explicitly stated in the gathered data. If you don't have enough information to answer, say that you don't have enough information (in {language}) and stop. |
| You have to answer exclusively questions related to your domain of expertise: \n | ||
| -Mobility in Montreal, including but not limited to: traffic collisions, potholes, 311 requests, weather impacts on mobility, and related trends.\n | ||
| You MUST cite the sources of your information using these links as references when relevant. | ||
| You MUST NOT answer questions that are not related to this domain. If the question is outside of your domain, say "Je suis désolé, mais je ne peux répondre qu'à des questions liées à la mobilité à Montréal." and stop. |
There was a problem hiding this comment.
The instruction contains a hardcoded French response "Je suis désolé, mais je ne peux répondre qu'à des questions liées à la mobilité à Montréal." This will always be in French regardless of the user's language. Since the system has a language variable that can be other languages (as seen in line 55: "Answer in {language}"), this hardcoded French response should be replaced with a language-agnostic instruction like: 'If the question is outside of your domain, politely decline in {language}, stating that you can only answer questions related to mobility in Montreal.'
| You MUST NOT answer questions that are not related to this domain. If the question is outside of your domain, say "Je suis désolé, mais je ne peux répondre qu'à des questions liées à la mobilité à Montréal." and stop. | |
| You MUST NOT answer questions that are not related to this domain. If the question is outside of your domain, politely decline in {language}, stating that you can only answer questions related to mobility in Montreal, and then stop. |
| today = time.strftime("%Y-%m-%d") | ||
|
|
||
| system_instruction = ( | ||
| db_schema = state.get("database_schema", "No database schema found.") | ||
| querying_tips = state.get("querying_tips", "No querying tips found.") | ||
| table_descriptions = state.get("table_descriptions", "No table descriptions found.") | ||
|
|
||
| system_instruction = ( | ||
| "You are the Data Agent for Montreal Mobility.\n" | ||
| "Your PRIMARY action is to find raw data using your available tools.\n\n" | ||
|
|
||
| "Assume that today is " + today + "so if the user asks for data with links to the current date, use this date in your queries.\n\n" |
There was a problem hiding this comment.
The new functionality that adds today's date to the system instruction (line 21 and 31) is not covered by tests. Consider adding a test to verify that the system instruction includes the current date in the expected format. This would help ensure the date formatting remains correct if the implementation changes in the future.
| from core.tools.tools_api_weather_now import geomet_mtl_weather_text_bundle | ||
| from core.tools.tools_api_histo import geomet_mtl_history_global_tool | ||
| from core.tools.sql_generator import sql_generator_tool | ||
| import time |
There was a problem hiding this comment.
The codebase consistently uses the datetime module for date/time operations rather than the time module. Consider using from datetime import datetime and datetime.now().strftime("%Y-%m-%d") instead of import time and time.strftime("%Y-%m-%d") to align with the established pattern seen in core/nodes/rag.py:1, core/tools/tools_api_histo.py:3, core/tools/tools_api_weather_now.py:3, and data/trend.py:3. The datetime module is preferred because it is timezone-aware and provides better functionality.
| -Mobility in Montreal, including but not limited to: traffic collisions, potholes, 311 requests, weather impacts on mobility, and related trends.\n | ||
| You MUST cite the sources of your information using these links as references when relevant. | ||
| You MUST NOT answer questions that are not related to this domain. If the question is outside of your domain, say "Je suis désolé, mais je ne peux répondre qu'à des questions liées à la mobilité à Montréal." and stop. | ||
| By the way, you can answer to questions like "hello", "what can you do?", "merci" etc but always keep in mind that your primary function is to provide analytical insights based on the data. |
There was a problem hiding this comment.
The instruction text has a grammatical error: "answer to questions" should be "answer questions". Additionally, the phrasing "you can answer to questions like" is awkward in English. Consider rephrasing to: "By the way, you can respond to greetings like 'hello', questions like 'what can you do?', and acknowledgments like 'merci', but always keep in mind that your primary function is to provide analytical insights based on the data."
| By the way, you can answer to questions like "hello", "what can you do?", "merci" etc but always keep in mind that your primary function is to provide analytical insights based on the data. | |
| By the way, you can respond to greetings like "hello", questions like "what can you do?", and acknowledgments like "merci", but always keep in mind that your primary function is to provide analytical insights based on the data. |
| "You are the Data Agent for Montreal Mobility.\n" | ||
| "Your PRIMARY action is to find raw data using your available tools.\n\n" | ||
|
|
||
| "Assume that today is " + today + "so if the user asks for data with links to the current date, use this date in your queries.\n\n" |
There was a problem hiding this comment.
The instruction has a grammatical error. The phrase "if the user asks for data with links to the current date" should be "if the user asks for data linked to the current date" or "if the user asks for data related to the current date". Additionally, there's a missing space after the closing quote and before "so" - it should be: "Assume that today is " + today + " so if the user asks for data linked to the current date, use this date in your queries.\n\n"
| "Assume that today is " + today + "so if the user asks for data with links to the current date, use this date in your queries.\n\n" | |
| "Assume that today is " + today + " so if the user asks for data linked to the current date, use this date in your queries.\n\n" |
No description provided.