Agent says dataset "doesn't exist" when there's no docs for it

[shawnxiao105-afk]

Following up from #57.

Against datahub docker quickstart sample data, ask:

tell me about SampleHiveDataset

The dataset is there (datahub get --urn 'urn:li:dataset:(urn:li:dataPlatform:hive,SampleHiveDataset,PROD)' returns it), but it has no description / glossary term / domain. The agent runs search_business_context, all four sub-searches come back empty, and it tells the user the dataset "doesn't exist in the catalog."

The SKILL.md for search_business_context actually says to fall through to search + get_entities when nothing is found, and both tools are wired in via build_langchain_tools. I tried calling search(query="SampleHiveDataset", type=DATASET) against quickstart GMS directly and it does return the dataset — so the fall-through path would work, the agent just isn't taking it when the empty result comes from the business-context fan-out.

Haven't dug into why the LLM stops there — could be a prompting thing, could be something else. Happy to look closer if useful.

[shawnxiao105-afk]

@shirshanka — spent more time on this and I think the root cause sits in prompts/system_prompt.md rather than the search_business_context skill body. Two things stood out.

Step 2 of the workflow gives a parallel OR. It currently reads:

Use search or list_tables to find candidate datasets.

Once Step 1 (search_documents / search_business_context) returns empty, both branches look equally valid. For "tell me about SampleHiveDataset", the "Dataset" keyword nudges the model toward the SQL workflow (execute_sql → list_tables) and it never circles back to a catalog search.

list_tables's tool description doesn't disambiguate it. It just says:

list_tables: list available tables (optionally filtered by schema)

Nothing tells the model that list_tables only sees the connected SQL engine, not the DataHub catalog. With Step 2's OR above, the model treats it as a peer of search for catalog discovery.

The search_business_context skill body has a "When all sub-searches return empty" fallback that says to call search — but it's appended after the main system prompt as a detail clause, competing against the higher-level Step 2 instruction. Tightening it more didn't feel like the right layer.

So two surgical changes I'd propose, both in system_prompt.md:

A — disambiguate Step 2:

Step 2 — Find the technical assets
Use `search` to find candidate datasets in the DataHub catalog. `list_tables`
only lists tables in the currently-connected SQL engine — it cannot tell you
whether an entity exists in the catalog. Reach for `list_tables` only after
you have a candidate URN from `search`, or for SQL-engine-only exploration.

B — sharpen the list_tables tool description:

- list_tables: list tables in the currently-connected {engine_name} SQL engine
  (NOT the DataHub catalog — use `search` for catalog-wide discovery)

For context: I had an earlier attempt at this (PR #63) that wired a deterministic dataset search into _search_business_context_impl itself. On reflection it expanded the skill's surface area for what's really a prompting problem — closed it. A+B stays at the prompt layer where the actual conflict lives.

Happy to open a PR with these two changes if you're OK with the direction, or push back if you'd rather see something else at that layer.