Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
280 changes: 201 additions & 79 deletions langchain/README.md
Original file line number Diff line number Diff line change
@@ -1,149 +1,264 @@
# TinyFish Web Agent for LangChain
# langchain-tinyfish

![Powered by TinyFish](https://img.shields.io/badge/Powered%20by-TinyFish-blue)
[![PyPI version](https://img.shields.io/pypi/v/langchain-tinyfish)](https://pypi.org/project/langchain-tinyfish/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)

This package provides a LangChain Tool to run the TinyFish Web Agent directly within your LangChain Agents and Workflows.
[TinyFish](https://tinyfish.ai) gives AI agents every level of web access through one platform. Four primitives, each built for a different layer:

TinyFish is a platform for executing complex, goal-oriented tasks on the live web. Unlike traditional scrapers or local browser automation, TinyFish uses a fleet of remote, AI-powered web agents that can navigate complex sites, handle anti-bot protection, and return clean, structured JSON data.
- **Search**: live web results, structured and ready for LLM consumption. Free.
- **Fetch**: renders pages in a real browser, returns clean markdown, HTML, or JSON. Token-efficient for LLM pipelines. Free.
- **Web Agent**: autonomous agents that navigate, authenticate, and extract from real websites. Uses credits.
- **Browser**: remote Chromium sessions with full CDP access. Bring your own Playwright or Puppeteer scripts. Uses credits.

This package wraps all four as LangChain tools. [Get your API key ->](https://agent.tinyfish.ai/api-keys)

## Installation

```bash
pip install langchain-tinyfish
```

## Configuration
```bash
export TINYFISH_API_KEY="your-api-key"
```

1. Get your TinyFish API key from [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys).
2. Set it as an environment variable:
## Tools

```bash
export TINYFISH_API_KEY="YOUR_API_KEY"
All four tools can be used standalone or passed to a LangChain agent:

```python
from langchain_tinyfish import (
TinyFishSearch,
TinyFishFetch,
TinyFishWebAutomation,
TinyFishBrowserSession,
)
```

Tool calls return strings. Successful responses are JSON strings produced from the TinyFish SDK response.

### Search

Search the web and get structured results with titles, snippets, and URLs. Free, no credits required.

```python
from langchain_tinyfish import TinyFishSearch

search = TinyFishSearch()
results = search.invoke({
"query": "latest LLM benchmarks 2025",
"location": "US",
})
print(results)
```

## Quick Start
Supported inputs:

| Parameter | Required | Description |
|-----------|----------|-------------|
| `query` | Yes | Search query |
| `location` | No | Optional location/country scope, such as `"US"` |
| `language` | No | Optional language code, such as `"en"` |

### Fetch

Here is a simple example of how to use the TinyFish Web Agent to extract the current stock price of NVIDIA from Yahoo Finance.
Extract clean content from up to 10 URLs at once. Returns markdown, HTML, or JSON. Free, no credits required.

```python
from langchain_tinyfish import TinyFishWebAutomation
from langchain_tinyfish import TinyFishFetch

fetch = TinyFishFetch()
content = fetch.invoke({
"urls": ["https://docs.tinyfish.ai"],
"format": "markdown",
"links": True,
})
print(content)
```

Supported inputs:

| Parameter | Required | Description |
|-----------|----------|-------------|
| `urls` | Yes | List of 1-10 URLs |
| `format` | No | `"markdown"`, `"html"`, or `"json"`; defaults to `"markdown"` |
| `links` | No | Include extracted page links |
| `image_links` | No | Include extracted image links |

tool = TinyFishWebAutomation()
### Web Agent

result = tool.invoke({
Run complex, goal-oriented tasks on live websites. TinyFish handles navigation, anti-bot protection, and returns structured JSON. Uses credits.

```python
from langchain_tinyfish import TinyFishWebAutomation

agent_tool = TinyFishWebAutomation()
result = agent_tool.invoke({
"url": "https://finance.yahoo.com/quote/NVDA/",
"goal": "Extract the current stock price of NVIDIA",
})

print(result)
# Output: {"stock_price": 950.02}
```

### Search, Fetch, and Browser Sessions
Supported inputs:

| Parameter | Required | Description |
|-----------|----------|-------------|
| `url` | Yes | Starting URL for the automation |
| `goal` | Yes | Natural-language instructions for what to do and what to return |

The package also exposes SDK-backed tools for TinyFish Search, Fetch, and Browser sessions:
When `TinyFishWebAutomation` runs inside a LangGraph execution context, it uses TinyFish's streaming endpoint and emits progress events through LangGraph's stream writer. Outside LangGraph, it falls back to a blocking TinyFish run.

### Browser Session

Launch a remote Chromium browser and get a CDP (Chrome DevTools Protocol) URL to connect with Playwright or Puppeteer. Sessions include remote browser infrastructure for direct CDP control. Uses credits.

```python
from langchain_tinyfish import (
TinyFishBrowserSession,
TinyFishFetch,
TinyFishSearch,
)
from langchain_tinyfish import TinyFishBrowserSession

search = TinyFishSearch()
fetch = TinyFishFetch()
browser = TinyFishBrowserSession()

search_results = search.invoke({"query": "TinyFish Web Agent docs"})
page_content = fetch.invoke({"urls": ["https://docs.tinyfish.ai"], "format": "markdown"})
session = browser.invoke({"url": "https://example.com"})
session = browser.invoke({
"url": "https://example.com",
"timeout_seconds": 300,
})
print(session)
# Returns a JSON string with: {"session_id": "...", "cdp_url": "wss://...", "base_url": "https://..."}
```

### With a LangChain Agent
Supported inputs:

| Parameter | Required | Description |
|-----------|----------|-------------|
| `url` | No | Optional target URL to open when the browser session starts |
| `timeout_seconds` | No | Optional inactivity timeout for the browser session |

## With a LangChain Agent

Give your agent access to multiple TinyFish tools. The agent decides which primitive to use based on the task.

Install the optional agent dependencies:

```bash
pip install langchain langchain-openai langgraph
```

```python
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
from langchain_tinyfish import TinyFishFetch, TinyFishSearch, TinyFishWebAutomation

llm = ChatOpenAI(model="gpt-4o")
tools = [
TinyFishWebAutomation(),
TinyFishSearch(),
TinyFishFetch(),
]
agent = create_react_agent(llm, tools)
from langchain.agents import create_agent
from langchain_tinyfish import (
TinyFishSearch,
TinyFishFetch,
TinyFishWebAutomation,
)

tools = [TinyFishSearch(), TinyFishFetch(), TinyFishWebAutomation()]
agent = create_agent(
model="openai:gpt-5.5",
Comment thread
coderabbitai[bot] marked this conversation as resolved.
tools=tools,
)

result = agent.invoke({
"messages": [("user", "Go to scrapeme.live/shop and extract the first 5 product names and prices")]
"messages": [
{
"role": "user",
"content": "Find the top 3 results for 'best open source LLMs' and extract the full content of the first result",
}
]
})

for message in result["messages"]:
print(message.content)
```

### Stealth Mode + Proxy
You can also pass the TinyFish tools to LangGraph's prebuilt agents, including `langgraph.prebuilt.create_react_agent`, if your application already uses LangGraph directly.

## Stealth Mode and Proxies

For sites with bot protection (Cloudflare, CAPTCHAs, etc.):
For Web Agent runs on sites with bot protection (Cloudflare, CAPTCHAs, etc.), configure stealth browsing and geo-targeted proxies:

```python
from langchain_tinyfish import TinyFishAPIWrapper, TinyFishWebAutomation

tool = TinyFishWebAutomation(
api_wrapper=TinyFishAPIWrapper(
browser_profile="stealth",
proxy_enabled=True,
proxy_country_code="US", # Also: GB, CA, DE, FR, JP, AU
)
wrapper = TinyFishAPIWrapper(
browser_profile="stealth",
proxy_enabled=True,
proxy_country_code="US", # Also: GB, CA, DE, FR, JP, AU
)
```

### Async Usage

```python
import asyncio
from langchain_tinyfish import TinyFishWebAutomation

async def main():
tool = TinyFishWebAutomation()
result = await tool.ainvoke({
"url": "https://example.com",
"goal": "Extract the page title",
})
print(result)

asyncio.run(main())
agent_tool = TinyFishWebAutomation(api_wrapper=wrapper)
```

## Use Cases

- **AI Agent Enablement:** Give your AI agent the ability to perform deep research on the web.
- **Workflow Automation:** Monitor a competitor's pricing page and get a Slack notification when it changes.
- **Data Extraction:** Extract job postings, product details, or contact information into a structured format.
`browser_profile` and proxy settings are currently applied by `TinyFishWebAutomation`. Search, Fetch, and Browser Session tools can share the same wrapper for API key and timeout configuration, but they do not pass Web Agent browser profile or proxy options.

## Configuration Options
## Configuration

All parameters are set on the `TinyFishAPIWrapper`:
All tools share a `TinyFishAPIWrapper` for API key and timeout configuration:

| Parameter | Default | Description |
|-----------|---------|-------------|
| `api_key` | `$TINYFISH_API_KEY` | Your TinyFish API key |
| `browser_profile` | `"lite"` | `"lite"` (fast) or `"stealth"` (anti-detection) |
| `proxy_enabled` | `False` | Enable proxy routing |
| `proxy_country_code` | `"US"` | Proxy country: US, GB, CA, DE, FR, JP, AU |
| `timeout` | `300` | Request timeout in seconds |
| `browser_profile` | `"lite"` | Web Agent browser profile: `"lite"` or `"stealth"` |
| `proxy_enabled` | `False` | Enable proxy routing for Web Agent runs |
| `proxy_country_code` | `"US"` | Proxy exit country for Web Agent runs: US, GB, CA, DE, FR, JP, AU |
| `timeout` | `300` | Request/poll timeout in seconds |

```python
from langchain_tinyfish import TinyFishAPIWrapper, TinyFishWebAutomation

wrapper = TinyFishAPIWrapper(
api_key="sk-mino-...",
browser_profile="stealth",
timeout=600,
)
tool = TinyFishWebAutomation(api_wrapper=wrapper)
```

## Parallel Web Agents

Every Web Agent run gets its own isolated remote browser. Run multiple agents concurrently with `asyncio.gather`:

```python
import asyncio
from langchain_tinyfish import TinyFishWebAutomation

async def main():
agent = TinyFishWebAutomation()

tasks = [
agent.ainvoke({"url": "https://example.com/page-1", "goal": "Extract the main heading"}),
agent.ainvoke({"url": "https://example.com/page-2", "goal": "Extract the main heading"}),
agent.ainvoke({"url": "https://example.com/page-3", "goal": "Extract the main heading"}),
]

results = await asyncio.gather(*tasks)
for result in results:
print(result)

asyncio.run(main())
```

## Async Usage

All tools support async with `ainvoke`:

```python
import asyncio
from langchain_tinyfish import TinyFishSearch, TinyFishFetch

async def main():
search = TinyFishSearch()
fetch = TinyFishFetch()

results = await search.ainvoke({"query": "TinyFish web agent"})
content = await fetch.ainvoke({
"urls": ["https://tinyfish.ai"],
"format": "markdown",
})

print(results)
print(content)

asyncio.run(main())
```

## Development

```bash
Expand All @@ -161,6 +276,13 @@ make lint
make integration_test
```

## Resources

- [TinyFish Documentation](https://docs.tinyfish.ai)
- [TinyFish Website](https://tinyfish.ai)
- [API Keys](https://agent.tinyfish.ai/api-keys)
- [Discord Community](https://discord.com/invite/tinyfish)

## Support

If you have any questions or need help, please reach out to [support@tinyfish.ai](mailto:support@tinyfish.ai) or join our [Discord community](https://discord.gg/agentql).
Questions or issues? Reach out at [support@tinyfish.ai](mailto:support@tinyfish.ai) or join our [Discord](https://discord.com/invite/tinyfish).
2 changes: 1 addition & 1 deletion langchain/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "langchain-tinyfish"
version = "0.1.0"
version = "0.1.1"
description = "LangChain integration for TinyFish Web Agent - AI-powered web automation"
readme = "README.md"
license = { text = "MIT" }
Expand Down