Skip to content

MCP_AND_TOOLS.md

Latest commit

 

History

History
184 lines (139 loc) · 7.96 KB

MCP_AND_TOOLS.md

File metadata and controls

184 lines (139 loc) · 7.96 KB
Raw

MCP & Tool Manager

ContextUI includes a built-in Model Context Protocol (MCP) server and a Tool Registry that lets AI agents interact with workflows, manage Python servers, and automate the UI.

Humans + Agents, Same App

ContextUI is built to be used by both humans and AI agents. A human interacts visually — clicking buttons, browsing tabs, dragging sliders. An agent interacts programmatically — calling MCP tools to launch workflows, automate UI, and manage servers.

Both work on the same workflows, the same tabs, the same data. An agent can build a workflow and a human can use it. A human can open a workflow and an agent can automate it. This is what makes ContextUI different — it's not a human tool with an API bolted on. Agents are first-class users.

Overview

There are two systems working together:

  • MCP Server — stdio-based server that AI agents connect to. Exposes tools for workflow management, Python backends, UI automation, and more.
  • Tool Registry — HTTP server on port 8800 that tracks all available tools (built-in + workflow-registered). The Tool Manager UI lets you enable/disable tools.

Connecting an Agent

Add the ContextUI MCP server to your agent's MCP configuration:

{
  "mcpServers": {
    "contextui-workflows": {
      "command": "node",
      "args": ["electron/mcp-server.cjs"],
      "env": {
        "NODE_ENV": "production"
      }
    }
  }
}

The server uses stdio transport — your MCP client starts the process and communicates over stdin/stdout.

Built-In MCP Tools

Workflow Management

Tool Description
list_workflows List workflow folders. Use folder param to list workflows in a specific subfolder.
read_workflow Read workflow source code by absolute path.
get_workflow_structure Get the complete directory tree of a workflow folder.
launch_workflow Open a workflow as a tab in the ContextUI app.
close_workflow Close a workflow tab. Omit path to close the active tab.

Python Server Management

Tool Description
python_list_venvs List available Python virtual environments.
python_start_server Start a Python server. Params: venvName, scriptPath, port, serverName.
python_stop_server Stop a running server by serverName.
python_server_status Get status of one or all running servers.
python_test_endpoint Make an HTTP request to a server endpoint. Params: url, method, body, headers.

UI Automation

Tool Description
ui_screenshot Capture a screenshot of the active workflow tab or full app window.
ui_get_dom Get the DOM tree of the current workflow. Params: depth, maxElements, selector.
ui_click Click an element by CSS selector, text content, or coordinates.
ui_type Type text into an input field. Params: selector, text, clear, pressEnter.
ui_select Select an option from a dropdown. Works with native <select> and custom dropdowns.
ui_drag Drag between coordinates or set a slider value directly.
ui_get_element Get element properties (visibility, bounds, text, enabled state).
ui_accessibility_audit Run an axe-core accessibility audit on the current workflow.

Tab Management

Tool Description
list_tabs List all open tabs with names, components, and active state.
switch_tab Switch to a tab by name or component. Use list_tabs first for exact names.

App Management (via Tool Registry)

The following tools are registered in the Tool Registry and available through the Tool Manager UI. They're managed via the registry's HTTP API rather than the MCP stdio server directly.

Tool Description
list_local_servers List all configured local servers with running status.
start_local_server Start a local server by ID, name, or port.
stop_local_server Stop a running local server.
list_html_apps List all configured HTML apps.
open_html_app Open an HTML app by ID or name.
list_mcp_servers List configured external MCP servers with connection status.
connect_mcp_server Connect to an external MCP server (auto-detects stdio/HTTP transport).
disconnect_mcp_server Disconnect from an external MCP server.

Tool Registry

The Tool Registry is an HTTP server running on port 8800 in the main Electron process. It serves as the central directory for all tools in the system.

What It Does

  • Registers built-in MCP tools automatically on startup
  • Accepts tool registrations from Python backend servers and external MCP servers
  • Provides tool discovery — list, search, and inspect available tools
  • Manages tool settings — enable/disable individual tools or entire namespaces

Registering Tools from a Python Server

Python backends can register their own tools with the registry. This makes them discoverable in the Tool Manager and callable by agents.

import requests

# Register a tool
requests.post("http://127.0.0.1:8800/tools/register", json={
    "serverName": "my-server",
    "namespace": "my-workflow",
    "baseUrl": "http://127.0.0.1:8810",
    "tools": [
        {
            "name": "analyze_data",
            "description": "Analyze uploaded data and return insights",
            "inputSchema": {
                "type": "object",
                "properties": {
                    "filePath": {
                        "type": "string",
                        "description": "Path to data file"
                    }
                },
                "required": ["filePath"]
            }
        }
    ]
})

Registry HTTP API

Endpoint Method Description
/tools GET List all registered tools
/providers GET List all tool providers (servers)
/tools/register POST Register tools from a provider
/tools/unregister POST Remove a provider and its tools
/tools/settings GET Get tool enable/disable settings
/tools/settings PUT Update tool settings
/health GET Health check

Tool Settings

Tool settings are persisted to ~/ContextUI/.tool-settings.json. When a tool or namespace is disabled, the MCP server won't expose it to agents.

Tool Manager UI

The Tool Manager is a built-in module accessible from the ContextUI sidebar. It provides a visual interface for:

  • Browsing tools — grouped by namespace (provider), expandable to see each tool's description and input schema
  • Toggling tools — enable or disable individual tools or entire namespaces
  • Viewing providers — see which servers are registered, their ports, and uptime
  • Refreshing — re-scan for newly registered or removed tools

This is useful for controlling which tools are available to your agent — if a workflow registers tools you don't want exposed, disable them here.

MCP Bridge (File-Based)

For processes that can't use stdio, ContextUI provides a file-based command bridge at ~/ContextUI/.mcp-bridge/. Write a JSON command file and the app processes it:

{
  "type": "switch_tab",
  "tab": "KokoroTTS",
  "id": "unique-request-id"
}

The response appears as {id}.response.json in the same directory.

Tips

  • Use list_tabs before switch_tab — tab names must match exactly (case-insensitive).
  • Use ui_get_dom with higher depth — the default depth=3 is often too shallow. Use depth=10 maxElements=200 for full visibility.
  • Launch before automating — UI tools (ui_click, ui_type, etc.) require a workflow to be open. Call launch_workflow first.
  • ui_screenshot for verification — after clicking or typing, take a screenshot to confirm the UI updated as expected.
  • Closing tabs stops servers — if a workflow has a Python backend, closing the tab stops the server. Reopen the workflow and restart from the Setup tab.