ThoughtNet is an experimental semantic exploration system designed to help users inspect how ideas around a question are distributed across real-world discourse.
Instead of relying on keyword matching, ThoughtNet attempts to:
- decompose a user’s query into meaningful sub-thoughts,
- fetch related sentences from multiple web sources,
- group them using neural embeddings,
- and present the result as an interactive graph of clustered ideas.
Version 2.0 represents a major evolution from the initial MVP. While v1 focused on validating an end-to-end pipeline, v2 introduces a redesigned multi-page frontend, asynchronous backend workflows, and a more flexible semantic retrieval approach. The system remains a research prototype, but it now supports a wider variety of prompts and provides better transparency into how results are formed.
- Query Decomposition: Attempts to break complex user questions into simpler semantic sub-thoughts.
- Multi-Source Aggregation: Asynchronously fetches related content from Reddit, Hacker News, NewsAPI, and the open web.
- Sentence-Level Embeddings: Uses neural sentence embeddings to group semantically related ideas rather than relying on keywords.
- Force-Directed Graph: Ideas are visualized as nodes connected by semantic relationships.
- Node Interaction: Users can expand clusters and inspect the underlying source sentences.
- Source Transparency: Every cluster can be traced back to its raw evidence.
- Graph Export: Save the current graph structure as JSON.
- Snapshot Export: Capture a visual snapshot of the graph for reference or sharing.
Early versions of ThoughtNet (v1.x) relied on token-based matching and rigid prompt structures. As a result:
- many prompts failed to produce meaningful results,
- accuracy depended heavily on phrasing,
- and only a narrow class of prompts worked reliably.
In v2.0, semantic embeddings, linguistic preprocessing, and source transparency significantly improve robustness:
- a wider range of prompts now generate structured graphs,
- user intent is better preserved through sub-thought decomposition,
- and relevance can be inspected directly through exposed source sentences.
That said, accuracy is still imperfect:
- unrelated or weakly relevant content may appear,
- clustering quality varies with topic complexity,
- and deeper clause-level understanding has not yet been implemented.
v2.0 marks the point where ThoughtNet transitions from a fragile prototype into a usable exploratory system for simple to moderately complex prompts.
ThoughtNet operates as a multi-stage pipeline:
-
Decomposition Attempts to break the input query into semantic sub-components.
-
Aggregation Asynchronous harvesting of related sentences from multiple sources.
-
Embedding Conversion of text into vector representations using neural language models.
-
Clustering Unsupervised grouping (KMeans / DBSCAN) to identify thematic clusters.
-
Synthesis & Visualization Construction of a graph where each node represents a cluster backed by transparent source evidence.
The term recursive in ThoughtNet refers to iterative semantic decomposition, not recursive model execution.
ThoughtNet is guided by the following principles:
The system prioritizes sentence-level semantics instead of exact token matching. Even when results are noisy, the goal is to preserve intent rather than phrasing.
ThoughtNet does not present answers as facts. Every cluster is backed by visible source sentences, allowing users to inspect, question, or discard conclusions themselves.
Graph instability, imperfect clustering, and noisy sources are treated as signals for exploration rather than errors to hide.
The system evolves in layers:
- first capturing structure,
- then improving relevance,
- and later refining linguistic understanding.
This keeps the system flexible, inspectable, and debuggable as it matures.
- Mobile responsiveness is not yet configured; the UI is optimized for desktop use.
- Backend performance still falls short of the project’s long-term real-time vision.
- Graph rendering may behave unpredictably when zooming via browser controls (
Ctrl + +/-). - Navigation may occasionally become unresponsive after visiting the graph page and require a reload.
- Graph edges can become unstable when expanding multiple sub-thoughts simultaneously.
- There is no support for authentication or cloud-based history persistence.
- Transformer-based clause-level query extraction is not yet implemented.
| Component | Technologies |
|---|---|
| Backend | Python 3.10, Django REST Framework, Asyncio |
| ML & Data | SentenceTransformers, scikit-learn, NumPy |
| Networking | HTTPX, AsyncPRAW |
| Frontend | React (Vite), TailwindCSS |
| Visualization | React Force Graph |
- Python 3.10+
- Node.js 16+
git clone https://github.com/Mridul-23/ThoughtNet.git
cd ThoughtNetcd backend
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txtCreate a .env file in backend/:
REDDIT_CLIENT_ID=your_id
REDDIT_CLIENT_SECRET=your_secret
NEWS_API_KEY=your_keyRun the backend:
python manage.py runservercd ui
npm install
npm run devVisit:
http://localhost:5173
ThoughtNet was developed using an AI-assisted coding workflow, where large language models were used as collaborative tools for:
- architectural reasoning,
- backend pipeline refactoring,
- frontend component structuring,
- and iterative design discussion.
All final design decisions, implementation, and validation were performed manually, with AI used as a development aid rather than an autonomous agent.
This project is licensed under the MIT License. See the LICENSE file for details.
Mridul Narula Creator & Lead Developer