GrokDet

A Research Framework for Building and Advancing Visual Detection

---

👁️ Overview

GrokDet is a modular framework for visual detection research and engineering validation, built on top of MMDetection. It extends MMDetection's component system to bring object detection, instance segmentation, visual object tracking, and vision-language detection into a single, configuration-driven research environment.

Model construction, data loading, optimization, training, evaluation, distributed execution, and checkpointing are implemented as explicit, inspectable components to keep the path from configuration to evaluation transparent and easy to trace.

GrokDet is also the shared implementation platform for GrokCV. As the project evolves, methods developed by the group will be released here.

Important

GrokDet guarantees full runnability and verification only for methods developed and maintained by GrokCV. Most methods inherited from MMDetection are expected to work in principle, but given their sheer number and variety, we recommend users verify correctness independently before relying on results from these methods in critical applications.

🔬 GrokCV Research Hub

GrokDet will serve as the central repository for research developed by GrokCV. New methods will be continuously integrated.

Method	Task	Paper	Brief Introduction
AdaScope	Infrared Small Target Detection		An asynchronous coarse-to-fine detector for infrared small targets with arbitrary spatial distributions. AdaScope reformulates coarse localization as spatial presence classification and employs a reinforcement-learning refiner to adaptively allocate contextual regions for accurate detection.
DeRefNet	Closely-Spaced Infrared Small Target Unmixing		The pioneering endeavor to address CSIST unmixing within a multi-frame paradigm, utilizing a Temporal Deformable Feature Alignment module to adaptively aggregate inter-frame information for precise sub-pixel localization. Furthermore, it establishes SeqCSIST, the first sequential benchmark dataset and toolkit tailored for unmixing under continuous frames.
DISTA-Net	Closely-Spaced Infrared Small Target Unmixing		The first deep learning model tailored for CSIST unmixing, which reformulates sparse reconstruction into a dynamic deep unfolding framework conditioned on input data for precise sub-pixel localization. It also establishes the first open-source ecosystem comprising CSIST-100K dataset, CSO-mAP metric, and GrokCSO toolkit.
DISTA-Net++	Closely-Spaced Infrared Small Target Unmixing		An informed, continuous paradigm that redefines CSIST unmixing, departing from blind, discrete sub-pixel separation. Built upon a dynamic deep unfolding backbone, it integrates a Count-Guided Prior to semantically regularize the solution space and regresses off-grid offsets, decoupling localization precision from grid resolution.
FOCUS	Closely-Spaced Infrared Small Target Unmixing		A fast, lightweight one-stage architecture that achieves a paradigm shift from iterative deep unfolding networks to feed-forward inference for CSIST unmixing. It employs a decoupled coarse-to-fine flow with sparsity regularization and flux conservation, achieving 60x faster inference speeds while preserving precise sub-pixel localization.
Poseidon-300K	Underwater Object Detection Dataset		A large-scale underwater perception benchmark comprising 84K+ images, 300K+ object instances across 19 categories, and 258K underwater-specific VQA pairs. It also introduces HistEqDet, a learnable feature-enhancement detector designed to improve feature discriminability under severe underwater degradation.

✨ Highlights

	Capability	What it means in GrokDet
🧩	Explicit component system	Models, datasets, transforms, and evaluators are organized through Explicit Catalogs. Supported components are visible, inspectable, and friendly to IDE navigation and static analysis.
🔍	Transparent execution path	The complete route from configuration parsing to training is exposed in train.py, making the framework easier to read, debug, and modify.
⚙️	Lightweight runtime	GrokDet owns the orchestration of epoch- and iteration-based training, validation, hooks, logging, distributed execution, and checkpoints.
🧱	Composable experiments	Python configurations combine models, data pipelines, optimization strategies, and runtime behavior into concise, reproducible experiments.
🌐	Multiple vision tasks	The repository covers detection, segmentation, tracking, and vision-language detection while leaving room for new task families.
🚀	Research-first development	New ideas can be integrated as reusable components, evaluated through shared infrastructure, and released as complete research projects.

🏗️ Architecture

flowchart LR
    A[Experiment Config] --> B[train.py / test.py]
    B --> C[GrokDet Runtime]
    C --> D[Explicit Builders]
    D --> E[Component Catalog]
    E --> F[Models]
    E --> G[Datasets & Transforms]
    E --> H[Evaluation]
    C --> I[Hooks & Logging]
    C --> J[Checkpoint & Distributed]

    style A fill:#eef6ff,stroke:#1677ff,color:#102a43
    style B fill:#e6fffb,stroke:#13c2c2,color:#102a43
    style C fill:#fff7e6,stroke:#fa8c16,color:#102a43
    style E fill:#f9f0ff,stroke:#722ed1,color:#102a43

Loading

The main execution path is intentionally compact:

experiment config
  └── train.py / test.py
      ├── runtime                 # loops, hooks, state, logging, checkpoints
      ├── utils.builder           # domain-specific construction entry points
      ├── utils.explicit_catalogs # explicit and inspectable component catalogs
      ├── models                  # detection, segmentation, tracking, V-L models
      ├── datasets                # datasets, samplers, and transformations
      └── evaluation              # evaluators, metrics, and functional utilities

A detailed account of the current implementation and its capability boundaries is available in the framework analysis.

🚀 Getting Started

1. Clone the repository

git clone https://github.com/GrokCV/GrokDet.git
cd GrokDet

2. Create an environment

Python 3.10+ is recommended for the example setup:

conda create -n grokdet python=3.12 -y
conda activate grokdet

3. Install PyTorch

Choose a PyTorch build that matches your CUDA environment. For CUDA 11.8:

pip install torch==2.7.1 torchvision==0.22.1 \
  --index-url https://download.pytorch.org/whl/cu118

For CUDA 12.8:

pip install torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/cu128

GrokDet has also been tested successfully on an NVIDIA GeForce RTX 5090 with CUDA 13.x.

4. Install dependencies and build GrokDet

pip install scipy pyyaml opencv-python pycocotools tqdm matplotlib \
  future tensorboard rich termcolor addict yapf shapely terminaltables \
  fairscale jsonlines nltk pycocoevalcap transformers seaborn lap \
  motmetrics scikit-image ninja

The command below builds the C++/CUDA operators from source during installation, which may take several minutes. Make sure that your PyTorch build, CUDA Toolkit, compiler, and other build dependencies are mutually compatible; you may need to adjust their versions or the build configuration for your local environment.

pip install -v -e . --no-build-isolation

5. Prepare data

Configurations use COCO-style datasets by default:

data/coco/
├── annotations/
├── train2017/
└── val2017/

🏋️ Training

Single-GPU training:

python train.py <config>

Multi-GPU training:

torchrun --nproc_per_node=4 train.py <config> --launcher pytorch

Continue an interrupted experiment or initialize a new one from pretrained weights:

python train.py <config> --resume-from CHECKPOINT.pth
python train.py <config> --load-from PRETRAINED.pth

🧪 Testing and Evaluation

# Evaluate a checkpoint
python test.py <config> CHECKPOINT.pth

# Export predictions
python test.py <config> CHECKPOINT.pth --out predictions.pkl

# Render predictions
python test.py <config> CHECKPOINT.pth --show-dir work_dirs/visualizations

🧩 Extending GrokDet

GrokDet encourages methods to be expressed as composable components and assembled through configurations.

1. Implement the component in the appropriate package, such as models/backbones/ or models/detectors/.
2. Import it explicitly from the package's __init__.py and add it to __all__.
3. Register a new component category in utils/explicit_catalogs.py when necessary.
4. Add a reproducible experiment configuration under configs/.
5. Validate construction, one training step, evaluation, and checkpoint recovery.

# models/backbones/__init__.py
from .my_backbone import MyBackbone

__all__ = [
    # ...
    'MyBackbone',
]

The component can then be selected directly from a configuration:

model = dict(
    type='FasterRCNN',
    backbone=dict(
        type='MyBackbone',
        # ...
    ),
    # ...
)

🤝 Contributing

Bug reports, documentation improvements, configuration updates and fixes are welcome. To keep GrokDet maintainable, a contribution should include clear component exports, a reproducible configuration, a minimal runnable example, appropriate tests, and notes about environment compatibility.

Please open an issue to discuss problems or substantial changes.

🙏 Acknowledgements

GrokDet originated from MMDetection.Its initial codebase is derived from MMDetection and shares most of its core code assets. It also relies on low-level utilities from MMCV and MMEngine. We thank the contributors of these projects and the broader open-source computer vision community.

📄 License

Except where otherwise noted, GrokDet is released under the Apache License 2.0. The project contains substantially modified material derived from MMDetection, MMCV, and MMEngine. Original copyright and attribution notices remain applicable.

Some vendored MMCV operations are governed by separate NVIDIA terms and are not relicensed under Apache-2.0. See NOTICE and THIRD_PARTY_LICENSES.md for attribution, exceptions, and third-party licensing details. Dataset, model-weight, and external-asset licenses must be considered separately.

---

Grok the model. Detect the world.

If GrokDet supports your research, consider giving the project a ⭐ and following its future releases.