AIGNLAI · Oct 15, 2026 · Oct 15, 2026
diff --git a/README.md b/README.md
@@ -67,6 +67,7 @@ Useful links:
 
 ## What's New
 
+- **2026-10**: Added **VL-RouterBench** support for vision-language model routing through the existing ORBIT pipeline. The adapter preserves named outcomes and prepared splits, resolves TSV image assets, and records estimated-token reference costs. See [preparation and protocol](docs/vl_routerbench.md).
 - **2026-10**: Added adapters for **xRouteBench**, **MMR-Bench V2**, and **R2-Bench**, allowing existing routers to evaluate newer released outcomes through the same ORBIT pipeline. R2's model-budget pairs are ordinary routing candidates; release-specific cost and input limitations are documented in [the adapter guide](docs/new_benchmarks.md).
 - **2026-10**: Added **RouteFM**, the frozen in-context foundation router that transfers to new candidate pools through behavioral context, and **SAVERouter**, which adaptively acquires fixed-K sparse feedback and accounts for supervision expenditure. [[RouteFM Paper]](https://arxiv.org/abs/2609.37362) [[RouteFM Code]](https://github.com/LAMDA-Model-Reuse/RouteFM) [[SAVERouter Paper]](https://arxiv.org/abs/2609.37402) [[SAVERouter Code]](https://github.com/LAMDA-Model-Reuse/SaveRouter)
 - **2026-08**: ORBIT has been accepted by the *Frontiers of Computer Science* (FCS) special column **[Code & Data](https://journal.hep.com.cn/fcs/EN/subject/showCollection.do?subjectId=1710741206314)**. [[Paper]](https://doi.org/10.1007/s11704-026-61310-5)
@@ -105,10 +106,10 @@ ORBIT reproduces **30 representative LLM routing methods** across training-free,
 - **TRouter**: Learns task-aware query and model representations for performance-cost routing. [[Paper]](https://arxiv.org/abs/2604.09377)
 - **UniRoute**: Represents model capabilities through cluster-level prediction errors and learns a query-to-cluster router. [[Paper]](https://openreview.net/forum?id=ka82fvJ5f1)
 - **InferenceDynamics**: Builds parameter-free model indexes from ranked capability and knowledge profiles for structured model-query matching. [[Paper]](https://arxiv.org/abs/2505.16303)
-- **ProfileRouter**: Implements RouteProfile's training-free Emb-GNN profile construction and cosine-similarity SimRouter using public model descriptions, family metadata, and optional reported benchmark scores. [[Paper]](https://arxiv.org/abs/2605.00180)
+- **ProfileRouter**: Implements RouteProfile's training-free Emb-GNN profile construction and cosine-similarity SimRouter using public metadata. Also supports an explicitly labeled CLIP-aligned multimodal extension. [[Paper]](https://arxiv.org/abs/2605.00180) [[Multimodal adapter]](docs/multimodal_adapters.md)
 - **CarrotRouter**: Uses CARROT's KNN plug-in estimators for query-level performance and cost, then applies its original cost-aware rate-optimal scalarization. [[Paper]](https://arxiv.org/abs/2502.03261)
 - **EARAMRouter**: Trains independent provider-side success predictors and allocates queries through EA-RAM's largest-positive-surplus reverse auction with runner-up externality payments. [[Paper]](https://arxiv.org/abs/2608.12719)
-- **RouteFM**: Uses the official frozen foundation router and anonymous per-candidate behavioral context to predict target-query quality and relative cost without target-domain parameter updates. ORBIT defaults to the released BGE text checkpoint. [[Paper]](https://arxiv.org/abs/2609.37362) [[Code]](https://github.com/LAMDA-Model-Reuse/RouteFM)
+- **RouteFM**: Uses the official frozen foundation router and anonymous per-candidate behavioral context to predict target-query quality and relative cost without target-domain parameter updates. Defaults to BGE text routing; Qwen multimodal routing uses joint Qwen3-VL-Embedding-8B features. [[Paper]](https://arxiv.org/abs/2609.37362) [[Code]](https://github.com/LAMDA-Model-Reuse/RouteFM) [[Configuration]](docs/multimodal_adapters.md)
 - **SAVERouter**: Adaptively reveals exactly K model outcomes per training query, then combines hierarchical group-model shrinkage with query-level residual refinement and native sparse cost estimates. [[Paper]](https://arxiv.org/abs/2609.37402) [[Code]](https://github.com/LAMDA-Model-Reuse/SaveRouter)
 
 ---
@@ -196,6 +197,7 @@ python main.py --dataset Routerbench --method SaveRouter
 python main.py --dataset XRouteBench --method knn
 python main.py --dataset MMRBenchV2 --method GraphRouter
 python main.py --dataset R2Bench --method CarrotRouter
+python main.py --dataset VLRouterBench --method GraphRouter
 ```
 
 Dataset and method names are matched case-insensitively. Device selection defaults to
@@ -223,6 +225,11 @@ ORBIT includes a compact benchmark suite spanning text-only and multimodal routi
 | **xRouteBench** | Text / text-view | 18 | Official train/test outcomes with recorded token usage and frozen pricing. |
 | **MMR-Bench V2** | Multimodal | 44 | Versioned 18-task release; common successful support and output-only reference costs. |
 | **R2-Bench** | Text | 157 actions | 10 physical models with released nominal output-budget settings. |
+| **VL-RouterBench** | Multimodal | 17 | 14-task upstream exports; train/dev/test IDs and explicit estimated-token reference costs. |
+
+> VL-RouterBench requires one-time preparation of its raw archive with upstream
+> Steps 1 and 2. ORBIT reads named JSONL/Parquet/CSV exports, not positional or
+> pickle matrices. See [data preparation, image handling and evaluation limitations](docs/vl_routerbench.md).
 
 > Note: RouterEval provides 12 dataset files with nested model pools. To ensure a consistent evaluation protocol, ORBIT deterministically sorts the common models from Group A's six files and aligns performance and cost columns to that canonical order. The bundled metadata currently contains 239 model descriptions; description-based routers validate coverage by model name and report any missing entries instead of silently misaligning them.
 

diff --git a/configs/benchmarks/vlrouterbench.json b/configs/benchmarks/vlrouterbench.json
@@ -0,0 +1,32 @@
+{
+  "_comment": "Prepared upstream Steps 1 and 2; see docs/vl_routerbench.md. No pickle/matrix loading.",
+  "modality": "text+image",
+  "voting": "weighted",
+  "seed": 42,
+  "description_path": "./configs/description/vlrouterbench.json",
+  "device": "auto",
+  "dataset": {
+    "name": "VLRouterBench",
+    "dataset_dir": "./data/vl_routerbench",
+    "revision": "6e44fbf50e170f8e72734b49f63f2b52a8866377",
+    "cost_scaling": "train-max",
+    "local_files_only": true,
+    "missing_policy": "error",
+    "text_overlap_policy": "error",
+    "estimated_output_policy": "allow",
+    "include_responses": false,
+    "multi_image": "contact-sheet",
+    "image_cell_size": 512,
+    "split": {
+      "mode": "official"
+    }
+  },
+  "embeddings": {
+    "text_model": "ViT-B/16",
+    "image_model": "ViT-B/16",
+    "batch_size": 128,
+    "normalize": true,
+    "training": false,
+    "out_dim": 1024
+  }
+}
diff --git a/configs/datasets/vlrouterbench.json b/configs/datasets/vlrouterbench.json
@@ -0,0 +1,92 @@
+{
+  "hf_id": "KinghtH/VL-RouterBench",
+  "revision": "6e44fbf50e170f8e72734b49f63f2b52a8866377",
+  "upstream_revision": "f5a41ce83d6efe67489b35bb7a07ae72976a9aba",
+  "pricing_source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/pricing.yaml",
+  "models": {
+    "GPT4o": {
+      "input_price_per_1m": 2.5,
+      "output_price_per_1m": 10
+    },
+    "GeminiFlash2-5": {
+      "input_price_per_1m": 0.3,
+      "output_price_per_1m": 2.4
+    },
+    "Gemma3-27B": {
+      "input_price_per_1m": 0.35,
+      "output_price_per_1m": 0.5
+    },
+    "InternVL2_5-78B": {
+      "input_price_per_1m": 1,
+      "output_price_per_1m": 1.5
+    },
+    "Janus-Pro-1B": {
+      "input_price_per_1m": 0.05,
+      "output_price_per_1m": 0.05
+    },
+    "Janus-Pro-7B": {
+      "input_price_per_1m": 0.18,
+      "output_price_per_1m": 0.25
+    },
+    "Kimi-VL-A3B-Thinking-2506": {
+      "input_price_per_1m": 0.2,
+      "output_price_per_1m": 0.25
+    },
+    "MiMo-VL-7B-RL": {
+      "input_price_per_1m": 0.2,
+      "output_price_per_1m": 0.3
+    },
+    "Phi-3.5-Vision": {
+      "input_price_per_1m": 0.1,
+      "output_price_per_1m": 0.1
+    },
+    "Pixtral-12B": {
+      "input_price_per_1m": 0.25,
+      "output_price_per_1m": 0.35
+    },
+    "Qianfan-VL-8B": {
+      "input_price_per_1m": 0.18,
+      "output_price_per_1m": 0.25
+    },
+    "Qwen2.5-VL-32B-Instruct": {
+      "input_price_per_1m": 0.4,
+      "output_price_per_1m": 0.6
+    },
+    "Qwen2.5-VL-72B-Instruct": {
+      "input_price_per_1m": 0.8,
+      "output_price_per_1m": 1.2
+    },
+    "SmolVLM2": {
+      "input_price_per_1m": 0.06,
+      "output_price_per_1m": 0.06
+    },
+    "deepseek_vl2": {
+      "input_price_per_1m": 0.35,
+      "output_price_per_1m": 0.5
+    },
+    "deepseek_vl2_tiny": {
+      "input_price_per_1m": 0.05,
+      "output_price_per_1m": 0.05
+    },
+    "llava_next_vicuna_7b": {
+      "input_price_per_1m": 0.2,
+      "output_price_per_1m": 0.2
+    }
+  },
+  "benchmarks": {
+    "MMMU_DEV_VAL": "vqa_mc",
+    "MathVista_MINI": "math",
+    "MathVision_MINI": "math",
+    "MathVerse_MINI": "math",
+    "MMBench_DEV_EN_V11": "vqa_mc",
+    "RealWorldQA": "vqa_oe",
+    "MMStar": "vqa_mc",
+    "HallusionBench": "vqa_mc",
+    "TextVQA_VAL": "vqa_oe",
+    "ChartQA_TEST": "ocr_qa",
+    "DocVQA_VAL": "ocr_qa",
+    "InfoVQA_VAL": "ocr_qa",
+    "AI2D_TEST": "vqa_mc",
+    "OCRBench": "ocr_qa"
+  }
+}
diff --git a/configs/description/tasks/VLRouterBench.json b/configs/description/tasks/VLRouterBench.json
@@ -0,0 +1,16 @@
+{
+  "MMMU_DEV_VAL": "Multimodal multi-task understanding (dev + validation)",
+  "MathVista_MINI": "Math visual reasoning and understanding (MINI version)",
+  "MathVision_MINI": "Math visual reasoning (MINI version)",
+  "MathVerse_MINI": "Math reasoning (MINI version)",
+  "MMBench_DEV_EN_V11": "Multimodal benchmark (dev set, English V11)",
+  "RealWorldQA": "Real-world question answering",
+  "MMStar": "MMStar multimodal evaluation",
+  "HallusionBench": "Hallucination detection benchmark",
+  "TextVQA_VAL": "TextVQA (validation set)",
+  "ChartQA_TEST": "Chart question answering (test set)",
+  "DocVQA_VAL": "Document question answering (validation set)",
+  "InfoVQA_VAL": "InfoVQA (validation set)",
+  "AI2D_TEST": "AI2D diagram understanding (test set)",
+  "OCRBench": "OCR capability benchmark"
+}
diff --git a/configs/description/vlrouterbench.json b/configs/description/vlrouterbench.json
@@ -0,0 +1,121 @@
+[
+  {
+    "id": 0,
+    "model_name": "GPT4o",
+    "family": "GPT",
+    "description": "GPT-4o OpenAI flagship multimodal model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 1,
+    "model_name": "GeminiFlash2-5",
+    "family": "Gemini",
+    "description": "Gemini Flash 2.5 Google efficient multimodal model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 2,
+    "model_name": "Gemma3-27B",
+    "family": "Gemma",
+    "description": "Gemma3-27B vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 3,
+    "model_name": "InternVL2_5-78B",
+    "family": "InternVL",
+    "description": "InternVL2.5-78B multimodal model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 4,
+    "model_name": "Janus-Pro-1B",
+    "family": "Janus",
+    "description": "Janus-Pro-1B lightweight vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 5,
+    "model_name": "Janus-Pro-7B",
+    "family": "Janus",
+    "description": "Janus-Pro-7B vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 6,
+    "model_name": "Kimi-VL-A3B-Thinking-2506",
+    "family": "Kimi",
+    "description": "Kimi-VL-A3B-Thinking-2506 thinking-oriented vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 7,
+    "model_name": "MiMo-VL-7B-RL",
+    "family": "MiMo",
+    "description": "MiMo-VL-7B-RL vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 8,
+    "model_name": "Phi-3.5-Vision",
+    "family": "Phi",
+    "description": "Phi-3.5-Vision multimodal model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 9,
+    "model_name": "Pixtral-12B",
+    "family": "Pixtral",
+    "description": "Pixtral-12B vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 10,
+    "model_name": "Qianfan-VL-8B",
+    "family": "Qianfan",
+    "description": "Qianfan-VL-8B vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 11,
+    "model_name": "Qwen2.5-VL-32B-Instruct",
+    "family": "Qwen",
+    "description": "Qwen2.5-VL-32B-Instruct vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 12,
+    "model_name": "Qwen2.5-VL-72B-Instruct",
+    "family": "Qwen",
+    "description": "Qwen2.5-VL-72B-Instruct vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 13,
+    "model_name": "SmolVLM2",
+    "family": "SmolVLM",
+    "description": "SmolVLM2 lightweight vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 14,
+    "model_name": "deepseek_vl2",
+    "family": "DeepSeek",
+    "description": "DeepSeek-VL2 vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 15,
+    "model_name": "deepseek_vl2_tiny",
+    "family": "DeepSeek",
+    "description": "DeepSeek-VL2-Tiny lightweight vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  },
+  {
+    "id": 16,
+    "model_name": "llava_next_vicuna_7b",
+    "family": "LLaVA",
+    "description": "LLaVA-Next-Vicuna-7B vision-language model. A released VL-RouterBench vision-language candidate; this historical identifier is not assumed to be a current API endpoint.",
+    "source": "https://github.com/K1nght/VL-RouterBench/blob/f5a41ce83d6efe67489b35bb7a07ae72976a9aba/config/models.yaml"
+  }
+]
diff --git a/configs/examples/profile_multimodal.json b/configs/examples/profile_multimodal.json
@@ -0,0 +1,3 @@
+{
+  "modality": "text+image"
+}
diff --git a/configs/examples/routefm_qwen.json b/configs/examples/routefm_qwen.json
@@ -0,0 +1,23 @@
+{
+  "modality": "text+image",
+  "embeddings": {
+    "text_model": null,
+    "image_model": null,
+    "joint_model": "Qwen/Qwen3-VL-Embedding-8B",
+    "out_dim": 4096,
+    "normalize": true,
+    "training": false,
+    "batch_size": 2,
+    "qwen": {
+      "backend": "local",
+      "model_dir": "./Qwen3-VL-Embedding-8B"
+    }
+  },
+  "routefm": {
+    "encoder": "qwen",
+    "checkpoint": null,
+    "cache_dir": null,
+    "context_size": 8,
+    "target_batch_size": 128
+  }
+}
diff --git a/configs/routers/ProfileRouter.json b/configs/routers/ProfileRouter.json
@@ -3,7 +3,14 @@
   "modality": "text",
   "profile": {
     "aggregation_hops": 4,
-    "normalization": "sym"
+    "normalization": "sym",
+    "multimodal_embeddings": {
+      "text_model": "ViT-B/16",
+      "image_model": "ViT-B/16",
+      "out_dim": 512,
+      "batch_size": 8,
+      "text_weight": 0.5
+    }
   },
   "embeddings": {
     "text_model": "allenai/longformer-base-4096",