feat(serve-ui): integrate cache, flat, and flat-structured storage modes in the Web UI

FORK_NOTES.md

@@ -0,0 +1,62 @@
+# Fork Notes
+
+This file documents fork-specific behavior added on top of upstream v3.x.
+
+## Scope
+
+This fork focuses on Web UI storage behavior and cache management across multiple layouts.
+
+## Storage Modes in Web UI (serve)
+
+The server supports three default storage modes for Web UI downloads:
+
+- cache (default)
+  - HuggingFace-compatible cache layout with hub plus friendly view
+  - Best for Python ecosystem compatibility
+- flat
+  - Plain files directly under the configured cache root
+  - No symlinks
+- flat-structured
+  - Plain files under <cacheRoot>/<owner>/<repo>/
+  - No symlinks
+
+Set default mode at startup:
+
+- hfdownloader serve
+- hfdownloader serve --flat
+- hfdownloader serve --flat-structured
+
+The Settings page can also update cache directory and default storage mode.
+
+## Analyze and Download Behavior
+
+Analyze and Download actions in the Web UI respect the selected storage mode.
+The command preview on Analyze includes storage-mode flags so displayed commands match runtime behavior.
+
+## Cache Page Coverage
+
+The Cache page and Cache API discover repos across all supported layouts:
+
+- HuggingFace cache layout (hub/models--..., hub/datasets--...)
+- flat-structured layout (<cacheRoot>/<owner>/<repo>)
+- flat-mode indexed downloads via .hfd-flat-index manifests
+
+## Deletion Behavior
+
+Delete from Cache supports all layouts above and includes cleanup for partial/interrupted downloads.
+
+- flat-mode delete removes indexed files and multipart artifacts (.part and .part-*)
+- flat-structured and cache-mode delete remove repository directories with safety checks
+
+## Flat-Mode Filename Rules
+
+To reduce root-level collisions in flat mode:
+
+- .gitattributes is skipped
+- README.md is renamed to <repo-name>.README.md
+- generic root artifacts like mmproj* and imatrix* are prefixed with <repo-name>.
+
+## Compatibility Notes
+
+- Empty top-level cache directories (for example hub after delete) may remain and are generally harmless.
+- Existing flat downloads created before flat indexing was introduced may not appear until re-downloaded.

README.md

@@ -282,6 +282,25 @@ hfdownloader download TheBloke/Mistral-7B-Instruct-v0.2-GGUF \
 Both spellings are interchangeable; pick whichever reads better in your
 scripts. They are mutually exclusive on a single command line.
 
+### Web UI Storage Modes (serve)
+
+When using the Web UI, the default storage mode is controlled by `serve`
+flags and can be changed from Settings:
+
+```bash
+# Default: HuggingFace cache layout (hub/ + friendly view)
+hfdownloader serve
+
+# Flat: write files directly to cache root (no hub/, no symlinks)
+hfdownloader serve --flat
+
+# Flat-structured: write files to <cacheRoot>/<owner>/<repo>/
+hfdownloader serve --flat-structured
+```
+
+The Web UI Settings page stores both the cache directory and default storage
+mode, and Analyze/Download actions follow that mode.
+
 ### Manifest Tracking
 
 Every download creates `hfd.yaml` so you know exactly what you have:
@@ -332,7 +351,7 @@ Browse everything you've downloaded with stats, search, and filters:
 | **Jobs** | Real-time WebSocket progress, pause/resume/cancel, download history |
 | **Cache** | Browse downloaded repos, disk usage stats, search & filter |
 | **Mirror** | Configure targets, compare differences, push/pull sync |
-| **Settings** | Token, connections, proxy, verification mode |
+| **Settings** | Token, cache directory, default storage mode, connections, proxy, verification mode |
 
 ### Server Options
 

docs/API.md

@@ -119,6 +119,7 @@ Start a new download job.
 | `excludes` | string[] | No | `[]` | Exclude patterns |
 | `appendFilterSubdir` | boolean | No | `false` | Create filter subdirs |
 | `dryRun` | boolean | No | `false` | Plan only |
+| `storageMode` | string | No | `cache` | Storage mode: `cache`, `flat`, `flat-structured` |
 
 **Filter Syntax**
 
@@ -147,7 +148,7 @@ Or as separate field:
   "isDataset": false,
   "filters": ["q4_k_m"],
   "excludes": [],
-  "outputDir": "/home/user/.cache/huggingface/hub",
+  "outputDir": "/home/user/.cache/huggingface",
   "status": "queued",
   "progress": {
     "totalFiles": 0,
@@ -208,7 +209,7 @@ Get download plan without starting download.
 
 **Request Body**
 
-Same as `/api/download`.
+Same as `/api/download`, including optional `storageMode`.
 
 **Response** `200 OK`
 
@@ -261,7 +262,7 @@ List all download jobs.
       "isDataset": false,
       "filters": ["q4_k_m"],
       "excludes": [],
-      "outputDir": "/home/user/.cache/huggingface/hub",
+      "outputDir": "/home/user/.cache/huggingface",
       "status": "running",
       "progress": {
         "totalFiles": 3,
@@ -462,7 +463,8 @@ Get current server settings.
 ```json
 {
   "token": "********mnop",
-  "cacheDir": "/home/user/.cache/huggingface/hub",
+  "cacheDir": "/home/user/.cache/huggingface",
+  "storageMode": "cache",
   "connections": 8,
   "maxActive": 3,
   "multipartThreshold": "32MiB",
@@ -491,14 +493,15 @@ Update server settings.
 | Field | Type | Description |
 |-------|------|-------------|
 | `token` | string | HuggingFace token |
+| `cacheDir` | string | Cache root directory used by server |
+| `storageMode` | string | Default mode: `cache`, `flat`, `flat-structured` |
 | `connections` | integer | Connections per file |
 | `maxActive` | integer | Max concurrent downloads |
 | `multipartThreshold` | string | Min size for multipart |
 | `verify` | string | Verification: none, size, sha256 |
 | `retries` | integer | Retry attempts |
 
 **Security Restrictions**
-- `cacheDir` cannot be changed via API
 - `modelsDir` cannot be changed via API
 - `datasetsDir` cannot be changed via API
 
@@ -518,6 +521,8 @@ curl -X POST http://localhost:8080/api/settings \
   -H "Content-Type: application/json" \
   -d '{
     "token": "hf_xxxxx",
+    "cacheDir": "/home/user/ai_models",
+    "storageMode": "flat-structured",
     "connections": 16,
     "maxActive": 8
   }'
@@ -644,6 +649,12 @@ curl "http://localhost:8080/api/analyze/owner/repo?revision=v1.0"
 
 List cached repositories.
 
+This endpoint scans all supported storage layouts:
+
+- HuggingFace cache layout (`hub/models--*`, `hub/datasets--*`)
+- Flat-structured layout (`<cacheRoot>/<owner>/<repo>/`)
+- Flat-mode indexed downloads (`<cacheRoot>/.hfd-flat-index/*.yaml`)
+
 **Query Parameters**
 
 | Parameter | Type | Description |
@@ -666,8 +677,17 @@ List cached repositories.
       "path": "/home/user/.cache/huggingface/hub/datasets--facebook--flores"
     }
   ],
-  "count": 2,
-  "cacheDir": "/home/user/.cache/huggingface/hub"
+  "stats": {
+    "totalModels": 1,
+    "totalDatasets": 1,
+    "totalSize": 123456789,
+    "totalSizeHuman": "117.7 MiB",
+    "totalFiles": 42
+  },
+  "cacheDir": "/home/user/.cache/huggingface",
+  "scannedCacheDirs": [
+    "/home/user/.cache/huggingface"
+  ]
 }
 ```
 

docs/CLI.md

@@ -246,6 +246,8 @@ hfdownloader serve [flags]
 | `--verify` | | string | `size` | Verification mode |
 | `--retries` | | int | `4` | Retry attempts |
 | `--endpoint` | | string | | Custom HF endpoint |
+| `--flat` | | bool | `false` | Default Web UI storage mode: flat files at cache root |
+| `--flat-structured` | | bool | `false` | Default Web UI storage mode: files under `<cacheRoot>/<owner>/<repo>/` |
 | `--auth-user` | | string | | Basic auth username |
 | `--auth-pass` | | string | | Basic auth password |
 | `--models-dir` | | string | `./Models` | Legacy models directory |
@@ -271,8 +273,24 @@ hfdownloader serve --endpoint https://hf-mirror.com
 
 # High-performance settings
 hfdownloader serve -c 16 --max-active 8
+
+# Default Web UI downloads to flat files at cache root
+hfdownloader serve --flat
+
+# Default Web UI downloads to owner/repo directories
+hfdownloader serve --flat-structured
 ```
 
+#### Storage Mode Defaults
+
+`serve` controls the default storage mode used by the Web UI:
+
+- Cache mode (default): HF-compatible `hub/` layout plus friendly symlinks
+- Flat mode (`--flat`): plain files directly under cache root
+- Flat-structured mode (`--flat-structured`): plain files under `<cacheRoot>/<owner>/<repo>/`
+
+Users can change this default later in the Web UI Settings page.
+
 #### Server Features
 
 - **Web UI** at `http://localhost:8080`

internal/assets/static/css/style.css

@@ -376,6 +376,21 @@ a {
   border: 1px solid var(--color-border);
   border-radius: var(--radius-lg);
   overflow: hidden;
+  display: flex;
+  flex-direction: column;
+}
+
+.card:has(.card-body.scrollable) {
+  overflow: visible;
+}
+
+.card {
+  background: var(--gradient-surface);
+  border: 1px solid var(--color-border);
+  border-radius: var(--radius-lg);
+  overflow: hidden;
+  display: flex;
+  flex-direction: column;
 }
 
 .card-header {
@@ -384,6 +399,7 @@ a {
   gap: 12px;
   padding: 20px 24px;
   border-bottom: 1px solid var(--color-border);
+  flex-shrink: 0;
 }
 
 .card-header h2, .card-header h3 {
@@ -399,6 +415,39 @@ a {
 
 .card-body {
   padding: 24px;
+  flex: 1;
+}
+
+.card-body.scrollable {
+  max-height: 500px;
+  overflow-y: auto;
+  overflow-x: hidden;
+  flex: 1;
+  min-height: 0;
+}
+
+.card-body.scrollable::-webkit-scrollbar {
+  width: 8px;
+}
+
+.card-body.scrollable::-webkit-scrollbar-track {
+  background: rgba(255, 255, 255, 0.05);
+  border-radius: 4px;
+}
+
+.card-body.scrollable::-webkit-scrollbar-thumb {
+  background: rgba(255, 255, 255, 0.2);
+  border-radius: 4px;
+}
+
+.card-body.scrollable::-webkit-scrollbar-thumb:hover {
+  background: rgba(255, 255, 255, 0.3);
+}
+
+/* Firefox scrollbar */
+.card-body.scrollable {
+  scrollbar-width: thin;
+  scrollbar-color: rgba(255, 255, 255, 0.2) rgba(255, 255, 255, 0.05);
 }
 
 /* ============================================

internal/assets/static/index.html

@@ -177,6 +177,14 @@ <h2>Model</h2>
                 <label for="modelExclude">Exclude</label>
                 <input type="text" id="modelExclude" placeholder=".md, fp16">
               </div>
+              <div class="form-group">
+                <label for="modelStorageMode">Storage Mode</label>
+                <select id="modelStorageMode">
+                  <option value="cache">HF Cache (Default)</option>
+                  <option value="flat">Flat Files</option>
+                  <option value="flat-structured">Flat with Structure (owner/model)</option>
+                </select>
+              </div>
               <div class="form-actions">
                 <button type="button" class="btn btn-secondary" id="previewModelBtn">Preview</button>
                 <button type="submit" class="btn btn-primary">Download</button>
@@ -212,6 +220,14 @@ <h2>Dataset</h2>
                 <label for="datasetExclude">Exclude</label>
                 <input type="text" id="datasetExclude" placeholder=".md, raw">
               </div>
+              <div class="form-group">
+                <label for="datasetStorageMode">Storage Mode</label>
+                <select id="datasetStorageMode">
+                  <option value="cache">HF Cache (Default)</option>
+                  <option value="flat">Flat Files</option>
+                  <option value="flat-structured">Flat with Structure (owner/model)</option>
+                </select>
+              </div>
               <div class="form-actions">
                 <button type="button" class="btn btn-secondary" id="previewDatasetBtn">Preview</button>
                 <button type="submit" class="btn btn-primary">Download</button>
@@ -597,16 +613,27 @@ <h1>Settings</h1>
             <div class="card-header">
               <h3>Storage</h3>
             </div>
-            <div class="card-body">
+            <div class="card-body scrollable">
               <div class="form-group">
-                <label>HuggingFace Cache Directory</label>
-                <div id="cacheDir" class="cache-dir-display"></div>
+                <label for="cacheDir">HuggingFace Cache Directory (HF_HOME)</label>
+                <input type="text" id="cacheDir" placeholder="~/.cache/huggingface">
                 <p class="form-hint">
-                  Downloads use the standard HuggingFace cache structure for Python compatibility.
+                  Directory where HuggingFace cache structure will be created (equivalent to HF_HOME environment variable).
+                  Changes are saved to config file.
                 </p>
-                <p class="form-hint" style="margin-top: 8px;">
-                  <strong>To change:</strong> Set the <code>HF_HOME</code> environment variable before starting the server,
-                  or use <code>--cache-dir</code> when launching <code>hfdownloader serve</code>.
+              </div>
+              <div class="form-group" style="margin-top: 16px;">
+              <label for="storageMode">Default Storage Mode</label>
+              <select id="storageMode">
+                <option value="cache">HF Cache (Default)</option>
+                <option value="flat">Flat Files</option>
+                <option value="flat-structured">Flat with Structure (owner/model)</option>
+              </select>
+                <p class="form-hint">
+                  Default: <strong>HF Cache</strong> (HuggingFace cache structure)<br>
+                  <strong>Flat:</strong> Files saved directly to output directory<br>
+                  <strong>Flat Structured:</strong> Files in owner/model subdirectories<br>
+                <em>Server CLI flags <code>--flat</code> or <code>--flat-structured</code> override this setting while the server is running.</em>
                 </p>
               </div>
             </div>
@@ -691,7 +718,7 @@ <h3>Advanced</h3>
             <div class="card-header">
               <h3>Proxy Settings</h3>
             </div>
-            <div class="card-body">
+            <div class="card-body scrollable">
               <div class="form-group">
                 <label for="proxyUrl">Proxy URL</label>
                 <input type="text" id="proxyUrl" placeholder="http://proxy:8080 or socks5://proxy:1080">