torch.AcceleratorError in surya layout encoder on MPS (Apple Silicon) with 66-page PDF

[DanielRamosAcosta]

🧨 Describe the Bug

marker_single crashes with torch.AcceleratorError: index 4755 is out of bounds: 0, range 0 to 1 during the layout recognition phase when processing a 66-page PDF on Apple Silicon (MPS backend). The error is reproducible and occurs consistently around page 14–16.

📄 Input Document

The PDF that triggers the crash is publicly available:
https://www.defensordelpueblo.es/wp-content/uploads/2025/03/Defensor-del-Pueblo_Informe-anual-2024.pdf

(2.7 MB, 66 pages — Spanish Ombudsman Annual Report 2024)

📤 Output Trace / Stack Trace

Click to expand

2026-03-01 09:40:26,850 [WARNING] surya: `TableRecEncoderDecoderModel` is not compatible with mps backend. Defaulting to cpu instead
Recognizing Layout:  24%|██▍       | 16/66 [01:12<03:47,  4.55s/it]
Traceback (most recent call last):
  File "marker/scripts/convert_single.py", line 38, in convert_single_cli
    rendered = converter(fpath)
  File "marker/converters/pdf.py", line 195, in __call__
    document = self.build_document(temp_path)
  File "marker/converters/pdf.py", line 182, in build_document
    document = DocumentBuilder(self.config)(provider, layout_builder, line_builder, ocr_builder)
  File "marker/builders/document.py", line 33, in __call__
    layout_builder(document, provider)
  File "marker/builders/layout.py", line 56, in __call__
    layout_results = self.surya_layout(document.pages)
  File "marker/builders/layout.py", line 88, in surya_layout
    layout_results = self.layout_model(
        [p.get_image(highres=False) for p in pages],
        batch_size=int(self.get_batch_size()),
    )
  File "surya/layout/__init__.py", line 51, in __call__
    self.foundation_predictor.prediction_loop(...)
  File "surya/foundation/__init__.py", line 780, in prediction_loop
    updated_inputs, outputs, merge_idxs = self.prefill(
        current_inputs, max_lookahead_tokens=0
    )
  File "surya/foundation/__init__.py", line 556, in prefill
    image_embeddings = self.model.get_image_embeddings(
        pixel_values=image_tiles, ...
    )
  File "surya/common/surya/__init__.py", line 258, in get_image_embeddings
    chunk_embeddings = self.vision_encoder.embed_images(
        image_batch=chunk_pixels.unsqueeze(0).to(device=self.device),
        grid_thw=chunk_grid_thw.unsqueeze(0).to(device=self.device),
    )
  File "surya/common/surya/encoder/__init__.py", line 798, in embed_images
    return super().forward(hidden_states=image_batch, grid_thw=grid_thw)
  File "surya/common/surya/encoder/__init__.py", line 765, in forward
    hidden_states = blk(hidden_states, cu_seqlens=cu_seqlens, position_embeddings=position_embeddings)
  File "surya/common/surya/encoder/__init__.py", line 595, in forward
    hidden_states = hidden_states + self.attn(self.norm1(hidden_states), ...)
  File "surya/common/surya/encoder/__init__.py", line 544, in forward
    self.unpack_qkv_with_mask(q, k, v, cu_seqlens)
  File "surya/common/surya/encoder/__init__.py", line 438, in unpack_qkv_with_mask
    max_seq_len = seq_lengths.max().item()
torch.AcceleratorError: index 4755 is out of bounds: 0, range 0 to 1

⚙️ Environment

• Marker version: 1.10.2
• Surya version: 0.17.1
• Python version: 3.13.5
• PyTorch version: 2.10.0
• Transformers version: 4.57.6
• Operating System: macOS 15.5 (Darwin 24.6.0), Apple Silicon (MPS)

✅ Expected Behavior

The PDF should convert to markdown without crashing. A shorter 29-page PDF from the same source (the Anexo version) converts successfully.

📟 Command or Code Used

Click to expand

curl -sL -o /tmp/test.pdf "https://www.defensordelpueblo.es/wp-content/uploads/2025/03/Defensor-del-Pueblo_Informe-anual-2024.pdf"
marker_single /tmp/test.pdf --output_format markdown --disable_image_extraction --output_dir /tmp/output

📎 Additional Context

• The error is reproducible across multiple runs, always crashing at the same point (~page 14–16 of 66).
• surya logs a warning at startup: TableRecEncoderDecoderModel is not compatible with mps backend. Defaulting to cpu instead — but the layout model still runs on MPS and crashes.
• Likely related to a specific page layout (complex table or chart) that causes a tensor index to go out of bounds in the vision encoder's attention mechanism.

[sanjanaj-pr]

Hi, I have been facing the same issue since yesterday. This used to work for me earlier, on the same documents that it's failing on now. I'm running this on Apple M5.

[sanjanaj-pr]

@DanielRamosAcosta It seems to be working for me now, not really sure what went wrong.

[kiki830621]

Additional reproduction case

Same crash here with a 70-page scanned Chinese PDF on Apple Silicon (M4 Pro).

• marker-pdf: 1.10.2
• surya-ocr: 0.17.1
• PyTorch: 2.7.0
• macOS: Darwin 25.3.0

The crash occurs in both the layout recognition phase and the OCR text recognition phase, at varying pages depending on batch size.

torch.AcceleratorError: index 7984 is out of bounds: 2, range 0 to 2184

Root cause analysis

The bug is in surya-ocr, not marker. Specifically in surya/common/surya/encoder/__init__.py:438:

def unpack_qkv_with_mask(self, q, k, v, cu_seqlens):
    seq_lengths = cu_seqlens[1:] - cu_seqlens[:-1]
    max_seq_len = seq_lengths.max().item()  # ← MPS kernel crashes here

This is a PyTorch MPS backend bug — the .max() kernel produces an incorrect index for certain tensor shapes. The computation itself is trivial (find max of a small 1D tensor), but the MPS kernel implementation has an edge case.

Suggested fix (in surya-ocr)

Option 1 — Try an alternative MPS kernel (zero CPU):

max_seq_len = torch.amax(seq_lengths, dim=0).item()
# or
max_seq_len = seq_lengths.reshape(-1).max().item()

Different tensor ops may route through different MPS kernels that don't have this bug.

Option 2 — Surgical .cpu() fallback (negligible perf cost):

max_seq_len = seq_lengths.cpu().max().item()

This only moves a tiny tensor (tens of elements) to CPU. The heavy attention/matmul computations all stay on MPS, so performance impact is effectively zero.

Option 3 — Upgrade PyTorch if a newer version fixes this MPS kernel.

Workarounds I tested

Workaround	Result
TORCH_DEVICE=cpu	✅ Works but extremely slow (~10 min per 70 pages, unusable)
PYTORCH_ENABLE_MPS_FALLBACK=1	❌ Didn't help (the crash is in a supported MPS op, just buggy)
--page_range (batch 10 pages)	⚠️ Reduces frequency but doesn't eliminate the crash
Lower --highres_image_dpi 96	❌ Still crashes

Recommendation

This should also be reported upstream:

1. surya-ocr (VikParuchuri/surya) — to apply the fix in encoder/__init__.py
2. PyTorch — as an MPS backend bug in the .max() kernel for certain tensor shapes

[kiki830621]

Follow-up: Batch size workaround results

After extensive testing with a 70-page scanned Chinese PDF, I found that reducing the --page_range batch size reliably works around the MPS crash. Here are the results:

Batch size vs crash rate

Batch size	Crash rate	Notes
70 pages (full)	100%	Always crashes
10 pages	~50%	Some batches succeed, some crash
5 pages	~20%	Most succeed, occasional crash
2-3 pages	0%	All succeeded

Successful strategy: adaptive batch with retry

1. Start with 5 pages per batch
2. If a batch crashes on MPS, retry with 2-3 pages
3. Merge all batch outputs into a single markdown file

This completed all 70 pages using MPS (GPU) without any CPU fallback.

Feature suggestion

It would be great if marker could handle this internally:

# Pseudocode for adaptive batch processing
def convert_with_retry(pdf_path, pages, batch_size=5):
    for batch in chunk(pages, batch_size):
        try:
            result = convert(pdf_path, page_range=batch)
        except torch.AcceleratorError:
            # Retry with smaller batch
            for mini_batch in chunk(batch, max(1, batch_size // 2)):
                result = convert(pdf_path, page_range=mini_batch)
    return merge(results)

This would make marker resilient to MPS tensor shape edge cases without falling back to CPU (which is ~10x slower).