[Contribution] SolarOpenForCausalLM Support

[lifelongeeek]

Description

This PR contributes NeuronX Distributed Inference (NxDI) support for Solar Open 100B MoE (upstage/Solar-Open-100B) following the contrib/ contribution pattern.

Solar Open shares the GLM-4.5 MoE routing architecture but is distinct in that all layers are MoE (first_k_dense_replace=0), uses full RoPE (partial_rotary_factor=1.0), and has no QK-norm or attention bias. The model has been merged into transformers main but is not yet in the current stable release; config and weights are loaded via a direct JSON loader until the stable release ships.

Model Information

Model Name: Solar Open 100B (SolarOpenForCausalLM)

Model Architecture: Decoder-only MoE transformer. All layers are MoE (first_k_dense_replace=0), full RoPE (partial_rotary_factor=1.0), no QK-norm, no attention bias.

Purpose: Text generation

Checklist

Please ensure your PR includes the following items. Refer to the contrib/CONTRIBUTING.md for detailed guidelines.

Required Components

• Accuracy Test (ex. test/integration/test_model.py)

  • At least one integration test that validates model accuracy
  • Uses logit validation or equivalent accuracy verification
  • Test can compile and run the model on Neuron

• README.md with the following sections:

  • Usage Example: Clear code example showing how to use the model
  • Compatibility Matrix: Table showing tested Neuron SDK versions and instance types (Trn1/Trn2/Inf2)
  • Example Checkpoints: Links to compatible model checkpoints (e.g., HuggingFace Hub)
  • Testing Instructions: Command to run the test suite for the model

• Source Code (src/)

  • Modeling code following NxD Inference patterns
  • Properly structured in the contrib folder hierarchy

Optional Components

• Unit Tests (CPU or Neuron-based)
  • Tests for individual modeling components
  • Located in test/unit/ directory

Folder Structure

contrib/models/solar_open/
├── src/solar_open/
│   ├── __init__.py
│   └── modeling_solar_open.py      # NeuronSolarOpenForCausalLM + config + loader
├── test/
│   ├── conftest.py
│   ├── integration/
│   │   ├── config_solar_open_2layers.json
│   │   ├── utils.py                # SolarOpenReferenceModel, get_neuron_config
│   │   └── test_model.py           # smoke, logit accuracy, performance
│   └── unit/
│       ├── test_router.py          # 10 tests
│       ├── test_attention.py       # 8 tests
│       └── test_decoder.py         # 16 tests
├── examples/generation_solar_open_demo.py
└── README.md

Top-level production benchmark script:

examples/generation_solar_open.py   # tp_degree=32, moe_tp_degree=4, moe_ep_degree=8, seq_len=65536

Testing

How did you test this change?

• Unit tests for all model components (router, attention, decoder)
• Integration test validating logit accuracy against a CPU reference model on a 2-layer toy config
• End-to-end text generation on a trn1.32xlarge instance with the full 100B model

Test Results:

Unit tests (CPU):

$ python -m pytest contrib/models/solar_open/test/unit/ -v
...
40 passed in 6.30s

Integration tests (Neuron hardware, 2-layer toy model):

$ python -m pytest contrib/models/solar_open/test/integration/ -v
...
5 passed in 20s

End-to-end generation (trn1.32xlarge, 100B full model):

Prompt: "Could you explain the concept of quantum computing in simple terms?"

================== NxDI Hardware Generation Result (100B Full-Layer) ==================
<|think|>The user asks: "Could you explain the concept of quantum computing in simple terms?"
This is a straightforward request for an explanation. The system prompt says we are Solar Open 100B,
a large language model trained by Upstage AI, Korean startup, knowledge cutoff 2025-07, current date
2026-03-09. There's no disallowed content. So we can comply. Provide a simple explanation of quantum
computing, perhaps analogies, basic concepts: qubits, superposition, entanglement, quantum gates,
quantum algorithms, differences from classical computing, potential applications, challenges. Keep it
simple, accessible. Possibly mention that it's still early stage, but promising. Use Korean? The user
wrote in English. So respond in English. Provide a clear, simple explanation. Possibly include analogies
like coin flipping, Schrödinger's cat, etc. Also mention that quantum computers can solve certain problems
faster, like factoring large numbers (Shor's algorithm), searching unsorted databases (Grover's algorithm),
simulating quantum systems. Also mention that building quantum computers is hard due to decoherence, error
correction, need for low temperature, etc. Provide a simple summary.

We need to ensure we comply with policy. There's no disallowed content. So we can comply.

We should be mindful of the date: knowledge cutoff 2025-07, current date 2026-03-09. So we can mention that
as of 2025, quantum computing is still in early stages, but progress is being made. Possibly mention recent
developments up to 2025. But we must not hallucinate beyond knowledge cutoff. We can say "as of my last
training data up to July 2025..."
======================================================================================

Compatibility

Tested with:

• Neuron SDK Version(s): 2.27
• Instance Type(s): trn1.32xlarge (full 100B model), trn2.3xlarge (2-layer integration test)
• PyTorch Version: 2.8.0
• Python Version: 3.12.3

Related Issues

• Model checkpoint: https://huggingface.co/upstage/Solar-Open-100B
• Solar Open in transformers main: https://github.com/huggingface/transformers/blob/main/src/transformers/models/solar_open/modeling_solar_open.py

vLLM Integration

• This model/feature is intended for use with vLLM
• Documentation includes vLLM registration instructions

For vLLM integration details, see: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/nxd-inference/developer_guides/onboarding-models.html#nxdi-onboarding-models-vllm

---

By submitting this PR, I confirm that:

• I have read and followed the contributing guidelines
• This is a community contribution and may have limited testing compared to officially-supported models
• The code follows best practices and is well-documented
• All required components listed above are included

[ahimsh-aws]

nit: unavailable file - ../../examples/generation_solar_open.py

[ahimsh-aws]

nit: As you mentioned in the generation code. It is available in transformers (>= 5.0.0).

[ahimsh-aws]

Thanks for the contribution! Just some nit comments to address.