learning/ai-posttraining.html at main · BobWang21/learning · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
<!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Training - AI</title>
    <link rel="stylesheet" href="styles.css">
    <style>
        .ai-section {
            background: white;
            border-radius: 8px;
            padding: 30px;
            margin-bottom: 30px;
            box-shadow: 0 2px 8px rgba(0, 0, 0, 0.05);
        }

        .ai-section h2 {
            color: #1e40af;
            border-bottom: 2px solid #e2e8f0;
            padding-bottom: 15px;
            margin-top: 0;
        }

        .ai-section h3 {
            color: #7c3aed;
            margin-top: 25px;
            margin-bottom: 15px;
        }

        .highlight-box {
            background: linear-gradient(135deg, rgba(30, 64, 175, 0.05), rgba(124, 58, 237, 0.05));
            border-left: 4px solid #1e40af;
            padding: 15px;
            margin: 15px 0;
            border-radius: 4px;
        }

        .back-link {
            display: inline-block;
            margin-bottom: 20px;
            padding: 10px 20px;
            background: #1e40af;
            color: white;
            text-decoration: none;
            border-radius: 6px;
            transition: all 0.3s ease;
            font-weight: 600;
        }

        .back-link:hover {
            background: #0c4a6e;
            transform: translateY(-2px);
        }

        .breadcrumb {
            color: #64748b;
            font-size: 0.95rem;
            margin-bottom: 20px;
        }

        .breadcrumb a {
            color: #1e40af;
            text-decoration: none;
        }

        .breadcrumb a:hover {
            text-decoration: underline;
        }

        .article-grid {
            display: grid;
            grid-template-columns: repeat(5, 1fr);
            gap: 20px;
        }

        .article-card {
            background: white;
            border: 1px solid #e2e8f0;
            border-radius: 8px;
            padding: 20px;
            transition: all 0.3s ease;
            cursor: pointer;
            height: 100%;
            display: flex;
            flex-direction: column;
        }

        .article-card:hover {
            transform: translateY(-5px);
            box-shadow: 0 8px 20px rgba(0, 0, 0, 0.1);
            border-color: #1e40af;
        }

        .article-card h3 {
            color: #1e40af;
            margin-bottom: 10px;
            font-size: 1.2rem;
        }

        .article-card p {
            color: #64748b;
            font-size: 0.95rem;
            line-height: 1.6;
            margin-bottom: 15px;
            flex-grow: 1;
        }

        .article-link {
            display: inline-block;
            color: #1e40af;
            text-decoration: none;
            font-weight: 600;
            padding: 8px 16px;
            border-radius: 4px;
            background: rgba(30, 64, 175, 0.1);
            transition: all 0.3s ease;
        }

        .article-link:hover {
            background: rgba(30, 64, 175, 0.2);
            transform: translateX(5px);
        }

        .article-card-link {
            text-decoration: none !important;
            display: block;
            color: inherit;
        }

        .article-card-link:hover .article-card {
            transform: translateY(-5px);
            box-shadow: 0 8px 20px rgba(0, 0, 0, 0.1);
            border-color: #1e40af;
        }

        .article-card-link h3 {
            text-decoration: none !important;
        }

        .article-card-link p {
            text-decoration: none !important;
        }

        .article-card-link:visited {
            text-decoration: none !important;
        }

        .article-card-link:active {
            text-decoration: none !important;
        }

        @media (max-width: 1200px) {
            .article-grid {
                grid-template-columns: repeat(3, 1fr);
            }
        }

        @media (max-width: 768px) {
            .article-grid {
                grid-template-columns: repeat(2, 1fr);
            }
        }

        @media (max-width: 480px) {
            .article-grid {
                grid-template-columns: 1fr;
            }
        }
    </style>
</head>
<body>
    <div class="container">
                <div class="sidebar">
            <div class="logo">📚 索引</div>
            <ul class="toc">
                <li><a href="index.html">🏠 首页</a></li>
                <li><a href="index.html#ai">🤖 AI</a>
                    <ul>
                        <li><a href="ai-model.html">1. Model</a>
                            <ul>
                                <li><a href="ai-model-gpt-principles.html">GPT 模型原理</a></li>
                                <li><a href="ai-model-attention-mechanism.html">注意力机制</a></li>
                            </ul>
                        </li>
                        <li><a href="ai-posttraining.html">Training</a>
                            <ul>
                                <li><a href="ai-pretraining.html">Pre-training</a></li>
                                <li><a href="ai-posttraining-overview.html">Post-training 全景指南</a></li>
                                <li><a href="ai-posttraining-peft.html">PEFT 详解</a></li>
                            </ul>
                        </li>
                        <li><a href="ai-agent.html">4. Agent</a>
                            <ul>
                                <li><a href="ai-agent-llm-survey.html">LLM Agent Survey</a></li>
                                <li><a href="ai-agent-agentic-reasoning.html">Agentic Reasoning</a></li>
                                <li><a href="ai-agent-memory.html">Memory</a></li>
                                <li><a href="ai-agent-self-evolving.html">Self-Evolving</a></li>
                                <li><a href="ai-agent-multi-agent.html">Multi-Agent Systems</a></li>
                                <li><a href="ai-agent-agentic-rl.html">Agentic RL</a></li>
                                <li><a href="ai-agent-knowledge-graph.html">Knowledge Graph</a></li>
                                <li><a href="ai-agent-rag.html">RAG</a></li>
                                <li><a href="ai-agent-tree-of-thoughts.html">Tree of Thoughts</a></li>
                                <li><a href="ai-agent-function-calling.html">Tools</a></li>
                            </ul>
                        </li>
                    </ul>
                </li>
                <li><a href="decision.html">🔄 端到端</a></li>
                <li><a href="models.html">⏱️ 预测</a></li>
            </ul>
        </div>

        <main class="content">
            <header class="header">
                <h1>Training</h1>
                <p class="subtitle">Pre-training + Mid-training + Post-training 全流程解析</p>
            </header>

            <div class="breadcrumb">
                <a href="index.html">首页</a> > <a href="index.html#ai">AI</a> > Training
            </div>

            <div class="page-toc">
    <h4 style="margin-bottom: 15px; color: #1e40af;">📑 页面目录</h4>
    <ul style="list-style: none; padding: 0; margin: 0;">
        <li style="margin-bottom: 8px;"><a href="#training" style="color: #1e40af; text-decoration: none; font-weight: 600;">Training</a></li>
        <li style="margin-left: 20px; margin-bottom: 6px;"><a href="#pre-training" style="color: #7c3aed; text-decoration: none; font-size: 0.95rem;">Pre-training</a></li>
        <li style="margin-left: 20px; margin-bottom: 6px;"><a href="#mid-training" style="color: #7c3aed; text-decoration: none; font-size: 0.95rem;">Mid-training</a></li>
        <li style="margin-left: 20px; margin-bottom: 6px;"><a href="#post-training" style="color: #7c3aed; text-decoration: none; font-size: 0.95rem;">Post-training</a></li>
    </ul>
</div>


            <section class="ai-section">
                <h2 id="training">Training</h2>
                <p>大语言模型训练全流程：Pre-training → Mid-training → Post-training</p>

                <h3 id="pre-training">Pre-training</h3>
                <p>预训练是 LLM 训练的第一阶段，通过海量无标注文本数据让模型学习语言规律和世界知识。模型在大量语料上进行自监督学习（如 Next Token Prediction），产出一个"什么都知道一点、但什么都不太好用"的基座模型。</p>
                <div class="article-grid">
                    <a href="ai-pretraining.html" class="article-card-link">
                        <div class="article-card">
                            <h3>Pre-training</h3>
                            <p>系统学习预训练阶段的关键技术与方法，包括 tokenization、BPE 算法等基础知识。</p>
                        </div>
                    </a>
                </div>

                <h3 id="mid-training">Mid-training</h3>
                <p>训练与领域适应是连接预训练与后训练的中间阶段。在预训练完成后，通过持续训练（Continue Training）在特定领域数据上进一步优化模型，使其在特定领域（如代码、数学、科学）表现更好。Mid-training 通常使用比预训练更小但更高质量的数据集。</p>
                <p>相关技术包括：领域自适应预训练（Domain-Adaptive Pretraining）、任务导向的持续训练等。这一阶段在 DeepSeek、Codex 等专业模型中尤为关键。</p>

                <h3 id="post-training">Post-training</h3>
                <p>后训练是将基座模型"精装修"为可用产品的关键阶段，涵盖监督微调（SFT）、偏好优化（RLHF/DPO）和强化学习训练（PPO/GRPO）等核心技术。</p>
                <div class="article-grid">
                    <a href="ai-posttraining-overview.html" class="article-card-link">
                        <div class="article-card">
                            <h3>Post-Training 全景指南</h3>
                            <p>从 RLHF 到 GRPO 再到 Agentic RL，系统梳理后训练的核心方法和最新进展。</p>
                        </div>
                    </a>
                    <a href="ai-posttraining-peft.html" class="article-card-link">
                        <div class="article-card">
                            <h3>PEFT 详解</h3>
                            <p>参数高效微调方法详解，包括 LoRA、QLoRA、Prefix Tuning、Prompt Tuning 等技术。</p>
                        </div>
                    </a>
                </div>

                <a href="index.html#ai" class="back-link">← 返回 AI</a>
            </section>

            <footer class="footer">
                <p>更新时间：2026-04-13</p>
                <p><a href="index.html">← 返回首页</a></p>
            </footer>
        </main>
    </div>
    <!-- 在线编辑器 - Quill.js + GitHub API -->
    <link rel="stylesheet" href="https://cdn.quilljs.com/1.3.7/quill.snow.css">
    <link rel="stylesheet" href="editor.css">
    <script src="https://cdn.quilljs.com/1.3.7/quill.min.js"></script>
    <script src="editor.js"></script>
</body>
</html>