Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/plugin.yml
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,15 @@ jobs:
shell: bash
run: make -C gxplug check GCC="$GXPLUG_GCC"

# The five plugins B05 is written around, built and loaded on this channel too. The
# lesson shows a recording made on one machine, so this is what stops it describing
# plugins that only work where they were written. It also covers the three things
# gxplug itself never exercises: a pass inserted into the pipeline, a gate overridden
# from outside, and a plugin being refused at load time on purpose.
- name: the five example plugins do what B05 says they do
shell: bash
run: make -C gxplug examples-check GCC="$GXPLUG_GCC"

# A build that produces no events is a plugin that loaded and did nothing, which the
# byte comparison above would happily call a pass.
- name: the stream is real, and reaches RTL
Expand Down
54 changes: 53 additions & 1 deletion GLOSSARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This file is generated from `gxray/glossary.py`. Edit that and run `just build-g

## Index

[DejaGnu](#dejagnu) | [GENERIC](#generic) | [GIMPLE](#gimple) | [IRA](#ira) | [LRA](#lra) | [RTL](#rtl) | [RTX](#rtx) | [SSA](#ssa) | [SSA name](#ssa-name) | [allocno](#allocno) | [alternative](#alternative) | [assembler directive](#assembler-directive) | [back end](#back-end) | [basic block](#basic-block) | [bootstrap](#bootstrap) | [bubbling](#bubbling) | [build config](#build-config) | [cc1](#cc1) | [checking build](#checking-build) | [collect2](#collect2) | [constraint](#constraint) | [control flow graph](#control-flow-graph) | [cross compiler](#cross-compiler) | [current function](#current-function) | [debug counter](#debug-counter) | [default definition](#default-definition) | [define_insn](#define_insn) | [definition](#definition) | [directive](#directive) | [dominance](#dominance) | [driver](#driver) | [dump file](#dump-file) | [edge](#edge) | [effective target](#effective-target) | [excess errors](#excess-errors) | [expand](#expand) | [final](#final) | [front end](#front-end) | [garbage collector](#garbage-collector) | [gate](#gate) | [generated file](#generated-file) | [gengtype](#gengtype) | [gimplification](#gimplification) | [hard register](#hard-register) | [immediate dominator](#immediate-dominator) | [inferior call](#inferior-call) | [insn](#insn) | [interference](#interference) | [live range](#live-range) | [loop](#loop) | [machine description](#machine-description) | [machine mode](#machine-mode) | [middle end](#middle-end) | [mode iterator](#mode-iterator) | [optimization level](#optimization-level) | [out of SSA](#out-of-ssa) | [out of tree build](#out-of-tree-build) | [output template](#output-template) | [param](#param) | [pass](#pass) | [pass manager](#pass-manager) | [phi node](#phi-node) | [poly_int](#poly_int) | [port](#port) | [pretty printer](#pretty-printer) | [pseudo register](#pseudo-register) | [register allocation](#register-allocation) | [register class](#register-class) | [register pressure](#register-pressure) | [section](#section) | [spec](#spec) | [spill](#spill) | [stage comparison](#stage-comparison) | [stamp file](#stamp-file) | [sum file](#sum-file) | [target hook](#target-hook) | [target triple](#target-triple) | [temporary](#temporary) | [three address form](#three-address-form) | [torture options](#torture-options) | [tree](#tree) | [use](#use) | [wide_int](#wide_int)
[DejaGnu](#dejagnu) | [GENERIC](#generic) | [GIMPLE](#gimple) | [IRA](#ira) | [LRA](#lra) | [RTL](#rtl) | [RTX](#rtx) | [SSA](#ssa) | [SSA name](#ssa-name) | [TODO flags](#todo-flags) | [allocno](#allocno) | [alternative](#alternative) | [assembler directive](#assembler-directive) | [back end](#back-end) | [basic block](#basic-block) | [bootstrap](#bootstrap) | [bubbling](#bubbling) | [build config](#build-config) | [cc1](#cc1) | [checking build](#checking-build) | [collect2](#collect2) | [constraint](#constraint) | [control flow graph](#control-flow-graph) | [cross compiler](#cross-compiler) | [current function](#current-function) | [debug counter](#debug-counter) | [default definition](#default-definition) | [define_insn](#define_insn) | [definition](#definition) | [directive](#directive) | [dominance](#dominance) | [driver](#driver) | [dump file](#dump-file) | [edge](#edge) | [effective target](#effective-target) | [excess errors](#excess-errors) | [expand](#expand) | [final](#final) | [front end](#front-end) | [garbage collector](#garbage-collector) | [gate](#gate) | [generated file](#generated-file) | [gengtype](#gengtype) | [gimplification](#gimplification) | [hard register](#hard-register) | [immediate dominator](#immediate-dominator) | [inferior call](#inferior-call) | [insn](#insn) | [interference](#interference) | [live range](#live-range) | [loop](#loop) | [machine description](#machine-description) | [machine mode](#machine-mode) | [middle end](#middle-end) | [mode iterator](#mode-iterator) | [optimization level](#optimization-level) | [out of SSA](#out-of-ssa) | [out of tree build](#out-of-tree-build) | [output template](#output-template) | [param](#param) | [pass](#pass) | [pass manager](#pass-manager) | [pass positioning](#pass-positioning) | [phi node](#phi-node) | [plugin](#plugin) | [plugin ABI](#plugin-abi) | [plugin event](#plugin-event) | [poly_int](#poly_int) | [port](#port) | [pretty printer](#pretty-printer) | [pseudo register](#pseudo-register) | [pseudo-event](#pseudo-event) | [register allocation](#register-allocation) | [register class](#register-class) | [register pressure](#register-pressure) | [section](#section) | [spec](#spec) | [spill](#spill) | [stage comparison](#stage-comparison) | [stamp file](#stamp-file) | [sum file](#sum-file) | [target hook](#target-hook) | [target triple](#target-triple) | [temporary](#temporary) | [three address form](#three-address-form) | [torture options](#torture-options) | [tree](#tree) | [use](#use) | [wide_int](#wide_int)

## Reading the source

Expand Down Expand Up @@ -713,3 +713,55 @@ Also written `dg-require-effective-target`, `target-supports.exp`, `dg-skip-if`.
A line is a state and a name, `PASS: gcc.dg/pr1234.c (test for excess errors)`, and one test file produces several of them. Five of the thirteen states make a run red and the other eight are information, and `XFAIL` in particular means a known failure that failed as expected. The right way to read one is against a previous one: a `PASS` that became a `FAIL` is a regression, and a name that is in the old file and not the new one is a test that stopped being run, which the summary block at the bottom cannot show you and which is the more dangerous of the two.

Also written `.sum`, `.log`, `XFAIL`. Taught in B04. See also [excess errors](#excess-errors), [effective target](#effective-target).

## Extending the compiler

Words you need to load code of your own into GCC, put a pass where you want it, and understand why the thing you are writing against is not an API. B05 is the lesson.

### plugin

**A shared object loaded into `cc1` at startup, which registers functions to be called at named points during a compilation.**

`-fplugin=./thing.so` and the compiler proper `dlopen`s it with `RTLD_NOW`, looks for a symbol called `plugin_is_GPL_compatible` by name, and calls `plugin_init`. Everything a plugin can do follows from being inside the same process: it sees the real IR, calls the real functions, and crashes the real compiler. There are three ways it can be refused, all of them before it runs a line of its own code, and all three stop the compilation rather than warning and carrying on. It is the only supported way to observe or change what GCC does without patching GCC.

Also written `-fplugin=`, `plugin_init`, `-fplugin-arg-`. Taught in B05. See also [plugin event](#plugin-event), [plugin ABI](#plugin-abi). In the source: [`gcc/plugin.cc:699@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/plugin.cc#L699).

### plugin event

**A named point in a compilation where GCC calls whatever a plugin registered for it.**

`gcc/plugin.def` is a file of twenty six one-line macro calls, and the order of that file is the ABI, because the enumerator's value is the index into the callback table. Twenty three of them are fired, from twenty nine places, and what arrives with each one is a `void *` whose real type is written in the call site and nowhere else. Firing an event is a walk down a linked list of callbacks in registration order, with no return value read, so a plugin cannot refuse an event or stop another plugin from seeing it.

Also written `DEFEVENT`, `register_callback`, `invoke_plugin_callbacks`. Taught in B05. See also [plugin](#plugin), [pseudo-event](#pseudo-event). In the source: [`gcc/plugin.def:20@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/plugin.def#L20).

### pseudo-event

**One of the three names in the event list that is never fired, and is instead acted on at the moment you register for it.**

`PLUGIN_PASS_MANAGER_SETUP`, `PLUGIN_INFO` and `PLUGIN_REGISTER_GGC_ROOTS` are handled inside `register_callback` itself: it asserts the callback is null and uses the user data argument straight away. So registering a pass is spelled the same way as registering a callback and does something else entirely, which is the single most confusing thing about the mechanism, and the reason a plugin that passes a function pointer alongside a `register_pass_info` gets an assertion failure rather than a diagnostic.

Also written `PLUGIN_PASS_MANAGER_SETUP`, `register_callback`. Taught in B05. See also [plugin event](#plugin-event), [pass positioning](#pass-positioning). In the source: [`gcc/plugin.cc:458@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/plugin.cc#L458).

### pass positioning

**The four fields that say where in the pipeline a plugin's pass goes: which pass to hang it on, which run of that pass, and before, after or instead of.**

`struct register_pass_info` holds the new pass, a `reference_pass_name`, a `ref_pass_instance_number` where zero means every instance and one means the first, and a `pos_op` of `PASS_POS_INSERT_AFTER`, `PASS_POS_INSERT_BEFORE` or `PASS_POS_REPLACE`. The name to give is the pass name and not the dump name: `-fdump-tree-cddce1` is the pass called `cddce` on its first instance, and a reference to `cddce1` matches nothing. A reference that matches nothing is a fatal error at registration, which is the one mistake in this area that tells you about itself.

Also written `register_pass_info`, `PASS_POS_INSERT_AFTER`, `position_pass`. Taught in B05. See also [pass](#pass), [pseudo-event](#pseudo-event). In the source: [`gcc/tree-pass.h:328@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/tree-pass.h#L328).

### plugin ABI

**The unwritten contract between a plugin and the compiler that loads it, which is every private header GCC has and every option it was configured with.**

`plugin_default_version_check` compares five fields, not one: the base version, the datestamp, the development phase, the revision and the whole configuration argument string. Two compilers of the same version built with different `--enable` flags have incompatible plugin ABIs, because those flags change struct layouts. That is why a plugin has to be built by the compiler that will load it, why the check is the plugin's own job rather than the compiler's, and why there is no such thing as shipping a binary plugin.

Also written `plugin_default_version_check`, `plugin-version.h`, `gcc_version`. Taught in B05. See also [plugin](#plugin), [build config](#build-config). In the source: [`gcc/plugin.cc:1013@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/plugin.cc#L1013).

### TODO flags

**What a pass tells the pass manager to do after it has run, returned as a bit mask from `execute`.**

`TODO_cleanup_cfg`, `TODO_update_ssa`, `TODO_verify_all` and their neighbours. A pass that changed nothing returns zero. A pass that modified the IR and returned zero has left the compiler believing things that are no longer true, and the failure shows up several passes later in code that did nothing wrong, which is the most expensive mistake a first plugin can make. There is a second set, `todo_flags_start`, that runs before the pass instead, and `TODO_mark_first_instance` in it is how the pass manager knows which run of a repeated pass is the first.

Also written `todo_flags_finish`, `TODO_update_ssa`, `pass_data`. Taught in B05. See also [pass](#pass), [pass positioning](#pass-positioning). In the source: [`gcc/tree-pass.h:238@releases/gcc-16.2.0`](https://github.com/gcc-mirror/gcc/blob/releases/gcc-16.2.0/gcc/tree-pass.h#L238).
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,9 @@ Lessons do not invent a fresh example each time either. There are three programs
| B02 | [Building it three times, and the one thing that proves](https://github.com/tamnd/gcc-internals/blob/main/lessons/b02-the-bootstrap/b02.ipynb) | Why a compiler is built three times and compared twice, what that comparison can and cannot catch, the six files it is told to ignore, and a real stage comparison failure induced on purpose so you can see what it looks like before it happens to you | M2 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/gcc-internals/blob/main/lessons/b02-the-bootstrap/b02.ipynb) |
| B03 | [A debugger on the compiler, and the counter for when you cannot have one](https://github.com/tamnd/gcc-internals/blob/main/lessons/b03-the-debugger/b03.ipynb) | A real gdb session against a real cc1, recorded command by command: what the compiler's own gdbinit did before you typed anything, how to stop on one pass out of several hundred, how to read a function from a breakpoint, and how to find the exact transformation that broke your code by bisecting a debug counter instead | M2 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/gcc-internals/blob/main/lessons/b03-the-debugger/b03.ipynb) |
| B04 | [Forty thousand tests, and how to run one of them](https://github.com/tamnd/gcc-internals/blob/main/lessons/b04-the-test-suite/b04.ipynb) | The GCC test suite taken apart: what a directive is, why the command line is not what the file says, the subtractive rule that makes a test fail for output nobody asked about, the six compilations every torture test costs, how to run one file without running the other thirty five thousand, and the one question worth asking of a .sum | M2 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/gcc-internals/blob/main/lessons/b04-the-test-suite/b04.ipynb) |
| B05 | [Sixty lines of C++, and you are inside the compiler](https://github.com/tamnd/gcc-internals/blob/main/lessons/b05-the-plugin/b05.ipynb) | GCC's plugin mechanism from the outside in: the three things a plugin has to have, the three ways it is refused, what an event actually is, a GIMPLE pass of your own inserted after ssa with its own dump file, switching one of GCC's passes off from outside and watching the assembly move, and why the thing you are writing against is not an API | M2 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/gcc-internals/blob/main/lessons/b05-the-plugin/b05.ipynb) |

16 of 96 written.
17 of 96 written.
<!-- nbbuild:end index -->

This table is generated from the lessons, by the same command that builds them, so it cannot list a lesson that does not exist or miss one that does. T05 is the pilot and it is deliberately in the middle of Part I rather than at the start, because everything the course promises has to be true of a hard lesson before it is worth writing the easy ones.
Expand Down
40 changes: 40 additions & 0 deletions citations.lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -1123,6 +1123,10 @@
"hash": "32cf5ac12ec10166",
"text": " invoke_plugin_callbacks (PLUGIN_NEW_PASS, new_pass);"
},
"gcc/passes.cc:1377@releases/gcc-16.2.0": {
"hash": "e2440f87e01d0e17",
"text": "/* Insert the pass at the proper position. Return true if the pass"
},
"gcc/passes.cc:1480@releases/gcc-16.2.0": {
"hash": "51646e5bb483387d",
"text": "register_pass (struct register_pass_info *pass_info)"
Expand Down Expand Up @@ -1287,6 +1291,10 @@
"hash": "bc750d58492694fc",
"text": "\t NEXT_PASS (pass_remove_cgraph_callee_edges);"
},
"gcc/plugin.cc:1004@releases/gcc-16.2.0": {
"hash": "adaccefb6d38829e",
"text": "/* The default version check. Compares every field in VERSION. */"
},
"gcc/plugin.cc:1007@releases/gcc-16.2.0": {
"hash": "d5e15bb6d9a8357f",
"text": "plugin_default_version_check (struct plugin_gcc_version *gcc_version,"
Expand All @@ -1295,6 +1303,10 @@
"hash": "4269ae3ebcee8a0c",
"text": ""
},
"gcc/plugin.cc:1013@releases/gcc-16.2.0": {
"hash": "62d69450ab9d232d",
"text": " if (strcmp (gcc_version->basever, plugin_version->basever))"
},
"gcc/plugin.cc:1043@releases/gcc-16.2.0": {
"hash": "ac5b6b08ed253871",
"text": "default_plugin_dir_name (void)"
Expand Down Expand Up @@ -1343,6 +1355,10 @@
"hash": "5bf83c681ec1db72",
"text": "get_named_event_id (const char *name, enum insert_option insert)"
},
"gcc/plugin.cc:450@releases/gcc-16.2.0": {
"hash": "bcf5f3c03b95cac3",
"text": "void"
},
"gcc/plugin.cc:458@releases/gcc-16.2.0": {
"hash": "2684a70ad2d92b09",
"text": " case PLUGIN_PASS_MANAGER_SETUP:"
Expand Down Expand Up @@ -1379,6 +1395,10 @@
"hash": "dec9f0ac799e2524",
"text": "# define DEFEVENT(NAME) GCC_PLUGIN_STRINGIFY1 (NAME),"
},
"gcc/plugin.cc:582@releases/gcc-16.2.0": {
"hash": "797c574a6f4323c9",
"text": " {"
},
"gcc/plugin.cc:596@releases/gcc-16.2.0": {
"hash": "3c735a762c2877de",
"text": " gcc_assert (false);"
Expand All @@ -1391,10 +1411,22 @@
"hash": "de7650830617d45d",
"text": "try_init_one_plugin (struct plugin_name_args *plugin)"
},
"gcc/plugin.cc:699@releases/gcc-16.2.0": {
"hash": "12f26bafa7944fcd",
"text": " /* We use RTLD_NOW to accelerate binding and detect any mismatch"
},
"gcc/plugin.cc:713@releases/gcc-16.2.0": {
"hash": "2c32f30d11101c3a",
"text": " /* Check the plugin license. */"
},
"gcc/plugin.cc:716@releases/gcc-16.2.0": {
"hash": "e1b04ee5a1c4bbf3",
"text": "\t\t \"plugin %s is not licensed under a GPL-compatible license:\""
},
"gcc/plugin.cc:731@releases/gcc-16.2.0": {
"hash": "cfd0fa2a390035aa",
"text": " /* Call the plugin-provided initialization routine with the arguments. */"
},
"gcc/plugin.cc:738@releases/gcc-16.2.0": {
"hash": "d042a48384209bbc",
"text": " /* leak dl_handle on purpose to ensure the plugin is loaded for the"
Expand All @@ -1415,6 +1447,10 @@
"hash": "df5740881a86dc4c",
"text": " fnotice (stderr, \"*** WARNING *** there are active plugins, do not report\""
},
"gcc/plugin.def:20@releases/gcc-16.2.0": {
"hash": "468bf48d8250da5b",
"text": "/* Called before parsing the body of a function. */"
},
"gcc/plugin.h:153@releases/gcc-16.2.0": {
"hash": "f1cdd28a83b226c7",
"text": "extern int plugin_is_GPL_compatible;"
Expand Down Expand Up @@ -1759,6 +1795,10 @@
"hash": "8480add6167edcd4",
"text": "#define PROP_gimple \\"
},
"gcc/tree-pass.h:238@releases/gcc-16.2.0": {
"hash": "f9a020dff3f3bbd5",
"text": "/* To-do flags. */"
},
"gcc/tree-pass.h:240@releases/gcc-16.2.0": {
"hash": "78c56868081ddba0",
"text": "#define TODO_do_not_ggc_collect\t\t(1 << 1)"
Expand Down
Loading
Loading