Skip to content

build: include pre-generated parser files and manual regeneration target#1003

Merged
wismill merged 2 commits intoxkbcommon:masterfrom
wismill:build/manual-parser-files-regeneration
Apr 22, 2026
Merged

build: include pre-generated parser files and manual regeneration target#1003
wismill merged 2 commits intoxkbcommon:masterfrom
wismill:build/manual-parser-files-regeneration

Conversation

@wismill
Copy link
Copy Markdown
Member

@wismill wismill commented Mar 24, 2026

This change introduces pre-generated parser.c and parser.h files into the source tree and adds a manual mechanism to regenerate them.

  • Added a yacc-wrapper script to ensure Bison runs from the project root. This guarantees that `#line`` directives in generated files use relative paths, keeping the source tree sanitized and portable.
  • Implement a 'regenerate' custom target (not built by default) to allow developers to manually sync the parser after a parser.y update.
  • Include the initial commit of the generated parser.c`` and parser.h` to allow out-of-the-box compilation without a yacc dependency.
  • Do not regenerate the parser files in CI, except for some Linux setup.

This approach prevents “Bison lock-in” while maintaining the ability to update the grammar as needed.

Fixes #997

@whot @bluetech @vazub

TODO:

  • Add changelog entry
  • Add license for code from parser.y
  • Remove bison installation in CI

@wismill wismill added this to the 1.14.0 milestone Mar 24, 2026
@wismill wismill requested review from bluetech and whot March 24, 2026 15:54
@wismill wismill added compile-keymap Indicates a need for improvements or additions to keymap compilation build system labels Mar 24, 2026
@wismill wismill force-pushed the build/manual-parser-files-regeneration branch from 203e980 to 87f92e2 Compare March 25, 2026 11:34
@wismill
Copy link
Copy Markdown
Member Author

wismill commented Apr 8, 2026

@whot @bluetech a quick review would be greatly appreciated 🙂, since it is a significant change of the build.

@bluetech
Copy link
Copy Markdown
Member

I have a hazy memory that we used to do this once (we also used to use flex back then), then removed it. I believe Debian at least has a policy of generating the files itself (to provide the ability to modify the source code in its preferred form), so the ability will need to be maintained at least.

Overall this adds complexity for a rather thin benefit, as bison is pretty close to ubiquitous. But if you want to help @vazub avoid bison, that's OK. Ideally byacc would add the detailed error feature, then we could just restore byacc support...

@wismill wismill force-pushed the build/manual-parser-files-regeneration branch 20 times, most recently from ae6e650 to e6c6de0 Compare April 14, 2026 09:53
@wismill
Copy link
Copy Markdown
Member Author

wismill commented Apr 14, 2026

I believe Debian at least has a policy of generating the files itself (to provide the ability to modify the source code in its preferred form), so the ability will need to be maintained at least.

It seems Fedora has some policy about it too. So let’s be a good FLOSS citizen and keep bison requirement by default (see hereinafter).

@bluetech By the same reasoning it seems that we should also make the build depend on gperf for keywords.c.

@wismill
Copy link
Copy Markdown
Member Author

wismill commented Apr 14, 2026

I added the build option enable-parser-generation enable-parser-auto-generation:

  • If true (default), the build requires bison and auto-generates the parser files.
  • Otherwise bison is optional and the build uses pre-generated files.

There is also a manual target src/xkbcomp/update-parser to regenerate bundled parser files, which depends on bison availability, but not of the value of enable-parser-generation.

@wismill wismill force-pushed the build/manual-parser-files-regeneration branch 3 times, most recently from 5c1441f to b281fc8 Compare April 22, 2026 10:23
@wismill
Copy link
Copy Markdown
Member Author

wismill commented Apr 22, 2026

Rebased, changed the meson option name and added a check in the CI to ensure pre-generated files are up-to-date.

wismill added 2 commits April 22, 2026 13:17
This change introduces pre-generated `parser.c` and `parser.h` files
into the source tree and adds a manual mechanism to regenerate them.

- Added the initial version of the generated `parser.c` and `parser.h`
  to allow out-of-the-box compilation without a yacc dependency.
- Added the meson option `enable-parser-auto-generation` (enabled by
  default) to choose between auto-generated and pre-generated parser
  files.
- Implemented the `src/xkbcomp/update-parser` custom target (not built
  by default) to allow developers to manually sync the pre-generated
  parser files after a `parser.y` update.
- Added a `yacc-wrapper` script to ensure Bison runs from the project
  root and adds a license header. This also guarantees that `#line`
  directives in generated files use relative paths, keeping the source
  tree sanitized and portable.

This approach prevents “Bison lock-in” while maintaining the ability
to update the grammar as needed.
- Ensure pre-generated parser files work everywhere.
- Ensure pre-generated parser files update works on Linux.
  For some reason the Windows CI fails on the same task and I could
  not figure out why, so I removed it.
- Ensure pre-generated files are up-to-date.
@wismill wismill force-pushed the build/manual-parser-files-regeneration branch from b281fc8 to 0d909d7 Compare April 22, 2026 11:19
@wismill wismill merged commit 470e490 into xkbcommon:master Apr 22, 2026
6 checks passed
@wismill wismill deleted the build/manual-parser-files-regeneration branch April 22, 2026 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build system compile-keymap Indicates a need for improvements or additions to keymap compilation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide pre-generated parse.c/h files in source release bundles

2 participants