[BUG] C++ shared library linking fails on Ubuntu: -shared flag stripped in transformation block

[msftsiwei]

Link to bug which I opened for setuptools: pypa/setuptools#5123 (comment)

---

setuptools version

79.0.1 through 80.9.0 (and likely later versions)

Python version

Python 3.10, 3.11, 3.12 (all affected)

OS

Ubuntu 22.04, 24.04 (affected); Debian-based distributions with standard c++ symlink (affected); Azure Linux 3, RHEL (NOT affected due to different compiler wrapper naming)

Additional environment information

• gcc/g++ version: 11.x, 13.x
• Compiler setup: Standard Ubuntu installation with c++ → /etc/alternatives/c++ → /usr/bin/g++
• Issue first appears: setuptools 79.0.1 (released 2025-04-23)
• Last working version: setuptools 79.0.0 (released 2025-04-20)

Description

When building C++ shared libraries (.so files) on Ubuntu-based systems, setuptools 79.0.1+ incorrectly strips the -shared linker flag during the transformation block execution in setuptools/_distutils/compilers/C/unix.py. This causes the linker to attempt building an executable instead of a shared library, resulting in "undefined reference to main" errors.

Root cause: Line 54 of unix.Compiler.link() passes the wrong parameter to _linker_params():

# Line 52-54 (current, buggy code)
_, linker_exe_ne = _split_env(self.linker_exe_cxx)
params = _linker_params(linker_na, linker_exe_ne)  # ← WRONG parameter!

The function _linker_params(linker_cmd, compiler_cmd) expects the second parameter to be a pure compiler command (e.g., ['c++']), but the code passes linker_exe_ne which includes linker parameters (e.g., ['c++', '-shared']). When both lists are identical, _linker_params() returns an empty list, stripping the -shared flag.

Why it only affects Ubuntu:

• Ubuntu uses simple c++ → g++ symlinks, resulting in linker_so_cxx = ['c++', '-shared']
• Azure Linux 3 uses x86_64-pc-linux-gnu-c++ wrapper with additional flags, causing the commands to differ enough that the bug's fallback logic (pivot=1) accidentally preserves the flags

Expected behavior

C++ shared libraries should link successfully with the -shared flag preserved throughout the transformation block. The linker command should be ['c++', '-shared', 'obj1.o', 'obj2.o', '-o', 'library.so'].

How to Reproduce

Minimal reproducer

# 1. Create test environment
docker run -it --rm ubuntu:24.04 bash

# 2. Install dependencies
apt-get update && apt-get install -y python3-pip python3-dev gcc g++

# 3. Install affected setuptools version
pip3 install --break-system-packages setuptools==80.9.0 "tree-sitter<0.21"

# 4. Create test directory structure
mkdir -p /tmp/test/src
cd /tmp/test

# 5. Create minimal C parser
cat > src/parser.c << 'EOF'
void *tree_sitter_test(void) { return 0; }
EOF

# 6. Create minimal C++ scanner (triggers C++ linker path)
cat > src/scanner.cc << 'EOF'
extern "C" {
    void *tree_sitter_test_external_scanner_create() { return 0; }
    void tree_sitter_test_external_scanner_destroy(void *p) {}
    unsigned tree_sitter_test_external_scanner_serialize(void *p, char *b) { return 0; }
    void tree_sitter_test_external_scanner_deserialize(void *p, const char *b, unsigned n) {}
    bool tree_sitter_test_external_scanner_scan(void *p, void *l, const int *s) { return false; }
}
EOF

# 7. Run build (will fail)
python3 << 'PYTHON'
from tree_sitter import Language
Language.build_library('/tmp/test.so', ['/tmp/test'])
PYTHON

Expected error output

distutils.errors.LinkError: command '/usr/bin/c++' failed with exit code 1
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/Scrt1.o: in function `_start':
(.text+0x1b): undefined reference to `main'
collect2: error: ld returned 1 exit status

Workaround to verify fix

Apply this monkey patch before importing:

from distutils.compilers.C import unix
from distutils.compilers.C.unix import _split_env, _split_aix, _linker_params

original_link = unix.Compiler.link

def fixed_link(self, target_desc, objects, output_filename, *args, **kwargs):
    # ... re-implement link() but change line 54 to:
    # params = _linker_params(linker_na, compiler_cxx_ne)  # ← Use compiler_cxx_ne
    pass  # (full implementation in attached fix_setuptools_80.py)

unix.Compiler.link = fixed_link

# Now tree-sitter build succeeds
from tree_sitter import Language
Language.build_library('/tmp/test.so', ['/tmp/test'])  # ✅ Works

Output

Full error output

$ python3 -c "from tree_sitter import Language; Language.build_library('/tmp/test.so', ['/tmp/test'])"

running build_ext
building 'tree_sitter_test' extension
creating /tmp/tmpXXXXX/tmp/test/src
cc -fPIC -c /tmp/test/src/parser.c -o /tmp/tmpXXXXX/tmp/test/src/parser.o
c++ -fPIC -c /tmp/test/src/scanner.cc -o /tmp/tmpXXXXX/tmp/test/src/scanner.o
c++ /tmp/tmpXXXXX/tmp/test/src/parser.o /tmp/tmpXXXXX/tmp/test/src/scanner.o -o /tmp/test.so

                ^^^ NOTE: Missing -shared flag!

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/Scrt1.o: in function `_start':
(.text+0x1b): undefined reference to `main'
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.12/dist-packages/tree_sitter/__init__.py", line 62, in build_library
    compiler.link_shared_object(
  File "/usr/local/lib/python3.12/dist-packages/setuptools/_distutils/compilers/C/base.py", line 812, in link_shared_object
    self.link(
  File "/usr/local/lib/python3.12/dist-packages/setuptools/_distutils/compilers/C/unix.py", line 309, in link
    raise LinkError(msg)
distutils.compilers.C.errors.LinkError: command '/usr/bin/c++' failed with exit code 1

---

Proposed Fix

File: setuptools/_distutils/compilers/C/unix.py
Location: Lines 48-55 (transformation block in Compiler.link() method)

Change line 54 from:

params = _linker_params(linker_na, linker_exe_ne)

To:

params = _linker_params(linker_na, compiler_cxx_ne)

Rationale:

• _linker_params(linker_cmd, compiler_cmd) is designed to extract linker parameters by removing the compiler command prefix
• Line 51 already computes compiler_cxx_ne which contains only the compiler name (e.g., ['c++'])
• Line 52's linker_exe_ne incorrectly includes linker parameters (e.g., ['c++', '-shared'])
• When both parameters to _linker_params() are identical, it returns [] (empty), stripping all flags

Verification:

Before fix:

linker_na = ['c++', '-shared']
linker_exe_ne = ['c++', '-shared']
params = _linker_params(linker_na, linker_exe_ne)  # → []
linker = [] + [] + ['c++'] + []  # → ['c++'] ❌ Missing -shared

After fix:

linker_na = ['c++', '-shared']
compiler_cxx_ne = ['c++']
params = _linker_params(linker_na, compiler_cxx_ne)  # → ['-shared']
linker = [] + [] + ['c++'] + ['-shared']  # → ['c++', '-shared'] ✅

---

Additional Context

Affected Projects

This bug affects any project building C++ extensions with mixed C/C++ source files:

• tree-sitter language bindings (html, php, ruby parsers)
• Custom Python extensions with C++ components
• Any project using distutils/setuptools to link C++ shared libraries on Ubuntu

Regression Timeline

• Last working: setuptools 79.0.0 (released 2025-04-20) ✅
• First broken: setuptools 79.0.1 (released 2025-04-23) ❌
• Current: setuptools 80.9.0 (released 2025-05-27) ❌ (still broken)

The C++ transformation block was introduced in setuptools 72.2.0+ (commit 2c93711) to support C++ linking, but the parameter bug was introduced in 79.0.1.

Workarounds

1. Downgrade: pip install 'setuptools<79.0.1' or pip install setuptools==79.0.0
2. Runtime patch: Import fix before distutils usage (see reproduction section)
3. Use Azure Linux/RHEL: Bug doesn't manifest due to compiler wrapper naming

Testing

Tested across:

• ✅ setuptools 69.0.3, 72.0.0, 75.0.0, 79.0.0 → All work
• ❌ setuptools 79.0.1, 80.0.0, 80.9.0 → All fail
• ✅ Same code with one-line fix → All work