[C++ interop] swiftc not compatible with latest libc++ headers

[junov-google]

Description

There was a recent refactor in libc++ where the _LIBCPP_COMPRESSED_PAIR macro was updated to use the [[__no_unique_address__]] attribute, which it seems is not handled correctly by the swift compiler.

This change is documented in the release notes here: https://releases.llvm.org/20.1.0/projects/libcxx/docs/ReleaseNotes.html#id4 (4th bullet point in the Improvements section)

Attempting to instantiate template classes that use _LIBCPP_COMPRESSED_PAIR internally, such as std::vector, leads to compiler errors due to missing member initializers in constructors for members that should not require initialization because they have zero storage.

For classes that do compile correctly despite their use of _LIBCPP_COMPRESSED_PAIR, I suspect there may be other problems down the road due to ABI incompatibilities caused by inconsistent handling of [[__no_unique_address__]] between clang and swiftc.

Reproduction

C++ Header:

#include <vector>
std::vector<int> GetOneTwoThree();

C++ Implementation:

#include "get_one_two_three.h"
std::vector<int> GetOneTwoThree() {
  return {1, 2, 3};
}

Module map:

module GetOneTwoThree {
  header "get_one_two_three.h"
  export*
}

Swift file:

import GetOneTwoThree
public func testingOneTwoThree() {
  let array = GetOneTwoThree()
  print(array.size())
}

Download a recent version of stdc++ (20.0 or later)

Use the following swiftc compiler flags, replacing all occurrences of (...) with the path to your local copy of stdc++:

-cxx-interoperability-mode=default -Xcc -nostdinc++ -Xcc -isystem/(...)/libc++/src/include
-Xcc -isystem/(...)/libc++abi/src/include -Xcc -I/(...)/libc++ -explicit-module-build -Xcc -fmodule-map-file=get_one_two_three.modulemap -Xcc -std=c++20

Note, bug is also reproducible in -std=c++23 mode.

Compile error:

libc++/src/include/__split_buffer:71:55: constructor for 'std::__split_buffer_pointer_layout<std::__split_buffer<int, std::allocator<int> &, std::__split_buffer_pointer_layout>, int, std::allocator<int> &>' must explicitly initialize the reference member '__alloc_'

Expected behavior

This should compile without errors

Environment

Apple Swift version 6.3-dev (LLVM bff1370bd79c983, Swift 57cf4ce)
Target: arm64-apple-macosx15.0

Additional information

No response

[junov-google]

It should be noted that compilation succeeds when using language mode -std=c++17 or an earlier standard, which turns off the use of [[__no_unique_address__]] in the stdc++ headers (because it's a C++20 feature).

Unfortunately, that's not a viable workaround for many projects, including the one I'm working on.

[junov-google]

This is the exact command line i'm using to repro the error, using the stdc++ from the Chromium project's hermetic toolchain:

swiftc get_one_two_three.swift -cxx-interoperability-mode=default -Xcc -nostdinc++ -Xcc -isystem/Users/junov/dev/bling/src/third_party/libc++/src/include -Xcc -isystem/Users/junov/dev/bling/src/third_party/libc++abi/src/include -Xcc -I/Users/junov/dev/bling/src/buildtools/third_party/libc++ -explicit-module-build -Xcc -fmodule-map-file=./get_one_two_three.modulemap -Xcc -std=c++20 -Xcc -D_LIBCPP_HARDENING_MODE_DEFAULT=_LIBCPP_HARDENING_MODE_NONE

[egorzhdan]

Interesting... This diagnostic is emitted by Clang, so I wonder if there is a way to trigger it without using Swift at all, perhaps with a modular build.
We've fixed some issues in the way Swift compiler handles no_unique_address fields in the past (a7c1744, 939ee49), but it looks like there's more.

[junov-google]

I'll try to come up with a further reduced test case that does not have stdc++ in the loop. I have confirmed that there is no bug with the object layout. The attribute actually works. The issue seems to be that the check that generates the compiler error is missing some logic to accept that zero-size members are uninitialized. This is weird because standalone clang does not generate this error, but swiftc does.

[junov-google]

I've investigated this more and I now believe that the no_unique_address attribute is a red herring. The actual problem seems to be related to class template instantiation. __split_buffer_pointer_layout in libc++ has a default constructor that fails to compile in clang if it is instantiated.

Here is an example, without stdc++ in the loop, that illustrates the problem.

template<class T>
class MyClass {
  public:
    T a_;
    int b_;
    MyClass() : b_(0) {}
    MyClass(T a) : a_(a), b_(0) {}
};

If T is a reference type, then it will be impossible to instantiate the default constructor because the initialization of member 'a_' is mandatory when T is a reference type.

When using std::vector<int>, for example, __split_buffer_pointer_layout is instantiated with the allocator_type template parameter equal to std::allocator<int> &, a reference type. Therefore, the initialization of member allocator_type __alloc_ is required in all constructors. The default constructor of __split_buffer_pointer_layout looks like this:

_LIBCPP_CONSTEXPR_SINCE_CXX20 _LIBCPP_HIDE_FROM_ABI __split_buffer_pointer_layout() : __back_cap_(nullptr) {}

Because it does not initialize __alloc_, any attempt to instantiate that constructor will fail to build. It is unclear to me why the constructor is implemented this way, and why allocator_type is sometimes a reference type and sometimes not a reference type. What seems clear to me is that the __split_buffer_pointer_layout default constructor is only meant to be used (and therefore instantiated) in cases where allocator_type is not a reference type.

What I think is happening is that swiftc's C++ interop module is trying to implicitly instantiate this constructor when it shouldn't be. I see two possible solutions to prevent this from happening:

1. Add requires (!std::is_reference_v<allocator_type>) to the definition of __split_buffer_pointer_layout(). I tried this in my local build, and it solves the problem
2. Make swiftc respect the __exclude_from_explicit_instantiation__ attribute when automatically generating template instantiations. Perhaps that would not be semantically correct because these instantiations are not technically explicit, but this is the attribute that stdc++ uses here (via the _LIBCPP_HIDE_FROM_ABI macro).

[junov-google]

FWIW, this problem exists in the "next", but not in the "stable/21.x" branch of Apple's llvm/libc++. I'd like to fix this upstream in order to unblock swift/C++ interop work in the Chromium project. From there, it should propagate to Apple's fork and contribute to getting the "next" branch ready for use in swift-project. Any objections?