MangoHUD crashes Godot on startup in self-compiled builds only

[Calinou]

Godot version

4.0.1.stable (self-compiled), 4.1.dev 2a275b9, 4.3.dev 8f0c20e

System information

Fedora 37, GeForce RTX 4090 (NVIDIA 525.89.02)

Issue description

MangoHud crashes Godot on startup in self-compiled builds only. This does not occur with official Godot 4.0 or 4.0.1 binaries.

The editor and project manager aren't affected by this issue, as they forcibly disable MangoHud. The Compatibility renderer also doesn't exhibit this issue, as MangoHud cannot be implicitly loaded in OpenGL applications (you must run it with mangohud). I can't get MangoHud to show up when using the Compatibiity rendering method and running Godot with mangohud, but that's a separate issue.

See #74182 (comment) where I originally found this.

I'm using MangoHud 0.6.8 from Fedora repositories, globally injected as a Vulkan layer (so I don't need to run every game with mangohud to be able to pop up the overlay).

The MangoHud configuration file does exist, and MangoHUD displays correctly on official Godot builds (and in other Vulkan applications).

When running from a terminal, output looks like this:

$ bin/godot.linuxbsd.editor.x86_64 --path ~/Documents/Godot/godot-reflection/
Godot Engine v4.1.dev.custom_build.2a275b985 - https://godotengine.org
[2023-03-28 03:56:03.400] [MANGOHUD] [info] [config.cpp:118] skipping config: '/home/hugo/Documents/Git/godotengine/godot/bin/MangoHud.conf' [ not found ]
[2023-03-28 03:56:03.400] [MANGOHUD] [info] [config.cpp:118] skipping config: '/home/hugo/.config/MangoHud/godot.linuxbsd.editor.x86_64.conf' [ not found ]
[2023-03-28 03:56:03.400] [MANGOHUD] [info] [config.cpp:123] parsing config: '/home/hugo/.config/MangoHud/MangoHud.conf'
free(): invalid pointer
[1]    89142 IOT instruction (core dumped)  bin/godot.linuxbsd.editor.x86_64 --path ~/Documents/Godot/godot-reflection/

This can be worked around by using export MANGOHUD=0 before running Godot, which disables MangoHUD as expected.

Backtrace

#0  0x00007ffff70afe5c in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007ffff705fa76 in raise () from /lib64/libc.so.6
#2  0x00007ffff70497fc in abort () from /lib64/libc.so.6
#3  0x00007ffff70a408e in __libc_message () from /lib64/libc.so.6
#4  0x00007ffff70b9b9c in malloc_printerr () from /lib64/libc.so.6
#5  0x00007ffff70bb9dc in _int_free () from /lib64/libc.so.6
#6  0x00007ffff70be363 in free () from /lib64/libc.so.6
#7  0x0000000009558ada in std::locale::_Impl::~_Impl() ()
#8  0x0000000009558d26 in std::locale::~locale() ()
#9  0x00007fffd46a0b38 in parseConfigFile(overlay_params&) () from /usr/lib64/mangohud/libMangoHud.so
#10 0x00007fffd469a06a in parse_overlay_config () from /usr/lib64/mangohud/libMangoHud.so
#11 0x00007fffd46b140b in overlay_CreateInstance(VkInstanceCreateInfo const*, VkAllocationCallbacks const*, VkInstance_T**) ()
   from /usr/lib64/mangohud/libMangoHud.so
#12 0x00007fffd55a59d0 in loader_create_instance_chain () from /lib64/libvulkan.so.1
#13 0x00007fffd55ba1f3 in vkCreateInstance () from /lib64/libvulkan.so.1
#14 0x0000000003c59aad in VulkanContext::_create_instance (this=0xb09eac0) at drivers/vulkan/vulkan_context.cpp:1042
#15 0x0000000003c5ea69 in VulkanContext::initialize (this=0xb09eac0) at drivers/vulkan/vulkan_context.cpp:2173
#16 0x000000000267b07b in DisplayServerX11::DisplayServerX11 (this=0xb035c50, p_rendering_driver=..., 
    p_mode=DisplayServer::WINDOW_MODE_WINDOWED, p_vsync_mode=DisplayServer::VSYNC_ENABLED, p_flags=0, p_position=0x0, 
    p_resolution=..., p_screen=-2, r_error=@0x7fffffff7ffc: OK) at platform/linuxbsd/x11/display_server_x11.cpp:5485
#17 0x0000000002678499 in DisplayServerX11::create_func (p_rendering_driver=..., p_mode=DisplayServer::WINDOW_MODE_WINDOWED, 
    p_vsync_mode=DisplayServer::VSYNC_ENABLED, p_flags=0, p_position=0x0, p_resolution=..., p_screen=-2, r_error=@0x7fffffff7ffc: OK)
    at platform/linuxbsd/x11/display_server_x11.cpp:4869
#18 0x000000000726b54e in DisplayServer::create (p_index=0, p_rendering_driver=..., p_mode=DisplayServer::WINDOW_MODE_WINDOWED, 
    p_vsync_mode=DisplayServer::VSYNC_ENABLED, p_flags=0, p_position=0x0, p_resolution=..., p_screen=-2, r_error=@0x7fffffff7ffc: OK)
    at servers/display_server.cpp:906
#19 0x00000000026d766f in Main::setup2 (p_main_tid_override=0) at main/main.cpp:2001
#20 0x00000000026d3406 in Main::setup (
    execpath=0x7fffffffd10a "/home/hugo/Documents/Git/godotengine/godot/bin/godot.linuxbsd.editor.x86_64", argc=2, 
    argv=0x7fffffffcbc0, p_second_phase=true) at main/main.cpp:1879
#21 0x0000000002643e05 in main (argc=3, argv=0x7fffffffcbb8) at platform/linuxbsd/godot_linuxbsd.cpp:61

Steps to reproduce

• Install MangoHud
• Run a project that uses Forward+ or Mobile rendering method from the editor, or directly from the command line.

Minimal reproduction project

https://github.com/Calinou/godot-reflection

[Calinou]

I can still reproduce this as of 4.6.dev, also with https://github.com/PancakeTAS/lsfg-vk which is another Vulkan layer:

================================================================
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.6.dev.custom_build (06faefc9f4d7169ebb580abf8f7dcfc2809f90f6)
Dumping the backtrace. Please include this when reporting the bug on: https://github.com/godotengine/godot/issues
[1] /lib64/libc.so.6(+0x1a070) [0x7fe9f8c9a070] (??:0)
[2] std::codecvt<char, char, __mbstate_t>::do_unshift(__mbstate_t&, char*, char*, char*&) const (??:0)
[3] std::istream& std::istream::_M_extract<long>(long&) (??:0)
[4] /lib64/liblsfg-vk.so(+0x879b8) [0x7fe9a82e09b8] (??:0)
[5] /lib64/liblsfg-vk.so(+0x63cfb) [0x7fe9a82bccfb] (??:0)
[6] /lib64/liblsfg-vk.so(+0x4b807) [0x7fe9a82a4807] (??:0)
[7] /lib64/liblsfg-vk.so(+0x46746) [0x7fe9a829f746] (??:0)
[8] /lib64/liblsfg-vk.so(+0x347bd) [0x7fe9a828d7bd] (??:0)
[9] /lib64/liblsfg-vk.so(+0x29afa) [0x7fe9a8282afa] (??:0)
[10] /lib64/liblsfg-vk.so(+0x1d652) [0x7fe9a8276652] (??:0)
[11] /lib64/liblsfg-vk.so(+0xa8d9f) [0x7fe9a8301d9f] (??:0)
[12] /lib64/ld-linux-x86-64.so.2(+0x5437) [0x7fe9f8f92437] (??:0)
[13] /lib64/ld-linux-x86-64.so.2(+0x550d) [0x7fe9f8f9250d] (??:0)
[14] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0x145) [0x7fe9f8f8e7b5] (??:0)
[15] /lib64/ld-linux-x86-64.so.2(+0xc109) [0x7fe9f8f99109] (??:0)
[16] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa6) [0x7fe9f8f8e716] (??:0)
[17] /lib64/ld-linux-x86-64.so.2(+0xc522) [0x7fe9f8f99522] (??:0)
[18] /lib64/libc.so.6(+0x6e2d4) [0x7fe9f8cee2d4] (??:0)
[19] /lib64/ld-linux-x86-64.so.2(_dl_catch_exception+0xa6) [0x7fe9f8f8e716] (??:0)
[20] /lib64/ld-linux-x86-64.so.2(+0x1869) [0x7fe9f8f8e869] (??:0)
[21] /lib64/libc.so.6(+0x6ddc3) [0x7fe9f8ceddc3] (??:0)
[22] /lib64/libc.so.6(dlopen+0x6b) [0x7fe9f8cee38b] (??:0)
[23] /lib64/libvulkan.so.1(+0x222f2) [0x7fe9df7552f2] (??:0)
[24] /lib64/libvulkan.so.1(+0x239fc) [0x7fe9df7569fc] (??:0)
[25] /lib64/libvulkan.so.1(vkCreateInstance+0xfac) [0x7fe9df76a4cc] (??:0)
[26] gde() [0x1c39a81] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_context_driver_vulkan.cpp:898 (discriminator 1))
[27] gde() [0x1c58bd1] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_context_driver_vulkan.cpp:734)
[28] gde() [0x1c5e966] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_context_driver_vulkan.cpp:931 (discriminator 1))
[29] gde() [0x4fe789] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/x11/display_server_x11.cpp:7051 (discriminator 1))
[30] gde() [0x500ce1] (/home/hugo/Documents/Git/godotengine/godot/servers/display/display_server.h:44)
[31] gde() [0x5999ab] (/home/hugo/Documents/Git/godotengine/godot/main/main.cpp:3137 (discriminator 1))
[32] gde() [0x5aea06] (/home/hugo/Documents/Git/godotengine/godot/main/main.cpp:2759)
[33] gde() [0x423579] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/godot_linuxbsd.cpp:107)
[34] /lib64/libc.so.6(+0x3575) [0x7fe9f8c83575] (??:0)
[35] /lib64/libc.so.6(__libc_start_main+0x88) [0x7fe9f8c83628] (??:0)
[36] gde() [0x46b765] (??:?)
-- END OF C++ BACKTRACE --
================================================================

As a workaround, run Godot with the DISABLE_LSFG=1 environment variable so the implicit layer doesn't inject.

However, there are some notes of interest:

• This is not reproducible when opening the project manager with --rendering-driver vulkan.
• This is also reproducible with 4.5.1.stable official binaries, as long as I run a project that uses Forward+ directly (I'm using the test project from GitHub Actions: Test project exporting on CI #109146). Not all Forward+ projects crash on startup though; for instance, the 3D platformer demo works fine.