Skip to content

Conversation

@Starsam80
Copy link

Taken from the upstream mesa definition here. Below is a "backtrace", or my best effort in trying to explain where it went wrong.

  • gamescope crashes with
[gamescope] [Error] vulkan: vkGetMemoryFdKHR failed (VkResult: -2)
[gamescope] [Error] vulkan: failed to allocate buffer for KMS
vulkan_make_output failed
  • mesa/RADV returns -2 (VK_ERROR_OUT_OF_DEVICE_MEMORY) because of these lines
  • libdrm calls DRM_IOCTL_PRIME_HANDLE_TO_FD which results in -1 (EPERM (Operation not permitted))
  • the kernel seems to return that error because of this (the bo->flags & AMDGPU_GEM_CREATE_VM_ALWAYS_VALID in particular)
  • mesa sets that flag here
  • This is where I get a little confused (and where the UB happens)... I set a breakpoint on this line, and flags gets set to 0x660 which is at least RADEON_FLAG_NO_INTERPROCESS_SHARING | RADEON_FLAG_PREFER_LOCAL_BO. (I have yet to look at the disassembly for a reasonable explanation)
  • wsi_info in mesa comes from the VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA gamescope sets in these lines
  • wsi_info->dma_buf_sync_file is undefined because the gamescope struct does not have that field

@Starsam80
Copy link
Author

I've been so focused on tracking this bug down that I forgot to check the other structs. wsi_image_create_info also is different, but seems to be backwards compatible to some extent. Should I change that one too? I've also been wondering if gamescope should even be using "mesa-internal" structs, it just seems like a bad idea (like the reason for the existence of this PR). Should I just remove the structs from gamescope?

@misyltoad
Copy link
Collaborator

The structs are unfortunately needed to make Gamescope work without modifiers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants