Possible false positive for depth attachments and dynamic rendering

[SaschaWillems]

I'm currently running into the following validation error with the latest SDK when doing dynamic rendering:

validation layer: type { Validation } msg: vkQueueSubmit(): WRITE_AFTER_WRITE hazard detected. vkCmdPipelineBarrier2 (from VkCommandBuffer 0x121daa1bae0 submitted on the current VkQueue 0x121959d22f0) writes to VkImage 0xf000000000f, which was previously written by vkCmdEndRendering (from VkCommandBuffer 0x121da9f4990 submitted on VkQueue 0x121959d22f0).
No sufficient synchronization is present to ensure that a layout transition does not conflict with a prior write (VK_ACCESS_2_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT) at VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT.
Validation Error: [ SYNC-HAZARD-WRITE-AFTER-WRITE ] | MessageID = 0x5c0ec5d6
(Warning - This VUID has now been reported 10 times, which is the duplicate_message_limit value, this will be the last time reporting it).
vkQueueSubmit(): WRITE_AFTER_WRITE hazard detected. vkCmdPipelineBarrier2 (from VkCommandBuffer 0x121da9f4990 submitted on the current VkQueue 0x121959d22f0) writes to VkImage 0xf000000000f, which was previously written by vkCmdEndRendering (from VkCommandBuffer 0x121daa1bae0 submitted on VkQueue 0x121959d22f0).
No sufficient synchronization is present to ensure that a layout transition does not conflict with a prior write (VK_ACCESS_2_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT) at VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT.
Objects: 2
    [0] VkQueue 0x121959d22f0
    [1] VkCommandBuffer 0x121da9f4990

With the following barrier:

        vk::ImageMemoryBarrier2 depthBarrier = {
            .srcStageMask = vk::PipelineStageFlagBits2::eTopOfPipe,
            .srcAccessMask = {},
            .dstStageMask = vk::PipelineStageFlagBits2::eEarlyFragmentTests | vk::PipelineStageFlagBits2::eLateFragmentTests,
            .dstAccessMask = vk::AccessFlagBits2::eDepthStencilAttachmentRead | vk::AccessFlagBits2::eDepthStencilAttachmentWrite,
            .oldLayout = vk::ImageLayout::eUndefined,
            .newLayout = vk::ImageLayout::eDepthStencilAttachmentOptimal,
            .srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
            .dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
            .image = depthImage,
            .subresourceRange = {
                .aspectMask = vk::ImageAspectFlagBits::eDepth,
                .baseMipLevel = 0,
                .levelCount = 1,
                .baseArrayLayer = 0,
                .layerCount = 1
            }
        };
        vk::DependencyInfo depthDependencyInfo = {
            .dependencyFlags = {},
            .imageMemoryBarrierCount = 1,
            .pImageMemoryBarriers = &depthBarrier
        };
        commandBuffers[currentFrame].pipelineBarrier2(depthDependencyInfo);

(from our Khronos Tutorial).

Same happens in my own samples, no matter how I adjust that barrier.

Oddly removing that barrier all along also gets rid of the validation layer error. That seems kinda weird and might be an issue with the layers?

[artem-lunarg]

@SaschaWillems We fixed some barrier regression issues after SDK release, do you have an opportunity to test on the latest code?

[SaschaWillems]

Just built the layers from source and it's still present. It might be an issue with the barrier setup in the tutorial/samples after all, but what puzzles me is the fact that the validation error goes away by removing the whole depth barrier. That sounds odd.

[artem-lunarg]

If that’s regression I’d like to fix this asap, which sample can i use to reproduce? (we changed barrier code recently, so it is likely issue on our side)

[SaschaWillems]

Tutorial is here: https://github.com/KhronosGroup/Vulkan-Tutorial, you can test with chapter 27 (ignore the color attachment sync issue, we're working on that)

Or my dynamic rendering sample: https://github.com/SaschaWillems/Vulkan/tree/master/examples/dynamicrendering

But it might be an application issue after all.

[artem-lunarg]

@SaschaWillems I tested your dynamicrendering sample with the latest vvl code which reports the error, but also with the older sdks 1.4.309.0 and 1.4.321.0 and they also reproduce the same error, so at least that's not a regression. Now I'm going to check the barriers setup.

[artem-lunarg]

This fixes it:

The previous frame writes to depth image in vkCmdEndRendering (I guess storeOp) but then on the next frame vkCmdPipelineBarrier layout transition does not make sure those writes are available (0 src access mask). So even if we do have VK_IMAGE_LAYOUT_UNDEFINED as an oldImage layout I think there is danger that image layout write operations can race with non synchronized previous accesses. But that's an interesting question if VK_IMAGE_LAYOUT_UNDEFINED can tolerate some accesses that are not made available, so I'll ask a question on the khronos board.

[artem-lunarg]

https://gitlab.khronos.org/vulkan/vulkan/-/issues/4502

[SaschaWillems]

Thank you very much!

Any idea why the validation error goes away if you completely remove that barrier? Is that due to the color attachment barrier?

[artem-lunarg]

Without a barrier, we don't have a layout transition write that participated in the race, so this specific error goes away.

Still this raises two questions:

a) Without a barrier we have sequence of storeOp (from previous frame) and loadOp (from next frame). There is ordering withtin a single render pass instance but in this case it's storeOp and loadOp from different render pass instances, and I think these accesses should be synchronized. Need to clarify this. But that's a potential SyncVal issue.

b) If I remove the barrier and run the program with Core validation I don't get any errors, but it looks like that barrier is a single place that transitions depth attachment to the expected layout. In that's correct I would expect error about image layout mismatch. That's potential Core validation issue.