Persistent map

[swq0553]

OpenGL Persistent Mapping
Persistent mapped buffers in OpenGL, introduced with the ARB_buffer_storage extension in OpenGL 4.4, allow a buffer to remain mapped indefinitely. This technique is particularly useful for scenarios requiring frequent updates of data from the CPU to the GPU, such as particle systems or video frame updates. It reduces driver overhead and synchronization issues, making it a key component of the "Approaching Zero Driver Overhead" (AZDO) techniques.

Persistent mapping enables you to map a buffer once and retain the pointer for continuous updates without unmapping. This eliminates the need for repeated mapping and unmapping operations, reducing the number of OpenGL calls and synchronization overhead.

[LukasBanana]

Thank you for another contribution.

It seems D3D12 and Vulkan support this feature out of the box, but the tricky part is synchronization and even if we only support this for GL initially, I would like to get a description from you what the expectation of this feature is, i.e. how do we expect clients to use this in conjunction with command buffer recording? I left a longer comment about this below.

[...] retain the pointer for continuous updates without unmapping.

That would be the part that I need more details in the documentation for this feature. It's not clear to me how you can write to the memory mapped buffer while the GL commands are reading from it. Remember that the command recording of LLGL::CommandBuffer is not guaranteed to be in sync with any other calls, especially not from a memory mapped buffer or anything else coming from LLGL::RenderSystem:

cmdBuffer->DoThis(myBuffer);
renderSystem->WriteToBuffer(myBuffer); // <-- Not guaranteed to be in sync with DoThis/DoThat
cmdBuffer->DoThat(myBuffer); 

Your unit test doesn't cover the synchronization part yet, because there's no command buffer that uses the buffer. It's only mapped, read, written etc. but what does "persistent" mapping mean here? Is it meant to be persitent across command buffer recording? Otherwise, I wouldn't see the advantage over simply mapping and unmapping. Maybe start with another unit test such as SceneUpdate and replace the constant buffer sceneCbuffer with a persistently mapped buffer.

[LukasBanana]

Thanks for adding a new test for this. Please also add a comment here that describes what the purpose of this test is and how this test operates. I have to do the same for several more existing tests, but any new tests should have a description.

[LukasBanana]

Does this pointer have to be reset in this function?

Also, does it make sense to effectively ignore this function if it is wrongfully invoked, i.e. shouldn't this be an error if someone calls UnmapBuffer() while it isn't mapped? Since this function is not directly called by the client, but via GLRenderSystem, this should probably be an assertion, e.g. something like this:

LLGL_ASSERT(mapped_ == nullptr, "Cannot unmap buffer that hasn't been mapped");

[LukasBanana]

Can you please add a few more descriptive details what the expectation of this feature is?
I'm thinking specifically about the command buffer model in LLGL. In OpenGL, this doesn't exist (as an explicit interface at least) but LLGL's deferred command buffer emulates that. Even if this feature is only supported for the GL backend, it should fit into this concept.

So for your description, what can the client programmer expect how the memory mapping behaves in a deferred command buffer? Can the client write (and perhaps read) to and from such a buffer while a command buffer is using it? I don't see the use of GL_CLIENT_MAPPED_BUFFER_BARRIER_BIT here but I think this needs to be included for such buffers for the glMemoryBarrier() calls (see GLCommandBuffer::FlushAndGetMemoryBarriers() and GLCommandBuffer::InvalidateMemoryBarriersForStorageResource()).

LLGL doesn't provide explicit synchronization as of right now, so this needs to be handled by the backend. At the same time, the LLGL interface is not designed to use RenderSystem functions interleaved with CommandBuffer calls. You can do that, but LLGL doesn't make any guarantees they will work in conjunction, i.e. MapBuffer/UnmapBuffer should not be called while you are recording a command buffer that uses such buffers, at least LLGL doesn't guarantee those calls will be in the correct order (since a deferred command buffer is submitted later).

[LukasBanana]

For your failing WebAssembly jobs, you might want to integrate 0c38290. C++17 was added as requirement for Embind.

[swq0553]

persistent map not work well for some case,
eg:In GLDeferredCommandBuffer the buffer is same with the last change.
but one can change many time and do some work use this buffer.

[LukasBanana]

In GLDeferredCommandBuffer the buffer is same with the last change.

I presume this is what I mentioned with respect to command buffer recording not being in sync with RenderSystem calls, including mapping buffers.
Maybe we can revise some of those GL specific buffer flags and incorporate somehow.