EnTT and multithreading

[Lumengine]

Hi,
I'm using EnTT as the main representation of my scene for 3D simulations. I’d like to multithread scene modifications. Let’s say I have many worker threads that may need to modify any component in the registry.

How would you approach this?

I’ve already protected entity creation, but when it comes to modifying the registry, I was thinking about using a queue of commands that are dequeued and executed on a single thread. However, this could result in a large number of small lambdas that might hurt performance.

I also heard about using a local registry per thread that gets merged back into the main one at the end of the frame. Is that a viable approach?

I’d be happy to get any feedback on this.
Thanks,

Baptiste

[skypjack]

You don't have to guard a registry as a whole. Quite the opposite. As long as you don't write modify the same pools from different threads, you're fine.
For example, if thread 1 works on type T and thread 2 works on type U, you can run them in parallel. Similarly, you can also spawn multiple threads on the same type T if they update different instances (i.e. you construct a view for T, get the size and split it in N parts, than send these parts to N different threads).
Also, keep in mind that entities are just yet another pool in the registry. That is, you can have a thread that creates new entities and assigns component T to them, while a different thread iterates and updates all Us.

[Lumengine]

@skypjack
Thanks for the explanation!

In my case, the workload comes from streaming meshes from a third-party library, and each mesh requires creating a new entity and assigning a MeshComponent to it.
This happens on multiple worker threads.

From what I understand, entity creation and writing to the same component pool (MeshComponent) cannot safely happen concurrently from multiple threads.

Is that correct?
Does this mean multiple threads cannot create entities and add the same component type at the same time?

If that's the case, how would you approach this kind of scenario?

[skypjack]

As I said, entities are just another type. So, what you're asking is: can multiple threads work on the same pool concurrently?
It boils down to: can multiple threads modify the same, let's say, vector concurrently? Of course not. I'm sorry.
In this case, you've to lock or use command queues, and so on. There are many different solutions to this problem already.
Side note: unless a single worker handles tons of meshes, I don't think that thread local registries are a good idea. My two cents.

[Lumengine]

Yes, so I think I understood correctly.

[skypjack]

Another option is to preallocate enough entities and components, then dispatch them to the right threads.
In this case, you can use a dedicated storage for example. When all threads return, you can merge its stuff with the main registry.
However, I'd start with a lock which is the simplest solution ever. Then I'd refine my approach if it turns to be a bottleneck.
I'm usually against finding overengineered solutions unless I have a real, measurable need for them. 🙂

[Lumengine]

I totally agree. I'll try first a simple approach and see if it fits my needs in term of performances.