Investigate fluent mapping/entity builder performance #4275

^{Originally posted by boris-turk September 12, 2023}
Hi,

I have a rather exotic performance issue with linq2db version 5.2.2. I am working on a large ERP system where we have about 150 POCO classes. If I get it correctly it is best practice to create a single MappingSchema (or a few of them) and fill it with appropriate mappings before executing the first linq2db query. I use fluent API to do that (via EntityMappingBuilder<> class).

Here comes the first issue: our method which defines the mappings for a single POCO class takes about 300 milliseconds to execute for a typical POCO class, which is a bit slow. I profiled a release build with DotTrace. This screenshot clearly shows that most of the time is lost in various linq2db internal methods (e.g., GetMemberEx, GetMappingAttributes, etc):

I also realized that the garbage collector is firing up frequently while defining the mappings, which is in a way reasonable (plenty of short lived objects being allocated). This all leads me to the conclusion that I simply cannot speed up significantly the initial mapping configuration. With about 150 POCO classes it looks like we are destined to wait more than 30 seconds before the mapping configuration is done with its work.

Our ERP system is a desktop application and customers want to have the startup as fast as possible. That's why I tried to parallelize the mapping configuration, i.e., split it to chunks of POCO classes and define on parallel threads the mappings for each chunk.

This leads to my main problem. We use a single MappingSchema through the whole application. It is supposed to hold a cache of all EntityDescriptor instances (one EntityDescriptor for one POCO class). I forgot to say before that I also force the building of EntityDescriptor instances on a parallel thread during system startup, since the creation of those classes is also slow (reflection, delegate compiling). But as soon as I call the EntityMappingBuilder<>.Build method to "flush" the mappings of one POCO class, the MappingSchema.ConfigurationID is reset through this call chain:

EntityMappingBuilder<>.Build -> FluentMappingBuilder.Build -> MappingSchema.AddMetadataReader -> MappingSchema.ResetID

This basically means that all of the EntityDescriptor instances built so far and cached in the EntityDescriptorsCache structure are no longer accessible (the changed MappingSchema.ConfigurationID is part of the cache key).

Long story short: is there a way to incrementally fill the mappings into MappingSchema instance while at the same time keeping its EntityDescriptorsCache structure intact?

Thanks a lot for any kind of help / suggestions. Best regards,

Boris Turk