Describe the feature

Improve preload efficiency by fixing O(n²) assignment logic and enabling concurrent preloading of independent relations.

Motivation

1. O(n²) nested loop in many-to-many/has-many preload assignment — callbacks/preload.go:319-348

The preload implementation has a nested loop: for each result row, iterate identity map entries and call rel.Field.ReflectValueOf + reflect.Append. For large result sets with many-to-many relations, this is O(results × parents).

Fix: Use index-based assignment or pre-group by foreign key using a map instead of nested iteration.

2. Sequential preloading — independent relations loaded one at a time

When preloading multiple independent relations at the same level (e.g., db.Preload("Pets").Preload("Toys").Preload("Languages")), each relation is loaded sequentially with a separate Find() call. For 5 relations, this adds 5 round-trips to the database.

Fix: For independent preloads at the same level, use errgroup.Group or sync.WaitGroup to load them concurrently. Ensure they use separate *gorm.DB instances (from Session()) to avoid shared state.

3. No batch splitting for large parent result sets

When preloading with very large parent result sets, the IN-clause can grow very large (e.g., WHERE foreign_id IN (1,2,...10000)). Some databases have limits on IN-clause size or degrade in performance.

Fix: Split the IN-clause into batches (e.g., 500 per batch) for the preload query.

Impact

• Finding 1 affects any preload with large result sets — the O(n²) behavior causes noticeable slowdown for N > 1000.
• Finding 2 adds N round-trips for N preloaded relations. Concurrent loading reduces this to 1 round-trip (the slowest relation).
• Finding 3 prevents database errors and improves query plan efficiency for large datasets.

Related Issues

• Performance architecture review of GORM codebase