A primary key is not helping w.r.t. performance; instead, insert the data sorted.

https://duckdb.org/docs/guides/performance/overview