fix(server): timeouts batch-inserting cache action items #8176
Conversation
✅ CLA Check PassedThank you @pepicrft! Your contribution is ready to be reviewed. The CLA requirement has been satisfied. |
✅ CLA Check PassedThank you @pepicrft! Your contribution is ready to be reviewed. The CLA requirement has been satisfied. |
fortmarek
left a comment
There was a problem hiding this comment.
We used them in the past to check if a cache object existed, but we moved away from that to use the storage as a source of truth, so this is no longer needed.
Should we just drop that table altogether if we don't use it anymore?
| %{ | ||
| type: "rich_text_section", | ||
| elements: [ | ||
| %{type: "text", text: "📦 Cache events: ", style: %{bold: true}}, | ||
| %{type: "text", text: format_numbers(cache_event_numbers)} | ||
| ] |
There was a problem hiding this comment.
I think it would be good to track how many cache hits we keep getting, but we don't need to use a table for this. In fact, we don't need that in the slack reporter at all, but would consider including it as part of posthog analytics. Not high priority, though, so for now, we can just get rid of this.
✅ CLA Check PassedThank you @pepicrft! Your contribution is ready to be reviewed. The CLA requirement has been satisfied. |
I noted in our logs that the timeouts happened when we tried to batch-insert cache action items into the database.
It turns out the larger the table is, the more expensive that operation will become because of the indexes and the foreign key, so the operation was likely being terminated by the DB.
To fix the issue I've removed the foreign key to
project_idand some redundant indexes, and reduced the batch size from 100 to 20. I also removed dropped the insertion of cache events. We used them in the past to check if a cache object existed, but we moved away from that to use the storage as a source of truth, so this is no longer needed. There was still one usage, our daily Slack usage report, but we don't need it.