Skip to content

Enh crawler fixes#274

Merged
daoudclarke merged 49 commits into
mainfrom
enh-crawler-fixes
Jul 5, 2025
Merged

Enh crawler fixes#274
daoudclarke merged 49 commits into
mainfrom
enh-crawler-fixes

Conversation

@daoudclarke

Copy link
Copy Markdown
Contributor

A few fixes for #269 - many thanks to @thiswillbeyourgithub for their work here.

  • HashedBatch was used for batches sent by the old crawler. We need to send the version with Results instead
  • Move some variables from env_vars so that we don't have to import it when running the server
  • HttpError is the correct exception

Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
because loguru takes care of the tracing

Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
thiswillbeyourgithub and others added 16 commits June 18, 2025 18:37
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub
<26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
Signed-off-by: thiswillbeyourgithub <26625900+thiswillbeyourgithub@users.noreply.github.com>
@daoudclarke daoudclarke mentioned this pull request Jun 23, 2025
@daoudclarke

daoudclarke commented Jul 5, 2025

Copy link
Copy Markdown
Contributor Author

Tested and deployed to beta.mwmbl.org, all looks good. Many thanks to @thiswillbeyourgithub and apologies for the delay in merging 😳

@daoudclarke daoudclarke merged commit ded1aaa into main Jul 5, 2025
1 check passed
@daoudclarke daoudclarke deleted the enh-crawler-fixes branch July 5, 2025 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants