Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Jun 15, 2026

• Merge pull request #337 from mwmbl/fix-robots-txt-spamming

  Show description for 53ae9c9
  daoudclarke

  authored
  53ae9c9

  Copy full SHA for 53ae9c9

Commits on Jun 14, 2026

• Fix crawler re-crawling every URL via bloom filter dedup bug

  Show description for 6a6e286
  

  daoudclarke

  and

  claude

  committed
  6a6e286

  Copy full SHA for 6a6e286

• Make redis parameter required in crawl_url and robots_allowed

  Show description for c2f399a
  daoudclarke

  committed
  c2f399a

  Copy full SHA for c2f399a

• Simplify: make redis parameter optional with None default

  Show description for 306b09a
  daoudclarke

  committed
  306b09a

  Copy full SHA for 306b09a

• Fix robots.txt spamming by adding Redis caching

  Show description for a11be6e
  daoudclarke

  committed
  a11be6e

  Copy full SHA for a11be6e

Commits on Jun 13, 2026

• Merge pull request #336 from mwmbl/broken-polar-module

  Show description for 11543b5
  daoudclarke

  authored
  11543b5

  Copy full SHA for 11543b5

• Fix test mocks for billing endpoints

  Show description for 00fb343
  daoudclarke

  committed
  00fb343

  Copy full SHA for 00fb343

• Add tests for platform billing endpoints

  Show description for a2ac5ee
  daoudclarke

  committed
  a2ac5ee

  Copy full SHA for a2ac5ee

• Fix polar-sdk import: rename SubscriptionUpdateProduct to CustomerSubscriptionUpdateProduct

  Show description for 5fb9543
  daoudclarke

  committed
  5fb9543

  Copy full SHA for 5fb9543

• Merge pull request #335 from mwmbl/fix-robots-not-respecting-jak2k

  Show description for 10ad9d0
  daoudclarke

  authored
  10ad9d0

  Copy full SHA for 10ad9d0

• Fix robots.txt compliance

  Show description for fa8b8ba
  daoudclarke

  committed
  fa8b8ba

  Copy full SHA for fa8b8ba

Commits on Jun 4, 2026

• Merge pull request #332 from mwmbl/store-super-search-results

  Show description for dafab31
  daoudclarke

  authored
  dafab31

  Copy full SHA for dafab31

• Index super search results

  Show description for 13ddd92
  daoudclarke

  committed
  13ddd92

  Copy full SHA for 13ddd92

Commits on Jun 2, 2026

• Merge pull request #331 from mwmbl/super-search-experiment

  Show description for 35d7516
  daoudclarke

  authored
  35d7516

  Copy full SHA for 35d7516

• Move Super Search auth into search_auth, fix JWT user lookup

  Show description for 8953bef
  

  daoudclarke

  and

  claude

  committed
  8953bef

  Copy full SHA for 8953bef

• Simplify link resolution using urljoin

  Show description for 5fcef60
  

  daoudclarke

  and

  claude

  committed
  5fcef60

  Copy full SHA for 5fcef60

Commits on Jun 1, 2026

• Fix Super Search review issues + harden crawler against SSRF

  Show description for e6c9321
  

  daoudclarke

  and

  claude

  committed
  e6c9321

  Copy full SHA for e6c9321

• Apply heuristic majority-terms filter in Rust LTR predict

  Show description for 2a2de6c
  

  daoudclarke

  and

  claude

  committed
  2a2de6c

  Copy full SHA for 2a2de6c

• Document Super Search SSE events via Ninja schemas

  Show description for 63d3f7e
  

  daoudclarke

  and

  claude

  committed
  63d3f7e

  Copy full SHA for 63d3f7e

Commits on May 31, 2026

• Merge branch 'main' into super-search-experiment

  daoudclarke

  committed
  d32cd53

  Copy full SHA for d32cd53

• Use whole-field heuristic score for super search promotion

  Show description for 4f03ea9
  

  daoudclarke

  and

  claude

  committed
  4f03ea9

  Copy full SHA for 4f03ea9

Commits on May 30, 2026

• Fix super search: add promoted pages to results, deduplicate emissions, log secondary failures

  Show description for f0488e0
  

  daoudclarke

  and

  claude

  committed
  f0488e0

  Copy full SHA for f0488e0

• Stream intermediate results after each source completes in super search

  Show description for 39e221f
  

  daoudclarke

  and

  claude

  committed
  39e221f

  Copy full SHA for 39e221f

• Fix tests broken by HN source changes and monthly limit increase

  Show description for bcef93c
  

  daoudclarke

  and

  claude

  committed
  bcef93c

  Copy full SHA for bcef93c

• Merge pull request #330 from mwmbl/use-curated-domains-in-crawler

  Show description for 3b6dc5b
  daoudclarke

  authored
  3b6dc5b

  Copy full SHA for 3b6dc5b

• Add logging for curated domain retrieval

  daoudclarke

  committed
  0c28488

  Copy full SHA for 0c28488

• Emit results event on super search timeout

  Show description for b2dedaf
  

  daoudclarke

  and

  claude

  committed
  b2dedaf

  Copy full SHA for b2dedaf

• Remove tags=story filter from HN search to include comments

  Show description for 5e41e2b
  

  daoudclarke

  and

  claude

  committed
  5e41e2b

  Copy full SHA for 5e41e2b

• Wrap HN query terms in double quotes to prevent prefix matching

  Show description for 56f0096
  

  daoudclarke

  and

  claude

  committed
  56f0096

  Copy full SHA for 56f0096

Commits on May 29, 2026

• Fix substring term matching in super search filters

  Show description for 2e95a99
  

  daoudclarke

  and

  claude

  committed
  2e95a99

  Copy full SHA for 2e95a99

• Use URL term overlap heuristic for outbound link selection in super search

  Show description for 3a0b26f
  

  daoudclarke

  and

  claude

  committed
  3a0b26f

  Copy full SHA for 3a0b26f

• Fall back to OG meta tags for title/extract on JS-first pages

  Show description for 8c06859
  

  daoudclarke

  and

  claude

  committed
  8c06859

  Copy full SHA for 8c06859

• Fix crawler missing image-only anchor links (e.g. icon nav links)

  Show description for 0947329
  

  daoudclarke

  and

  claude

  committed
  0947329

  Copy full SHA for 0947329

• Add mwmbl source

  daoudclarke

  committed
  8b8a733

  Copy full SHA for 8b8a733

• Clean HN extracts through the justext pipeline before indexing

  Show description for 52db7d5
  

  daoudclarke

  and

  claude

  committed
  52db7d5

  Copy full SHA for 52db7d5