Add support for scheduled/recurring web crawling jobs with change detection, monitoring, and notification capabilities. This feature would enable users to automatically track changes on websites over time, making Deepcrawl suitable for production monitoring use cases.

Currently, Deepcrawl only supports one-time URL processing. Users who need to:
• Monitor price changes on e-commerce sites
• Track content updates on competitor websites
• Monitor product availability
• Aggregate news from multiple sources
• Track SEO changes over time

…must manually trigger crawls repeatedly, which is not scalable for production workflows.

Maybe I can help with this.