A Data analyst, I have a strong interest in tech in general and AI architectures in particular; currently, I am exploring Graph Deep Learning.
I think the real value of scraping is when you do it at scale, and for that I need to know a lot. That's why I am learning:
- Foundational Computer Networking: Basic Usage of Wireshark, OSI Model, DNS, TCP & UDP, SSL and TLS, IP Address, MAC Address, ARP Protocol, Routers, Switches, Hubs, etc.
- API Design: How REST and GraphQL APIs work, as they are the most used on the internet.
- Tools & Advanced Python: Scrapy, Playwright, and advanced Python (context managers, asynchronous programming, dunder methods, generators).
- How Browsers Work: HTTP Protocol, Rendering methods, DOM, CSSOM, etc.
- Data Engineering: Implementing the ETL pipeline (Airflow and Data Modeling).