Popular repositories Loading

1. mqtt-client mqtt-client Public

   Forked from fusesource/mqtt-client

   A Java MQTT Client

   Java

2. webmagic webmagic Public

   Forked from code4craft/webmagic

   A scalable web crawler framework.

   Java

3. Distributed_spider_pku_java Distributed_spider_pku_java Public

   Forked from PkuJavaGroupCzz/Distributed_spider_pku_java

   1. 主要分为三个模块，一个爬虫抓取模块，一个是数据处理模块，一个是用户模块。 2. 爬虫抓取模块主要是从直播吧、新浪体育、网易体育上爬取有关足球的新闻和用户关于足球的评论，利用集群HADOOP抓取网页，分析得出URL集，提取特征URL 3. 网页linux脚本过滤得到原始网页，然后二次过滤得到文本，并使用分布式储存。 4. 处理模块主要是根据训练集规则一和规则二，得到分词器，然后对文本进行…

   Java

4. WebCollector WebCollector Public

   Forked from CrawlScript/WebCollector

   WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.

   Java

5. scrapy scrapy Public

   Forked from scrapy/scrapy

   Scrapy, a fast high-level web crawling & scraping framework for Python.

   Python

6. material material Public

   Forked from Daemonite/material

   HTML5 UI design based on Google Material

   HTML