DynamicSpider

Build a dynamic spider which can handle the dom parse and fetch urls

The aim for this code is to learn how to use gevent in python and refer the main idea from https://github.com/pnigos/vulcan

Dependency:

1. splinter

2. phantomjs

3. gevent

4. requests

5. MySQLdb

6. urlparse

7. beautiful soup

8. lxml

9. purl