Item with general/unknown/dynamic fields. Variation of https://github.com/WilliamKinaan/ScrapyDynamicItems - extracting only essential part. See [https://medium.com/@alex_ber/scrapy-item-with-general-fields-7552bd6e4622] for description. See CHANGELOG.md for detail description.
python3 -m pip install -U scrapy-itempython3 -m pip install -U https://github.com/alex-ber/scrapy-item/archive/master.zipOptionally installing tests requirements.
python3 -m pip install -U https://github.com/alex-ber/scrapy-item/archive/master.zip#egg=alex-ber-utils[tests]Or explicitly:
wget https://github.com/alex-ber/scrapy-item/archive/master.zip -O master.zip; unzip master.zip; rm master.zipAnd then installing from source (see below).
python3 -m pip install -r req.txt # only installs "required" (relaxed)python3 -m pip install . # only installs "required"python3 -m pip install .[tests] # installs dependencies for testspython3 -m pip install -r requirements.txt # only installs "required"python3 -m pip install -r requirements-tests.txt # installs dependencies for testsFrom the directory with setup.py
python3 setup.py test #run all testsor
pytestSee https://docs.python.org/3.1/distutils/uploading.html
python3 setup.py sdist uploadscrapy-item requires the following modules.
- Python 3.6+