Skip to content

Streaming sites #628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
soxoj merged 2 commits into from
Sep 10, 2022
Merged

Streaming sites #628

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ccaf95d
Bump to 0.4.4
soxoj Sep 3, 2022
383f0e0
Added new sites, new error solution caption
soxoj Sep 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,89 @@

## [Unreleased]

## [0.4.4] - 2022-09-03
* Fixed some false positives by @soxoj in https://github.com/soxoj/maigret/pull/433
* Drop Python 3.6 support by @soxoj in https://github.com/soxoj/maigret/pull/434
* Bump xhtml2pdf from 0.2.5 to 0.2.7 by @dependabot in https://github.com/soxoj/maigret/pull/409
* Bump reportlab from 3.6.6 to 3.6.9 by @dependabot in https://github.com/soxoj/maigret/pull/403
* Bump markupsafe from 2.0.1 to 2.1.1 by @dependabot in https://github.com/soxoj/maigret/pull/389
* Bump pycountry from 22.1.10 to 22.3.5 by @dependabot in https://github.com/soxoj/maigret/pull/384
* Bump pypdf2 from 1.26.0 to 1.27.4 by @dependabot in https://github.com/soxoj/maigret/pull/438
* Update GH actions by @soxoj in https://github.com/soxoj/maigret/pull/439
* Bump tqdm from 4.63.0 to 4.64.0 by @dependabot in https://github.com/soxoj/maigret/pull/440
* Bump jinja2 from 3.0.3 to 3.1.1 by @dependabot in https://github.com/soxoj/maigret/pull/441
* Bump soupsieve from 2.3.1 to 2.3.2 by @dependabot in https://github.com/soxoj/maigret/pull/436
* Bump pypdf2 from 1.26.0 to 1.27.4 by @dependabot in https://github.com/soxoj/maigret/pull/442
* Bump pyvis from 0.1.9 to 0.2.0 by @dependabot in https://github.com/soxoj/maigret/pull/443
* Bump pypdf2 from 1.27.4 to 1.27.6 by @dependabot in https://github.com/soxoj/maigret/pull/448
* Bump typing-extensions from 4.1.1 to 4.2.0 by @dependabot in https://github.com/soxoj/maigret/pull/447
* Bump soupsieve from 2.3.2 to 2.3.2.post1 by @dependabot in https://github.com/soxoj/maigret/pull/444
* Bump pypdf2 from 1.27.6 to 1.27.7 by @dependabot in https://github.com/soxoj/maigret/pull/449
* Bump pypdf2 from 1.27.7 to 1.27.8 by @dependabot in https://github.com/soxoj/maigret/pull/450
* XMind 8 report warning and some docs update by @soxoj in https://github.com/soxoj/maigret/pull/452
* False positive fixes 24.04.22 by @soxoj in https://github.com/soxoj/maigret/pull/455
* Bump pypdf2 from 1.27.8 to 1.27.9 by @dependabot in https://github.com/soxoj/maigret/pull/456
* Bump pytest from 7.0.1 to 7.1.2 by @dependabot in https://github.com/soxoj/maigret/pull/457
* Bump jinja2 from 3.1.1 to 3.1.2 by @dependabot in https://github.com/soxoj/maigret/pull/460
* Ubisoft forums addition by @fen0s in https://github.com/soxoj/maigret/pull/461
* Add BYOND, Figma, BeatStars by @fen0s in https://github.com/soxoj/maigret/pull/462
* fix Figma username definition, add a bunch of sites by @fen0s in https://github.com/soxoj/maigret/pull/464
* Bump pypdf2 from 1.27.9 to 1.27.10 by @dependabot in https://github.com/soxoj/maigret/pull/465
* Bump pypdf2 from 1.27.10 to 1.27.12 by @dependabot in https://github.com/soxoj/maigret/pull/466
* Sites fixes 05 05 22 by @soxoj in https://github.com/soxoj/maigret/pull/469
* Bump pyvis from 0.2.0 to 0.2.1 by @dependabot in https://github.com/soxoj/maigret/pull/472
* Social analyzer websites, also fixing presense strs by @fen0s in https://github.com/soxoj/maigret/pull/471
* Updated logic of false positive risk estimating by @soxoj in https://github.com/soxoj/maigret/pull/475
* Improved usability of external progressbar func by @soxoj in https://github.com/soxoj/maigret/pull/476
* New sites added, some tags/rank update by @soxoj in https://github.com/soxoj/maigret/pull/477
* Added new sites by @soxoj in https://github.com/soxoj/maigret/pull/480
* Added new forums, updated ranks, some utils improvements by @soxoj in https://github.com/soxoj/maigret/pull/481
* Disabled sites with false positives results by @soxoj in https://github.com/soxoj/maigret/pull/482
* Bump certifi from 2021.10.8 to 2022.5.18.1 by @dependabot in https://github.com/soxoj/maigret/pull/488
* Bump psutil from 5.9.0 to 5.9.1 by @dependabot in https://github.com/soxoj/maigret/pull/490
* Bump pypdf2 from 1.27.12 to 1.28.1 by @dependabot in https://github.com/soxoj/maigret/pull/491
* Bump pypdf2 from 1.28.1 to 1.28.2 by @dependabot in https://github.com/soxoj/maigret/pull/493
* added and fixed some websites in data.json by @kustermariocoding in https://github.com/soxoj/maigret/pull/494
* Bump pypdf2 from 1.28.2 to 2.0.0 by @dependabot in https://github.com/soxoj/maigret/pull/504
* Bump pefile from 2021.9.3 to 2022.5.30 by @dependabot in https://github.com/soxoj/maigret/pull/499
* Updated sites list, added disabled Anilist by @soxoj in https://github.com/soxoj/maigret/pull/502
* Bump lxml from 4.8.0 to 4.9.0 by @dependabot in https://github.com/soxoj/maigret/pull/503
* Compatibility with Python 10 by @soxoj in https://github.com/soxoj/maigret/pull/509
* feat: add .log & .bak files to gitignore in https://github.com/soxoj/maigret/pull/511
* fix some sites and delete abandoned by @fen0s in https://github.com/soxoj/maigret/pull/526
* Fixesjulyfirst by @fen0s in https://github.com/soxoj/maigret/pull/533
* yazbel, aboutcar, zhihu by @fen0s in https://github.com/soxoj/maigret/pull/531
* Fixes july third by @fen0s in https://github.com/soxoj/maigret/pull/535
* Update data.json by @fen0s in https://github.com/soxoj/maigret/pull/539
* Update data.json by @fen0s in https://github.com/soxoj/maigret/pull/540
* Bump reportlab from 3.6.9 to 3.6.11 by @dependabot in https://github.com/soxoj/maigret/pull/543
* Bump requests from 2.27.1 to 2.28.1 by @dependabot in https://github.com/soxoj/maigret/pull/530
* Bump pypdf2 from 2.0.0 to 2.5.0 by @dependabot in https://github.com/soxoj/maigret/pull/542
* Bump xhtml2pdf from 0.2.7 to 0.2.8 by @dependabot in https://github.com/soxoj/maigret/pull/522
* Bump lxml from 4.9.0 to 4.9.1 by @dependabot in https://github.com/soxoj/maigret/pull/538
* disable yandex music + set utf8 encoding by @fen0s in https://github.com/soxoj/maigret/pull/562
* fix false positives by @fen0s in https://github.com/soxoj/maigret/pull/577
* disable Instagram, fix two false positives by @fen0s in https://github.com/soxoj/maigret/pull/578
* Bump certifi from 2022.5.18.1 to 2022.6.15 by @dependabot in https://github.com/soxoj/maigret/pull/551
* August15 by @fen0s in https://github.com/soxoj/maigret/pull/591
* Bump pytest-httpserver from 1.0.4 to 1.0.5 by @dependabot in https://github.com/soxoj/maigret/pull/583
* Bump typing-extensions from 4.2.0 to 4.3.0 by @dependabot in https://github.com/soxoj/maigret/pull/549
* Bump colorama from 0.4.4 to 0.4.5 by @dependabot in https://github.com/soxoj/maigret/pull/548
* Bump chardet from 4.0.0 to 5.0.0 by @dependabot in https://github.com/soxoj/maigret/pull/550
* Bump cloudscraper from 1.2.60 to 1.2.63 by @dependabot in https://github.com/soxoj/maigret/pull/600
* Bump flake8 from 4.0.1 to 5.0.4 by @dependabot in https://github.com/soxoj/maigret/pull/598
* Bump attrs from 21.4.0 to 22.1.0 by @dependabot in https://github.com/soxoj/maigret/pull/597
* Bump pytest-asyncio from 0.18.2 to 0.19.0 by @dependabot in https://github.com/soxoj/maigret/pull/601
* Bump pypdf2 from 2.5.0 to 2.10.4 by @dependabot in https://github.com/soxoj/maigret/pull/606
* Bump pytest from 7.1.2 to 7.1.3 by @dependabot in https://github.com/soxoj/maigret/pull/613
* Update sites.md -Gitmemory.com suppression by @C3n7ral051nt4g3ncy in https://github.com/soxoj/maigret/pull/610
* Bump cloudscraper from 1.2.63 to 1.2.64 by @dependabot in https://github.com/soxoj/maigret/pull/614
* Bump pycountry from 22.1.10 to 22.3.5 by @dependabot in https://github.com/soxoj/maigret/pull/607
* add ProtonMail, disable 3 broken sites by @fen0s in https://github.com/soxoj/maigret/pull/619
* Bump tqdm from 4.64.0 to 4.64.1 by @dependabot in https://github.com/soxoj/maigret/pull/618

**Full Changelog**: https://github.com/soxoj/maigret/compare/v0.4.3...v0.4.4

## [0.4.3] - 2022-04-13
* Added Sites to data.json by @kustermariocoding in https://github.com/soxoj/maigret/pull/386
* added new Websites to data.json by @kustermariocoding in https://github.com/soxoj/maigret/pull/390
Expand Down
4 changes: 2 additions & 2 deletions docs/source/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
copyright = '2021, soxoj'
author = 'soxoj'

release = '0.4.3'
version = '0.4.3'
release = '0.4.4'
version = '0.4.4'

# -- General configuration

Expand Down
4 changes: 2 additions & 2 deletions docs/source/development.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ PyPi package.

.. code-block:: console

git add ...
git commit -m 'Bump to 0.4.0'
git add -p
git commit -m 'Bump to YOUR VERSION'
git push origin head


Expand Down
2 changes: 1 addition & 1 deletion maigret/__version__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Maigret version file"""

__version__ = '0.4.3'
__version__ = '0.4.4'
1 change: 1 addition & 0 deletions maigret/errors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ def desc(self):
'Bot protection': 'Try to switch to another IP address',
'Censorship': 'switch to another internet service provider',
'Request timeout': 'Try to increase timeout or to switch to another internet service provider',
'Connecting failure': 'Try to decrease number of parallel connections (e.g. --n 10)',
}

# TODO: checking for reason
Expand Down
2 changes: 1 addition & 1 deletion maigret/maigret.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ def setup_arguments_parser(settings: Settings):
type=int,
dest="connections",
default=settings.max_connections,
help="Allowed number of concurrent connections.",
help=f"Allowed number of concurrent connections (default {settings.max_connections}).",
)
parser.add_argument(
"--no-recursion",
Expand Down
140 changes: 135 additions & 5 deletions maigret/resources/data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13094,8 +13094,8 @@
"usernameUnclaimed": "noonewouldeverusethis7"
},
"Rap-royalty": {
"disabled": true,
"urlSubpath": "/forum",
"disabled": true,
"tags": [
"forum",
"music",
Expand Down Expand Up @@ -17060,7 +17060,7 @@
"video"
],
"headers": {
"Authorization": "jwt eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NjIyMDE5NjAsInVzZXJfaWQiOm51bGwsImFwcF9pZCI6NTg0NzksInNjb3BlcyI6InB1YmxpYyIsInRlYW1fdXNlcl9pZCI6bnVsbH0.Omx8x6yyzU_AdxxnVSzo7jv9JPLTrKCrIiridIW8YCY"
"Authorization": "jwt eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NjI4NDkxODAsInVzZXJfaWQiOm51bGwsImFwcF9pZCI6NTg0NzksInNjb3BlcyI6InB1YmxpYyIsInRlYW1fdXNlcl9pZCI6bnVsbH0.DAmAaaKixSd_WQ9-7PiTZDmyK61SHEYluYC-qdcJtkE"
},
"activation": {
"url": "https://vimeo.com/_rv/viewer",
Expand Down Expand Up @@ -18245,10 +18245,19 @@
"tags": [
"ru"
],
"checkType": "status_code",
"checkType": "message",
"absenceStrs": [
".zen-ui-page-404"
],
"presenseStrs": [
"zen_object_id"
],
"alexaRank": 50,
"urlMain": "https://zen.yandex.ru",
"url": "https://zen.yandex.ru/{username}",
"urlMain": "https://dzen.ru",
"url": "https://dzen.ru/channel/{username}",
"headers": {
"Cookie": "Session_id=noauth:1; yandex_login=; ys=c_chck.1; mda2_beacon=1; sso_status=sso.passport.yandex.ru:synchronized; _yasc=1; _ym_uid=1; _ym_d=1; _ym_isad=2; yandexuid=1"
},
"source": "Yandex",
"usernameClaimed": "tema",
"usernameUnclaimed": "noonewouldeverusethis77777"
Expand Down Expand Up @@ -33625,6 +33634,127 @@
],
"usernameUnclaimed": "noonewouldeverusethis7",
"alexaRank": 3482358
},
"goodgame.ru": {
"absenceStrs": [
"not-found-wrap",
"images/404.gif"
],
"presenseStrs": [
"name",
"streamer_name",
"user",
" role=",
"streamer"
],
"url": "https://goodgame.ru/channel/{username}",
"urlMain": "https://goodgame.ru",
"usernameClaimed": "Nikichar",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 55420,
"tags": [
"ru",
"streaming"
]
},
"breakers.tv": {
"absenceStrs": [
"Channel Not Found - Breakers.TV"
],
"presenseStrs": [
"profile",
"copy_username",
"View Profile",
"copy_username_chat",
"chatterUserBoxName"
],
"url": "https://breakers.tv/{username}",
"urlMain": "https://breakers.tv",
"usernameClaimed": "friendlyboxbreaks",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 987478,
"tags": [
"streaming",
"us"
]
},
"AfreecaTV": {
"absenceStrs": [
"Blog does not exist."
],
"presenseStrs": [
"profile_text",
"profile_image",
"name",
"station_name",
"user_nick"
],
"url": "http://bjapi.afreecatv.com/api/{username}/station",
"urlMain": "http://bjapi.afreecatv.com",
"usernameClaimed": "showsaovivo",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 905,
"tags": [
"streaming"
]
},
"Picarto": {
"absenceStrs": [
"We are the world\\u2019s leading live streaming platform for creative minds. Come join us"
],
"presenseStrs": [
"\"success\":true"
],
"url": "https://ptvintern.picarto.tv/metadescription/{username}",
"urlMain": "https://ptvintern.picarto.tv",
"usernameClaimed": "tamarinfrog",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 15844,
"tags": [
"art",
"streaming"
]
},
"stripchat.global": {
"presenseStrs": [
"profile email",
"setVersionName",
",SITE_NAME=",
"input[name=",
"project"
],
"url": "https://stripchat.global/{username}",
"urlMain": "https://stripchat.global",
"usernameClaimed": "lunagirl13",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 117062,
"tags": [
"webcam"
]
},
"dlive.tv": {
"absenceStrs": [
"Channel not found"
],
"presenseStrs": [
"username",
"profile-part",
"profile-about"
],
"url": "https://dlive.tv/{username}",
"urlMain": "https://dlive.tv",
"usernameClaimed": "TomTourettes",
"usernameUnclaimed": "noonewouldeverusethis7",
"checkType": "message",
"alexaRank": 17235,
"tags": [
"streaming"
]
}
},
"engines": {
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
requires = rf.read().splitlines()

setup(name='maigret',
version='0.4.3',
version='0.4.4',
description='Collect a dossier on a person by username from a huge number of sites',
long_description=long_description,
long_description_content_type="text/markdown",
Expand Down
Loading