Repository navigation
Migrate FlexGet to Loguru’s predefined log levels #5144
Description
Activity
@paranoidi, I noticed that your Pyright fix PR suppresses all warnings related to
logger.verbose. However, I think a better approach would be to migrate FlexGet to Loguru’s predefined log levels instead of suppressing these warnings.If you agree with this approach, I can prepare a PR to migrate FlexGet to Loguru’s predefined log levels.
Where would the verbose logs go? That's the level that's automatically used when FlexGet is run interactively as opposed to in cron or daemon mode.
Note
FlexGet currently uses the following log levels:
critical,error,warning,info,verbose,debug, andtrace.
Loguru’s predefined log levels are:
critical,error,warning,succeed,info,debug, andtrace.@gazpachoking A simple approach would be to map FlexGet’s seven log levels directly to Loguru’s seven predefined levels:
info→succeedandverbose→info.This mapping is straightforward, but not all existing
infologs represent successful operations. We could therefore review the affected logs and:- move non-successful
infologs to the newinfolevel; - move successful
verboselogs tosucceed.
If you think this additional review could cause confusion, we could instead keep the mapping as a simple replacement. The downside is that some
succeedlogs may not actually represent successful operations.- move non-successful
I dunno, I feel like succeeded doesn't fit our purposes quite right. Can we maybe move the logger import to somewhere internal and override when TYPE_CHECKING to add the method?
I dunno, I feel like succeeded doesn't fit our purposes quite right.
I counted the occurrences of
logger.infoandlogger.verbosein each file and found that 65 files uselogger.info, 60 uselogger.verbose, and 33 use both.The vast majority of files use only one of the two, so I think it is reasonable to conclude that most plugin authors do not distinguish between the semantics of
logger.infoandlogger.verbose, but simply choose whichever they prefer. Therefore, it seems reasonable to move the logs that represent completed operations to the newlogger.successlevel, while keeping the remaining logs atlogger.info.successhas a stronger semantic meaning, which should also make it easier for plugin authors to distinguish between different log levels.Table of the number of
infoandverboseoccurrences in each fileFile info verbose flexget\components\archive\archive.py0 1 flexget\components\archive\cli.py5 3 flexget\components\archive\db.py2 1 flexget\components\archives\decompress.py2 0 flexget\components\archives\utils.py0 2 flexget\components\backlog\backlog.py0 1 flexget\components\backlog\db.py2 0 flexget\components\bittorrent\convert_magnet.py1 0 flexget\components\bittorrent\torrent_scrub.py1 0 flexget\components\bittorrent\trackers.py3 0 flexget\components\estimate_release\estimators\est_released_movies.py0 1 flexget\components\ftp\ftp_download.py2 0 flexget\components\ftp\ftp_list.py0 1 flexget\components\ftp\sftp_client.py0 5 flexget\components\history\db.py0 1 flexget\components\imdb\from_imdb.py5 2 flexget\components\imdb\imdb.py1 0 flexget\components\imdb\imdb_lookup.py0 5 flexget\components\imdb\imdb_watchlist.py0 3 flexget\components\irc\irc.py3 2 flexget\components\managed_lists\list_add.py1 1 flexget\components\managed_lists\list_clear.py1 1 flexget\components\managed_lists\list_match.py1 1 flexget\components\managed_lists\list_remove.py1 1 flexget\components\managed_lists\lists\couchpotato_list.py2 6 flexget\components\managed_lists\lists\imdb_list.py0 1 flexget\components\managed_lists\lists\radarr_list.py0 5 flexget\components\managed_lists\lists\sonarr_list.py0 3 flexget\components\managed_lists\lists\subtitle_list.py2 0 flexget\components\managed_lists\lists\thetvdb_list.py0 2 flexget\components\notify\notification_framework.py0 1 flexget\components\notify\notifiers\discord.py4 0 flexget\components\notify\notifiers\ifttt.py1 0 flexget\components\notify\notifiers\matrix.py1 0 flexget\components\notify\notifiers\mqtt.py0 11 flexget\components\notify\notifiers\telegram.py1 0 flexget\components\notify\notify.py0 1 flexget\components\pending_approval\db.py1 0 flexget\components\pending_approval\pending_approval.py0 1 flexget\components\rejected\db.py3 1 flexget\components\rejected\remember_rejected.py1 0 flexget\components\scheduler\scheduler.py3 0 flexget\components\seen\db.py6 0 flexget\components\seen\seen.py1 0 flexget\components\seen\seen_movies.py1 0 flexget\components\series\configure_series.py1 1 flexget\components\series\db.py10 5 flexget\components\series\gen_series.py3 0 flexget\components\series\next_series_episodes.py0 2 flexget\components\series\next_series_seasons.py0 2 flexget\components\series\series.py3 7 flexget\components\series\series_begin.py1 0 flexget\components\series\series_remove.py2 0 flexget\components\sites\sites\allyoulike.py0 1 flexget\components\sites\sites\filelist_api.py0 2 flexget\components\sites\sites\lostfilm.py8 6 flexget\components\sites\sites\newznab.py4 1 flexget\components\sites\sites\rlsbb.py0 1 flexget\components\sites\sites\site_1337x.py1 0 flexget\components\sites\sites\site_rutracker.py1 0 flexget\components\sites\sites\solidtorrents.py1 0 flexget\components\sites\sites\uindex.py0 2 flexget\components\sites\urlrewrite_search.py0 2 flexget\components\sites\urlrewriting.py1 0 flexget\components\status\db.py0 2 flexget\components\thetvdb\api_tvdb.py1 1 flexget\components\thetvdb\thetvdb_lookup.py0 1 flexget\components\tmdb\api_tmdb.py1 1 flexget\components\trakt\db.py0 1 flexget\components\trakt\trakt_list.py1 3 flexget\db_schema.py2 0 flexget\entry.py1 0 flexget\ipc.py0 1 flexget\manager.py16 2 flexget\plugins\cli\check.py0 4 flexget\plugins\cli\explain_sql.py2 0 flexget\plugins\cli\perf_tests.py3 0 flexget\plugins\cli\performance.py4 0 flexget\plugins\clients\aria2.py0 1 flexget\plugins\clients\deluge.py6 3 flexget\plugins\clients\nzbget.py2 0 flexget\plugins\clients\pyload.py2 0 flexget\plugins\clients\qbittorrent.py14 0 flexget\plugins\clients\rtorrent.py6 5 flexget\plugins\clients\transmission.py10 0 flexget\plugins\daemon\web_server.py4 0 flexget\plugins\filter\content_filter.py7 0 flexget\plugins\filter\content_size.py2 0 flexget\plugins\filter\delay.py0 2 flexget\plugins\filter\exists.py0 1 flexget\plugins\filter\exists_movie.py0 4 flexget\plugins\filter\limit_new.py1 1 flexget\plugins\filter\only_new.py0 1 flexget\plugins\filter\proper_movies.py1 1 flexget\plugins\filter\rottentomatoes.py1 0 flexget\plugins\filter\timeframe.py1 0 flexget\plugins\generic\cron_env.py1 0 flexget\plugins\generic\db_analyze.py1 0 flexget\plugins\generic\db_vacuum.py1 0 flexget\plugins\generic\log_start.py2 0 flexget\plugins\generic\welcome.py2 0 flexget\plugins\input\anidb_list.py0 4 flexget\plugins\input\betaseries_list.py0 1 flexget\plugins\input\discover.py1 7 flexget\plugins\input\filesystem.py8 2 flexget\plugins\input\filmweb_watchlist.py0 3 flexget\plugins\input\from_task.py0 2 flexget\plugins\input\gazelle.py2 0 flexget\plugins\input\html.py1 3 flexget\plugins\input\inputs.py0 1 flexget\plugins\input\letterboxd.py0 1 flexget\plugins\input\next_sonarr_episodes.py0 2 flexget\plugins\input\npo_watchlist.py3 4 flexget\plugins\input\rlslog.py0 1 flexget\plugins\input\rss.py0 3 flexget\plugins\input\sickbeard.py2 0 flexget\plugins\input\tail.py6 0 flexget\plugins\input\text.py3 0 flexget\plugins\input\torznab.py2 0 flexget\plugins\internal\api_bluray.py0 1 flexget\plugins\internal\api_rottentomatoes.py0 1 flexget\plugins\metainfo\assume_quality.py0 3 flexget\plugins\modify\manipulate.py0 4 flexget\plugins\modify\regex_extract.py1 0 flexget\plugins\modify\sort_by_weight.py0 1 flexget\plugins\operate\debug_warnings.py1 0 flexget\plugins\operate\interval.py3 1 flexget\plugins\operate\proxy.py0 1 flexget\plugins\operate\run_task.py1 0 flexget\plugins\operate\sleep.py0 1 flexget\plugins\operate\spy_headers.py5 0 flexget\plugins\operate\verbose.py0 1 flexget\plugins\operate\verbose_details.py0 3 flexget\plugins\output\download.py5 0 flexget\plugins\output\exec.py3 1 flexget\plugins\output\file_operations.py15 3 flexget\plugins\output\html.py0 1 flexget\plugins\output\mock_output.py0 1 flexget\plugins\output\rss.py2 1 flexget\plugins\output\rtorrent_magnet.py2 0 flexget\plugins\output\sabnzbd.py2 0 flexget\plugins\output\sns.py2 0 flexget\plugins\output\subtitles_periscope.py1 0 flexget\plugins\output\subtitles_subliminal.py2 2 flexget\plugins\output\symlink.py0 2 flexget\plugins\output\utorrent.py3 0 flexget\plugins\services\kodi_library.py2 0 flexget\plugins\services\myepisodes.py3 0 flexget\plugins\services\pogcal_acquired.py0 4 flexget\task.py5 0 flexget\task_queue.py0 1 flexget\tray_icon.py0 2 flexget\utils\cached_input.py0 3 flexget\utils\lazy_dict.py1 0 flexget\utils\log.py1 1 flexget\utils\simple_persistence.py1 0 flexget\utils\tools.py0 1 flexget\webserver.py3 0 Total 281 205 Code used for the statistics
```py from pathlib import Path import re import sysdef count_logger_calls(directory: str):
info_pattern = re.compile(r"\blogger.info\s*(")
verbose_pattern = re.compile(r"\blogger.verbose\s*(")results = [] for path in Path(directory).rglob("*.py"): try: content = path.read_text(encoding="utf-8") except UnicodeDecodeError: content = path.read_text(encoding="utf-8", errors="ignore") info_count = len(info_pattern.findall(content)) verbose_count = len(verbose_pattern.findall(content)) if info_count or verbose_count: results.append((path, info_count, verbose_count)) return resultsif name == "main":
directory = sys.argv[1] if len(sys.argv) > 1 else "flexget"results = count_logger_calls(directory) total_info = 0 total_verbose = 0 print(f"{'File':80} {'info':>8} {'verbose':>8}") print("-" * 100) only_info, only_verbose, both=0,0,0 for path, info, verbose in sorted(results): print(f"{str(path):80} {info:8} {verbose:8}") total_info += info total_verbose += verbose if info and not verbose: only_info += 1 elif verbose and not info: only_verbose += 1 elif verbose and info: both += 1 print("-" * 100) print(f"{'Total':80} {total_info:8} {total_verbose:8}") print(f"The number of files using `logger.info` is {only_info}, the number using `logger.verbose` is {only_verbose}, and the number using both is {both}.")</details>Can we maybe move the logger import to somewhere internal and override when TYPE_CHECKING to add the method?
I did some quick testing, and this approach should work fine. However, all
loggerimports will need to be changed fromfrom loguru import loggertofrom flexget.log import logger.
Update: This is a bit hacky. Although I did a quick test and it seems to work, there could still be obstacles to implementing it fully.There is actually an even simpler solution: change
logger.verbose(message)tologger.log("verbose", message), avoiding the need for monkey-patching.So there are three ways to solve this issue now:
- Migrate FlexGet’s log levels to Loguru’s predefined log levels.
- Use the approach you mentioned: move the logger import to somewhere internal and override it under
TYPE_CHECKINGto add the method. - Change
logger.verbose(message)tologger.log("verbose", message).
@gazpachoking Which approach would you prefer?
Summary
All uses of
logger.verbosecurrently trigger the following Pyright error:Instead of suppressing these errors individually, I suggest migrating FlexGet away from the custom
verboselevel and using Loguru’s predefined levels.There are several reasons for this:
1.
verbosehas unclear semanticsFlexGet currently uses:
It is difficult to consistently distinguish the intended semantics of
info,verbose,debug, andtrace.In particular,
verbosedescribes the amount of detail rather than the nature of the event. Loguru'ssucceed, on the other hand, has a much clearer and more specific meaning.Therefore, rather than mechanically replacing every
verbosewithsucceed, I suggest reviewing the existinginfoandverbosemessages:succeedinfodebug/traceThis would make the logging levels more consistent and easier to understand.
2. Custom levels cause Pyright errors
verboseis a custom Loguru level and is not part of Loguru's typedLoggerAPI. As a result, everylogger.verbose(...)call produces a Pyright error.Loguru's predefined levels are already included in its type definitions, so using them would eliminate these errors without requiring individual
# type: ignorecomments or other suppressions.3. Custom log levels should generally be avoided
The Python logging documentation recommends avoiding custom logging levels when the existing levels are sufficient, and specifically warns that custom levels can be problematic for libraries because different libraries may define levels with conflicting meanings.
Although this documentation is for the standard
loggingmodule, the same principle applies here: if Loguru already provides suitable predefined levels, there is little reason for FlexGet to maintain its own custom level.Proposal
I suggest removing FlexGet's custom
verboselevel and migrating its current uses to Loguru's predefined levels based on their actual semantics:succeedfor messages indicating successful operationsinfofor other informational messages currently usinginfoorverbosedebug/tracefor messages that are primarily diagnosticThis would improve the semantic clarity of FlexGet's logging, eliminate the Pyright errors caused by
logger.verbose, and avoid maintaining a custom log level when suitable predefined levels already exist.