Skip to content

Python shelve __del__ ignored exception #83383

Closed
Closed
Python shelve __del__ ignored exception#83383
Assignees
serhiy-storchaka
Labels
3.13only security fixesstdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Description

@libbkmz
libbkmz
mannequin
opened on Jan 3, 2020
BPO 39202
Nosy @libbkmz
Files
  • test_python_shelve_issue.py: python testcases via unittests library to reproduce the issue
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-01-03.16:19:53.484>
labels = ['3.8', 'type-feature', 'library']
title = 'Python shelve __del__ ignored exception'
updated_at = <Date 2020-01-03.16:19:53.484>
user = 'https://github.com/libbkmz'

    bugs.python.org fields:

    activity = <Date 2020-01-03.16:19:53.484>
actor = 'libbkmz'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-01-03.16:19:53.484>
creator = 'libbkmz'
dependencies = []
files = ['48826']
hgrepos = []
issue_num = 39202
keywords = []
message_count = 1.0
messages = ['359242']
nosy_count = 1.0
nosy_names = ['libbkmz']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue39202'
versions = ['Python 3.8']

    Linked PRs

    Activity

    1. libbkmz commented on Jan 3, 2020

      @libbkmz
      libbkmzmannequin
      on Jan 3, 2020
      MannequinAuthor

      I'm using my own implementation of the memoize by shelve module. In the attachment, there are 2 simple test cases which pass but the console there are a lot of messages like that:

      Exception ignored in: <function Shelf.__del__ at 0x0000023202F363A0>
Traceback (most recent call last):
  File "C:\Miniconda2\envs\38_common\lib\shelve.py", line 162, in __del__
    self.close()
  File "C:\Miniconda2\envs\38_common\lib\shelve.py", line 144, in close
    self.sync()
  File "C:\Miniconda2\envs\38_common\lib\shelve.py", line 172, in sync
    self.dict.sync()
  File "C:\Miniconda2\envs\38_common\lib\dbm\dumb.py", line 129, in _commit
    with self._io.open(self._dirfile, 'w', encoding="Latin-1") as f:
PermissionError: [Errno 13] Permission denied: 'C:\\project\\tests\\test_memoize_tmp_t5tai08p\\memoize_test_file.dat.dir'

Exception ignored in: <function _Database.close at 0x0000023202EF9040>
Traceback (most recent call last):
  File "C:\Miniconda2\envs\38_common\lib\dbm\dumb.py", line 274, in close
    self._commit()
  File "C:\Miniconda2\envs\38_common\lib\dbm\dumb.py", line 129, in _commit
    with self._io.open(self._dirfile, 'w', encoding="Latin-1") as f:
PermissionError: [Errno 13] Permission denied: 'C:\\project\\tests\\test_memoize_tmp_t5tai08p\\memoize_test_file.dat.dir'

      Basically, the main issue can be explained like that - Python dbm.dumb._Database should maintain self._modified the attribute in the right way(set it to False) after the _commit method.

      Later I will try to make changes in the dbm.dumb module and run Python internal tests for that modification to see any regression, if not will add PR here.

    2. libbkmz
      added
      3.8 (EOL)end of life
      stdlibStandard Library Python modules in the Lib/ directory
      type-featureA feature request or enhancement
      on Jan 3, 2020
    3. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    4. erlend-aasland
      added this to dbm and shelve issueson Jan 25, 2024
    5. serhiy-storchaka
      moved this to In Progress in dbm and shelve issueson Jan 25, 2024
    6. serhiy-storchaka
      added a commit that references this issue on Jan 25, 2024

      pythongh-83383: Always mark the dbm.dumb database as unmodified after…

      068d971
    7. serhiy-storchaka
      added
      3.13only security fixes
      and removed
      3.8 (EOL)end of life
      on Jan 25, 2024

    8. serhiy-storchaka commented on Jan 25, 2024

      @serhiy-storchaka
      serhiy-storchaka
      on Jan 25, 2024
      Member

      It is not a bug. If you do not close the database explicitly, it will be implicitly closed in the __del__() method. And this happens when the local variable memoize is rebound, which happens only after a new database referring to the same files is opened. Opening multiple databases that use the same files is not supported.

      But it is good idea to mark the database as unmodified after synchronization or creation and avoid any writing in __del__() if there was no modifications since that moment. It helps to save data, even if the use of the database is not perfect.

    9. serhiy-storchaka
      self-assigned this
      on Jan 25, 2024
    10. serhiy-storchaka
      added a commit that references this issue on Feb 4, 2024

      gh-83383: Always mark the dbm.dumb database as unmodified after open(…

      fc06096
    11. serhiy-storchaka
      closed this as completedon Feb 4, 2024
    12. github-project-automation
      moved this from In Progress to Done in dbm and shelve issueson Feb 4, 2024
    13. aisk
      added a commit that references this issue on Feb 11, 2024

      pythongh-83383: Always mark the dbm.dumb database as unmodified after…

      ce82e49
    14. fsc-eriker
      added a commit that references this issue on Feb 14, 2024

      pythongh-83383: Always mark the dbm.dumb database as unmodified after…

      16baf0e
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.13only security fixesstdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions