This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: TimedRotatingFileHandler deletes wrong files
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Sami Salonen, iritkatriel, miss-islington, vinay.sajip
Priority: normal Keywords: patch

Created on 2021-12-13 12:12 by Sami Salonen, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 30093 merged vinay.sajip, 2021-12-13 21:55
PR 30094 merged miss-islington, 2021-12-14 00:53
PR 30095 merged miss-islington, 2021-12-14 00:54
PR 30103 merged vinay.sajip, 2021-12-14 16:07
PR 30104 merged miss-islington, 2021-12-14 16:46
PR 30105 merged miss-islington, 2021-12-14 16:46
Messages (10)
msg408445 - (view) Author: Sami Salonen (Sami Salonen) Date: 2021-12-13 12:12
https://bugs.python.org/issue45628 breaks file rotation.

I name my log files after module name, run the modules in separate processes, use the same log config in them, and store the log files in one common directory. For example, for modules `a.b` and `a.b.c` I have the following log files:

a.b.log
a.b.c.log

Now after bpo-45628 fix, `getFilesToDelete` treats the rotated log files from those modules the same, and deletes wrong files. If `backupCount` is set to e.g. 7, only total of 7 files are stored instead of 7 of each.
msg408446 - (view) Author: Sami Salonen (Sami Salonen) Date: 2021-12-13 12:14
Breaking PR: https://github.com/python/cpython/pull/29310
msg408506 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 00:53
New changeset cb589d1b6bad4b75852c2e2a471a3800d5efdca7 by Vinay Sajip in branch 'main':
bpo-46063: Improve algorithm for computing which rolled-over log file… (GH-30093)
https://github.com/python/cpython/commit/cb589d1b6bad4b75852c2e2a471a3800d5efdca7
msg408509 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 01:18
New changeset f84e2f6c0aca97c59ec8ce21715ae9bd89893307 by Miss Islington (bot) in branch '3.10':
[3.10] bpo-46063: Improve algorithm for computing which rolled-over log file… (GH-30093) (GH-30094)
https://github.com/python/cpython/commit/f84e2f6c0aca97c59ec8ce21715ae9bd89893307
msg408510 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 01:19
New changeset 94234228abbb84945a48049a7515dea960bc9834 by Miss Islington (bot) in branch '3.9':
[3.9] bpo-46063: Improve algorithm for computing which rolled-over log file… (GH-30093) (GH-30095)
https://github.com/python/cpython/commit/94234228abbb84945a48049a7515dea960bc9834
msg408521 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-14 11:18
% ./python.exe -We -m test -v test_logging -m test_compute_files_to_delete 
Raised RLIMIT_NOFILE: 256 -> 1024
== CPython 3.11.0a3+ (heads/main:9130a4d620, Dec 14 2021, 11:12:05) [Clang 13.0.0 (clang-1300.0.29.3)]
== macOS-11.6-x86_64-i386-64bit little-endian
== cwd: /Users/iritkatriel/src/cpython/build/test_python_86072æ
== CPU count: 12
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 5.20 Run tests sequentially
0:00:00 load avg: 5.20 [1/1] test_logging
test_compute_files_to_delete (test.test_logging.TimedRotatingFileHandlerTest) ... Warning -- Unraisable exception
Exception ignored in: <_io.FileIO name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/d.e.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/Lib/test/test_logging.py", line 5469, in test_compute_files_to_delete
    for i, prefix in enumerate(prefixes):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/d.e.log' mode='a' encoding='UTF-8'>
Warning -- Unraisable exception
Exception ignored in: <_io.FileIO name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/a.b.c.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/Lib/test/test_logging.py", line 5469, in test_compute_files_to_delete
    for i, prefix in enumerate(prefixes):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/a.b.c.log' mode='a' encoding='UTF-8'>
Warning -- Unraisable exception
Exception ignored in: <_io.FileIO name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/a.b.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/Lib/test/test_logging.py", line 5469, in test_compute_files_to_delete
    for i, prefix in enumerate(prefixes):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/a.b.log' mode='a' encoding='UTF-8'>
Warning -- Unraisable exception
Exception ignored in: <_io.FileIO name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/d.e.f.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython/Lib/test/test_logging.py", line 5469, in test_compute_files_to_delete
    for i, prefix in enumerate(prefixes):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/kf/0v7kz3ps62dg11v9rq0sz35m0000gn/T/test_logging_w3nn9vsi/d.e.f.log' mode='a' encoding='UTF-8'>
ok

----------------------------------------------------------------------
Ran 1 test in 0.016s

OK
test_logging failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment:
    test_logging
msg408522 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 11:22
Ah ... forgot to set delay=True for the handlers. Will look at this soon.
msg408548 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 16:46
New changeset 850aefc2c651110a784cd5478af9774b1f6287a3 by Vinay Sajip in branch 'main':
bpo-46063: Add 'delay=True' to file handler initialization. (GH-30103)
https://github.com/python/cpython/commit/850aefc2c651110a784cd5478af9774b1f6287a3
msg408552 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 17:27
New changeset 908fd691f96403a3c30d85c17dd74ed1f26a60fd by Miss Islington (bot) in branch '3.10':
[3.10] bpo-46063: Add 'delay=True' to file handler initialization. (GH-30103) (GH-30104)
https://github.com/python/cpython/commit/908fd691f96403a3c30d85c17dd74ed1f26a60fd
msg408553 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2021-12-14 17:27
New changeset 17260e44b5ed3508e3c15f1b7ded761879e91d3e by Miss Islington (bot) in branch '3.9':
[3.9] bpo-46063: Add 'delay=True' to file handler initialization. (GH-30103) (GH-30105)
https://github.com/python/cpython/commit/17260e44b5ed3508e3c15f1b7ded761879e91d3e
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90221
2021-12-14 17:27:26vinay.sajipsetmessages: + msg408553
2021-12-14 17:27:09vinay.sajipsetmessages: + msg408552
2021-12-14 16:46:38vinay.sajipsetmessages: + msg408548
2021-12-14 16:46:27miss-islingtonsetpull_requests: + pull_request28328
2021-12-14 16:46:22miss-islingtonsetpull_requests: + pull_request28327
2021-12-14 16:07:48vinay.sajipsetpull_requests: + pull_request28326
2021-12-14 11:22:49vinay.sajipsetmessages: + msg408522
2021-12-14 11:18:12iritkatrielsetnosy: + iritkatriel
messages: + msg408521
2021-12-14 10:01:53vinay.sajipsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-12-14 01:19:54vinay.sajipsetmessages: + msg408510
2021-12-14 01:18:02vinay.sajipsetmessages: + msg408509
2021-12-14 00:54:01miss-islingtonsetpull_requests: + pull_request28318
2021-12-14 00:53:56miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request28317
2021-12-14 00:53:45vinay.sajipsetmessages: + msg408506
2021-12-13 21:55:25vinay.sajipsetkeywords: + patch
stage: patch review
pull_requests: + pull_request28316
2021-12-13 15:40:19iritkatrielsetnosy: + vinay.sajip
2021-12-13 12:14:04Sami Salonensetmessages: + msg408446
2021-12-13 12:12:03Sami Salonencreate