When running tests, the tempoary directory is not left clean: there are temporary files which are not removed.

I wrote a PoC change to detect such bugs and I found that at least test_shutil and test_urllib leak such temporary files/directories.

Test leaking a temporary file in test_urllib:
test.test_urllib.urlretrieve_HttpTests.test_short_content_raises_ContentTooShortError_without_reporthook

I'm not sure of my fix:

diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py
index f6ce9cb6d5..fdddd6e2d8 100644
--- a/Lib/urllib/request.py
+++ b/Lib/urllib/request.py
@@ -283,6 +283,7 @@ def urlretrieve(url, filename=None, reporthook=None, data=None):
                     reporthook(blocknum, bs, size)
 
     if size >= 0 and read < size:
+        urlcleanup()
         raise ContentTooShortError(
             "retrieval incomplete: got only %i out of %i bytes"
             % (read, size), result)

Using PR 14415, the following test of test_multiprocessing_spawn emits a false alarm because multiprocessing use "Finalizer" objects which are only finalized "later":
test.test_multiprocessing_spawn.WithManagerTestMyManager.test_mymanager_context_prestarted


Workaround, call explicitly _run_finalizers():

diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
index eef262d723..bfecbab9ee 100644
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -5651,6 +5651,7 @@ def install_tests_in_module_dict(remote_globs, start_method):
         if need_sleep:
             time.sleep(0.5)
         multiprocessing.process._cleanup()
+        multiprocessing.util._run_finalizers()
         test.support.gc_collect()
 
     remote_globs['setUpModule'] = setUpModule

New changeset 4c26abd14f1b7242998eb2f7756aa375e0fe714f by Victor Stinner in branch 'master':
bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416)
https://github.com/python/cpython/commit/4c26abd14f1b7242998eb2f7756aa375e0fe714f

New changeset 7fe81ce47ff6725e2e4d667d499e23dac19834af by Miss Islington (bot) in branch '3.8':
bpo-37421: Fix test_shutil: don't leak temporary files (GH-14416)
https://github.com/python/cpython/commit/7fe81ce47ff6725e2e4d667d499e23dac19834af

About urllib.request, I created: bpo-37475 "What is urllib.request.urlcleanup() function?".

New changeset 039fb49c185570ab7b02f13fbdc51c859cfd831e by Victor Stinner in branch 'master':
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
https://github.com/python/cpython/commit/039fb49c185570ab7b02f13fbdc51c859cfd831e

New changeset 632cb36084dc9d13f1cdb31a0e7e3ba80745a51a by Miss Islington (bot) in branch '3.8':
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
https://github.com/python/cpython/commit/632cb36084dc9d13f1cdb31a0e7e3ba80745a51a

New changeset 7cb9204ee1cf204f6f507d99a60f7c5bb359eebb by Victor Stinner in branch 'master':
bpo-37421: urllib.request tests call urlcleanup() (GH-14529)
https://github.com/python/cpython/commit/7cb9204ee1cf204f6f507d99a60f7c5bb359eebb

New changeset b71d8d67959f3b5efbdfe00066589ac0d8f98aad by Victor Stinner in branch 'master':
bpo-37421: test_winconsoleio doesn't leak temp file anymore (GH-14562)
https://github.com/python/cpython/commit/b71d8d67959f3b5efbdfe00066589ac0d8f98aad

New changeset 684cb47fffb7af3ac50cb077f6d2a095c9ce20b4 by Victor Stinner in branch 'master':
bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563)
https://github.com/python/cpython/commit/684cb47fffb7af3ac50cb077f6d2a095c9ce20b4

New changeset 74c9dd57771f4f061ee83b069c8e7b37de41246b by Victor Stinner in branch 'master':
bpo-37421: Fix test_distutils.test_build_ext() (GH-14564)
https://github.com/python/cpython/commit/74c9dd57771f4f061ee83b069c8e7b37de41246b

New changeset a2a807f75dc162dfb45fb297aee4961de8008f84 by Miss Islington (bot) in branch '3.8':
bpo-37421: test_winconsoleio doesn't leak temp file anymore (GH-14562)
https://github.com/python/cpython/commit/a2a807f75dc162dfb45fb297aee4961de8008f84

New changeset 79665c698fb8f97475e03c4231067db1ae47addb by Miss Islington (bot) in branch '3.8':
bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563)
https://github.com/python/cpython/commit/79665c698fb8f97475e03c4231067db1ae47addb

New changeset 0aefba7f99f0fd9bcb3328a5919e07f9e03676b0 by Miss Islington (bot) in branch '3.8':
bpo-37421: Fix test_distutils.test_build_ext() (GH-14564)
https://github.com/python/cpython/commit/0aefba7f99f0fd9bcb3328a5919e07f9e03676b0

New changeset 9d40554e0da09a44a8547f3f3a2b9dedfeaf7928 by Victor Stinner in branch 'master':
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
https://github.com/python/cpython/commit/9d40554e0da09a44a8547f3f3a2b9dedfeaf7928

New changeset 2d438fc0b748b64d518ea8876af3f6963c6d7d60 by Miss Islington (bot) in branch '3.7':
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
https://github.com/python/cpython/commit/2d438fc0b748b64d518ea8876af3f6963c6d7d60

New changeset 957656ee1d6fb42664274ef2f440a10d26870e2a by Miss Islington (bot) in branch '3.8':
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
https://github.com/python/cpython/commit/957656ee1d6fb42664274ef2f440a10d26870e2a

./python -u -m test test_multiprocessing_spawn -R 3:3 still fail:

ERROR: test_context (test.test_multiprocessing_spawn.TestStartMethod)
ERROR: test_set_get (test.test_multiprocessing_spawn.TestStartMethod)

PR 14601 fix these tests when test_multiprocessing_spawn is run more than once.

New changeset 8fbeb14312b4c1320d31ad86e69749515879d1c3 by Victor Stinner in branch 'master':
bpo-37421: multiprocessing tests now stop ForkServer (GH-14601)
https://github.com/python/cpython/commit/8fbeb14312b4c1320d31ad86e69749515879d1c3

I created a follow-up issue: bpo-37507 "multiprocessing: Add a stop() method to ForkServer".

New changeset 229f6e85f8b4d57a2e742e0d3fc361c5bd15f1cb by Miss Islington (bot) in branch '3.8':
bpo-37421: multiprocessing tests now stop ForkServer (GH-14601)
https://github.com/python/cpython/commit/229f6e85f8b4d57a2e742e0d3fc361c5bd15f1cb

New changeset e676244235895aeb6ec3b81ca3ccf4a70e487919 by Victor Stinner in branch 'master':
bpo-37421: test_concurrent_futures stops ForkServer (GH-14643)
https://github.com/python/cpython/commit/e676244235895aeb6ec3b81ca3ccf4a70e487919

New changeset cdada40b23b1f7f527797ba7cb14c25820b05981 by Victor Stinner (Miss Islington (bot)) in branch '3.8':
bpo-37421: test_concurrent_futures stops ForkServer (GH-14643) (GH-14645)
https://github.com/python/cpython/commit/cdada40b23b1f7f527797ba7cb14c25820b05981

It seems like tests don't leak temporary files anymore:

$ mkdir ~/TEMP
$ TMPDIR=~/TEMP TEMPDIR=~/TEMP ./python -m test -v -r -u all,-gui 
$ ls ~/TEMP
# empty directory

I close the issue.

New changeset fd32a0e2ee445dd7156323d216627112e66b0a69 by Victor Stinner in branch '3.7':
[3.7] bpo-38546: Backport multiprocessing tests fixes from master (GH-19689)
https://github.com/python/cpython/commit/fd32a0e2ee445dd7156323d216627112e66b0a69