On non-Windows platforms, if a user attempts to use asyncio.get_event_loop() in a child process created by multiprocessing.Process using the fork context, and an asyncio event loop is also being used in the main process, the same _UnixSelectorEventLoop object will be used by both processes. This, of course, won't work properly; the child will raise a "RuntimeError: Event loop is running" exception as soon as it tries using the loop object.

However, this may or may not actually make it back to the parent: If the parent is expecting to get items from a queue from that child publishes to, rather than yielding from it immediately, the program will deadlock. Even if the child is yielded from, it may not be immediately obvious why "Event loop is running" was raised, and the behavior is inconsistent with the behavior if a method other than os.fork is used to create the child process, since the child will get a new event loop in that case.

So, it'd be better if _UnixDefaultEventLoopPolicy detected that get_event_loop was being called in a child process, and either 

1) Created a new loop for the child (this would make the behavior appear consistent no matter what platform/method for launching children is used)
2) Raised an exception stating that no default event loop exists for this process, similar to the assert used for threads currently.

I've attached a test script that demonstrates the different between forked/spawned processes, and a patch that implements #1 above.

Good point.  Asyncio definitely should not share event loops across forked processes.  However, I don't like the dependency on multiprocessing (even though it's in the stdlib) -- can't the policy just use os.getpid()?

Also, I've got a feeling that maybe the pid should be part of the policy state instead of the loop state?  The policy could just reset self._local when the pid doesn't match.

Yep, agreed on both points. The latter suggestion also has the benefit of not requiring any test changes. Here's an updated patch.

I think there should still be a new unittest -- we're adding a behavior we're promising, so we should test it.

See aslo issue #21998: "asyncio: a new self-pipe should be created in the child process after fork".

I've added a unit test that spawns a new forked process via multiprocessing, and verifies that the loop returned by get_event_loop is not the same as the one we have in the parent.

re: #21998, perhaps it's time to revive #16500? Without that, I'm not sure what can be done aside from documenting the need to call "loop = asyncio.get_event_loop()" in the child immediately after forking.

A simple pid check in the policy should be enough.

Hmm, I'm not sure what you mean. What check in the policy would prevent this issue you described in #21998?:

import asyncio, os
loop = asyncio.get_event_loop()
pid = os.fork()
if pid:
    print("parent", loop._csock.fileno(), loop._ssock.fileno())
else:
    print("child", loop._csock.fileno(), loop._ssock.fileno())

Output:
---
parent 6 5
child 6 5

Are any other changes needed here? I'm still not completely clear on what Victor meant with his last comment.

This issue looks to be a duplicate of #21998.

handle-mp_unix2.patch looks more to a workaround than a real issue. When I write asyncio code, I prefer to pass explicitly the loop, so get_event_loop() should never be called. IMO the methods of the event loop should detect the fork and handle the fork directly.

See also the https://pypi.python.org/pypi/mpworker project

I believe this is now worse due to https://github.com/python/asyncio/pull/452

before I was able to simply create a new run loop from sub-processes however you will now get the error "Cannot run the event loop while another loop is running".  The state of the run loop should not be preserved in sub-processes either.

The approach in Dan's patches looks sane to me and I agree the issue needs fixing.  Dan, would you like to submit a PR for this?  We're using Github for patch submissions now.

> The approach in Dan's patches looks sane to me and I agree the issue needs fixing.  Dan, would you like to submit a PR for this?  We're using Github for patch submissions now.


Mind that the patch is very outdated and we already have these checks in get_event_loop.

It seems there's a check in the top-level get_event_loop() function but not in the DefaultEventLoopPolicy.get_event_loop() method.

(also: wow, that stuff is complicated)

> It seems there's a check in the top-level get_event_loop() function but not in the DefaultEventLoopPolicy.get_event_loop() method.

Yes, as users aren't supposed to work with policies directly.

> (also: wow, that stuff is complicated)

Yep. Please assign me to review if you submit a PR.

New changeset 5d97b7bcc19496617bf8c448d2f149cc28c73bc7 by Yury Selivanov in branch 'master':
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
https://github.com/python/cpython/commit/5d97b7bcc19496617bf8c448d2f149cc28c73bc7

New changeset 2a7eb0b531656f4a77d85078e6e009e4b3639ef9 by Miss Islington (bot) in branch '3.7':
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
https://github.com/python/cpython/commit/2a7eb0b531656f4a77d85078e6e009e4b3639ef9

another related issue: https://bugs.python.org/issue33688

Even though I committed a version of Dan's patch to 3.7 and 3.8, I've finally decided to revert it and do it properly instead.  We should strive to implement a proper solution, not commit some half-working code.

A concrete plan (for Python 3.8 probably):

1. Fix BaseDefaultEventLoopPolicy to track PID in its 'get_event_loop()' and 'set_event_loop()' methods.  If a PID has changed since the last invocation: reset its internal local state.

2. Fix _UnixDefaultEventLoopPolicy to check for PID change in 'get_child_watcher()' and 'set_child_watcher()'.

3. Fix child watcher / event loops to track PID changes too to avoid listening for child processes of their parent process.

4. Look at how libuv and other event loops implement fork support. Ideally we should be able to shutdown selectors (epoll, kqueue) in forked processes in such a way that their parent process isn't affected.

5. Think how we can make 'asyncio.run' fork friendly. Ideally it should initialize its own child watcher and remove it when its done.

New changeset 99279ad823a758288e4e41962abfc4dad8943ce8 by Yury Selivanov in branch 'master':
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (#7232)
https://github.com/python/cpython/commit/99279ad823a758288e4e41962abfc4dad8943ce8

New changeset af9cda9845666e2f704177a431d29f91efbf828a by Yury Selivanov in branch '3.7':
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (GH-7233)
https://github.com/python/cpython/commit/af9cda9845666e2f704177a431d29f91efbf828a