Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions Doc/c-api/gen.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,6 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`.
The C structure used for generator objects.


.. c:type:: PySendResult

The enum value used to represent different results of :c:func:`PyGen_Send`.


.. c:var:: PyTypeObject PyGen_Type

The type object corresponding to generator objects.
Expand Down
14 changes: 14 additions & 0 deletions Doc/c-api/iter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,17 @@ something like this::
else {
/* continue doing useful work */
}


.. c:type:: PySendResult

The enum value used to represent different results of :c:func:`PyIter_Send`.


.. c:function:: PySendResult PyIter_Send(PyObject *iter, PyObject *arg, PyObject **presult)

Sends the *arg* value into the iterator *iter*. Returns:

- ``PYGEN_RETURN`` if iterator returns. Return value is returned via *presult*.
- ``PYGEN_NEXT`` if iterator yields. Yielded value is returned via *presult*.
- ``PYGEN_ERROR`` if iterator has raised and exception. *presult* is set to ``NULL``.
5 changes: 5 additions & 0 deletions Doc/data/refcounts.dat
Original file line number Diff line number Diff line change
Expand Up @@ -1081,6 +1081,11 @@ PyIter_Check:PyObject*:o:0:
PyIter_Next:PyObject*::+1:
PyIter_Next:PyObject*:o:0:

PyIter_Send:int:::
PyIter_Send:PyObject*:iter:0:
PyIter_Send:PyObject*:arg:0:
PyIter_Send:PyObject**:presult:+1:

PyList_Append:int:::
PyList_Append:PyObject*:list:0:
PyList_Append:PyObject*:item:+1:
Expand Down
4 changes: 4 additions & 0 deletions Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,10 @@ New Features
search function.
(Contributed by Hai Shi in :issue:`41842`.)

* The :c:func:`PyIter_Send` and :c:func:`PyGen_Send` functions were added to allow
sending value into iterator without raising ``StopIteration`` exception.
(Contributed by Vladimir Matveev in :issue:`41756`.)

Porting to Python 3.10
----------------------

Expand Down
16 changes: 16 additions & 0 deletions Include/abstract.h
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,22 @@ PyAPI_FUNC(int) PyIter_Check(PyObject *);
NULL with an exception means an error occurred. */
PyAPI_FUNC(PyObject *) PyIter_Next(PyObject *);

typedef enum {
PYGEN_RETURN = 0,
PYGEN_ERROR = -1,
PYGEN_NEXT = 1,
} PySendResult;

/* Takes generator, coroutine or iterator object and sends the value into it.
Returns:
- PYGEN_RETURN (0) if generator has returned.
'result' parameter is filled with return value
- PYGEN_ERROR (-1) if exception was raised.
'result' parameter is NULL
- PYGEN_NEXT (1) if generator has yielded.
'result' parameter is filled with yielded value. */
PyAPI_FUNC(PySendResult) PyIter_Send(PyObject *, PyObject *, PyObject **);


/* === Number Protocol ================================================== */

Expand Down
8 changes: 1 addition & 7 deletions Include/genobject.h
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ extern "C" {
#endif

#include "pystate.h" /* _PyErr_StackItem */
#include "abstract.h" /* PySendResult */

/* _PyGenObject_HEAD defines the initial segment of generator
and coroutine objects. */
Expand Down Expand Up @@ -41,16 +42,9 @@ PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(PyFrameObject *,
PyObject *name, PyObject *qualname);
PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *);
PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **);
PyAPI_FUNC(PyObject *) _PyGen_Send(PyGenObject *, PyObject *);
PyObject *_PyGen_yf(PyGenObject *);
PyAPI_FUNC(void) _PyGen_Finalize(PyObject *self);

typedef enum {
PYGEN_RETURN = 0,
PYGEN_ERROR = -1,
PYGEN_NEXT = 1,
} PySendResult;

/* Sends the value into the generator or the coroutine. Returns:
- PYGEN_RETURN (0) if generator has returned.
'result' parameter is filled with return value
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add `PyIter_Send` function to allow sending value into
generator/coroutine/iterator without raising StopIteration exception to
signal return.
9 changes: 1 addition & 8 deletions Modules/_asynciomodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ _Py_IDENTIFIER(add_done_callback);
_Py_IDENTIFIER(call_soon);
_Py_IDENTIFIER(cancel);
_Py_IDENTIFIER(get_event_loop);
_Py_IDENTIFIER(send);
_Py_IDENTIFIER(throw);


Expand Down Expand Up @@ -2695,13 +2694,7 @@ task_step_impl(TaskObj *task, PyObject *exc)

int gen_status = PYGEN_ERROR;
if (exc == NULL) {
if (PyGen_CheckExact(coro) || PyCoro_CheckExact(coro)) {
gen_status = PyGen_Send((PyGenObject*)coro, Py_None, &result);
}
else {
result = _PyObject_CallMethodIdOneArg(coro, &PyId_send, Py_None);
gen_status = gen_status_from_result(&result);
}
gen_status = PyIter_Send(coro, Py_None, &result);
}
else {
result = _PyObject_CallMethodIdOneArg(coro, &PyId_throw, exc);
Expand Down
3 changes: 2 additions & 1 deletion Modules/_testcapimodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -5028,6 +5028,7 @@ dict_get_version(PyObject *self, PyObject *args)
static PyObject *
raise_SIGINT_then_send_None(PyObject *self, PyObject *args)
{
_Py_IDENTIFIER(send);
PyGenObject *gen;

if (!PyArg_ParseTuple(args, "O!", &PyGen_Type, &gen))
Expand All @@ -5044,7 +5045,7 @@ raise_SIGINT_then_send_None(PyObject *self, PyObject *args)
because we check for signals before every bytecode operation.
*/
raise(SIGINT);
return _PyGen_Send(gen, Py_None);
return _PyObject_CallMethodIdOneArg((PyObject *)gen, &PyId_send, Py_None);
}


Expand Down
24 changes: 24 additions & 0 deletions Objects/abstract.c
Original file line number Diff line number Diff line change
Expand Up @@ -2671,6 +2671,30 @@ PyIter_Next(PyObject *iter)
return result;
}

PySendResult
PyIter_Send(PyObject *iter, PyObject *arg, PyObject **result)
{
_Py_IDENTIFIER(send);
assert(result != NULL);

if (PyGen_CheckExact(iter) || PyCoro_CheckExact(iter)) {
return PyGen_Send((PyGenObject *)iter, arg, result);
}

if (arg == Py_None && PyIter_Check(iter)) {
*result = Py_TYPE(iter)->tp_iternext(iter);
}
else {
*result = _PyObject_CallMethodIdOneArg(iter, &PyId_send, arg);
}
if (*result != NULL) {
return PYGEN_NEXT;
}
if (_PyGen_FetchStopIterationValue(result) == 0) {
return PYGEN_RETURN;
}
return PYGEN_ERROR;
}

/*
* Flatten a sequence of bytes() objects into a C array of
Expand Down
8 changes: 1 addition & 7 deletions Objects/genobject.c
Original file line number Diff line number Diff line change
Expand Up @@ -308,12 +308,6 @@ gen_send(PyGenObject *gen, PyObject *arg)
return gen_send_ex(gen, arg, 0, 0);
}

PyObject *
_PyGen_Send(PyGenObject *gen, PyObject *arg)
{
return gen_send(gen, arg);
}

PyDoc_STRVAR(close_doc,
"close() -> raise GeneratorExit inside generator.");

Expand Down Expand Up @@ -1012,7 +1006,7 @@ PyDoc_STRVAR(coro_close_doc,
"close() -> raise GeneratorExit inside coroutine.");

static PyMethodDef coro_methods[] = {
{"send",(PyCFunction)_PyGen_Send, METH_O, coro_send_doc},
{"send",(PyCFunction)gen_send, METH_O, coro_send_doc},
{"throw",(PyCFunction)gen_throw, METH_VARARGS, coro_throw_doc},
{"close",(PyCFunction)gen_close, METH_NOARGS, coro_close_doc},
{NULL, NULL} /* Sentinel */
Expand Down
21 changes: 7 additions & 14 deletions Python/ceval.c
Original file line number Diff line number Diff line change
Expand Up @@ -2213,24 +2213,17 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
case TARGET(YIELD_FROM): {
PyObject *v = POP();
PyObject *receiver = TOP();
int is_gen_or_coro = PyGen_CheckExact(receiver) || PyCoro_CheckExact(receiver);
int gen_status;
if (tstate->c_tracefunc == NULL && is_gen_or_coro) {
gen_status = PyGen_Send((PyGenObject *)receiver, v, &retval);
PySendResult gen_status;
if (tstate->c_tracefunc == NULL) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we fold the tracing into the genobject/PyIter_Send inmplementation to avoid code duplication?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would also make ceval a tad smaller which is always a good thing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it is feasible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a call of the trace function in between calls of __next__/send and _PyGen_FetchStopIterationValue. It cannot be called after PyIter_Send because the original exception (which can be a subclass of StopIteration or StopIteration with additional attributes) is already silenced in PyIter_Send. Therefore we should pass the trace function to PyIter_Send, and this would complicate the API.

We can introduce a private function _PyIter_SendWithTrace which accepts also the trace function argument. But would not it make the code more complicated? In any case it is better to left to a separate PR or separate issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about exporting the call_exc_trace function and calling it directly from genobject.c; the trace function can easily be read from the thread state.

gen_status = PyIter_Send(receiver, v, &retval);
} else {
if (is_gen_or_coro) {
retval = _PyGen_Send((PyGenObject *)receiver, v);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_PyGen_Send() can be removed now, is not?

_Py_IDENTIFIER(send);
if (v == Py_None && PyIter_Check(receiver)) {
retval = Py_TYPE(receiver)->tp_iternext(receiver);
}
else {
_Py_IDENTIFIER(send);
if (v == Py_None) {
retval = Py_TYPE(receiver)->tp_iternext(receiver);
}
else {
retval = _PyObject_CallMethodIdOneArg(receiver, &PyId_send, v);
}
retval = _PyObject_CallMethodIdOneArg(receiver, &PyId_send, v);
}

if (retval == NULL) {
if (tstate->c_tracefunc != NULL
&& _PyErr_ExceptionMatches(tstate, PyExc_StopIteration))
Expand Down