Skip to content

Conversation

@thorwhalen
Copy link

@thorwhalen thorwhalen commented Jul 7, 2020

Important Note: A single test (but replicated in many systems) fails, but I believe the test is not correct (see #21379 (comment) for detals)

WHAT
Adding 'defaults', 'kwdefaults' to WRAPPER_ASSIGNMENTS so that actual defaults can be consistent with signature defaults.

REASON

When using functools.wraps, the signature claims one set of defaults, but the (wrapped) function uses the original (wrappee) defaults.

Why might that be the desirable default?

CODE DEMO

from functools import wraps
from inspect import signature

def g(a: float, b=10):
    return a * b

def f(a: int, b=1):
    return a * b

assert f(3) == 3

f = wraps(g)(f)

assert str(signature(f)) == '(a: float, b=10)'  # signature says that b defaults to 10
assert f.__defaults__ == (1,)  # ... but it's a lie!
assert f(3) == 3 != g(3) == 30  # ... and one that can lead to problems!

Why is this so? Because functools.wraps updates the __signature__ (including annotations and defaults), __annotations__, but not __defaults__, which python apparently looks to in order to assign defaults.

One solution is to politely ask wraps to include these defaults.

from functools import wraps, WRAPPER_ASSIGNMENTS, partial

my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))

def g(a: float, b=10):
    return a * b

def f(a: int, b=1):
    return a * b

assert f(3) == 3

f = my_wraps(g)(f)

assert f(3) == 30 == g(3)
assert f.__defaults__ == (10,)  # ... because now got g defaults!

Wouldn't it be better to get this out of the box?

When would I want the defaults that are actually used be different than those mentioned in the signature?

https://bugs.python.org/issue41232

**WHAT**
Adding '__defaults__',  '__kwdefaults__' to `WRAPPER_ASSIGNMENTS` so that actual defaults can be consistent with signature defaults.

**REASON**

When using `functools.wraps`, the signature claims one set of defaults, but the (wrapped) function uses the original (wrappee) defaults.

Why might that be the desirable default?

**CODE DEMO**

```python
from functools import wraps
from inspect import signature

def g(a: float, b=10):
    return a * b

def f(a: int, b=1):
    return a * b

assert f(3) == 3

f = wraps(g)(f)

assert str(signature(f)) == '(a: float, b=10)'  # signature says that b defaults to 10
assert f.__defaults__ == (1,)  # ... but it's a lie!
assert f(3) == 3 != g(3) == 30  # ... and one that can lead to problems!
```

Why is this so? Because `functools.wraps` updates the `__signature__` (including annotations and defaults), `__annotations__`, but not `__defaults__`, which python apparently looks to in order to assign defaults.

One solution is to politely ask wraps to include these defaults.

```python
from functools import wraps, WRAPPER_ASSIGNMENTS, partial

my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))

def g(a: float, b=10):
    return a * b

def f(a: int, b=1):
    return a * b

assert f(3) == 3

f = my_wraps(g)(f)

assert f(3) == 30 == g(3)
assert f.__defaults__ == (10,)  # ... because now got g defaults!
```

Wouldn't it be better to get this out of the box?

When would I want the defaults that are actually used be different than those mentioned in the signature?
@thorwhalen thorwhalen requested a review from rhettinger as a code owner July 7, 2020 21:57
@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@thorwhalen

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@thorwhalen
Copy link
Author

Further, note that even with the additional 'defaults', and 'kwdefaults', functools.wraps breaks when keyword only arguments involved:

from functools import wraps, WRAPPER_ASSIGNMENTS, partial

# First, I need to add `__defaults__` and `__kwdefaults__` to wraps, because they don't come for free...
my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))

def g(a: float, b=10):
    return a * b

def f(a: int,  *, b=1):
    return a * b

# all is well (for now)...
assert f(3) == 3
assert g(3) == 30

This:

my_wraps(g)(f)(3)

raises TypeError (missing required positional argument 'b'), expected

Note that wraps(g)(f)(3) doesn't throw a TypeError, but the output is not consistent with the signature (inspect.signature(wraps(g)(f)) is (a: float, b=10), so 3 should be multiplied by 10). This is because defaults wasn't updated.

See for example, that third-party from boltons.funcutils import wraps works as expected. And so do (the very popular) wrapt and decorator packages.

Note that the boltons version works for wraps(f)(g), but not wraps(g)(f) in my example.

See: https://stackoverflow.com/questions/62782709/pythons-functools-wraps-breaks-when-keyword-only-arguments-involved

@thorwhalen thorwhalen changed the title Updated WRAPPER_ASSIGNMENTS bpo-41232: Updated WRAPPER_ASSIGNMENTS Jul 7, 2020
@thorwhalen
Copy link
Author

thorwhalen commented Jul 8, 2020

Tests are failing, so I had a look at them, and my conclusion is that the tests are the mistake here.
Please see if you agree.

I'm usint python 3.82 on a Mac.

The tests fail here:

# In test/test_inspect.py:849
def test_argspec_api_ignores_wrapped(self):
    # Issue 20684: low level introspection API must ignore __wrapped__
    @functools.wraps(mod.spam)
    def ham(x, y):
        pass
    # Basic check
    self.assertArgSpecEquals(ham, ['x', 'y'], formatted='(x, y)')  # <--- this what fails

Note that mod.spam is this:

def spam(a, /, b, c, d=3, e=4, f=5, *g, **h):
    eggs(b + d, c + f)

The assertArgSpecEquals fails on the comparison of expected and actual defaults (line 776):

self.assertEqual(defaults, defaults_e)

where defaults_e == None and defaults == (3,), but neither seem to be correct. In my opinion,
the correct value should be ham.__defaults__ == spam.__defaults__ == (3, 4, 5).

In the current version of functools.wraps, __defaults__ and __kwdefaults__ are not even copied, which leads to defaults_e == defaults == ham.__defaults__ == None, so the test passes. In my opinion, it shouldn't.

Here are two code blocks that may help you to investigate:

# With the current functools.py
# WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
#                        '__annotations__', '__defaults__', '__kwdefaults__')

import inspect
import functools
from test import inspect_fodder as mod

@functools.wraps(mod.spam)
def ham(x, y):
    pass

assert ham.__defaults__ == None == inspect.getargspec(ham).defaults 
# With proposed change, adding '__defaults__', '__kwdefaults__' to WRAPPER_ASSIGNMENTS:
# WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
#                        '__annotations__', '__defaults__', '__kwdefaults__')


import inspect
import functools
from test import inspect_fodder as mod

@functools.wraps(mod.spam)
def ham(x, y):
    pass

assert ham.__defaults__ == (3, 4, 5)
assert inspect.getargspec(ham).defaults == (3,)

@csabella
Copy link
Contributor

@thorwhalen, please sign the CLA. Thank you!

@CoolCat467
Copy link

@thorwhalen Reminding you what csabella said

@thorwhalen
Copy link
Author

Where do I find this?

@Mariatta
Copy link
Member

Please read the message posted by @the-knights-who-say-ni above. Not only you need to sign the CLA, you'll also need to create an account on the bug tracker, and associate your GitHub username there.

@github-actions
Copy link

This PR is stale because it has been open for 30 days with no activity. If the CLA is not signed within 14 days, it will be closed. See also https://devguide.python.org/pullrequest/#licensing

@github-actions github-actions bot added the stale Stale PR or inactive for long period of time. label Feb 19, 2022
@github-actions
Copy link

github-actions bot commented Mar 6, 2022

Closing this stale PR because the CLA is still not signed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants