Support caching target attributes in Proxy to avoid resolving when using hash() or isinstance()#551
Conversation
The cache_defaults flag and target parameter, when combined, will compute and cache the __class__ and __hash__ values of the proxy such that the proxy can be hashed or used with isinstance() without needing to resolve the proxy. This is particularly important for certain serializers or workflow systems (like Dask) that want to do some custom management of values which could be proxies.
There's quite a few special attributes between the base Proxy and the reference Proxy implementations. To keep it clear what is a special Proxy attribute and an attribute of the target, I've changed all the special Proxy attributes to be prefixed with __proxy_*. This is similar to how pydantic v2 prefixes all the pydantic specific attributes and methods with "model_". This is not a breaking change because these are all internal implementation details of the Proxy. There were a few places where __factory__ was accessed so I've added a get_factory() function to make that the preferred way of getting the factory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the
cache_defaultsandtargetparameters toProxy. When used together, the proxy caches default values of__class__and__hash__so thathash(proxy)andisinstance(proxy, ...)can be used without needing the resolve the proxy. Thepopulate_targetflag ofStoremethods now defaults to also settingcache_defaults=True.Because this change added more special attributes to the
Proxyclass, I renamed all special attributes (__wrapped__,__factory__,__resolved__, etc.) to be prefixed by__proxy_*to better distinguish what is a property ofProxyversus the target. This is not considered a breaking change because those properties are an internal implementation detail of theProxyclass.Fixes
hashandisinstancethat does not resolve aProxy#549Type of Change
Testing
Added new unit tests for the features.
I also validated against this Dask example:
Here, when
populate_target=False, the proxy will be resolved three times: on the client when the proxy is serialized, on the scheduler when Dask does its input management, and on the worker when the proxy is actually used. Withpopulate_target=True, the proxy is only resolved once on the worker when actually used.Pull Request Checklist
Please confirm the PR meets the following requirements.
pre-commit(e.g., mypy, ruff, etc.).