Skip to content

Replace pseudo-extensible Foreign with a big union#6125

Merged
dolio merged 4 commits intotrunkfrom
topic/foreign-rep
Jan 20, 2026
Merged

Replace pseudo-extensible Foreign with a big union#6125
dolio merged 4 commits intotrunkfrom
topic/foreign-rep

Conversation

@dolio
Copy link
Contributor

@dolio dolio commented Jan 16, 2026

This replaces the existential-based Foreign type with an exhaustive enumeration of the wrapped types. The original intention was that the wrapped types would be extensible at runtime, but I'm not sure that's really possible, and in any case we haven't done it.

Doing this requires moving the type into the Stack module, because some cases recursively refer to Val. So the U.R.Foreign module is gone. The namespace still exists for other related stuff.

The intention of this change was to improve equality/ordering of wrapped values. Before, we had to do repeated testing of Reference values to determine what the wrapped types even were. This is expensive, and the expense is worse if you're low on the list. With the union, figuring out the type is a case analysis, and is much faster.

However, this also has other positive effects. Many uses of unsafeCoerce have been eliminated, and I believe I even found an unnoticed type error in one of the AVRO cases. Some compiler panics should get slightly better information, too, or a more useful panic will happen in place of a segfault. I was also able to easily improve the universal equality by making more wrapped values use (==) instead of pointer equality, since every case is now listed.

Base tests pass.

Benchmarks look mostly the same, except:

Mutate a local Remote.Ref 10k times
76.475683ms -> 41.219356ms

I assume that's looking up Bytes keys or something.

I was also running a benchmark locally that did comparisons in a loop, testing 16-byte Bytes values vs. two Nat values. Before these changes, the Bytes version was ~50% slower, and now they're approximately the same. There is an actual difference (Bytes being slower) if you look hard enough, but they're close enough to get lost in the noise unless you're doing almost exclusively comparisons.

This replaces the existential-based `Foreign` type with an exhaustive
enumeration of the wrapped types. The original intention was that the
wrapped types would be extensible at runtime, but I'm not sure that's
really possible, and in any case we haven't done it.

Doing this requires moving the type into the `Stack` module, because
some cases recursively refer to `Val`. So the `U.R.Foreign` module is
gone. The namespace still exists for other related stuff.

The intention of this change was to improve equality/ordering of
wrapped values. Before, we had to do repeated testing of `Reference`
values to determine what the wrapped types even were. This is
expensive, and the expense is worse if you're low on the list. With
the union, figuring out the type is a case analysis, and is much
faster.

However, this also has other positive effects. Many uses of
`unsafeCoerce` have been eliminated, and I believe I even found an
unnoticed type error in one of the AVRO cases. Some compiler panics
should get slightly better information, too, or a more useful panic
will happen in place of a segfault.
Copy link
Member

@pchiusano pchiusano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!

@pchiusano
Copy link
Member

@dolio CI failed with some weeder issue, otherwise looks good, you can merge when ready.

@dolio
Copy link
Contributor Author

dolio commented Jan 16, 2026

Another @ceedubs sanity check would be appreciated. One thing I encountered was that the pointer equality stopped working for one of the tls things in the transcripts (thought not everything). But I was actually able to replace it with normal equality. I tried to switch to doing as little pointer equality as possible. It'd be good to know if everything seems covered for cloud.

Copy link
Contributor

@ceedubs ceedubs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloud tests pass 👍

@dolio dolio merged commit bb58b2f into trunk Jan 20, 2026
32 checks passed
@dolio dolio deleted the topic/foreign-rep branch January 20, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants