API: Add ``matrix_norm``, ``vector_norm``, ``vecdot`` and ``matrix_transpose`` [Array API] by mtsokol · Pull Request #25155 · numpy/numpy

mtsokol · 2023-11-15T16:16:58Z

This PR contains the last Array API compatibility update for numpy.linalg. It adds:

numpy.linalg.matrix_norm
numpy.linalg.vector_norm
numpy.vecdot and numpy.linalg.vecdot
numpy.matrix_transpose and numpy.linalg.matrix_transpose

ngoldbaum · 2023-12-05T15:50:26Z

The mypy failure is real, I restarted the cygwin job.

ngoldbaum

Overall looks good, tests just need to be cleaned up. I see that the implementations are copy/pasted from the existing array API wrappers. This is probably fine as a first pass but I suspect there are more performant implementations we could use.

numpy/_core/numeric.py

ngoldbaum · 2023-12-05T15:56:13Z

doc/release/upcoming_changes/25155.new_feature.rst

+
+* `numpy.vecdot` and `numpy.linalg.vecdot`
+
+* `numpy.matrix_transpose` and `numpy.linalg.matrix_transpose`


Can you add a sentence or two describing what these functions do?

Sure! I added one sentence descriptions from Array API definitions.

ngoldbaum · 2023-12-08T18:06:37Z

numpy/_core/fromnumeric.py

+    transpose : Generic transpose method.
+
+    """
+    x = asarray(x)


Similar to the discussion in the other PR - should this be asanyarray?

I think there was roughly an agreement in #25101 (comment) that asanyarray can be used in place of asarray.

I replaced it for functions added here and outer that has been merged.

ngoldbaum · 2023-12-13T22:09:41Z

numpy/_core/numeric.py

+    x1_ = np.moveaxis(x1_, axis, -1)
+    x2_ = np.moveaxis(x2_, axis, -1)
+
+    res = x1_[..., None, :] @ x2_[..., None]


@seberg mentioned on the triage call today that this is supposed to do the hermitian conjugate according to the array API standard - does the matmul operator do that?

And if so, probably worth adding a test that has complex vectors.

Right, a conjugate was missing - I added it (and a test) according to Array API.

Note: np.vdot does conjugate on the first parameter (if it's complex), where np.vecdot does it on the second one (I mentioned it in the vecdot's docs).

no, the numpy way is correct, the array-api way is just a bug.

Ah, right (I see that pytorch also performs conjugate on the first argument). Let me change that here and fix it in the Array API.

Can you open an issue that this should have a dedicated ufunc? BLAS should have kernels which perform the complex conjugation on the fly, so at least for the complex conjugating case, this is really not ideal.

Note that wikipedia disagrees: the conjugate of the second argument is taken: https://en.wikipedia.org/wiki/Dot_product#Complex_vectors

Mathematicians and physicists have opposite conventions. Mathematicians use sesquilinear forms, physicists follow the Dirac bracket convention (I don't know who actually started the convention). It can make teaching awkward ...

But then BLAS agrees with what you have, so I guess wikipedia is confused too... https://netlib.org/lapack/explore-html/d1/dcc/group__dot_ga5c189335a4e6130a2206c190579b1571.html#ga5c189335a4e6130a2206c190579b1571

ngoldbaum · 2023-12-14T15:40:55Z

Alright, let's pull this one in. Thanks @mtsokol!

One note: in the release notes the code literals should be wrapped in double backticks, not single backticks. Can you go through all the release notes you've added and make sure they're all formatted correctly? Merging this anyway since that's a minor issue that can be fixed separately.

mtsokol · 2023-12-14T17:23:19Z

One note: in the release notes the code literals should be wrapped in double backticks, not single backticks. Can you go through all the release notes you've added and make sure they're all formatted correctly? Merging this anyway since that's a minor issue that can be fixed separately.

@ngoldbaum I did this formatting on purpose. Double backticks give code format, where single backticks create a link to NumPy member's documentation.

For example see: https://github.com/numpy/numpy/blob/4b5ae68c78a222eba2e07cc1e34bd29d5cfeb30d/doc/release/upcoming_changes/22539.deprecation.rst?plain=1 from a PR shipped with 1.25.0 release.

which resulted in:

ngoldbaum · 2023-12-14T17:27:45Z

Ah, thanks for the explanation! Gotta love sphinx/rst trivia like that.

mhvk · 2023-12-17T17:14:31Z

numpy/_core/numeric.py

+            f"but they are: {x1_shape[axis]} and {x2_shape[axis]}."
+        )
+
+    x1_, x2_ = np.broadcast_arrays(x1, x2)


Darn, another one. Would it still be possible to add subok=True here to allow subclasses to "just work"?

mhvk · 2023-12-17T21:51:46Z

numpy/_core/numeric.py

+    ndim = builtins.max(x1.ndim, x2.ndim)
+    x1_shape = (1,)*(ndim - x1.ndim) + tuple(x1.shape)
+    x2_shape = (1,)*(ndim - x2.ndim) + tuple(x2.shape)
+    if x1_shape[axis] != x2_shape[axis]:


Note that this use of axis is contrary to how a gufunc would interpret it: there axis is taken for every element separately before broadcasting, i.e., for vectors with shapes (3, 4) and (3, 2, 4) and axis=0, the output would have shape (2, 4). You can see this from

from numpy._core.umath_tests import inner1d a = np.arange(12).reshape(3, 4) b = np.arange(24).reshape(3, 2, 4) inner1d(a, b, axis=0).shape # (2, 4)

I think we should stick with this, otherwise this is going to be very confusing in the future (alternatively, one would need to change how axis and axes are interpreted in gufuncs...)

mtsokol self-assigned this Nov 15, 2023

github-actions bot added the 30 - API label Nov 15, 2023

mtsokol added the Numpy 2.0 API Changes label Nov 15, 2023

mtsokol mentioned this pull request Nov 15, 2023

Tracking issue: Add array API standard support to the main namespace #25076

Closed

23 tasks

mtsokol force-pushed the array-api-linalg branch 2 times, most recently from 4f392bf to 0f6b8b1 Compare November 15, 2023 17:08

mtsokol force-pushed the array-api-linalg branch from 0f6b8b1 to 32526b4 Compare November 29, 2023 12:06

mtsokol added this to the 2.0.0 release milestone Nov 29, 2023

ngoldbaum reviewed Dec 5, 2023

View reviewed changes

mtsokol force-pushed the array-api-linalg branch 5 times, most recently from 8102ddb to 665d6c6 Compare December 8, 2023 14:18

charris changed the title ~~API: Add matrix_norm, vector_norm, vecdot and matrix_transpose [Array API]~~ API: Add matrix_norm, vector_norm, vecdot and matrix_transpose [Array API] Dec 8, 2023

ngoldbaum reviewed Dec 8, 2023

View reviewed changes

mtsokol force-pushed the array-api-linalg branch 2 times, most recently from b3011a7 to daa0350 Compare December 11, 2023 18:13

ngoldbaum reviewed Dec 13, 2023

View reviewed changes

mtsokol force-pushed the array-api-linalg branch from daa0350 to f7b5640 Compare December 14, 2023 11:49

API: add matrix_norm, vector_norm, vecdot and matrix_transpose

dd913aa

mtsokol force-pushed the array-api-linalg branch from f7b5640 to dd913aa Compare December 14, 2023 12:53

ngoldbaum merged commit 8162caf into numpy:main Dec 14, 2023

mtsokol deleted the array-api-linalg branch December 14, 2023 16:05

mtsokol mentioned this pull request Dec 14, 2023

ENH: Add a dedicated ufunc for numpy.vecdot #25393

Closed

mhvk reviewed Dec 17, 2023

View reviewed changes

mhvk mentioned this pull request Dec 17, 2023

BUG: Vecdot subclasses and axis #25411

Closed

This was referenced Dec 18, 2023

Support new additions to numpy (np.vecdot, np.matrix_transpose, etc.) astropy/astropy#15760

Merged

(NEP 18) Implement and test array functions new in numpy 2.0 yt-project/unyt#482

Closed

mhvk mentioned this pull request Dec 18, 2023

ENH: define a gufunc for vecdot (with BLAS support) #25416

Merged

This was referenced Jan 4, 2024

ENH: Add np.linalg.matrix_transpose #22767

Closed

ENH: Add matrix_transpose function #24099

Open


		* `numpy.vecdot` and `numpy.linalg.vecdot`

		* `numpy.matrix_transpose` and `numpy.linalg.matrix_transpose`

Uh oh!

Comments

Conversation

mtsokol commented Nov 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngoldbaum commented Dec 5, 2023

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum Dec 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsokol Dec 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ngoldbaum commented Dec 14, 2023

Uh oh!

mtsokol commented Dec 14, 2023

Uh oh!

ngoldbaum commented Dec 14, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mtsokol commented Nov 15, 2023 •

edited

Loading

ngoldbaum Dec 8, 2023 •

edited

Loading

mtsokol Dec 14, 2023 •

edited

Loading