The NumPy 1.24.0 release continues the ongoing work to improve the handling andpromotion of dtypes, increase the execution speed, and clarify thedocumentation. There are also a large number of new and expired deprecationsdue to changes in promotion and cleanups. This might be called a deprecationrelease. Highlights are
Many new deprecations, check them out.
Many expired deprecations,
New F2PY features and fixes.
New “dtype” and “casting” keywords for stacking functions.
See below for the details,
This release supports Python versions 3.8-3.11.
Deprecations#
Deprecate fastCopyAndTranspose and PyArray_CopyAndTranspose#
The numpy.fastCopyAndTranspose
function has been deprecated. Use thecorresponding copy and transpose methods directly:
arr.T.copy()
The underlying C function PyArray_CopyAndTranspose
has also been deprecatedfrom the NumPy C-API.
(gh-22313)
Conversion of out-of-bound Python integers#
Attempting a conversion from a Python integer to a NumPy value will now alwayscheck whether the result can be represented by NumPy. This means the followingexamples will fail in the future and give a DeprecationWarning
now:
np.uint8(-1)np.array([3000], dtype=np.int8)
Many of these did succeed before. Such code was mainly useful for unsignedintegers with negative values such as np.uint8(-1)
givingnp.iinfo(np.uint8).max
.
Note that conversion between NumPy integers is unaffected, so thatnp.array(-1).astype(np.uint8)
continues to work and use C integer overflowlogic. For negative values, it will also work to view the array:np.array(-1, dtype=np.int8).view(np.uint8)
.In some cases, using np.iinfo(np.uint8).max
or val % 2**8
may alsowork well.
In rare cases input data may mix both negative values and very large unsignedvalues (i.e. -1
and 2**63
). There it is unfortunately necessaryto use %
on the Python value or use signed or unsigned conversiondepending on whether negative values are expected.
(gh-22385)
Deprecate msort
#
The numpy.msort
function is deprecated. Use np.sort(a, axis=0)
instead.
(gh-22456)
np.str0
and similar are now deprecated#
The scalar type aliases ending in a 0 bit size: np.object0
, np.str0
,np.bytes0
, np.void0
, np.int0
, np.uint0
as well as np.bool8
are now deprecated and will eventually be removed.
(gh-22607)
Expired deprecations#
The
normed
keyword argument has been removed fromnp.histogram, np.histogram2d, and np.histogramdd.Usedensity
instead. Ifnormed
was passed byposition,density
is now used.(gh-21645)
Ragged array creation will now always raise a
ValueError
unlessdtype=object
is passed. This includes very deeply nested sequences.(gh-22004)
Support for Visual Studio 2015 and earlier has been removed.
Support for the Windows Interix POSIX interop layer has been removed.
(gh-22139)
Support for Cygwin < 3.3 has been removed.
(gh-22159)
The mini() method of
np.ma.MaskedArray
has been removed. Use eithernp.ma.MaskedArray.min()
ornp.ma.minimum.reduce()
.The single-argument form of
np.ma.minimum
andnp.ma.maximum
has beenremoved. Usenp.ma.minimum.reduce()
ornp.ma.maximum.reduce()
instead.(gh-22228)
Passing dtype instances other than the canonical (mainly native byte-order)ones to
dtype=
orsignature=
in ufuncs will now raise aTypeError
. We recommend passing the strings"int8"
or scalar typesnp.int8
since the byte-order, datetime/timedelta unit, etc. are neverenforced. (Initially deprecated in NumPy 1.21.)(gh-22540)
The
dtype=
argument to comparison ufuncs is now applied correctly. Thatmeans that onlybool
andobject
are valid values anddtype=object
is enforced.See Also3. An Informal Introduction to PythonTop 100 Model-Based Development Using MATLAB Simulink Interview Questions - CS Electrical & ElectronicsNumPy 1.12.0 Release Notes — NumPy v2.0 ManualNumPy 1.19.0 Release Notes — NumPy v2.0 Manual(gh-22541)
The deprecation for the aliases
np.object
,np.bool
,np.float
,np.complex
,np.str
, andnp.int
is expired (introduces NumPy1.20). Some of these will now give a FutureWarning in addition to raising anerror since they will be mapped to the NumPy scalars in the future.(gh-22607)
Compatibility notes#
array.fill(scalar)
may behave slightly different#
numpy.ndarray.fill
may in some cases behave slightly different now due tothe fact that the logic is aligned with item assignment:
arr = np.array([1]) # with any dtype/valuearr.fill(scalar)# is now identical to:arr[0] = scalar
Previously casting may have produced slightly different answers when usingvalues that could not be represented in the target dtype
or when the targethad object
dtype.
(gh-20924)
Subarray to object cast now copies#
Casting a dtype that includes a subarray to an object will now ensure a copy ofthe subarray. Previously an unsafe view was returned:
arr = np.ones(3, dtype=[("f", "i", 3)])subarray_fields = arr.astype(object)[0]subarray = subarray_fields[0] # "f" fieldnp.may_share_memory(subarray, arr)
Is now always false. While previously it was true for the specific cast.
(gh-21925)
Returned arrays respect uniqueness of dtype kwarg objects#
When the dtype
keyword argument is used with array orasarray, the dtype of the returned array now always exactlymatches the dtype provided by the caller.
In some cases this change means that a view rather than the input array isreturned. The following is an example for this on 64bit Linux where long
and longlong
are the same precision but different dtypes
:
>>> arr = np.array([1, 2, 3], dtype="long")>>> new_dtype = np.dtype("longlong")>>> new = np.asarray(arr, dtype=new_dtype)>>> new.dtype is new_dtypeTrue>>> new is arrFalse
Before the change, the dtype
did not match because new is arr
wasTrue
.
(gh-21995)
DLPack export raises BufferError
#
When an array buffer cannot be exported via DLPack a BufferError
is nowalways raised where previously TypeError
or RuntimeError
was raised.This allows falling back to the buffer protocol or __array_interface__
whenDLPack was tried first.
(gh-22542)
NumPy builds are no longer tested on GCC-6#
Ubuntu 18.04 is deprecated for GitHub actions and GCC-6 is not available onUbuntu 20.04, so builds using that compiler are no longer tested. We still testbuilds using GCC-7 and GCC-8.
(gh-22598)
New Features#
New attribute symbol
added to polynomial classes#
The polynomial classes in the numpy.polynomial
package have a newsymbol
attribute which is used to represent the indeterminate of thepolynomial. This can be used to change the value of the variable whenprinting:
>>> P_y = np.polynomial.Polynomial([1, 0, -1], symbol="y")>>> print(P_y)1.0 + 0.0·y¹ - 1.0·y²
Note that the polynomial classes only support 1D polynomials, so operationsthat involve polynomials with different symbols are disallowed when the resultwould be multivariate:
>>> P = np.polynomial.Polynomial([1, -1]) # default symbol is "x">>> P_z = np.polynomial.Polynomial([1, 1], symbol="z")>>> P * P_zTraceback (most recent call last) ...ValueError: Polynomial symbols differ
The symbol can be any valid Python identifier. The default is symbol=x
,consistent with existing behavior.
(gh-16154)
F2PY support for Fortran character
strings#
F2PY now supports wrapping Fortran functions with:
character (e.g.
character x
)character array (e.g.
character, dimension(n) :: x
)character string (e.g.
character(len=10) x
)and character string array (e.g.
character(len=10), dimension(n, m) :: x
)
arguments, including passing Python unicode strings as Fortran character stringarguments.
(gh-19388)
New function np.show_runtime
#
A new function numpy.show_runtime
has been added to display the runtimeinformation of the machine in addition to numpy.show_config
which displaysthe build-related information.
(gh-21468)
strict
option for testing.assert_array_equal
#
The strict
option is now available for testing.assert_array_equal
.Setting strict=True
will disable the broadcasting behaviour for scalars andensure that input arrays have the same data type.
(gh-21595)
New parameter equal_nan
added to np.unique
#
np.unique
was changed in 1.21 to treat all NaN
values as equal andreturn a single NaN
. Setting equal_nan=False
will restore pre-1.21behavior to treat NaNs
as unique. Defaults to True
.
(gh-21623)
casting
and dtype
keyword arguments for numpy.stack
#
The casting
and dtype
keyword arguments are now available fornumpy.stack
. To use them, write np.stack(..., dtype=None,casting='same_kind')
.
casting
and dtype
keyword arguments for numpy.vstack
#
The casting
and dtype
keyword arguments are now available fornumpy.vstack
. To use them, write np.vstack(..., dtype=None,casting='same_kind')
.
casting
and dtype
keyword arguments for numpy.hstack
#
The casting
and dtype
keyword arguments are now available fornumpy.hstack
. To use them, write np.hstack(..., dtype=None,casting='same_kind')
.
(gh-21627)
The bit generator underlying the singleton RandomState can be changed#
The singleton RandomState
instance exposed in the numpy.random
moduleis initialized at startup with the MT19937
bit generator. The new functionset_bit_generator
allows the default bit generator to be replaced with auser-provided bit generator. This function has been introduced to provide amethod allowing seamless integration of a high-quality, modern bit generator innew code with existing code that makes use of the singleton-provided randomvariate generating functions. The companion function get_bit_generator
returns the current bit generator being used by the singleton RandomState
.This is provided to simplify restoring the original source of randomness ifrequired.
The preferred method to generate reproducible random numbers is to use a modernbit generator in an instance of Generator
. The function default_rng
simplifies instantiation:
>>> rg = np.random.default_rng(3728973198)>>> rg.random()
The same bit generator can then be shared with the singleton instance so thatcalling functions in the random
module will use the same bit generator:
>>> orig_bit_gen = np.random.get_bit_generator()>>> np.random.set_bit_generator(rg.bit_generator)>>> np.random.normal()
The swap is permanent (until reversed) and so any call to functions in therandom
module will use the new bit generator. The original can be restoredif required for code to run correctly:
>>> np.random.set_bit_generator(orig_bit_gen)
(gh-21976)
np.void
now has a dtype
argument#
NumPy now allows constructing structured void scalars directly bypassing the dtype
argument to np.void
.
(gh-22316)
Improvements#
F2PY Improvements#
The generated extension modules don’t use the deprecated NumPy-C API anymore
Improved
f2py
generated exception messagesNumerous bug and
flake8
warning fixesvarious CPP macros that one can use within C-expressions of signature filesare prefixed with
f2py_
. For example, one should usef2py_len(x)
instead oflen(x)
A new construct
character(f2py_len=...)
is introduced to supportreturning assumed length character strings (e.g.character(len=*)
) fromwrapper functions
A hook to support rewriting f2py
internal data structures after reading allits input files is introduced. This is required, for instance, for BC of SciPysupport where character arguments are treated as character strings arguments inC
expressions.
(gh-19388)
IBM zSystems Vector Extension Facility (SIMD)#
Added support for SIMD extensions of zSystem (z13, z14, z15), through theuniversal intrinsics interface. This support leads to performance improvementsfor all SIMD kernels implemented using the universal intrinsics, including thefollowing operations: rint, floor, trunc, ceil, sqrt, absolute, square,reciprocal, tanh, sin, cos, equal, not_equal, greater, greater_equal, less,less_equal, maximum, minimum, fmax, fmin, argmax, argmin, add, subtract,multiply, divide.
(gh-20913)
NumPy now gives floating point errors in casts#
In most cases, NumPy previously did not give floating point warnings or errorswhen these happened during casts. For examples, casts like:
np.array([2e300]).astype(np.float32) # overflow for float32np.array([np.inf]).astype(np.int64)
Should now generally give floating point warnings. These warnings should warnthat floating point overflow occurred. For errors when converting floatingpoint values to integers users should expect invalid value warnings.
Users can modify the behavior of these warnings using np.errstate
.
Note that for float to int casts, the exact warnings that are given maybe platform dependent. For example:
arr = np.full(100, fill_value=1000, dtype=np.float64)arr.astype(np.int8)
May give a result equivalent to (the intermediate cast means no warning isgiven):
arr.astype(np.int64).astype(np.int8)
May return an undefined result, with a warning set:
RuntimeWarning: invalid value encountered in cast
The precise behavior is subject to the C99 standard and its implementation inboth software and hardware.
(gh-21437)
F2PY supports the value attribute#
The Fortran standard requires that variables declared with the value
attribute must be passed by value instead of reference. F2PY now supports thisuse pattern correctly. So integer, intent(in), value :: x
in Fortran codeswill have correct wrappers generated.
(gh-21807)
Added pickle support for third-party BitGenerators#
The pickle format for bit generators was extended to allow each bit generatorto supply its own constructor when during pickling. Previous versions of NumPyonly supported unpickling Generator
instances created with one of the coreset of bit generators supplied with NumPy. Attempting to unpickle aGenerator
that used a third-party bit generators would fail since theconstructor used during the unpickling was only aware of the bit generatorsincluded in NumPy.
(gh-22014)
arange() now explicitly fails with dtype=str#
Previously, the np.arange(n, dtype=str)
function worked for n=1
andn=2
, but would raise a non-specific exception message for other values ofn
. Now, it raises a TypeError informing that arange
does not supportstring dtypes:
>>> np.arange(2, dtype=str)Traceback (most recent call last) ...TypeError: arange() not supported for inputs with DType <class 'numpy.dtype[str_]'>.
(gh-22055)
numpy.typing
protocols are now runtime checkable#
The protocols used in numpy.typing.ArrayLike
and numpy.typing.DTypeLike
are now properly marked as runtime checkable, making them easier to use forruntime type checkers.
(gh-22357)
Performance improvements and changes#
Faster version of np.isin
and np.in1d
for integer arrays#
np.in1d
(used by np.isin
) can now switch to a faster algorithm (up to>10x faster) when it is passed two integer arrays. This is often automaticallyused, but you can use kind="sort"
or kind="table"
to force the old ornew method, respectively.
(gh-12065)
Faster comparison operators#
The comparison functions (numpy.equal
, numpy.not_equal
, numpy.less
,numpy.less_equal
, numpy.greater
and numpy.greater_equal
) are nowmuch faster as they are now vectorized with universal intrinsics. For a CPUwith SIMD extension AVX512BW, the performance gain is up to 2.57x, 1.65x and19.15x for integer, float and boolean data types, respectively (with N=50000).
(gh-21483)
Changes#
Better reporting of integer division overflow#
Integer division overflow of scalars and arrays used to provide aRuntimeWarning
and the return value was undefined leading to crashes atrare occasions:
>>> np.array([np.iinfo(np.int32).min]*10, dtype=np.int32) // np.int32(-1)<stdin>:1: RuntimeWarning: divide by zero encountered in floor_dividearray([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)
Integer division overflow now returns the input dtype’s minimum value and raisethe following RuntimeWarning
:
>>> np.array([np.iinfo(np.int32).min]*10, dtype=np.int32) // np.int32(-1)<stdin>:1: RuntimeWarning: overflow encountered in floor_dividearray([-2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648], dtype=int32)
(gh-21506)
masked_invalid
now modifies the mask in-place#
When used with copy=False
, numpy.ma.masked_invalid
now modifies theinput masked array in-place. This makes it behave identically tomasked_where
and better matches the documentation.
(gh-22046)
nditer
/NpyIter
allows all allocating all operands#
The NumPy iterator available through np.nditer
in Python and as NpyIter
in C now supports allocating all arrays. The iterator shape defaults to ()
in this case. The operands dtype must be provided, since a “common dtype”cannot be inferred from the other inputs.
(gh-22457)