.. currentmodule:: xarray
What's New
==========
.. ipython:: python
:suppress:
import numpy as np
import pandas as pd
import xarray as xray
import xarray
import xarray as xr
np.random.seed(123456)
.. _whats-new.2022.03.0:
v2022.03.0 (2 March 2022)
-------------------------
This release brings a number of small improvements, as well as a move to `calendar versioning `_ (:issue:`6176`).
Many thanks to the 16 contributors to the v2022.02.0 release!
Aaron Spring, Alan D. Snow, Anderson Banihirwe, crusaderky, Illviljan, Joe Hamman, Jonas Gliß,
Lukas Pilz, Martin Bergemann, Mathias Hauser, Maximilian Roos, Romain Caneill, Stan West, Stijn Van Hoey,
Tobias Kölling, and Tom Nicholas.
New Features
~~~~~~~~~~~~
- Enabled multiplying tick offsets by floats. Allows ``float`` ``n`` in
:py:meth:`CFTimeIndex.shift` if ``shift_freq`` is between ``Day``
and ``Microsecond``. (:issue:`6134`, :pull:`6135`).
By `Aaron Spring `_.
- Enbable to provide more keyword arguments to `pydap` backend when reading
OpenDAP datasets (:issue:`6274`).
By `Jonas Gliß `.
- Allow :py:meth:`DataArray.drop_duplicates` to drop duplicates along multiple dimensions at once,
and add :py:meth:`Dataset.drop_duplicates`. (:pull:`6307`)
By `Tom Nicholas `_.
Breaking changes
~~~~~~~~~~~~~~~~
- Renamed the ``interpolation`` keyword of all ``quantile`` methods (e.g. :py:meth:`DataArray.quantile`)
to ``method`` for consistency with numpy v1.22.0 (:pull:`6108`).
By `Mathias Hauser `_.
Deprecations
~~~~~~~~~~~~
Bug fixes
~~~~~~~~~
- Variables which are chunked using dask in larger (but aligned) chunks than the target zarr chunk size
can now be stored using `to_zarr()` (:pull:`6258`) By `Tobias Kölling `_.
- Multi-file datasets containing encoded :py:class:`cftime.datetime` objects can be read in parallel again (:issue:`6226`, :pull:`6249`, :pull:`6305`). By `Martin Bergemann `_ and `Stan West `_.
Documentation
~~~~~~~~~~~~~
- Delete files of datasets saved to disk while building the documentation and enable
building on Windows via `sphinx-build` (:pull:`6237`).
By `Stan West `_.
Internal Changes
~~~~~~~~~~~~~~~~
.. _whats-new.0.21.1:
v0.21.1 (31 January 2022)
-------------------------
This is a bugfix release to resolve (:issue:`6216`, :pull:`6207`).
Bug fixes
~~~~~~~~~
- Add `packaging` as a dependency to Xarray (:issue:`6216`, :pull:`6207`).
By `Sebastian Weigand `_ and `Joe Hamman `_.
.. _whats-new.0.21.0:
v0.21.0 (27 January 2022)
-------------------------
Many thanks to the 20 contributors to the v0.21.0 release!
Abel Aoun, Anderson Banihirwe, Ant Gib, Chris Roat, Cindy Chiao,
Deepak Cherian, Dominik Stańczak, Fabian Hofmann, Illviljan, Jody Klymak, Joseph
K Aicher, Mark Harfouche, Mathias Hauser, Matthew Roeschke, Maximilian Roos,
Michael Delgado, Pascal Bourgault, Pierre, Ray Bell, Romain Caneill, Tim Heap,
Tom Nicholas, Zeb Nicholls, joseph nowak, keewis.
New Features
~~~~~~~~~~~~
- New top-level function :py:func:`cross`. (:issue:`3279`, :pull:`5365`).
By `Jimmy Westling `_.
- ``keep_attrs`` support for :py:func:`where` (:issue:`4141`, :issue:`4682`, :pull:`4687`).
By `Justus Magin `_.
- Enable the limit option for dask array in the following methods :py:meth:`DataArray.ffill`, :py:meth:`DataArray.bfill`, :py:meth:`Dataset.ffill` and :py:meth:`Dataset.bfill` (:issue:`6112`)
By `Joseph Nowak `_.
Breaking changes
~~~~~~~~~~~~~~~~
- Rely on matplotlib's default datetime converters instead of pandas' (:issue:`6102`, :pull:`6109`).
By `Jimmy Westling `_.
- Improve repr readability when there are a large number of dimensions in datasets or dataarrays by
wrapping the text once the maximum display width has been exceeded. (:issue:`5546`, :pull:`5662`)
By `Jimmy Westling `_.
Deprecations
~~~~~~~~~~~~
- Removed the lock kwarg from the zarr and pydap backends, completing the deprecation cycle started in :issue:`5256`.
By `Tom Nicholas `_.
- Support for ``python 3.7`` has been dropped. (:pull:`5892`)
By `Jimmy Westling `_.
Bug fixes
~~~~~~~~~
- Preserve chunks when creating a :py:class:`DataArray` from another :py:class:`DataArray`
(:pull:`5984`). By `Fabian Hofmann `_.
- Properly support :py:meth:`DataArray.ffill`, :py:meth:`DataArray.bfill`, :py:meth:`Dataset.ffill` and :py:meth:`Dataset.bfill` along chunked dimensions (:issue:`6112`).
By `Joseph Nowak `_.
- Subclasses of ``byte`` and ``str`` (e.g. ``np.str_`` and ``np.bytes_``) will now serialise to disk rather than raising a ``ValueError: unsupported dtype for netCDF4 variable: object`` as they did previously (:pull:`5264`).
By `Zeb Nicholls `_.
- Fix applying function with non-xarray arguments using :py:func:`xr.map_blocks`.
By `Cindy Chiao `_.
- No longer raise an error for an all-nan-but-one argument to
:py:meth:`DataArray.interpolate_na` when using `method='nearest'` (:issue:`5994`, :pull:`6144`).
By `Michael Delgado `_.
- `dt.season `_ can now handle NaN and NaT. (:pull:`5876`).
By `Pierre Loicq `_.
- Determination of zarr chunks handles empty lists for encoding chunks or variable chunks that occurs in certain circumstances (:pull:`5526`). By `Chris Roat `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Replace ``distutils.version`` with ``packaging.version`` (:issue:`6092`).
By `Mathias Hauser `_.
- Removed internal checks for ``pd.Panel`` (:issue:`6145`).
By `Matthew Roeschke `_.
- Add ``pyupgrade`` pre-commit hook (:pull:`6152`).
By `Maximilian Roos `_.
.. _whats-new.0.20.2:
v0.20.2 (9 December 2021)
-------------------------
This is a bugfix release to resolve (:issue:`3391`, :issue:`5715`). It also
includes performance improvements in unstacking to a ``sparse`` array and a
number of documentation improvements.
Many thanks to the 20 contributors:
Aaron Spring, Alexandre Poux, Deepak Cherian, Enrico Minack, Fabien Maussion,
Giacomo Caria, Gijom, Guillaume Maze, Illviljan, Joe Hamman, Joseph Hardin, Kai
Mühlbauer, Matt Henderson, Maximilian Roos, Michael Delgado, Robert Gieseke,
Sebastian Weigand and Stephan Hoyer.
Breaking changes
~~~~~~~~~~~~~~~~
- Use complex nan when interpolating complex values out of bounds by default (instead of real nan) (:pull:`6019`).
By `Alexandre Poux `_.
Performance
~~~~~~~~~~~
- Significantly faster unstacking to a ``sparse`` array. :pull:`5577`
By `Deepak Cherian `_.
Bug fixes
~~~~~~~~~
- :py:func:`xr.map_blocks` and :py:func:`xr.corr` now work when dask is not installed (:issue:`3391`, :issue:`5715`, :pull:`5731`).
By `Gijom `_.
- Fix plot.line crash for data of shape ``(1, N)`` in _title_for_slice on format_item (:pull:`5948`).
By `Sebastian Weigand `_.
- Fix a regression in the removal of duplicate backend entrypoints (:issue:`5944`, :pull:`5959`)
By `Kai Mühlbauer `_.
- Fix an issue that datasets from being saved when time variables with units that ``cftime`` can parse but pandas can not were present (:pull:`6049`).
By `Tim Heap `_.
Documentation
~~~~~~~~~~~~~
- Better examples in docstrings for groupby and resampling reductions (:pull:`5871`).
By `Deepak Cherian `_,
`Maximilian Roos `_,
`Jimmy Westling `_ .
- Add list-like possibility for tolerance parameter in the reindex functions.
By `Antoine Gibek `_,
Internal Changes
~~~~~~~~~~~~~~~~
- Use ``importlib`` to replace functionality of ``pkg_resources`` in
backend plugins tests. (:pull:`5959`).
By `Kai Mühlbauer `_.
.. _whats-new.0.20.1:
v0.20.1 (5 November 2021)
-------------------------
This is a bugfix release to fix :issue:`5930`.
Bug fixes
~~~~~~~~~
- Fix a regression in the detection of the backend entrypoints (:issue:`5930`, :pull:`5931`)
By `Justus Magin `_.
Documentation
~~~~~~~~~~~~~
- Significant improvements to :ref:`api`. By `Deepak Cherian `_.
.. _whats-new.0.20.0:
v0.20.0 (1 November 2021)
-------------------------
This release brings improved support for pint arrays, methods for weighted standard deviation, variance,
and sum of squares, the option to disable the use of the bottleneck library, significantly improved performance of
unstack, as well as many bugfixes and internal changes.
Many thanks to the 40 contributors to this release!:
Aaron Spring, Akio Taniguchi, Alan D. Snow, arfy slowy, Benoit Bovy, Christian Jauvin, crusaderky, Deepak Cherian,
Giacomo Caria, Illviljan, James Bourbeau, Joe Hamman, Joseph K Aicher, Julien Herzen, Kai Mühlbauer,
keewis, lusewell, Martin K. Scherer, Mathias Hauser, Max Grover, Maxime Liquet, Maximilian Roos, Mike Taves, Nathan Lis,
pmav99, Pushkar Kopparla, Ray Bell, Rio McMahon, Scott Staniewicz, Spencer Clark, Stefan Bender, Taher Chegini,
Thomas Nicholas, Tomas Chor, Tom Augspurger, Victor Negîrneac, Zachary Blackwood, Zachary Moon, and Zeb Nicholls.
New Features
~~~~~~~~~~~~
- Add ``std``, ``var``, ``sum_of_squares`` to :py:class:`~core.weighted.DatasetWeighted` and :py:class:`~core.weighted.DataArrayWeighted`.
By `Christian Jauvin `_.
- Added a :py:func:`get_options` method to xarray's root namespace (:issue:`5698`, :pull:`5716`)
By `Pushkar Kopparla `_.
- Xarray now does a better job rendering variable names that are long LaTeX sequences when plotting (:issue:`5681`, :pull:`5682`).
By `Tomas Chor `_.
- Add an option (``"use_bottleneck"``) to disable the use of ``bottleneck`` using :py:func:`set_options` (:pull:`5560`)
By `Justus Magin `_.
- Added ``**kwargs`` argument to :py:meth:`open_rasterio` to access overviews (:issue:`3269`).
By `Pushkar Kopparla `_.
- Added ``storage_options`` argument to :py:meth:`to_zarr` (:issue:`5601`, :pull:`5615`).
By `Ray Bell `_, `Zachary Blackwood `_ and
`Nathan Lis `_.
- Added calendar utilities :py:func:`DataArray.convert_calendar`, :py:func:`DataArray.interp_calendar`, :py:func:`date_range`, :py:func:`date_range_like` and :py:attr:`DataArray.dt.calendar` (:issue:`5155`, :pull:`5233`).
By `Pascal Bourgault `_.
- Histogram plots are set with a title displaying the scalar coords if any, similarly to the other plots (:issue:`5791`, :pull:`5792`).
By `Maxime Liquet `_.
- Slice plots display the coords units in the same way as x/y/colorbar labels (:pull:`5847`).
By `Victor Negîrneac `_.
- Added a new :py:attr:`Dataset.chunksizes`, :py:attr:`DataArray.chunksizes`, and :py:attr:`Variable.chunksizes`
property, which will always return a mapping from dimension names to chunking pattern along that dimension,
regardless of whether the object is a Dataset, DataArray, or Variable. (:issue:`5846`, :pull:`5900`)
By `Tom Nicholas `_.
Breaking changes
~~~~~~~~~~~~~~~~
- The minimum versions of some dependencies were changed:
=============== ====== ====
Package Old New
=============== ====== ====
cftime 1.1 1.2
dask 2.15 2.30
distributed 2.15 2.30
lxml 4.5 4.6
matplotlib-base 3.2 3.3
numba 0.49 0.51
numpy 1.17 1.18
pandas 1.0 1.1
pint 0.15 0.16
scipy 1.4 1.5
seaborn 0.10 0.11
sparse 0.8 0.11
toolz 0.10 0.11
zarr 2.4 2.5
=============== ====== ====
- The ``__repr__`` of a :py:class:`xarray.Dataset`'s ``coords`` and ``data_vars``
ignore ``xarray.set_option(display_max_rows=...)`` and show the full output
when called directly as, e.g., ``ds.data_vars`` or ``print(ds.data_vars)``
(:issue:`5545`, :pull:`5580`).
By `Stefan Bender `_.
Deprecations
~~~~~~~~~~~~
- Deprecate :py:func:`open_rasterio` (:issue:`4697`, :pull:`5808`).
By `Alan Snow `_.
- Set the default argument for `roll_coords` to `False` for :py:meth:`DataArray.roll`
and :py:meth:`Dataset.roll`. (:pull:`5653`)
By `Tom Nicholas `_.
- :py:meth:`xarray.open_mfdataset` will now error instead of warn when a value for ``concat_dim`` is
passed alongside ``combine='by_coords'``.
By `Tom Nicholas `_.
Bug fixes
~~~~~~~~~
- Fix ZeroDivisionError from saving dask array with empty dimension (:issue: `5741`).
By `Joseph K Aicher `_.
- Fixed performance bug where ``cftime`` import attempted within various core operations if ``cftime`` not
installed (:pull:`5640`).
By `Luke Sewell `_
- Fixed bug when combining named DataArrays using :py:func:`combine_by_coords`. (:pull:`5834`).
By `Tom Nicholas `_.
- When a custom engine was used in :py:func:`~xarray.open_dataset` the engine
wasn't initialized properly, causing missing argument errors or inconsistent
method signatures. (:pull:`5684`)
By `Jimmy Westling `_.
- Numbers are properly formatted in a plot's title (:issue:`5788`, :pull:`5789`).
By `Maxime Liquet `_.
- Faceted plots will no longer raise a `pint.UnitStrippedWarning` when a `pint.Quantity` array is plotted,
and will correctly display the units of the data in the colorbar (if there is one) (:pull:`5886`).
By `Tom Nicholas `_.
- With backends, check for path-like objects rather than ``pathlib.Path``
type, use ``os.fspath`` (:pull:`5879`).
By `Mike Taves `_.
- ``open_mfdataset()`` now accepts a single ``pathlib.Path`` object (:issue: `5881`).
By `Panos Mavrogiorgos `_.
- Improved performance of :py:meth:`Dataset.unstack` (:pull:`5906`). By `Tom Augspurger `_.
Documentation
~~~~~~~~~~~~~
- Users are instructed to try ``use_cftime=True`` if a ``TypeError`` occurs when combining datasets and one of the types involved is a subclass of ``cftime.datetime`` (:pull:`5776`).
By `Zeb Nicholls `_.
- A clearer error is now raised if a user attempts to assign a Dataset to a single key of
another Dataset. (:pull:`5839`)
By `Tom Nicholas `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Explicit indexes refactor: avoid ``len(index)`` in ``map_blocks`` (:pull:`5670`).
By `Deepak Cherian `_.
- Explicit indexes refactor: decouple ``xarray.Index``` from ``xarray.Variable`` (:pull:`5636`).
By `Benoit Bovy `_.
- Fix ``Mapping`` argument typing to allow mypy to pass on ``str`` keys (:pull:`5690`).
By `Maximilian Roos `_.
- Annotate many of our tests, and fix some of the resulting typing errors. This will
also mean our typing annotations are tested as part of CI. (:pull:`5728`).
By `Maximilian Roos `_.
- Improve the performance of reprs for large datasets or dataarrays. (:pull:`5661`)
By `Jimmy Westling `_.
- Use isort's `float_to_top` config. (:pull:`5695`).
By `Maximilian Roos `_.
- Remove use of the deprecated ``kind`` argument in
:py:meth:`pandas.Index.get_slice_bound` inside :py:class:`xarray.CFTimeIndex`
tests (:pull:`5723`). By `Spencer Clark `_.
- Refactor `xarray.core.duck_array_ops` to no longer special-case dispatching to
dask versions of functions when acting on dask arrays, instead relying numpy
and dask's adherence to NEP-18 to dispatch automatically. (:pull:`5571`)
By `Tom Nicholas `_.
- Add an ASV benchmark CI and improve performance of the benchmarks (:pull:`5796`)
By `Jimmy Westling `_.
- Use ``importlib`` to replace functionality of ``pkg_resources`` such
as version setting and loading of resources. (:pull:`5845`).
By `Martin K. Scherer `_.
.. _whats-new.0.19.0:
v0.19.0 (23 July 2021)
----------------------
This release brings improvements to plotting of categorical data, the ability to specify how attributes
are combined in xarray operations, a new high-level :py:func:`unify_chunks` function, as well as various
deprecations, bug fixes, and minor improvements.
Many thanks to the 29 contributors to this release!:
Andrew Williams, Augustus, Aureliana Barghini, Benoit Bovy, crusaderky, Deepak Cherian, ellesmith88,
Elliott Sales de Andrade, Giacomo Caria, github-actions[bot], Illviljan, Joeperdefloep, joooeey, Julia Kent,
Julius Busecke, keewis, Mathias Hauser, Matthias Göbel, Mattia Almansi, Maximilian Roos, Peter Andreas Entschev,
Ray Bell, Sander, Santiago Soler, Sebastian, Spencer Clark, Stephan Hoyer, Thomas Hirtz, Thomas Nicholas.
New Features
~~~~~~~~~~~~
- Allow passing argument ``missing_dims`` to :py:meth:`Variable.transpose` and :py:meth:`Dataset.transpose`
(:issue:`5550`, :pull:`5586`)
By `Giacomo Caria `_.
- Allow passing a dictionary as coords to a :py:class:`DataArray` (:issue:`5527`,
reverts :pull:`1539`, which had deprecated this due to python's inconsistent ordering in earlier versions).
By `Sander van Rijn `_.
- Added :py:meth:`Dataset.coarsen.construct`, :py:meth:`DataArray.coarsen.construct` (:issue:`5454`, :pull:`5475`).
By `Deepak Cherian `_.
- Xarray now uses consolidated metadata by default when writing and reading Zarr
stores (:issue:`5251`).
By `Stephan Hoyer `_.
- New top-level function :py:func:`unify_chunks`.
By `Mattia Almansi `_.
- Allow assigning values to a subset of a dataset using positional or label-based
indexing (:issue:`3015`, :pull:`5362`).
By `Matthias Göbel `_.
- Attempting to reduce a weighted object over missing dimensions now raises an error (:pull:`5362`).
By `Mattia Almansi `_.
- Add ``.sum`` to :py:meth:`~xarray.DataArray.rolling_exp` and
:py:meth:`~xarray.Dataset.rolling_exp` for exponentially weighted rolling
sums. These require numbagg 0.2.1;
(:pull:`5178`).
By `Maximilian Roos `_.
- :py:func:`xarray.cov` and :py:func:`xarray.corr` now lazily check for missing
values if inputs are dask arrays (:issue:`4804`, :pull:`5284`).
By `Andrew Williams `_.
- Attempting to ``concat`` list of elements that are not all ``Dataset`` or all ``DataArray`` now raises an error (:issue:`5051`, :pull:`5425`).
By `Thomas Hirtz `_.
- allow passing a function to ``combine_attrs`` (:pull:`4896`).
By `Justus Magin `_.
- Allow plotting categorical data (:pull:`5464`).
By `Jimmy Westling `_.
- Allow removal of the coordinate attribute ``coordinates`` on variables by setting ``.attrs['coordinates']= None``
(:issue:`5510`).
By `Elle Smith `_.
- Added :py:meth:`DataArray.to_numpy`, :py:meth:`DataArray.as_numpy`, and :py:meth:`Dataset.as_numpy`. (:pull:`5568`).
By `Tom Nicholas `_.
- Units in plot labels are now automatically inferred from wrapped :py:meth:`pint.Quantity` arrays. (:pull:`5561`).
By `Tom Nicholas `_.
Breaking changes
~~~~~~~~~~~~~~~~
- The default ``mode`` for :py:meth:`Dataset.to_zarr` when ``region`` is set
has changed to the new ``mode="r+"``, which only allows for overriding
pre-existing array values. This is a safer default than the prior ``mode="a"``,
and allows for higher performance writes (:pull:`5252`).
By `Stephan Hoyer `_.
- The main parameter to :py:func:`combine_by_coords` is renamed to `data_objects` instead
of `datasets` so anyone calling this method using a named parameter will need to update
the name accordingly (:issue:`3248`, :pull:`4696`).
By `Augustus Ijams `_.
Deprecations
~~~~~~~~~~~~
- Removed the deprecated ``dim`` kwarg to :py:func:`DataArray.integrate` (:pull:`5630`)
- Removed the deprecated ``keep_attrs`` kwarg to :py:func:`DataArray.rolling` (:pull:`5630`)
- Removed the deprecated ``keep_attrs`` kwarg to :py:func:`DataArray.coarsen` (:pull:`5630`)
- Completed deprecation of passing an ``xarray.DataArray`` to :py:func:`Variable` - will now raise a ``TypeError`` (:pull:`5630`)
Bug fixes
~~~~~~~~~
- Fix a minor incompatibility between partial datetime string indexing with a
:py:class:`CFTimeIndex` and upcoming pandas version 1.3.0 (:issue:`5356`,
:pull:`5359`).
By `Spencer Clark `_.
- Fix 1-level multi-index incorrectly converted to single index (:issue:`5384`,
:pull:`5385`).
By `Benoit Bovy `_.
- Don't cast a duck array in a coordinate to :py:class:`numpy.ndarray` in
:py:meth:`DataArray.differentiate` (:pull:`5408`)
By `Justus Magin `_.
- Fix the ``repr`` of :py:class:`Variable` objects with ``display_expand_data=True``
(:pull:`5406`)
By `Justus Magin `_.
- Plotting a pcolormesh with ``xscale="log"`` and/or ``yscale="log"`` works as
expected after improving the way the interval breaks are generated (:issue:`5333`).
By `Santiago Soler `_
- :py:func:`combine_by_coords` can now handle combining a list of unnamed
``DataArray`` as input (:issue:`3248`, :pull:`4696`).
By `Augustus Ijams `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Run CI on the first & last python versions supported only; currently 3.7 & 3.9.
(:pull:`5433`)
By `Maximilian Roos `_.
- Publish test results & timings on each PR.
(:pull:`5537`)
By `Maximilian Roos `_.
- Explicit indexes refactor: add a ``xarray.Index.query()`` method in which
one may eventually provide a custom implementation of label-based data
selection (not ready yet for public use). Also refactor the internal,
pandas-specific implementation into ``PandasIndex.query()`` and
``PandasMultiIndex.query()`` (:pull:`5322`).
By `Benoit Bovy `_.
.. _whats-new.0.18.2:
v0.18.2 (19 May 2021)
---------------------
This release reverts a regression in xarray's unstacking of dask-backed arrays.
.. _whats-new.0.18.1:
v0.18.1 (18 May 2021)
---------------------
This release is intended as a small patch release to be compatible with the new
2021.5.0 ``dask.distributed`` release. It also includes a new
``drop_duplicates`` method, some documentation improvements, the beginnings of
our internal Index refactoring, and some bug fixes.
Thank you to all 16 contributors!
Anderson Banihirwe, Andrew, Benoit Bovy, Brewster Malevich, Giacomo Caria,
Illviljan, James Bourbeau, Keewis, Maximilian Roos, Ravin Kumar, Stephan Hoyer,
Thomas Nicholas, Tom Nicholas, Zachary Moon.
New Features
~~~~~~~~~~~~
- Implement :py:meth:`DataArray.drop_duplicates`
to remove duplicate dimension values (:pull:`5239`).
By `Andrew Huang `_.
- Allow passing ``combine_attrs`` strategy names to the ``keep_attrs`` parameter of
:py:func:`apply_ufunc` (:pull:`5041`)
By `Justus Magin `_.
- :py:meth:`Dataset.interp` now allows interpolation with non-numerical datatypes,
such as booleans, instead of dropping them. (:issue:`4761` :pull:`5008`).
By `Jimmy Westling `_.
- Raise more informative error when decoding time variables with invalid reference dates.
(:issue:`5199`, :pull:`5288`). By `Giacomo Caria `_.
Bug fixes
~~~~~~~~~
- Opening netCDF files from a path that doesn't end in ``.nc`` without supplying
an explicit ``engine`` works again (:issue:`5295`), fixing a bug introduced in
0.18.0.
By `Stephan Hoyer `_
Documentation
~~~~~~~~~~~~~
- Clean up and enhance docstrings for the :py:class:`DataArray.plot` and ``Dataset.plot.*``
families of methods (:pull:`5285`).
By `Zach Moon `_.
- Explanation of deprecation cycles and how to implement them added to contributors
guide. (:pull:`5289`)
By `Tom Nicholas `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Explicit indexes refactor: add an ``xarray.Index`` base class and
``Dataset.xindexes`` / ``DataArray.xindexes`` properties. Also rename
``PandasIndexAdapter`` to ``PandasIndex``, which now inherits from
``xarray.Index`` (:pull:`5102`).
By `Benoit Bovy `_.
- Replace ``SortedKeysDict`` with python's ``dict``, given dicts are now ordered.
By `Maximilian Roos `_.
- Updated the release guide for developers. Now accounts for actions that are automated via github
actions. (:pull:`5274`).
By `Tom Nicholas `_.
.. _whats-new.0.18.0:
v0.18.0 (6 May 2021)
--------------------
This release brings a few important performance improvements, a wide range of
usability upgrades, lots of bug fixes, and some new features. These include
a plugin API to add backend engines, a new theme for the documentation,
curve fitting methods, and several new plotting functions.
Many thanks to the 38 contributors to this release: Aaron Spring, Alessandro Amici,
Alex Marandon, Alistair Miles, Ana Paula Krelling, Anderson Banihirwe, Aureliana Barghini,
Baudouin Raoult, Benoit Bovy, Blair Bonnett, David Trémouilles, Deepak Cherian,
Gabriel Medeiros Abrahão, Giacomo Caria, Hauke Schulz, Illviljan, Mathias Hauser, Matthias Bussonnier,
Mattia Almansi, Maximilian Roos, Ray Bell, Richard Kleijn, Ryan Abernathey, Sam Levang, Spencer Clark,
Spencer Jones, Tammas Loughran, Tobias Kölling, Todd, Tom Nicholas, Tom White, Victor Negîrneac,
Xianxiang Li, Zeb Nicholls, crusaderky, dschwoerer, johnomotani, keewis
New Features
~~~~~~~~~~~~
- apply ``combine_attrs`` on data variables and coordinate variables when concatenating
and merging datasets and dataarrays (:pull:`4902`).
By `Justus Magin `_.
- Add :py:meth:`Dataset.to_pandas` (:pull:`5247`)
By `Giacomo Caria `_.
- Add :py:meth:`DataArray.plot.surface` which wraps matplotlib's `plot_surface` to make
surface plots (:issue:`2235` :issue:`5084` :pull:`5101`).
By `John Omotani `_.
- Allow passing multiple arrays to :py:meth:`Dataset.__setitem__` (:pull:`5216`).
By `Giacomo Caria `_.
- Add 'cumulative' option to :py:meth:`Dataset.integrate` and
:py:meth:`DataArray.integrate` so that result is a cumulative integral, like
:py:func:`scipy.integrate.cumulative_trapezoidal` (:pull:`5153`).
By `John Omotani `_.
- Add ``safe_chunks`` option to :py:meth:`Dataset.to_zarr` which allows overriding
checks made to ensure Dask and Zarr chunk compatibility (:issue:`5056`).
By `Ryan Abernathey `_
- Add :py:meth:`Dataset.query` and :py:meth:`DataArray.query` which enable indexing
of datasets and data arrays by evaluating query expressions against the values of the
data variables (:pull:`4984`).
By `Alistair Miles `_.
- Allow passing ``combine_attrs`` to :py:meth:`Dataset.merge` (:pull:`4895`).
By `Justus Magin `_.
- Support for `dask.graph_manipulation
`_ (requires dask >=2021.3)
By `Guido Imperiale `_
- Add :py:meth:`Dataset.plot.streamplot` for streamplot plots with :py:class:`Dataset`
variables (:pull:`5003`).
By `John Omotani `_.
- Many of the arguments for the :py:attr:`DataArray.str` methods now support
providing an array-like input. In this case, the array provided to the
arguments is broadcast against the original array and applied elementwise.
- :py:attr:`DataArray.str` now supports ``+``, ``*``, and ``%`` operators. These
behave the same as they do for :py:class:`str`, except that they follow
array broadcasting rules.
- A large number of new :py:attr:`DataArray.str` methods were implemented,
:py:meth:`DataArray.str.casefold`, :py:meth:`DataArray.str.cat`,
:py:meth:`DataArray.str.extract`, :py:meth:`DataArray.str.extractall`,
:py:meth:`DataArray.str.findall`, :py:meth:`DataArray.str.format`,
:py:meth:`DataArray.str.get_dummies`, :py:meth:`DataArray.str.islower`,
:py:meth:`DataArray.str.join`, :py:meth:`DataArray.str.normalize`,
:py:meth:`DataArray.str.partition`, :py:meth:`DataArray.str.rpartition`,
:py:meth:`DataArray.str.rsplit`, and :py:meth:`DataArray.str.split`.
A number of these methods allow for splitting or joining the strings in an
array. (:issue:`4622`)
By `Todd Jennings `_
- Thanks to the new pluggable backend infrastructure external packages may now
use the ``xarray.backends`` entry point to register additional engines to be used in
:py:func:`open_dataset`, see the documentation in :ref:`add_a_backend`
(:issue:`4309`, :issue:`4803`, :pull:`4989`, :pull:`4810` and many others).
The backend refactor has been sponsored with the "Essential Open Source Software for Science"
grant from the `Chan Zuckerberg Initiative `_ and
developed by `B-Open `_.
By `Aureliana Barghini `_ and `Alessandro Amici `_.
- :py:attr:`~core.accessor_dt.DatetimeAccessor.date` added (:issue:`4983`, :pull:`4994`).
By `Hauke Schulz `_.
- Implement ``__getitem__`` for both :py:class:`~core.groupby.DatasetGroupBy` and
:py:class:`~core.groupby.DataArrayGroupBy`, inspired by pandas'
:py:meth:`~pandas.core.groupby.GroupBy.get_group`.
By `Deepak Cherian `_.
- Switch the tutorial functions to use `pooch `_
(which is now a optional dependency) and add :py:func:`tutorial.open_rasterio` as a
way to open example rasterio files (:issue:`3986`, :pull:`4102`, :pull:`5074`).
By `Justus Magin `_.
- Add typing information to unary and binary arithmetic operators operating on
:py:class:`Dataset`, :py:class:`DataArray`, :py:class:`Variable`,
:py:class:`~core.groupby.DatasetGroupBy` or
:py:class:`~core.groupby.DataArrayGroupBy` (:pull:`4904`).
By `Richard Kleijn `_.
- Add a ``combine_attrs`` parameter to :py:func:`open_mfdataset` (:pull:`4971`).
By `Justus Magin `_.
- Enable passing arrays with a subset of dimensions to
:py:meth:`DataArray.clip` & :py:meth:`Dataset.clip`; these methods now use
:py:func:`xarray.apply_ufunc`; (:pull:`5184`).
By `Maximilian Roos `_.
- Disable the `cfgrib` backend if the `eccodes` library is not installed (:pull:`5083`).
By `Baudouin Raoult `_.
- Added :py:meth:`DataArray.curvefit` and :py:meth:`Dataset.curvefit` for general curve fitting applications. (:issue:`4300`, :pull:`4849`)
By `Sam Levang `_.
- Add options to control expand/collapse of sections in display of Dataset and
DataArray. The function :py:func:`set_options` now takes keyword arguments
``display_expand_attrs``, ``display_expand_coords``, ``display_expand_data``,
``display_expand_data_vars``, all of which can be one of ``True`` to always
expand, ``False`` to always collapse, or ``default`` to expand unless over a
pre-defined limit (:pull:`5126`).
By `Tom White `_.
- Significant speedups in :py:meth:`Dataset.interp` and :py:meth:`DataArray.interp`.
(:issue:`4739`, :pull:`4740`).
By `Deepak Cherian `_.
- Prevent passing `concat_dim` to :py:func:`xarray.open_mfdataset` when
`combine='by_coords'` is specified, which should never have been possible (as
:py:func:`xarray.combine_by_coords` has no `concat_dim` argument to pass to).
Also removes unneeded internal reordering of datasets in
:py:func:`xarray.open_mfdataset` when `combine='by_coords'` is specified.
Fixes (:issue:`5230`).
By `Tom Nicholas `_.
- Implement ``__setitem__`` for ``xarray.core.indexing.DaskIndexingAdapter`` if
dask version supports item assignment. (:issue:`5171`, :pull:`5174`)
By `Tammas Loughran `_.
Breaking changes
~~~~~~~~~~~~~~~~
- The minimum versions of some dependencies were changed:
============ ====== ====
Package Old New
============ ====== ====
boto3 1.12 1.13
cftime 1.0 1.1
dask 2.11 2.15
distributed 2.11 2.15
matplotlib 3.1 3.2
numba 0.48 0.49
============ ====== ====
- :py:func:`open_dataset` and :py:func:`open_dataarray` now accept only the first argument
as positional, all others need to be passed are keyword arguments. This is part of the
refactor to support external backends (:issue:`4309`, :pull:`4989`).
By `Alessandro Amici `_.
- Functions that are identities for 0d data return the unchanged data
if axis is empty. This ensures that Datasets where some variables do
not have the averaged dimensions are not accidentally changed
(:issue:`4885`, :pull:`5207`).
By `David Schwörer `_.
- :py:attr:`DataArray.coarsen` and :py:attr:`Dataset.coarsen` no longer support passing ``keep_attrs``
via its constructor. Pass ``keep_attrs`` via the applied function, i.e. use
``ds.coarsen(...).mean(keep_attrs=False)`` instead of ``ds.coarsen(..., keep_attrs=False).mean()``.
Further, coarsen now keeps attributes per default (:pull:`5227`).
By `Mathias Hauser `_.
- switch the default of the :py:func:`merge` ``combine_attrs`` parameter to
``"override"``. This will keep the current behavior for merging the ``attrs`` of
variables but stop dropping the ``attrs`` of the main objects (:pull:`4902`).
By `Justus Magin `_.
Deprecations
~~~~~~~~~~~~
- Warn when passing `concat_dim` to :py:func:`xarray.open_mfdataset` when
`combine='by_coords'` is specified, which should never have been possible (as
:py:func:`xarray.combine_by_coords` has no `concat_dim` argument to pass to).
Also removes unneeded internal reordering of datasets in
:py:func:`xarray.open_mfdataset` when `combine='by_coords'` is specified.
Fixes (:issue:`5230`), via (:pull:`5231`, :pull:`5255`).
By `Tom Nicholas `_.
- The `lock` keyword argument to :py:func:`open_dataset` and :py:func:`open_dataarray` is now
a backend specific option. It will give a warning if passed to a backend that doesn't support it
instead of being silently ignored. From the next version it will raise an error.
This is part of the refactor to support external backends (:issue:`5073`).
By `Tom Nicholas `_ and `Alessandro Amici `_.
Bug fixes
~~~~~~~~~
- Properly support :py:meth:`DataArray.ffill`, :py:meth:`DataArray.bfill`, :py:meth:`Dataset.ffill`, :py:meth:`Dataset.bfill` along chunked dimensions.
(:issue:`2699`).
By `Deepak Cherian `_.
- Fix 2d plot failure for certain combinations of dimensions when `x` is 1d and `y` is
2d (:issue:`5097`, :pull:`5099`).
By `John Omotani `_.
- Ensure standard calendar times encoded with large values (i.e. greater than
approximately 292 years), can be decoded correctly without silently overflowing
(:pull:`5050`). This was a regression in xarray 0.17.0.
By `Zeb Nicholls `_.
- Added support for `numpy.bool_` attributes in roundtrips using `h5netcdf` engine with `invalid_netcdf=True` [which casts `bool`s to `numpy.bool_`] (:issue:`4981`, :pull:`4986`).
By `Victor Negîrneac `_.
- Don't allow passing ``axis`` to :py:meth:`Dataset.reduce` methods (:issue:`3510`, :pull:`4940`).
By `Justus Magin `_.
- Decode values as signed if attribute `_Unsigned = "false"` (:issue:`4954`)
By `Tobias Kölling `_.
- Keep coords attributes when interpolating when the indexer is not a Variable. (:issue:`4239`, :issue:`4839` :pull:`5031`)
By `Jimmy Westling `_.
- Ensure standard calendar dates encoded with a calendar attribute with some or
all uppercase letters can be decoded or encoded to or from
``np.datetime64[ns]`` dates with or without ``cftime`` installed
(:issue:`5093`, :pull:`5180`).
By `Spencer Clark `_.
- Warn on passing ``keep_attrs`` to ``resample`` and ``rolling_exp`` as they are ignored, pass ``keep_attrs``
to the applied function instead (:pull:`5265`).
By `Mathias Hauser `_.
Documentation
~~~~~~~~~~~~~
- New section on :ref:`add_a_backend` in the "Internals" chapter aimed to backend developers
(:issue:`4803`, :pull:`4810`).
By `Aureliana Barghini `_.
- Add :py:meth:`Dataset.polyfit` and :py:meth:`DataArray.polyfit` under "See also" in
the docstrings of :py:meth:`Dataset.polyfit` and :py:meth:`DataArray.polyfit`
(:issue:`5016`, :pull:`5020`).
By `Aaron Spring `_.
- New sphinx theme & rearrangement of the docs (:pull:`4835`).
By `Anderson Banihirwe `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Enable displaying mypy error codes and ignore only specific error codes using
``# type: ignore[error-code]`` (:pull:`5096`).
By `Mathias Hauser `_.
- Replace uses of ``raises_regex`` with the more standard
``pytest.raises(Exception, match="foo")``;
(:pull:`5188`), (:pull:`5191`).
By `Maximilian Roos `_.
.. _whats-new.0.17.0:
v0.17.0 (24 Feb 2021)
---------------------
This release brings a few important performance improvements, a wide range of
usability upgrades, lots of bug fixes, and some new features. These include
better ``cftime`` support, a new quiver plot, better ``unstack`` performance,
more efficient memory use in rolling operations, and some python packaging
improvements. We also have a few documentation improvements (and more planned!).
Many thanks to the 36 contributors to this release: Alessandro Amici, Anderson
Banihirwe, Aureliana Barghini, Ayrton Bourn, Benjamin Bean, Blair Bonnett, Chun
Ho Chow, DWesl, Daniel Mesejo-León, Deepak Cherian, Eric Keenan, Illviljan, Jens
Hedegaard Nielsen, Jody Klymak, Julien Seguinot, Julius Busecke, Kai Mühlbauer,
Leif Denby, Martin Durant, Mathias Hauser, Maximilian Roos, Michael Mann, Ray
Bell, RichardScottOZ, Spencer Clark, Tim Gates, Tom Nicholas, Yunus Sevinchan,
alexamici, aurghs, crusaderky, dcherian, ghislainp, keewis, rhkleijn
Breaking changes
~~~~~~~~~~~~~~~~
- xarray no longer supports python 3.6
The minimum version policy was changed to also apply to projects with irregular
releases. As a result, the minimum versions of some dependencies have changed:
============ ====== ====
Package Old New
============ ====== ====
Python 3.6 3.7
setuptools 38.4 40.4
numpy 1.15 1.17
pandas 0.25 1.0
dask 2.9 2.11
distributed 2.9 2.11
bottleneck 1.2 1.3
h5netcdf 0.7 0.8
iris 2.2 2.4
netcdf4 1.4 1.5
pseudonetcdf 3.0 3.1
rasterio 1.0 1.1
scipy 1.3 1.4
seaborn 0.9 0.10
zarr 2.3 2.4
============ ====== ====
(:issue:`4688`, :pull:`4720`, :pull:`4907`, :pull:`4942`)
- As a result of :pull:`4684` the default units encoding for
datetime-like values (``np.datetime64[ns]`` or ``cftime.datetime``) will now
always be set such that ``int64`` values can be used. In the past, no units
finer than "seconds" were chosen, which would sometimes mean that ``float64``
values were required, which would lead to inaccurate I/O round-trips.
- Variables referred to in attributes like ``bounds`` and ``grid_mapping``
can be set as coordinate variables. These attributes are moved to
:py:attr:`DataArray.encoding` from :py:attr:`DataArray.attrs`. This behaviour
is controlled by the ``decode_coords`` kwarg to :py:func:`open_dataset` and
:py:func:`open_mfdataset`. The full list of decoded attributes is in
:ref:`weather-climate` (:pull:`2844`, :issue:`3689`)
- As a result of :pull:`4911` the output from calling :py:meth:`DataArray.sum`
or :py:meth:`DataArray.prod` on an integer array with ``skipna=True`` and a
non-None value for ``min_count`` will now be a float array rather than an
integer array.
Deprecations
~~~~~~~~~~~~
- ``dim`` argument to :py:meth:`DataArray.integrate` is being deprecated in
favour of a ``coord`` argument, for consistency with :py:meth:`Dataset.integrate`.
For now using ``dim`` issues a ``FutureWarning``. It will be removed in
version 0.19.0 (:pull:`3993`).
By `Tom Nicholas `_.
- Deprecated ``autoclose`` kwargs from :py:func:`open_dataset` are removed (:pull:`4725`).
By `Aureliana Barghini `_.
- the return value of :py:meth:`Dataset.update` is being deprecated to make it work more
like :py:meth:`dict.update`. It will be removed in version 0.19.0 (:pull:`4932`).
By `Justus Magin `_.
New Features
~~~~~~~~~~~~
- :py:meth:`~xarray.cftime_range` and :py:meth:`DataArray.resample` now support
millisecond (``"L"`` or ``"ms"``) and microsecond (``"U"`` or ``"us"``) frequencies
for ``cftime.datetime`` coordinates (:issue:`4097`, :pull:`4758`).
By `Spencer Clark `_.
- Significantly higher ``unstack`` performance on numpy-backed arrays which
contain missing values; 8x faster than previous versions in our benchmark, and
now 2x faster than pandas (:pull:`4746`).
By `Maximilian Roos `_.
- Add :py:meth:`Dataset.plot.quiver` for quiver plots with :py:class:`Dataset` variables.
By `Deepak Cherian `_.
- Add ``"drop_conflicts"`` to the strategies supported by the ``combine_attrs`` kwarg
(:issue:`4749`, :pull:`4827`).
By `Justus Magin `_.
- Allow installing from git archives (:pull:`4897`).
By `Justus Magin `_.
- :py:class:`~core.rolling.DataArrayCoarsen` and :py:class:`~core.rolling.DatasetCoarsen`
now implement a ``reduce`` method, enabling coarsening operations with custom
reduction functions (:issue:`3741`, :pull:`4939`).
By `Spencer Clark `_.
- Most rolling operations use significantly less memory. (:issue:`4325`).
By `Deepak Cherian `_.
- Add :py:meth:`Dataset.drop_isel` and :py:meth:`DataArray.drop_isel`
(:issue:`4658`, :pull:`4819`).
By `Daniel Mesejo `_.
- Xarray now leverages updates as of cftime version 1.4.1, which enable exact I/O
roundtripping of ``cftime.datetime`` objects (:pull:`4758`).
By `Spencer Clark `_.
- :py:func:`open_dataset` and :py:func:`open_mfdataset` now accept ``fsspec`` URLs
(including globs for the latter) for ``engine="zarr"``, and so allow reading from
many remote and other file systems (:pull:`4461`)
By `Martin Durant `_
- :py:meth:`DataArray.swap_dims` & :py:meth:`Dataset.swap_dims` now accept dims
in the form of kwargs as well as a dict, like most similar methods.
By `Maximilian Roos `_.
Bug fixes
~~~~~~~~~
- Use specific type checks in ``xarray.core.variable.as_compatible_data`` instead of
blanket access to ``values`` attribute (:issue:`2097`)
By `Yunus Sevinchan `_.
- :py:meth:`DataArray.resample` and :py:meth:`Dataset.resample` do not trigger
computations anymore if :py:meth:`Dataset.weighted` or
:py:meth:`DataArray.weighted` are applied (:issue:`4625`, :pull:`4668`). By
`Julius Busecke `_.
- :py:func:`merge` with ``combine_attrs='override'`` makes a copy of the attrs
(:issue:`4627`).
- By default, when possible, xarray will now always use values of
type ``int64`` when encoding and decoding ``numpy.datetime64[ns]`` datetimes. This
ensures that maximum precision and accuracy are maintained in the round-tripping
process (:issue:`4045`, :pull:`4684`). It also enables encoding and decoding standard
calendar dates with time units of nanoseconds (:pull:`4400`).
By `Spencer Clark `_ and `Mark Harfouche
`_.
- :py:meth:`DataArray.astype`, :py:meth:`Dataset.astype` and :py:meth:`Variable.astype` support
the ``order`` and ``subok`` parameters again. This fixes a regression introduced in version 0.16.1
(:issue:`4644`, :pull:`4683`).
By `Richard Kleijn `_ .
- Remove dictionary unpacking when using ``.loc`` to avoid collision with ``.sel`` parameters (:pull:`4695`).
By `Anderson Banihirwe `_.
- Fix the legend created by :py:meth:`Dataset.plot.scatter` (:issue:`4641`, :pull:`4723`).
By `Justus Magin `_.
- Fix a crash in orthogonal indexing on geographic coordinates with ``engine='cfgrib'``
(:issue:`4733` :pull:`4737`).
By `Alessandro Amici `_.
- Coordinates with dtype ``str`` or ``bytes`` now retain their dtype on many operations,
e.g. ``reindex``, ``align``, ``concat``, ``assign``, previously they were cast to an object dtype
(:issue:`2658` and :issue:`4543`).
By `Mathias Hauser `_.
- Limit number of data rows when printing large datasets. (:issue:`4736`, :pull:`4750`).
By `Jimmy Westling `_.
- Add ``missing_dims`` parameter to transpose (:issue:`4647`, :pull:`4767`).
By `Daniel Mesejo `_.
- Resolve intervals before appending other metadata to labels when plotting (:issue:`4322`, :pull:`4794`).
By `Justus Magin `_.
- Fix regression when decoding a variable with a ``scale_factor`` and ``add_offset`` given
as a list of length one (:issue:`4631`).
By `Mathias Hauser `_.
- Expand user directory paths (e.g. ``~/``) in :py:func:`open_mfdataset` and
:py:meth:`Dataset.to_zarr` (:issue:`4783`, :pull:`4795`).
By `Julien Seguinot `_.
- Raise DeprecationWarning when trying to typecast a tuple containing a :py:class:`DataArray`.
User now prompted to first call `.data` on it (:issue:`4483`).
By `Chun Ho Chow `_.
- Ensure that :py:meth:`Dataset.interp` raises ``ValueError`` when interpolating
outside coordinate range and ``bounds_error=True`` (:issue:`4854`,
:pull:`4855`).
By `Leif Denby `_.
- Fix time encoding bug associated with using cftime versions greater than
1.4.0 with xarray (:issue:`4870`, :pull:`4871`).
By `Spencer Clark `_.
- Stop :py:meth:`DataArray.sum` and :py:meth:`DataArray.prod` computing lazy
arrays when called with a ``min_count`` parameter (:issue:`4898`, :pull:`4911`).
By `Blair Bonnett `_.
- Fix bug preventing the ``min_count`` parameter to :py:meth:`DataArray.sum` and
:py:meth:`DataArray.prod` working correctly when calculating over all axes of
a float64 array (:issue:`4898`, :pull:`4911`).
By `Blair Bonnett `_.
- Fix decoding of vlen strings using h5py versions greater than 3.0.0 with h5netcdf backend (:issue:`4570`, :pull:`4893`).
By `Kai Mühlbauer `_.
- Allow converting :py:class:`Dataset` or :py:class:`DataArray` objects with a ``MultiIndex``
and at least one other dimension to a ``pandas`` object (:issue:`3008`, :pull:`4442`).
By `ghislainp `_.
Documentation
~~~~~~~~~~~~~
- Add information about requirements for accessor classes (:issue:`2788`, :pull:`4657`).
By `Justus Magin `_.
- Start a list of external I/O integrating with ``xarray`` (:issue:`683`, :pull:`4566`).
By `Justus Magin `_.
- Add concat examples and improve combining documentation (:issue:`4620`, :pull:`4645`).
By `Ray Bell `_ and
`Justus Magin `_.
- explicitly mention that :py:meth:`Dataset.update` updates inplace (:issue:`2951`, :pull:`4932`).
By `Justus Magin `_.
- Added docs on vectorized indexing (:pull:`4711`).
By `Eric Keenan `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Speed up of the continuous integration tests on azure.
- Switched to mamba and use matplotlib-base for a faster installation of all dependencies (:pull:`4672`).
- Use ``pytest.mark.skip`` instead of ``pytest.mark.xfail`` for some tests that can currently not
succeed (:pull:`4685`).
- Run the tests in parallel using pytest-xdist (:pull:`4694`).
By `Justus Magin `_ and `Mathias Hauser `_.
- Use ``pyproject.toml`` instead of the ``setup_requires`` option for
``setuptools`` (:pull:`4897`).
By `Justus Magin `_.
- Replace all usages of ``assert x.identical(y)`` with ``assert_identical(x, y)``
for clearer error messages (:pull:`4752`).
By `Maximilian Roos `_.
- Speed up attribute style access (e.g. ``ds.somevar`` instead of ``ds["somevar"]``) and
tab completion in IPython (:issue:`4741`, :pull:`4742`).
By `Richard Kleijn `_.
- Added the ``set_close`` method to ``Dataset`` and ``DataArray`` for backends
to specify how to voluntary release all resources. (:pull:`#4809`)
By `Alessandro Amici `_.
- Update type hints to work with numpy v1.20 (:pull:`4878`).
By `Mathias Hauser `_.
- Ensure warnings cannot be turned into exceptions in :py:func:`testing.assert_equal` and
the other ``assert_*`` functions (:pull:`4864`).
By `Mathias Hauser `_.
- Performance improvement when constructing DataArrays. Significantly speeds up
repr for Datasets with large number of variables.
By `Deepak Cherian `_.
.. _whats-new.0.16.2:
v0.16.2 (30 Nov 2020)
---------------------
This release brings the ability to write to limited regions of ``zarr`` files,
open zarr files with :py:func:`open_dataset` and :py:func:`open_mfdataset`,
increased support for propagating ``attrs`` using the ``keep_attrs`` flag, as
well as numerous bugfixes and documentation improvements.
Many thanks to the 31 contributors who contributed to this release: Aaron
Spring, Akio Taniguchi, Aleksandar Jelenak, alexamici, Alexandre Poux, Anderson
Banihirwe, Andrew Pauling, Ashwin Vishnu, aurghs, Brian Ward, Caleb, crusaderky,
Dan Nowacki, darikg, David Brochart, David Huard, Deepak Cherian, Dion Häfner,
Gerardo Rivera, Gerrit Holl, Illviljan, inakleinbottle, Jacob Tomlinson, James
A. Bednar, jenssss, Joe Hamman, johnomotani, Joris Van den Bossche, Julia Kent,
Julius Busecke, Kai Mühlbauer, keewis, Keisuke Fujii, Kyle Cranmer, Luke
Volpatti, Mathias Hauser, Maximilian Roos, Michaël Defferrard, Michal
Baumgartner, Nick R. Papior, Pascal Bourgault, Peter Hausamann, PGijsbers, Ray
Bell, Romain Martinez, rpgoldman, Russell Manser, Sahid Velji, Samnan Rahee,
Sander, Spencer Clark, Stephan Hoyer, Thomas Zilio, Tobias Kölling, Tom
Augspurger, Wei Ji, Yash Saboo, Zeb Nicholls,
Deprecations
~~~~~~~~~~~~
- :py:attr:`~core.accessor_dt.DatetimeAccessor.weekofyear` and :py:attr:`~core.accessor_dt.DatetimeAccessor.week`
have been deprecated. Use ``DataArray.dt.isocalendar().week``
instead (:pull:`4534`). By `Mathias Hauser `_.
`Maximilian Roos `_, and `Spencer Clark `_.
- :py:attr:`DataArray.rolling` and :py:attr:`Dataset.rolling` no longer support passing ``keep_attrs``
via its constructor. Pass ``keep_attrs`` via the applied function, i.e. use
``ds.rolling(...).mean(keep_attrs=False)`` instead of ``ds.rolling(..., keep_attrs=False).mean()``
Rolling operations now keep their attributes per default (:pull:`4510`).
By `Mathias Hauser `_.
New Features
~~~~~~~~~~~~
- :py:func:`open_dataset` and :py:func:`open_mfdataset`
now works with ``engine="zarr"`` (:issue:`3668`, :pull:`4003`, :pull:`4187`).
By `Miguel Jimenez `_ and `Wei Ji Leong `_.
- Unary & binary operations follow the ``keep_attrs`` flag (:issue:`3490`, :issue:`4065`, :issue:`3433`, :issue:`3595`, :pull:`4195`).
By `Deepak Cherian `_.
- Added :py:meth:`~core.accessor_dt.DatetimeAccessor.isocalendar()` that returns a Dataset
with year, week, and weekday calculated according to the ISO 8601 calendar. Requires
pandas version 1.1.0 or greater (:pull:`4534`). By `Mathias Hauser `_,
`Maximilian Roos `_, and `Spencer Clark `_.
- :py:meth:`Dataset.to_zarr` now supports a ``region`` keyword for writing to
limited regions of existing Zarr stores (:pull:`4035`).
See :ref:`io.zarr.appending` for full details.
By `Stephan Hoyer `_.
- Added typehints in :py:func:`align` to reflect that the same type received in ``objects`` arg will be returned (:pull:`4522`).
By `Michal Baumgartner `_.
- :py:meth:`Dataset.weighted` and :py:meth:`DataArray.weighted` are now executing value checks lazily if weights are provided as dask arrays (:issue:`4541`, :pull:`4559`).
By `Julius Busecke `_.
- Added the ``keep_attrs`` keyword to ``rolling_exp.mean()``; it now keeps attributes
per default. By `Mathias Hauser `_ (:pull:`4592`).
- Added ``freq`` as property to :py:class:`CFTimeIndex` and into the
``CFTimeIndex.repr``. (:issue:`2416`, :pull:`4597`)
By `Aaron Spring `_.
Bug fixes
~~~~~~~~~
- Fix bug where reference times without padded years (e.g. ``since 1-1-1``) would lose their units when
being passed by ``encode_cf_datetime`` (:issue:`4422`, :pull:`4506`). Such units are ambiguous
about which digit represents the years (is it YMD or DMY?). Now, if such formatting is encountered,
it is assumed that the first digit is the years, they are padded appropriately (to e.g. ``since 0001-1-1``)
and a warning that this assumption is being made is issued. Previously, without ``cftime``, such times
would be silently parsed incorrectly (at least based on the CF conventions) e.g. "since 1-1-1" would
be parsed (via ``pandas`` and ``dateutil``) to ``since 2001-1-1``.
By `Zeb Nicholls `_.
- Fix :py:meth:`DataArray.plot.step`. By `Deepak Cherian `_.
- Fix bug where reading a scalar value from a NetCDF file opened with the ``h5netcdf`` backend would raise a ``ValueError`` when ``decode_cf=True`` (:issue:`4471`, :pull:`4485`).
By `Gerrit Holl `_.
- Fix bug where datetime64 times are silently changed to incorrect values if they are outside the valid date range for ns precision when provided in some other units (:issue:`4427`, :pull:`4454`).
By `Andrew Pauling `_
- Fix silently overwriting the ``engine`` key when passing :py:func:`open_dataset` a file object
to an incompatible netCDF (:issue:`4457`). Now incompatible combinations of files and engines raise
an exception instead. By `Alessandro Amici `_.
- The ``min_count`` argument to :py:meth:`DataArray.sum()` and :py:meth:`DataArray.prod()`
is now ignored when not applicable, i.e. when ``skipna=False`` or when ``skipna=None``
and the dtype does not have a missing value (:issue:`4352`).
By `Mathias Hauser `_.
- :py:func:`combine_by_coords` now raises an informative error when passing coordinates
with differing calendars (:issue:`4495`). By `Mathias Hauser `_.
- :py:attr:`DataArray.rolling` and :py:attr:`Dataset.rolling` now also keep the attributes and names of of (wrapped)
``DataArray`` objects, previously only the global attributes were retained (:issue:`4497`, :pull:`4510`).
By `Mathias Hauser `_.
- Improve performance where reading small slices from huge dimensions was slower than necessary (:pull:`4560`). By `Dion Häfner `_.
- Fix bug where ``dask_gufunc_kwargs`` was silently changed in :py:func:`apply_ufunc` (:pull:`4576`). By `Kai Mühlbauer `_.
Documentation
~~~~~~~~~~~~~
- document the API not supported with duck arrays (:pull:`4530`).
By `Justus Magin `_.
- Mention the possibility to pass functions to :py:meth:`Dataset.where` or
:py:meth:`DataArray.where` in the parameter documentation (:issue:`4223`, :pull:`4613`).
By `Justus Magin `_.
- Update the docstring of :py:class:`DataArray` and :py:class:`Dataset`.
(:pull:`4532`);
By `Jimmy Westling `_.
- Raise a more informative error when :py:meth:`DataArray.to_dataframe` is
is called on a scalar, (:issue:`4228`);
By `Pieter Gijsbers `_.
- Fix grammar and typos in the :doc:`contributing` guide (:pull:`4545`).
By `Sahid Velji `_.
- Fix grammar and typos in the :doc:`user-guide/io` guide (:pull:`4553`).
By `Sahid Velji `_.
- Update link to NumPy docstring standard in the :doc:`contributing` guide (:pull:`4558`).
By `Sahid Velji `_.
- Add docstrings to ``isnull`` and ``notnull``, and fix the displayed signature
(:issue:`2760`, :pull:`4618`).
By `Justus Magin `_.
Internal Changes
~~~~~~~~~~~~~~~~
- Optional dependencies can be installed along with xarray by specifying
extras as ``pip install "xarray[extra]"`` where ``extra`` can be one of ``io``,
``accel``, ``parallel``, ``viz`` and ``complete``. See docs for updated
:ref:`installation instructions `.
(:issue:`2888`, :pull:`4480`).
By `Ashwin Vishnu `_, `Justus Magin
`_ and `Mathias Hauser
`_.
- Removed stray spaces that stem from black removing new lines (:pull:`4504`).
By `Mathias Hauser `_.
- Ensure tests are not skipped in the ``py38-all-but-dask`` test environment
(:issue:`4509`). By `Mathias Hauser `_.
- Ignore select numpy warnings around missing values, where xarray handles
the values appropriately, (:pull:`4536`);
By `Maximilian Roos `_.
- Replace the internal use of ``pd.Index.__or__`` and ``pd.Index.__and__`` with ``pd.Index.union``
and ``pd.Index.intersection`` as they will stop working as set operations in the future
(:issue:`4565`). By `Mathias Hauser