Changelog#

v2.9.0#

Highlights#

This release includes

Bug fixes#

  • Fixed usage of work_dir in some CMORizer scripts (#3192) Rémi Kazeroni

  • Realize data for scalar cube in recipe_carvalhais14nat to avert issue from dask latest (2023.6.0) (#3265) Valeriu Predoi

  • Fix failing mlr diagnostic test by adding new scikit-learn default tag (#3273) Rémi Kazeroni

  • Fix ordering of models in perfmetrics diagnostic script (#3275) Lisa Bock

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

  • Add a CircleCI-testing-specific recipe_python_for_CI.yml to avoid calling geolocator/Nominatim over CI (#3159) Valeriu Predoi

  • Check if Python minor version changed after Julia install in development installation test (#3213) Valeriu Predoi

  • Fix tests using deprecated esmvalcore._config module that has been removed in ESMValCore v2.9 (#3204) Valeriu Predoi

Installation#

Improvements#

v2.8.0#

Highlights#

Highlights from ESMValCore v2.8.0 here:

  • ESMValCore now supports wildcards in recipes and offers improved support for ancillary variables and dataset versioning.

  • Support for CORDEX datasets in a rotated pole coordinate system has been added.

  • Native ICON output is now made UGRID-compliant on-the-fly.

  • The Python API has been extended with the addition of three modules: esmvalcore.config, esmvalcore.dataset, and esmvalcore.local

  • The preprocessor multi_model_statistics() has been extended to support more use-cases.

This release includes:

Backwards incompatible changes#

Please read the descriptions of the linked pull requests for detailed upgrade instructions.

  • Deprecated features scheduled for removal in v2.8.0 or earlier have now been removed (#2941) Manuel Schlund. Removed esmvaltool.iris_helpers.var_name_constraint (has been deprecated in v2.6.0; please use iris.NameConstraint with the keyword argument var_name instead). Removed write_netcdf and write_plots from recipe_filer.py.

  • No files from the native6 project will be found if a non-existent version of a dataset is specified (#3041) Rémi Kazeroni. The tool now searches for exact version of native6 datasets. Therefore, it is necessary to make sure that the version number in the directory tree matches with the version number in the recipe to find the files.

  • The conversion of precipitation units from monitoring diagnostic is now done at the preprocessor stage (#3049) Manuel Schlund. To use the unit conversion for precipitation in the new version of this diagnostic, add it as a preprocessor for the precipitation dataset to the recipe.

Bug fixes#

Community#

Deprecations#

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

Installation#

Improvements#

v2.7.0#

Highlights#

  • This release has seen the inclusion of the code for figures 3.3, 3.4, 3.5, 3,13 and 3.15 of the IPCC AR6 WG1 report, see them in the new documentation

  • We have also included new diagnostics and recipe necessary to produce the plots and tables for the journal article “Climate model projections from the Scenario Model Intercomparison Project (ScenarioMIP) of CMIP6” by Tebaldi et al. in ESD 2020-68 from 2021; also see the recipe entry

  • We have also extended the support for MERRA2 observational dataset, by adding support for a large number of variables, including 3D variables, see the table of supported obs datasets

Backwards incompatible changes#

Bug fixes#

Community#

Deprecations#

  • Recipe recipe_esacci_oc.yml replace with new regrid scheme nearest_extrapolate (#2841) Valeriu Predoi

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

  • Pin netcdf4 != 1.6.1 since that is spitting large numbers of SegFaults (#2796) Valeriu Predoi

Installation#

Improvements#

v2.6.0#

Highlights#

Please also note the highlights from the corresponding ESMValCore release here. Thanks to that ESMValTool has gained the following features:

  • A new set of CMOR fixes is now available in order to load native EMAC model output and CMORize it on the fly.

  • The version number of ESMValCore is now automatically generated using setuptools_scm, which extracts Python package versions from git metadata.

This release includes

Bug fixes#

Community#

  • Collecting github user names for config-references (#2677) Lukas

Deprecations#

  • Deprecate the function esmvaltool.diag_scripts.shared.var_name_constraint. This function is scheduled for removal in v2.8.0. Please use iris.NameConstraint with the keyword argument var_name instead: this is an exact replacement. (#2655) Manuel Schlund

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

Installation#

Improvements#

  • Improved handling of weights in MLR diagnostics (#2625) Manuel Schlund

  • Fixed order of variables in perfemetrics plot of Anav13jclim recipe (#2706) Manuel Schlund

  • Added input file sorting to many diagnostic to make output exactly reproducible (#2710) Manuel Schlund

  • Removed ‘ancestors’ attributes before saving netcdf files in emergent constraints diagnostics (#2713) Manuel Schlund

v2.5.0#

Highlights#

Please also note the highlights from the corresponding ESMValCore release here. Thanks to that ESMValTool has gained the following features:

  • The new preprocessor extract_location can extract arbitrary locations on the Earth.

  • Time ranges can now be extracted using the ISO 8601 format.

  • The new preprocessor ensemble_statistics can calculate arbitrary statistics over all ensemble members of a simulation.

This release includes

Backwards incompatible changes#

  • Streamline observations download (#1657) Javier Vegas-Regidor. This change removes the cmorize_obs command which has previously been used to CMORize observations and other datasets. The new command esmvaltool data provides many new features apart from the CMORization (esmvaltool data format), for example, automatic downloading of observational datasets (esmvaltool data download). More details on this can be found here and here.

  • Dropped Python 3.7 (#2585) Manuel Schlund. ESMValTool v2.5.0 dropped support for Python 3.7. From now on Python >=3.8 is required to install ESMValTool. The main reason for this is that conda-forge dropped support for Python 3.7 for OSX and arm64 (more details are given here).

Bug fixes#

Deprecations#

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

  • Ensure dummy data for cmorize_obs_woa test are written to the correct directory (#2451) Emma Hogan

  • Add ESA CCI LST cmorizer and diagnostic (see previous section Diagnostics)

Automatic testing#

  • Run a nightly Github Actions workflow to monitor tests memory per test (configurable for other metrics too) and lists the slowest 100 tests (#2449) Valeriu Predoi

  • Fix individual pytest runs broken due to missing explicit imports from iris and adding a couple missing package markers (#2455) Valeriu Predoi

  • Add Python=3.10 to Github Actions and switch to Python=3.10 for the Github Action that builds the PyPi package (#2488) Valeriu Predoi

  • Switch all github actions from miniconda to mambaforge (#2498) Klaus Zimmermann

  • Pin flake8<4 to have actual FLAKE8 error printed if tests fail and not garbage (#2492) Valeriu Predoi

  • Implementing conda lock (#2193) Valeriu Predoi

  • [Docker] Update Docker container builds with correct installations of Julia (#2530) Valeriu Predoi

  • Update Linux condalock file (various pull requests) github-actions[bot]

Installation#

Improvements#

v2.4.0#

Highlights#

  • ESMValTool is moving from Conda to Mamba as the preferred installation method. This will speed up the installation and comes with some improvements behind the scenes. Read more about it at Move to Mamba and in the installation guide.

Please also note the highlights from the corresponding ESMValCore release here. Thanks to that ESMValTool has gained the following features:

  • Download any missing data that is available on the ESGF automatically.

  • Resume previous runs, reusing expensive pre-processing results.

This release includes

Bug fixes#

Deprecations#

  • Removed write_plots and write_netcdf from some NCL diagnostics (#2293) Manuel Schlund

  • Fixed provenance logging of all python diagnostics by removing ‘plot_file’ entry (#2296) Manuel Schlund

  • Do not deprecate classes Variable, Variables and Datasets on a specific version (#2286) Manuel Schlund

  • Remove obsolete write_netcdf option from ncl diagnostic scripts (#2387) Klaus Zimmermann

  • Remove write plots from ocean diagnostics (#2393) Valeriu Predoi

  • More removals of instances of write_plots from Python diagnostics (appears to be the final removal from Py diags) (#2394) Valeriu Predoi

Documentation#

Diagnostics#

  • Add all available CMIP5 and CMIP6 models to recipe_impact.yml (#2251) Bouwe Andela

  • Add Fig. 6, 7 and 9 of Bock20jgr (#2252) Lisa Bock

  • Generalize recipe_validation* diagnostic to work with identical control and experiment dataset names (#2284) Valeriu Predoi

  • Add missing preprocessor to recipe_gier2020bg and adapt to available data (#2399) Bettina Gier

  • Removed custom version of AtmosphereSigmaFactory in diagnostics (#2405) Manuel Schlund

Observational and re-analysis dataset support#

Automatic testing#

Installation#

Improvements#

v2.3.0#

This release includes

Bug fixes#

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

Installation#

Improvements#

v2.2.0#

Highlights#

ESMValTool is now using the recently released Iris 3. We acknowledge that this change may impact your work, as Iris 3 introduces several changes that are not backward-compatible, but we think that moving forward is the best decision for the tool in the long term.

This release includes

Bug fixes#

Deprecations#

  • Deprecate classes Variable, Variables and Datasets (#1944) Manuel Schlund

  • Python 3.9: remove pynio as dependency and replace with rasterio and pin Matplotlib>3.3.1 and pin cartopy>=0.18 (#1997) Valeriu Predoi

  • Removed write_plots and write_netcdf in some python diagnostics (#2036) Manuel Schlund

Documentation#

Diagnostics#

Observational and re-analysis dataset support#

Automatic testing#

Installation#

Improvements#

v2.1.1#

This release includes

Improvements#

Documentation#

v2.1.0#

This release includes

Diagnostics#

Documentation#

Improvements#

Observational and re-analysis dataset support#

v2.0.0#

This release includes

Bug fixes#

Diagnostics#

Documentation#

Improvements#

Observational and re-analysis dataset support#

v2.0.0b4#

This release includes

Bug fixes#

Diagnostics#

Documentation#

Improvements#

Observational and re-analysis dataset support#

For older releases, see the release notes on ESMValGroup/ESMValTool.