Changelog#
v2.9.0#
Highlights#
It is now possible to use the Dask distributed scheduler, which can significantly reduce the run-time of recipes. Configuration examples and advice are available in our documentation. More work on improving the computational performance is planned, so please share your experiences, good and bad, with this new feature in ESMValGroup/ESMValCore#1763.
This release includes
Backwards incompatible changes#
Remove deprecated configuration options (#2056) Bouwe Andela
The module
esmvalcore.experimental.config
has been removed. To upgrade, import the module fromesmvalcore.config
instead.The module
esmvalcore._config
has been removed. To upgrade, useesmvalcore.config
instead.The methods
esmvalcore.config.Session.to_config_user
andesmvalcore.config.Session.from_config_user
have been removed. To upgrade, useesmvalcore.config.Session
to access the configuration values directly.
Bug fixes#
Respect
ignore_warnings
settings from the project configuration inesmvalcore.dataset.Dataset.load()
(#2046) Manuel SchlundFixed usage of custom location for custom CMOR tables (#2052) Manuel Schlund
Fix issue with writing index.html when running a recipe with
--resume-from
(#2055) Bouwe AndelaFixed bug in ICON CMORizer that lead to shifted time coordinates (#2038) Manuel Schlund
Computational performance improvements#
Add support for configuring Dask distributed (#2049) Bouwe Andela
Make
esmvalcore.preprocessor.extract_levels()
lazy (#1761) Bouwe AndelaLazy implementation of
esmvalcore.preprocessor.multi_model_statistics()
andesmvalcore.preprocessor.ensemble_statistics()
(#968 and #2087) Peter Kalverla
Documentation#
Remove unneeded sphinxcontrib extension (#2047) Valeriu Predoi
Show ESMValTool logo on PyPI webpage (#2065) Valeriu Predoi
Fixes for datasets#
Pass the
esmvalcore.config.Session
to fixes (#1988) Manuel SchlundICON: Allowed specifying vertical grid information in recipe (#2067) Manuel Schlund
Allow specifying
raw_units
for CESM2, EMAC, and ICON CMORizers (#2043) Manuel SchlundICON: allow specifying horizontal grid file in recipe/extra facets (#2078) Manuel Schlund
Installation#
Drop support for Python 3.8 (#2053) Bouwe Andela
Add python 3.11 to Github Actions package (conda and PyPI) installation tests (#2083) Valeriu Predoi
Preprocessor#
Added
period='hourly'
foresmvalcore.preprocessor.climate_statistics()
andesmvalcore.preprocessor.anomalies()
(#2068) Manuel SchlundSupport IPCC AR6 regions in
esmvalcore.preprocessor.extract_shape()
(#2008) Manuel SchlundAutomatically enable
use_src_mask
parameter inesmvalcore.preprocessor.regrid()
for datasets with grid discontinuities (#2070) sloosvel
v2.8.1#
Highlights#
This release adds support for Python 3.11 and includes several bugfixes.
This release includes:
Bug fixes#
Pin numpy !=1.24.3 (#2011) Valeriu Predoi
Fix a bug in recording provenance for the
mask_multimodel
preprocessor (#1984) Manuel SchlundFix ICON hourly data rounding issues (#2022) Julian Bauer
Use the default SSL context when using the
extract_location
preprocessor (#2023) Emma HoganMake time-related CMOR fixes work with time dimensions time1, time2, time3 (#1971) Manuel Schlund
Always create a cache directory for storing ICON grid files (#2030) Manuel Schlund
Fixed altitude <–> pressure level conversion for masked arrays in the
extract_levels
preprocessor (#1999) Manuel SchlundAllowed ignoring of scalar time coordinates in the
multi_model_statistics
preprocessor (#1961) Manuel Schlund
Fixes for datasets#
Add support for hourly ICON data (#1990) Julian Bauer
Fix areacello in BCC-CSM2-MR (#1993) Rémi Kazeroni
Installation#
Add support for Python=3.11 (#1832) Valeriu Predoi
Modernize conda lock file creation workflow with mamba, Mambaforge etc (#2027) Valeriu Predoi
Pin libnetcdf!=4.9.1 (#2072) Rémi Kazeroni
Documentation#
Add changelog for v2.8.1 (#2079) Bouwe Andela
Automatic testing#
Use mocked geopy.geocoders.Nominatim to avoid ReadTimeoutError (#2005) Manuel Schlund
Update pre-commit hooks (#2020) Bouwe Andela
v2.8.0#
Highlights#
ESMValCore now supports wildcards in recipes and offers improved support for ancillary variables and dataset versioning thanks to contributions by Bouwe Andela. For details, see Automatically populating a recipe with all available datasets and Defining supplementary variables.
Support for CORDEX datasets in a rotated pole coordinate system has been added by sloosvel.
Native ICON output is now made UGRID-compliant on-the-fly to unlock the use of more sophisticated regridding algorithms, thanks to Manuel Schlund.
The Python API has been extended with the addition of three modules:
esmvalcore.config
,esmvalcore.dataset
, andesmvalcore.local
, all these features courtesy of Bouwe Andela. For details, see our new example Example notebooks.The preprocessor
multi_model_statistics()
has been extended to support more use-cases thanks to contributions by Manuel Schlund. For details, see Multi-model statistics.
This release includes:
Backwards incompatible changes#
Please read the descriptions of the linked pull requests for detailed upgrade instructions.
The algorithm for automatically defining the ancillary variables and cell measures has been improved (#1609) Bouwe Andela. If this does not work as expected, more examples of how to adapt your recipes are given here and in the corresponding sections of the recipe documentation and the preprocessor documentation.
Remove deprecated features scheduled for removal in v2.8.0 or earlier (#1826) Manuel Schlund. Removed
esmvalcore.iris_helpers.var_name_constraint
(has been deprecated in v2.6.0; please useiris.NameConstraint
with the keyword argumentvar_name
instead) and the optionalways_use_ne_mask
foresmvalcore.preprocessor.mask_landsea()
(has been deprecated in v2.5.0; the same behavior can now be achieved by specifyingsupplementary_variables
.No files will be found if a non-existent version of a dataset is specified (#1835) Bouwe Andela. If a
version
of a dataset is specified in the recipe, the tool will now search for exactly that version, instead of simply using the latest version. Therefore, it is necessary to make sure that the version number in the directory tree matches with the version number in the recipe to find the files.The default filename template for obs4MIPs has been updated to better match filenames used in this project in (#1866) Bouwe Andela. This may cause issues if you are storing all the files for obs4MIPs in a directory with no subdirectories per dataset.
Deprecations#
Please read the descriptions of the linked pull requests for detailed upgrade instructions.
Various configuration related options that are now available through
esmvalcore.config
have been deprecated (#1769) Bouwe Andela.The
fx_variables
preprocessor argument and related features have been deprecated (#1609) Bouwe Andela. See here for more information.Combined
offline
andalways_search_esgf
into a single optionsearch_esgf
(#1935) Manuel Schlund. The configuration option/command line argumentoffline
has been deprecated in favor ofsearch_esgf
. The previousoffline: true
is nowsearch_esgf: never
(the default); the previousoffline: false
is nowsearch_esgf: when_missing
. More details on how to adapt your workflow regarding these new options are given in #1935 and the documentation.esmvalcore.preprocessor.cleanup()
has been deprecated (#1949) Manuel Schlund. Please do not use this anymore in the recipe (it is not necessary).
Python API#
Support searching ESGF for a specific version of a file and add
esmvalcore.esgf.ESGFFile.facets
(#1822) Bouwe AndelaFix issues with searching for files on ESGF (#1863) Bouwe Andela
Move the
esmvalcore.experimental.config
module toesmvalcore.config
(#1769) Bouwe AndelaAdd
esmvalcore.local
, a module to search data on the local filesystem (#1835) Bouwe AndelaAdd
esmvalcore.dataset
module (#1877) Bouwe Andela
Bug fixes#
Import from
esmvalcore.config
in theesmvalcore.experimental
module (#1816) Bouwe AndelaAdded scalar coords of input cubes to output of esmpy_regrid (#1811) Manuel Schlund
Fix severe bug in
esmvalcore.preprocessor.mask_fillvalues()
(#1823) Manuel SchlundFix LWP of ICON on-the-fly CMORizer (#1839) Manuel Schlund
Fixed issue in irregular regridding regarding scalar coordinates (#1845) Manuel Schlund
Update product attributes and metadata.yml with cube metadata before saving files (#1837) Manuel Schlund
Remove an extra space character from a filename (#1883) Bouwe Andela
Improve resilience of ESGF search (#1869) Bouwe Andela
Fix issue with no files found if timerange start/end differs in length (#1880) Bouwe Andela
Add driver and sub_experiment tags to generate dataset aliases (#1886) sloosvel
Fixed time points of native CESM2 output (#1772) Manuel Schlund
Fix type hints for Python versions < 3.10 (#1897) Bouwe Andela
Fixed set_range_in_0_360 for dask arrays (#1919) Manuel Schlund
Made equalized attributes in concatenated cubes consistent across runs (#1783) Manuel Schlund
Fix issue with reading dates from files (#1936) Bouwe Andela
Add institute name used on ESGF for CMIP5 CanAM4, CanCM4, and CanESM2 (#1937) Bouwe Andela
Fix issue where data was not loaded and saved (#1962) Bouwe Andela
Fix type hints for Python 3.8 (#1795) Bouwe Andela
Update the institute facet of the CSIRO-Mk3L-1-2 model (#1966) Rémi Kazeroni
Fixed race condition that may result in errors in
esmvalcore.preprocessor.cleanup()
(#1949) Manuel SchlundUpdate notebook so it uses supplementaries instead of ancillaries (#1945) Bouwe Andela
Documentation#
Fix anaconda badge in README (#1759) Valeriu Predoi
Fix mistake in the documentation of
esmvalcore.esgf.find_files
(#1784) Bouwe AndelaSupport linking to “stable” ESMValTool version on readthedocs (#1608) Bouwe Andela
Updated ICON doc with information on usage of extract_levels preprocessor (#1903) Manuel Schlund
Add changelog for latest released version v2.7.1 (#1905) Valeriu Predoi
Update preprocessor.rst due to renaming of NCEP dataset to NCEP-NCAR-R1 (#1908) Birgit Hassler
Replace timerange nested lists in docs with overview table (#1940) Klaus Zimmermann
Updated section “backward compatibility” in contributing.rst (#1918) Axel Lauer
Add link to ESMValTool release procedure steps (#1957) Rémi Kazeroni
Synchronize documentation table of contents with ESMValTool (#1958) Bouwe Andela
Improvements#
Support wildcards in the recipe and improve support for ancillary variables and dataset versioning (#1609) Bouwe Andela. More details on how to adapt your recipes are given in the corresponding pull request description and in the corresponding sections of the recipe documentation and the preprocessor documentation.
Create a session directory with suffix “-1”, “-2”, etc if it already exists (#1818) Bouwe Andela
Message for users when they use esmvaltool executable from esmvalcore only (#1831) Valeriu Predoi
Order recipe output in index.html (#1899) Bouwe Andela
Improve reading facets from ESGF search results (#1920) Bouwe Andela
Fixes for datasets#
Fix rotated coordinate grids and tas and pr for CORDEX datasets (#1765) sloosvel
Made ICON output UGRID-compliant (on-the-fly) (#1664) Manuel Schlund
Fix automatic download of ICON grid file and make ICON UGRIDization optional (default: true) (#1922) Manuel Schlund
Add siconc fixes for EC-Earth3-Veg and EC-Earth3-Veg-LR models (#1771) Evgenia Galytska
Extension of ERA5 CMORizer (variable cl) (#1850) Axel Lauer
Add standard variable names for EMAC (#1853) FranziskaWinterstein
Installation#
Add all deps to the conda-forge environment and suppress installing and reinstalling deps with pip at readthedocs builds (#1786) Valeriu Predoi
Pin netCDF4<1.6.1 (#1805) Bouwe Andela
Unpin NetCF4 (#1814) Valeriu Predoi
Unpin flake8 (#1820) Valeriu Predoi
Pin esmpy<8.4 (#1871) Klaus Zimmermann
Update esmpy import for ESMF v8.4.0 (#1876) Bouwe Andela
Preprocessor#
Allow
esmvalcore.preprocessor.multi_model_statistics()
on cubes with arbitrary dimensions (#1808) Manuel SchlundSmarter removal of coordinate metadata in
esmvalcore.preprocessor.multi_model_statistics()
preprocessor (#1813) Manuel SchlundAllowed usage of
esmvalcore.preprocessor.multi_model_statistics()
on single cubes/products (#1849) Manuel SchlundAllowed usage of
esmvalcore.preprocessor.multi_model_statistics()
on cubes with identicalname()
andunits
(but e.g. different long_name) (#1921) Manuel SchlundAllowed ignoring scalar coordinates in
esmvalcore.preprocessor.multi_model_statistics()
(#1934) Manuel SchlundRefactored
esmvalcore.preprocessor.regrid()
and removed unnecessary code not needed anymore due to new iris version (#1898) Manuel SchlundDo not realise coordinates during CMOR check (#1912) sloosvel
Make
esmvalcore.preprocessor.extract_volume()
work with closed and mixed intervals and allow nearest value selection (#1930) sloosvel
Release#
Changelog for v2.8.0rc1 (#1952) Rémi Kazeroni
Increase version number for ESMValCore v2.8.0rc1 (#1955) Rémi Kazeroni
Changelog for v2.8.0rc2 (#1959) Rémi Kazeroni
Increase version number for ESMValCore v2.8.0rc2 (#1973) Rémi Kazeroni
Changelog for v2.8.0 (#1978) Rémi Kazeroni
Increase version number for ESMValCore v2.8.0 (#1983) Rémi Kazeroni
Automatic testing#
Set implicit optional to true in mypy config to avert side effects and test fails from new mypy version (#1790) Valeriu Predoi
Remove duplicate implicit_optional = True line in
setup.cfg
(#1791) Valeriu PredoiFix failing test due to missing sample data (#1797) Bouwe Andela
Remove outdated cmor_table facet from data finder tests (#1798) Bouwe Andela
Modernize tests for
esmvalcore.preprocessor.save()
(#1799) Bouwe AndelaNo more sequential tests since SegFaults were not noticed anymore (#1819) Valeriu Predoi
Update pre-commit configuration (#1821) Bouwe Andela
Updated URL of ICON grid file used for testing (#1914) Manuel Schlund
Variable Derivation#
v2.7.1#
Highlights#
This is a bugfix release where we unpin cf-units to allow the latest iris=3.4.0 to be installed. It also includes an update to the default configuration used when searching the ESGF for files, to account for a recent change of the CEDA ESGF index node hostname. The changelog contains only changes that were made to the main
branch.
Installation#
Set the version number on the development branches to one minor version more than the previous release (#1854) Bouwe Andela
Unpin cf-units (#1770) Bouwe Andela
Bug fixes#
Improve error handling if an esgf index node is offline (#1834) Bouwe Andela
Automatic testing#
Removed unnecessary test that fails with iris 3.4.0 (#1846) Manuel Schlund
Update CEDA ESGF index node hostname (#1838) Valeriu Predoi
v2.7.0#
Highlights#
We have a new preprocessor function called ‘rolling_window_statistics’ implemented by Liza Malinina
We have improved the support for native models, refactored native model fixes by adding common base class NativeDatasetFix, changed default DRS for reading native ICON output, and added tests for input/output filenames for ICON and EMAC on-the-fly CMORizer, all these features courtesy of Manuel Schlund
Performance of preprocessor functions that use time dimensions has been sped up by two orders of magnitude thanks to contributions by Bouwe Andela
This release includes:
Backwards incompatible changes#
Change default DRS for reading native ICON output (#1705) Manuel Schlund
Bug fixes#
Add support for regions stored as MultiPolygon to extract_shape preprocessor (#1670) Bouwe Andela
Fixed type annotations for Python 3.8 (#1700) Manuel Schlund
Core _io.concatenate() may fail due to case when one of the cubes is scalar - this fixes that (#1715) Valeriu Predoi
Pick up esmvalcore badge instead of esmvaltool one in README (#1749) Valeriu Predoi
Restore support for scalar cubes to time selection preprocessor functions (#1750) Bouwe Andela
Fix calculation of precipitation flux in EMAC on-the-fly CMORizer (#1755) Manuel Schlund
Deprecations#
Remove deprecation warning for regrid schemes already deprecated for v2.7.0 (#1753) Valeriu Predoi
Documentation#
Add Met Office Installation Method (#1692) mo-tgeddes
Add MO-paths to config file (#1709) mo-tgeddes
Update MO obs4MIPs paths in the user configuration file (#1734) mo-tgeddes
Update Making a release section of the documentation (#1689) sloosvel
Added changelog for v2.7.0 (#1746) Valeriu Predoi
update CITATION.cff file with 2.7.0 release info (#1757) Valeriu Predoi
Improvements#
New preprocessor function ‘rolling_window_statistics’ (#1702) Liza Malinina
Remove pytest_flake8 plugin and use flake8 instead (#1722) Valeriu Predoi
Added CESM2 CMORizer (#1678) Manuel Schlund
Speed up functions that use time dimension (#1713) Bouwe Andela
Modernize and minimize pylint configuration (#1726) Bouwe Andela
Fixes for datasets#
Refactored native model fixes by adding common base class NativeDatasetFix (#1694) Manuel Schlund
Installation#
Pin netCDF4 != 1.6.1 since that seems to throw a flurry of Segmentation Faults (#1724) Valeriu Predoi
Automatic testing#
Pin flake8<5.0.0 since Circle CI tests are failing copiously (#1698) Valeriu Predoi
Added tests for input/output filenames for ICON and EMAC on-the-fly CMORizer (#1718) Manuel Schlund
Fix failed tests for Python<3.10 resulting from typing (#1748) Manuel Schlund
v2.6.0#
Highlights#
A new set of CMOR fixes is now available in order to load native EMAC model output and CMORize it on the fly. For details, see Supported native models: EMAC.
The version number of ESMValCore is now automatically generated using setuptools_scm, which extracts Python package versions from git metadata.
This release includes
Deprecations#
Deprecate the function esmvalcore.var_name_constraint (#1592) Manuel Schlund. This function is scheduled for removal in v2.8.0. Please use
iris.NameConstraint
with the keyword argument var_name instead: this is an exact replacement.
Bug fixes#
Added start_year and end_year attributes to derived variables (#1547) Manuel Schlund
Show all results on recipe results webpage (#1560) Bouwe Andela
Regridding regular grids with similar coordinates (#1567) Tomas Lovato
Fix timerange wildcard search when deriving variables or downloading files (#1562) sloosvel
Apply clip_timerange to time dependent fx variables (#1603) sloosvel
Correctly handle requests.exceptions.ConnectTimeout when an ESGF index node is offline (#1638) Bouwe Andela
CMOR standard#
Added custom CMOR tables used for EMAC CMORizer (#1599) Manuel Schlund
Extended ICON CMORizer (#1549) Manuel Schlund
Add CMOR check exception for a basin coord named sector (#1612) David Hohn
Custom user-defined location for custom CMOR tables (#1625) Manuel Schlund
Containerization#
Community#
Add David Hohn to contributors’ list (#1586) Valeriu Predoi
Documentation#
[Github Actions Docs] Full explanation on how to use the GA test triggered by PR comment and added docs link for GA hosted runners (#1553) Valeriu Predoi
Update the command for building the documentation (#1556) Bouwe Andela
Update documentation on running the tool (#1400) Bouwe Andela
Add support for DKRZ-Levante (#1558) Rémi Kazeroni
Improved documentation on native dataset support (#1559) Manuel Schlund
Tweak extract_point preprocessor: explain what it returns if one point coord outside cube and add explicit test (#1584) Valeriu Predoi
Update CircleCI, readthedocs, and Docker configuration (#1588) Bouwe Andela
Remove support for Mistral in config-user.yml (#1620) Rémi Kazeroni
Add a note on transferring permissions to the release manager (#1645) Bouwe Andela
Add documentation on building and uploading Docker images (#1644) Bouwe Andela
Update documentation on ESMValTool module at DKRZ (#1647) Rémi Kazeroni
Expanded information on deprecations in changelog (#1658) Manuel Schlund
Improvements#
Removed trailing whitespace in custom CMOR tables (#1564) Manuel Schlund
Try searching multiple ESGF index nodes (#1561) Bouwe Andela
Add CMIP6 amoc derivation case and add a test (#1577) Valeriu Predoi
Added EMAC CMORizer (#1554) Manuel Schlund
Fixes for datasets#
Fixes of ocean variables in multiple CMIP6 datasets (#1566) Tomas Lovato
Ensure lat/lon bounds in FGOALS-l3 atmos variables are contiguous (#1571) sloosvel
Added AllVars fix for CMIP6’s ICON-ESM-LR (#1582) Manuel Schlund
Installation#
Removed package/meta.yml (#1540) Manuel Schlund
Pinned iris>=3.2.1 (#1552) Manuel Schlund
Use setuptools-scm to automatically generate the version number (#1578) Bouwe Andela
Pin cf-units to lower than 3.1.0 to temporarily avoid changes within new version related to calendars (#1659) Valeriu Predoi
Preprocessor#
Allowed special case for unit conversion of precipitation (kg m-2 s-1 <–> mm day-1) (#1574) Manuel Schlund
Add general extract_coordinate_points preprocessor (#1581) sloosvel
Add preprocessor accumulate_coordinate (#1281) Javier Vegas-Regidor
Add axis_statistics and improve depth_integration (#1589) sloosvel
Release#
Automatic testing#
Refresh CircleCI cache weekly (#1597) Bouwe Andela
Use correct cache restore key on CircleCI (#1598) Bouwe Andela
Install git and ssh before checking out code on CircleCI (#1601) Bouwe Andela
Test Github Actions dashboard badge from meercode.io (#1640) Valeriu Predoi
Improve esmvalcore.esgf unit test (#1650) Bouwe Andela
Variable Derivation#
Added derivation of hfns (#1594) Manuel Schlund
v2.5.0#
Highlights#
The new preprocessor
extract_location()
can extract arbitrary locations on the Earth using the geopy package that connects to OpenStreetMap. For details, see Extract location.Time ranges can now be extracted using the ISO 8601 format. In addition, wildcards are allowed, which makes the time selection much more flexible. For details, see Recipe section: Datasets.
The new preprocessor
ensemble_statistics()
can calculate arbitrary statistics over all ensemble members of a simulation. In addition, the preprocessormulti_model_statistics()
now accepts the keywordgroupy
, which allows the calculation of multi-model statistics over arbitrary multi-model ensembles. For details, see Ensemble statistics and Multi-model statistics.
This release includes
Backwards incompatible changes#
Update Cordex section in config-developer.yml (#1303) francesco-cmcc. This changes the naming convention of ESMValCore’s output files from CORDEX dataset. This only affects recipes that use CORDEX data. Most likely, no changes in diagnostics are necessary; however, if code relies on the specific naming convention of files, it might need to be adapted.
Dropped Python 3.7 (#1530) Manuel Schlund. ESMValCore v2.5.0 dropped support for Python 3.7. From now on Python >=3.8 is required to install ESMValCore. The main reason for this is that conda-forge dropped support for Python 3.7 for OSX and arm64 (more details are given here).
Bug fixes#
Added support of extra_facets to fx variables added by the preprocessor (#1399) Manuel Schlund
Augmented input for derived variables with extra_facets (#1412) Manuel Schlund
Correctly use masked arrays after unstructured_nearest regridding (#1414) Manuel Schlund
Fixing the broken derivation script for XCH4 (and XCO2) (#1428) Birgit Hassler
Ignore .pymon-journal file in test discovery (#1436) Valeriu Predoi
Fixed bug that caused automatic download to fail in rare cases (#1442) Manuel Schlund
Add new JULIA_LOAD_PATH to diagnostic task test (#1444) Valeriu Predoi
Fix provenance file permissions (#1468) Bouwe Andela
Fixed usage of statistics=std_dev option in multi-model statistics preprocessors (#1478) Manuel Schlund
Removed scalar coordinates p0 and ptop prior to merge in multi_model_statistics (#1471) Axel Lauer
Added dataset and alias attributes to multi_model_statistics output (#1483) Manuel Schlund
Fixed issues with multi-model-statistics timeranges (#1486) Manuel Schlund
Fixed output messages for CMOR logging (#1494) Manuel Schlund
Fixed clip_timerange if only a single time point is extracted (#1497) Manuel Schlund
Fixed chunking in multi_model_statistics (#1500) Manuel Schlund
Fixed renaming of auxiliary coordinates in multi_model_statistics if coordinates are equal (#1502) Manuel Schlund
Fixed timerange selection for automatic downloads (#1517) Manuel Schlund
Fixed chunking in multi_model_statistics (#1524) Manuel Schlund
Deprecations#
Renamed vertical regridding schemes (#1429) Manuel Schlund. Old regridding schemes are supported until v2.7.0. For details, see Vertical interpolation schemes.
Documentation#
Remove duplicate entries in changelog (#1391) Klaus Zimmermann
Documentation on how to use HPC central installations (#1409) Valeriu Predoi
Correct brackets in preprocessor documentation for list of seasons (#1420) Bouwe Andela
Add Python=3.10 to package info, update Circle CI auto install and documentation for Python=3.10 (#1432) Valeriu Predoi
Reverted unintentional change in .zenodo.json (#1452) Manuel Schlund
Synchronized config-user.yml with version from ESMValTool (#1453) Manuel Schlund
Solved issues in configuration files (#1457) Manuel Schlund
Add direct link to download conda lock file in the install documentation (#1462) Valeriu Predoi
CITATION.cff fix and automatic validation of citation metadata (#1467) Valeriu Predoi
Updated documentation on how to deprecate features (#1426) Manuel Schlund
Added reference hook to conda lock in documentation install section (#1473) Valeriu Predoi
Increased ESMValCore version to 2.5.0rc1 (#1477) Manuel Schlund
Added changelog for v2.5.0 release (#1476) Manuel Schlund
Increased ESMValCore version to 2.5.0rc2 (#1487) Manuel Schlund
Added some authors to citation and zenodo files (#1488) SarahAlidoost
Restored scipy intersphinx mapping (#1491) Manuel Schlund
Increased ESMValCore version to 2.5.0rc3 (#1504) Manuel Schlund
Fix download instructions for the MSWEP dataset (#1506) Rémi Kazeroni
Documentation updated for the new cmorizer framework (#1417) Rémi Kazeroni
Added tests for duplicates in changelog and removed duplicates (#1508) Manuel Schlund
Increased ESMValCore version to 2.5.0rc4 (#1519) Manuel Schlund
Add Github Actions Test badge in README (#1526) Valeriu Predoi
Increased ESMValCore version to 2.5.0rc5 (#1529) Manuel Schlund
Increased ESMValCore version to 2.5.0rc6 (#1532) Manuel Schlund
Fixes for datasets#
Added fix for AIRS v2.1 (obs4mips) (#1472) Axel Lauer
Preprocessor#
Added bias preprocessor (#1406) Manuel Schlund
Improve error messages when a preprocessor is failing (#1408) Manuel Schlund
Added option to explicitly not use fx variables in preprocessors (#1416) Manuel Schlund
Add extract_location preprocessor to extract town, city, mountains etc - anything specifiable by a location (#1251) Javier Vegas-Regidor
Add ensemble statistics preprocessor and ‘groupby’ option for multimodel (#673) sloosvel
Generic regridding preprocessor (#1448) Klaus Zimmermann
Automatic testing#
Add pandas as dependency :panda_face: (#1402) Valeriu Predoi
Fixed tests for python 3.7 (#1410) Manuel Schlund
Remove accessing .xml() cube method from test (#1419) Valeriu Predoi
Remove flag to use pip 2020 solver from Github Action pip install command on OSX (#1357) Valeriu Predoi
Add Python=3.10 to Github Actions and switch to Python=3.10 for the Github Action that builds the PyPi package (#1430) Valeriu Predoi
Pin flake8<4 to keep getting relevant error traces when tests fail with FLAKE8 issues (#1434) Valeriu Predoi
Implementing conda lock (#1164) Valeriu Predoi
Relocate pytest-monitor outputted database .pymon so .pymon-journal file should not be looked for by pytest (#1441) Valeriu Predoi
Switch to Mambaforge in Github Actions tests (#1438) Valeriu Predoi
Turn off conda lock file creation on any push on main branch from Github Action test (#1489) Valeriu Predoi
Add DRS path test for IPSLCM files (#1490) Stéphane Sénési
Add a test module that runs tests of iris I/O every time we notice serious bugs there (#1510) Valeriu Predoi
[Github Actions] Trigger Github Actions tests (run-tests.yml workflow) from a comment in a PR (#1520) Valeriu Predoi
Update Linux condalock file (various pull requests) github-actions[bot]
Installation#
Move nested-lookup dependency to environment.yml to be installed from conda-forge instead of PyPi (#1481) Valeriu Predoi
Pinned iris (#1511) Manuel Schlund
Updated dependencies (#1521) Manuel Schlund
Pinned iris<3.2.0 (#1525) Manuel Schlund
Improvements#
Allow to load all files, first X years or last X years in an experiment (#1133) sloosvel
Filter tasks earlier (#1264) Javier Vegas-Regidor
Added earlier validation for command line arguments (#1435) Manuel Schlund
Remove profile_diagnostic from diagnostic settings and increase test coverage of _task.py (#1404) Valeriu Predoi
Add output2 to the product extra facet of CMIP5 data (#1514) Rémi Kazeroni
Speed up ESGF search (#1512) Bouwe Andela
v2.4.0#
Highlights#
ESMValCore now has the ability to automatically download missing data from ESGF. For details, see Data Retrieval.
ESMValCore now also can resume an earlier run. This is useful to re-use expensive preprocessor results. For details, see Running.
This release includes
Bug fixes#
Crop on the ID-selected region(s) and not on the whole shapefile (#1151) Stef Smeets
Add ‘comment’ to list of removed attributes (#1244) Peter Kalverla
Speed up multimodel statistics and fix bug in peak computation (#1301) Bouwe Andela
No longer make plots of provenance (#1307) Bouwe Andela
No longer embed provenance in output files (#1306) Bouwe Andela
Removed automatic addition of areacello to obs4mips datasets (#1316) Manuel Schlund
Pin docutils <0.17 to fix bullet lists on readthedocs (#1320) Klaus Zimmermann
Fix obs4MIPs capitalization (#1328) Bouwe Andela
Fix Python 3.7 tests (#1330) Bouwe Andela
Handle fx variables in extract_levels and some time operations (#1269) sloosvel
Refactored mask regridding for irregular grids (fixes #772) (#865) Klaus Zimmermann
Fix da.broadcast_to call when the fx cube has different shape than target data cube (#1350) Valeriu Predoi
Fix extra facets (#1360) Bouwe Andela
Pin pip!=21.3 to avoid pypa/pip#10573 with editable installs (#1359) Klaus Zimmermann
Add a custom date2num function to deal with changes in cftime (#1373) Klaus Zimmermann
Removed custom version of AtmosphereSigmaFactory (#1382) Manuel Schlund
Deprecations#
Remove write_netcdf and write_plots from config-user.yml (#1300) Bouwe Andela
Documentation#
Add link to plot directory in index.html (#1256) Stef Smeets
Work around issue with yapf not following PEP8 (#1277) Bouwe Andela
Update the core development team (#1278) Bouwe Andela
Update the documentation of the provenance interface (#1305) Bouwe Andela
Update version number to first release candidate 2.4.0rc1 (#1363) Klaus Zimmermann
Update to new ESMValTool logo (#1374) Klaus Zimmermann
Update version number for third release candidate 2.4.0rc3 (#1384) Klaus Zimmermann
Update changelog for 2.4.0rc3 (#1385) Klaus Zimmermann
Update version number to final 2.4.0 release (#1389) Klaus Zimmermann
Update changelog for 2.4.0 (#1366) Klaus Zimmermann
Fixes for datasets#
Add fix for differing latitude coordinate between historical and ssp585 in MPI-ESM1-2-HR r2i1p1f1 (#1292) Bouwe Andela
Add fixes for time and latitude coordinate of EC-Earth3 r3i1p1f1 (#1290) Bouwe Andela
Apply latitude fix to all CCSM4 variables (#1295) Bouwe Andela
Fix lat and lon bounds for FGOALS-g3 mrsos (#1289) Thomas Crocker
Add fix for CIESM pr (#1344) Bouwe Andela
Fix DRS for IPSLCM : split attribute ‘freq’ into : ‘out’ and ‘freq’ (#1304) Stéphane Sénési - work
CMOR standard#
Remove history attribute from coords (#1276) Javier Vegas-Regidor
Increased flexibility of CMOR checks for datasets with generic alevel coordinates (#1032) Manuel Schlund
Automatically fix small deviations in vertical levels (#1177) Bouwe Andela
Adding standard names to the custom tables of the rlns and rsns variables (#1386) Rémi Kazeroni
Preprocessor#
Implemented fully lazy climate_statistics (#1194) Manuel Schlund
Run the multimodel statistics preprocessor last (#1299) Bouwe Andela
Automatic testing#
Improving test coverage for _task.py (#514) Valeriu Predoi
Upload coverage to codecov (#1190) Bouwe Andela
Improve codecov status checks (#1195) Bouwe Andela
Fix curl install in CircleCI (#1228) Javier Vegas-Regidor
Drop support for Python 3.6 (#1200) Valeriu Predoi
Allow more recent version of scipy (#1182) Manuel Schlund
Speed up conda build conda_build Circle test by using mamba solver via boa (and use it for Github Actions test too) (#1243) Valeriu Predoi
Fix numpy deprecation warnings (#1274) Bouwe Andela
Unpin upper bound for iris (previously was at <3.0.4) (#1275) Valeriu Predoi
Modernize conda_install test on Circle CI by installing from conda-forge with Python 3.9 and change install instructions in documentation (#1280) Valeriu Predoi
Run a nightly Github Actions workflow to monitor tests memory per test (configurable for other metrics too) (#1284) Valeriu Predoi
Speed up tests of tasks (#1302) Bouwe Andela
Fix upper case to lower case variables and functions for flake compliance in tests/unit/preprocessor/_regrid/test_extract_levels.py (#1347) Valeriu Predoi
Cleaned up a bit Github Actions workflows (#1345) Valeriu Predoi
Update circleci jobs: renaming tests to more descriptive names and removing conda build test (#1351) Klaus Zimmermann
Pin iris to latest >=3.1.0 (#1341) Valeriu Predoi
Installation#
Pin esmpy to anything but 8.1.0 since that particular one changes the CPU affinity (#1310) Valeriu Predoi
Improvements#
Add a more friendly and useful message when using default config file (#1233) Valeriu Predoi
Replace os.walk by glob.glob in data finder (only look for data in the specified locations) (#1261) Bouwe Andela
Machine-specific directories for auxiliary data in the config-user.yml file (#1268) Rémi Kazeroni
Add an option to download missing data from ESGF (#1217) Bouwe Andela
Speed up provenance recording (#1327) Bouwe Andela
Improve results web page (#1332) Bouwe Andela
Move institutes from config-developer.yml to default extra facets config and add wildcard support for extra facets (#1259) Bouwe Andela
Add support for re-using preprocessor output from previous runs (#1321) Bouwe Andela
Log fewer messages to screen and hide stack trace for known recipe errors (#1296) Bouwe Andela
Log ESMValCore and ESMValTool versions when running (#1263) Javier Vegas-Regidor
Add “grid” as a tag to the output file template for CMIP6 (#1356) Klaus Zimmermann
Implemented ICON project to read native ICON model output (#1079) Brei Soliño
v2.3.1#
This release includes
Bug fixes#
Update config-user.yml template with correct drs entries for CEDA-JASMIN (#1184) Valeriu Predoi
Enhancing MIROC5 fix for hfls and evspsbl (#1192) katjaweigel
Fix alignment of daily data with inconsistent calendars in multimodel statistics (#1212) Peter Kalverla
Pin cf-units, remove github actions test for Python 3.6 and fix test_access1_0 and test_access1_3 to use cf-units for comparisons (#1197) Valeriu Predoi
Fixed search for fx files when no
mip
is given (#1216) Manuel SchlundMake sure climate statistics always returns original dtype (#1237) Klaus Zimmermann
Bugfix for regional regridding when non-integer range is passed (#1231) Stef Smeets
Make sure area_statistics preprocessor always returns original dtype (#1239) Klaus Zimmermann
Add “.” (dot) as allowed separation character for the time range group (#1248) Klaus Zimmermann
Documentation#
Add a link to the instructions to use pre-installed versions on HPC clusters (#1186) Rémi Kazeroni
Bugfix release: set version to 2.3.1 (#1253) Klaus Zimmermann
Fixes for datasets#
Fixed time coordinate of MIROC-ESM (#1188) Manuel Schlund
Preprocessor#
Filter warnings about collapsing multi-model dimension in multimodel statistics preprocessor function (#1215) Bouwe Andela
Remove fx variables before computing multimodel statistics (#1220) sloosvel
Installation#
Pin lower bound for iris to 3.0.2 (#1206) Valeriu Predoi
Pin iris<3.0.4 to ensure we still (sort of) support Python 3.6 (#1252) Valeriu Predoi
Improvements#
Add test to verify behaviour for scalar height coord for tas in multi-model (#1209) Peter Kalverla
Sort missing years in “No input data available for years” message (#1225) Lee de Mora
v2.3.0#
This release includes
Bug fixes#
Extend preprocessor multi_model_statistics to handle data with “altitude” coordinate (#1010) Axel Lauer
Remove scripts included with CMOR tables (#1011) Bouwe Andela
Avoid side effects in extract_season (#1019) Javier Vegas-Regidor
Use nearest scheme to avoid interpolation errors with masked data in regression test (#1021) Stef Smeets
Move _get_time_bounds from preprocessor._time to cmor.check to avoid circular import with cmor module (#1037) Valeriu Predoi
Fix test that makes conda build fail (#1046) Valeriu Predoi
Fix ‘positive’ attribute for rsns/rlns variables (#1051) Lukas Brunner
Added preprocessor mask_multimodel (#767) Manuel Schlund
Fix bug when fixing bounds after fixing longitude values (#1057) sloosvel
Run conda build parallel AND sequential tests (#1065) Valeriu Predoi
Add key to id_prop (#1071) Lukas Brunner
Fix bounds after reversing coordinate values (#1061) sloosvel
Fixed –skip-nonexistent option (#1093) Manuel Schlund
Do not consider CMIP5 variable sit to be the same as sithick from CMIP6 (#1033) Bouwe Andela
Improve finding date range in filenames (enforces separators) (#1145) Stéphane Sénési - work
Fix lru cache decorator with explicit call to method (#1172) Valeriu Predoi
Update _volume.py (#1174) Lee de Mora
Deprecations#
Documentation#
Final changelog for 2.3.0 (#1163) Klaus Zimmermann
Set version to 2.3.0 (#1162) Klaus Zimmermann
Fix documentation build (#1006) Bouwe Andela
Add labels required for linking from ESMValTool docs (#1038) Bouwe Andela
Update contribution guidelines (#1047) Bouwe Andela
Fix basestring references in documentation (#1106) Javier Vegas-Regidor
Updated references master to main (#1132) Axel Lauer
Add instructions how to use the central installation at DKRZ-Mistral (#1155) Rémi Kazeroni
Fixes for datasets#
Added fixes for various CMIP5 datasets, variable cl (3-dim cloud fraction) (#1017) Axel Lauer
Added fixes for hybrid level coordinates of CESM2 models (#882) Manuel Schlund
Extending LWP fix for CMIP6 models (#1049) Axel Lauer
Add fixes for the net & up radiation variables from ERA5 (#1052) Lukas Brunner
Add derived variable rsus (#1053) Lukas Brunner
Supported mip-level fixes (#1095) Manuel Schlund
Fix erroneous use of grid_latitude and grid_longitude and cleaned ocean grid fixes (#1092) Manuel Schlund
Fix for pr of miroc5 (#1110) Rémi Kazeroni
Ocean depth fix for cnrm_esm2_1, gfdl_esm4, ipsl_cm6a_lr datasets + mcm_ua_1_0 (#1098) Tomas Lovato
Fix for uas variable of the MCM_UA_1_0 dataset (#1102) Rémi Kazeroni
Fixes for sos and siconc of BCC models (#1090) Rémi Kazeroni
Fixes for the siconc variable of CMIP6 models (#1105) Rémi Kazeroni
Fix for pr of EC_EARTH (#1116) Rémi Kazeroni
CMOR standard#
Preprocessor#
Update tests for multimodel statistics preprocessor (#1023) Stef Smeets
Raise in extract_season and extract_month if result is None (#1041) Javier Vegas-Regidor
Allow selection of shapes in extract_shape (#764) Javier Vegas-Regidor
Add option for regional regridding to regrid preprocessor (#1034) Stef Smeets
Load fx variables as cube cell measures / ancillary variables (#999) sloosvel
Check horizontal grid before regridding (#507) Benjamin Müller
Clip irregular grids (#245) Bouwe Andela
Use native iris functions in multi-model statistics (#1150) Peter Kalverla
Notebook API (experimental)#
Automatic testing#
Report coverage for tests that run on any pull request (#994) Bouwe Andela
Install ESMValTool sample data from PyPI (#998) Javier Vegas-Regidor
Fix tests for multi-processing with spawn method (i.e. macOSX with Python>3.8) (#1003) Barbara Vreede
Switch to running the Github Action test workflow every 3 hours in single thread mode to observe if Segmentation Faults occur (#1022) Valeriu Predoi
Revert to original Github Actions test workflow removing the 3-hourly test run with -n 1 (#1025) Valeriu Predoi
Avoid stale cache for multimodel statistics regression tests (#1030) Bouwe Andela
Add newer Python versions in OSX to Github Actions (#1035) Barbara Vreede
Add tests for type annotations with mypy (#1042) Stef Smeets
Run problematic cmor tests sequentially to avoid segmentation faults on CircleCI (#1064) Valeriu Predoi
Test installation of esmvalcore from conda-forge (#1075) Valeriu Predoi
Added additional test cases for integration tests of data_finder.py (#1087) Manuel Schlund
Pin cf-units and fix tests (cf-units>=2.1.5) (#1140) Valeriu Predoi
Fix failing CircleCI tests (#1167) Bouwe Andela
Fix test failing due to fx files chosen differently on different OS’s (#1169) Valeriu Predoi
Compare datetimes instead of strings in _fixes/cmip5/test_access1_X.py (#1173) Valeriu Predoi
Pin Python to 3.9 in environment.yml on CircleCI and skip mypy tests in conda build (#1176) Bouwe Andela
Installation#
Update yamale to version 3 (#1059) Klaus Zimmermann
Improvements#
Refactor diagnostics / tags management (#939) Stef Smeets
Support multiple paths in input_dir (#1000) Javier Vegas-Regidor
Generate HTML report with recipe output (#991) Stef Smeets
Add timeout to requests.get in _citation.py (#1091) SarahAlidoost
Add SYNDA drs for CMIP5 and CMIP6 (closes #582) (#583) Klaus Zimmermann
Add basic support for variable mappings (#1124) Klaus Zimmermann
Handle IPSL-CM6 (#1153) Stéphane Sénési - work
v2.2.0#
Highlights#
ESMValCore is now using the recently released Iris 3. We acknowledge that this change may impact your work, as Iris 3 introduces several changes that are not backward-compatible, but we think that moving forward is the best decision for the tool in the long term.
This release is also the first one including support for downloading CMIP6 data using Synda and we have also started supporting Python 3.9. Give it a try!
This release includes
Bug fixes#
Fix path settings for DKRZ/Mistral (#852) Bouwe Andela
Change logic for calling the diagnostic script to avoid problems with scripts where the executable bit is accidentally set (#877) Bouwe Andela
Add double quotes to script args in rerun screen message when using vprof profiling (#897) Valeriu Predoi
Simplify time handling in multi-model statistics preprocessor (#685) Peter Kalverla
Fix links to Iris documentation (#966) Javier Vegas-Regidor
Bugfix: Fix units for MSWEP data (#986) Stef Smeets
Deprecations#
Deprecate defining write_plots and write_netcdf in config-user file (#808) Bouwe Andela
Documentation#
Fix numbering of steps in release instructions (#838) Bouwe Andela
Add labels to changelogs of individual versions for easy reference (#899) Klaus Zimmermann
Make CircleCI badge specific to main branch (#902) Bouwe Andela
Fix docker build badge url (#906) Stef Smeets
Update github PR template (#909) Stef Smeets
Refer to ESMValTool GitHub discussions page in the error message (#900) Bouwe Andela
Support automatically closing issues (#922) Bouwe Andela
Fix checkboxes in PR template (#931) Stef Smeets
Change in config-user defaults and documentation with new location for esmeval OBS data on JASMIN (#958) Valeriu Predoi
Update Core Team info (#942) Axel Lauer
Update iris documentation URL for sphinx (#964) Bouwe Andela
Set version to 2.2.0 (#977) Javier Vegas-Regidor
Add first draft of v2.2.0 changelog (#983) Javier Vegas-Regidor
Add checkbox in PR template to assign labels (#985) Javier Vegas-Regidor
Change the order of the publication steps (#984) Javier Vegas-Regidor
Add instructions how to use esmvaltool from HPC central installations (#841) Valeriu Predoi
Fixes for datasets#
Fixing unit for derived variable rsnstcsnorm to prevent overcorrection2 (#846) katjaweigel
Changed latitude longitude fix from Tas to AllVars. (#916) katjaweigel
Fix for precipitation (pr) to use ERA5-Land cmorizer (#879) katjaweigel
Improve ACCESS-CM2 and ACCESS-ESM1-5 fixes and add CIESM and CESM2-WACCM-FV2 fixes for cl, clw and cli (#635) Axel Lauer
Add fixes for cl, cli, clw and tas for several CMIP6 models (#955) Manuel Schlund
Dataset fixes for MSWEP (#969) Stef Smeets
Dataset fixes for: ACCESS-ESM1-5, CanESM5, CanESM5 for carbon cycle (#947) Bettina Gier
Fixes for KIOST-ESM (CMIP6) (#904) Rémi Kazeroni
Fixes for AWI-ESM-1-1-LR (CMIP6, piControl) (#911) Rémi Kazeroni
CMOR standard#
CMOR check generic level coordinates in CMIP6 (#598) sloosvel
Update CMIP6 tables to 6.9.33 (#919) Javier Vegas-Regidor
Adding custom variables for tas uncertainty (#924) Lisa Bock
Remove monotonicity coordinate check for unstructured grids (#965) Javier Vegas-Regidor
Preprocessor#
Added clip_start_end_year preprocessor (#796) Manuel Schlund
Add support for downloading CMIP6 data with Synda (#699) Bouwe Andela
Add multimodel tests using real data (#856) Stef Smeets
Add plev/altitude conversion to extract_levels (#892) Axel Lauer
Adding the ability to derive xch4 (#783) Birgit Hassler
Add preprocessor function to resample time and compute x-hourly statistics (#696) Javier Vegas-Regidor
Fix duplication in preprocessors DEFAULT_ORDER introduced in #696 (#973) Javier Vegas-Regidor
Use consistent precision in multi-model statistics calculation and update reference data for tests (#941) Peter Kalverla
Refactor multi-model statistics code to facilitate ensemble stats and lazy evaluation (#949) Peter Kalverla
Add option to exclude input cubes in output of multimodel statistics to solve an issue introduced by #949 (#978) Peter Kalverla
Automatic testing#
Pin cftime>=1.3.0 to have newer string formatting in and fix two tests (#878) Valeriu Predoi
Switched miniconda conda setup hooks for Github Actions workflows (#873) Valeriu Predoi
Add test for latest version resolver (#874) Stef Smeets
Update codacy coverage reporter to fix coverage (#905) Niels Drost
Avoid hardcoded year in tests and add improvement to plev test case (#921) Bouwe Andela
Pin scipy to less than 1.6.0 until ESMValGroup/ESMValCore/issues/927 gets resolved (#928) Valeriu Predoi
Github Actions: change time when conda install test runs (#930) Valeriu Predoi
Remove redundant test line from test_utils.py (#935) Valeriu Predoi
Removed netCDF4 package from integration tests of fixes (#938) Manuel Schlund
Use new conda environment for installing ESMValCore in Docker containers (#951) Bouwe Andela
Notebook API (experimental)#
Implement importable config object in experimental API submodule (#868) Stef Smeets
Add loading and running recipes to the notebook API (#907) Stef Smeets
Add displaying and loading of recipe output to the notebook API (#957) Stef Smeets
Add functionality to run single diagnostic task to notebook API (#962) Stef Smeets
Improvements#
Create CODEOWNERS file (#809) Javier Vegas-Regidor
Remove code needed for Python <3.6 (#844) Bouwe Andela
Add requests as a dependency (#850) Bouwe Andela
Pin Python to less than 3.9 (#870) Valeriu Predoi
Remove numba dependency (#880) Manuel Schlund
Add Listing and finding recipes to the experimental notebook API (#901) Stef Smeets
Skip variables that don’t have dataset or additional_dataset keys (#860) Valeriu Predoi
Refactor logging configuration (#933) Stef Smeets
Xco2 derivation (#913) Bettina Gier
Working environment for Python 3.9 (pin to !=3.9.0) (#885) Valeriu Predoi
Print source file when using config get_config_user command (#960) Valeriu Predoi
Switch to Iris 3 (#819) Stef Smeets
Refactor tasks (#959) Stef Smeets
Restore task summary in debug log after #959 (#981) Bouwe Andela
Pin pre-commit hooks (#974) Stef Smeets
Improve error messages when data is missing (#917) Javier Vegas-Regidor
Set remove_preproc_dir to false in default config-user (#979) Valeriu Predoi
Move fiona to be installed from conda forge (#987) Valeriu Predoi
Re-added fiona in setup.py (#990) Valeriu Predoi
v2.1.0#
This release includes
Bug fixes#
Fix crash for FGOALS-g2 variables without longitude coordinate (#729) Bouwe Andela
Improve variable alias management (#595) Javier Vegas-Regidor
Fix area_statistics fx files loading (#798) Javier Vegas-Regidor
Fix units after derivation (#754) Manuel Schlund
Documentation#
Update v2.0.0 release notes with final additions (#722) Bouwe Andela
Update package description in setup.py (#725) Mattia Righi
Add installation instructions for pip installation (#735) Bouwe Andela
Improve config-user documentation (#740) Bouwe Andela
Update the zenodo file with contributors (#807) Valeriu Predoi
Improve command line run documentation (#721) Javier Vegas-Regidor
Update the zenodo file with contributors (continued) (#810) Valeriu Predoi
Improvements#
Reduce size of docker image (#723) Javier Vegas-Regidor
Add ‘test’ extra to installation, used by docker development tag (#733) Bouwe Andela
Correct dockerhub link (#736) Bouwe Andela
Create action-install-from-pypi.yml (#734) Valeriu Predoi
Add pre-commit for linting/formatting (#766) Stef Smeets
Run tests in parallel and when building conda package (#745) Bouwe Andela
Readable exclude pattern for pre-commit (#770) Stef Smeets
Github Actions Tests (#732) Valeriu Predoi
Remove isort setup to fix formatting conflict with yapf (#778) Stef Smeets
Fix yapf-isort import formatting conflict (Fixes #777) (#784) Stef Smeets
Sorted output for esmvaltool recipes list (#790) Stef Smeets
Replace vmprof with vprof (#780) Valeriu Predoi
Update CMIP6 tables to 6.9.32 (#706) Javier Vegas-Regidor
Default config-user path now set in config-user read function (#791) Javier Vegas-Regidor
Create Github Actions workflow to build and deploy on Test PyPi and PyPi (#820) Valeriu Predoi
Build and publish the esmvalcore package to conda via Github Actions workflow (#825) Valeriu Predoi
Fixes for datasets#
Fix siconca variable in EC-Earth3 and EC-Earth3-Veg models in amip simulation (#702) Evgenia Galytska
Preprocessor#
Move cmor_check_data to early in preprocessing chain (#743) Bouwe Andela
Add RMS iris analysis operator to statistics preprocessor functions (#747) Pep Cos
Add surface chlorophyll concentration as a derived variable (#720) sloosvel
Use dask to reduce memory consumption of extract_levels for masked data (#776) Valeriu Predoi
v2.0.0#
This release includes
Bug fixes#
Fixed derivation of co2s (#594) Manuel Schlund
Padding while cropping needs to stay within sane bounds for shapefiles that span the whole Earth (#626) Valeriu Predoi
Fix concatenation of a single cube (#655) Bouwe Andela
Fix mask fx dict handling not to fail if empty list in values (#661) Valeriu Predoi
Preserve metadata during anomalies computation when using iris cubes difference (#652) Valeriu Predoi
Avoid crashing when there is directory ‘esmvaltool’ in the current working directory (#672) Valeriu Predoi
Solve bug in ACCESS1 dataset fix for calendar. (#671) Peter Kalverla
Fix the syntax for adding multiple ensemble members from the same dataset (#678) SarahAlidoost
Fix bug that made preprocessor with fx files fail in rare cases (#670) Manuel Schlund
Add support for string coordinates (#657) Javier Vegas-Regidor
Fixed the shape extraction to account for wraparound shapefile coords (#319) Valeriu Predoi
Fixed bug in time weights calculation (#695) Manuel Schlund
Fix diagnostic filter (#713) Javier Vegas-Regidor
Documentation#
Add pandas as a requirement for building the documentation (#607) Bouwe Andela
Document default order in which preprocessor functions are applied (#633) Bouwe Andela
Add pointers about data loading and CF standards to documentation (#571) Valeriu Predoi
Config file populated with site-specific data paths examples (#619) Valeriu Predoi
Update Codacy badges (#643) Bouwe Andela
Update copyright info on readthedocs (#668) Bouwe Andela
Updated references to documentation (now docs.esmvaltool.org) (#675) Axel Lauer
Add all European grants to Zenodo (#680) Bouwe Andela
Update Sphinx to v3 or later (#683) Bouwe Andela
Increase version to 2.0.0 and add release notes (#691) Bouwe Andela
Update setup.py and README.md for use on PyPI (#693) Bouwe Andela
Suggested Documentation changes (#690) Steve Smith
Improvements#
Reduce the size of conda package (#606) Bouwe Andela
Add a few unit tests for DiagnosticTask (#613) Bouwe Andela
Make ncl or R tests not fail if package not installed (#610) Valeriu Predoi
Pin flake8<3.8.0 (#623) Valeriu Predoi
Log warnings for likely errors in provenance record (#592) Bouwe Andela
Unpin flake8 (#646) Bouwe Andela
More flexible native6 default DRS (#645) Bouwe Andela
Try to use the same python for running diagnostics as for esmvaltool (#656) Bouwe Andela
Fix test for lower python version and add note on lxml (#659) Valeriu Predoi
Added 1m deep average soil moisture variable (#664) bascrezee
Update docker recipe (#603) Javier Vegas-Regidor
Improve command line interface (#605) Javier Vegas-Regidor
Remove utils directory (#697) Bouwe Andela
Avoid pytest version that crashes (#707) Bouwe Andela
Options arg in read_config_user_file now optional (#716) Javier Vegas-Regidor
Produce a readable warning if ancestors are a string instead of a list. (#711) katjaweigel
Pin Yamale to v2 (#718) Bouwe Andela
Expanded cmor public API (#714) Manuel Schlund
Fixes for datasets#
Added various fixes for hybrid height coordinates (#562) Manuel Schlund
Extended fix for cl-like variables of CESM2 models (#604) Manuel Schlund
Added fix to convert “geopotential” to “geopotential height” for ERA5 (#640) Evgenia Galytska
Do not fix longitude values if they are too far from valid range (#636) Javier Vegas-Regidor
Preprocessor#
Implemented concatenation of cubes with derived coordinates (#546) Manuel Schlund
Fix derived variable ctotal calculation depending on project and standard name (#620) Valeriu Predoi
State of the art FX variables handling without preprocessing (#557) Valeriu Predoi
Add max, min and std operators to multimodel (#602) Javier Vegas-Regidor
Added preprocessor to extract amplitude of cycles (#597) Manuel Schlund
Overhaul concatenation and allow for correct concatenation of multiple overlapping datasets (#615) Valeriu Predoi
Change volume stats to handle and output masked array result (#618) Valeriu Predoi
Make multimodel work correctly with yearly data (#677) Valeriu Predoi
Optimize time weights in time preprocessor for climate statistics (#684) Valeriu Predoi
Add percentiles to multi-model stats (#679) Peter Kalverla
v2.0.0b9#
This release includes
Bug fixes#
Cast dtype float32 to output from zonal and meridional area preprocessors (#581) Valeriu Predoi
Improvements#
Unpin on Python<3.8 for conda package (run) (#570) Valeriu Predoi
Update pytest installation marker (#572) Bouwe Andela
Remove vmrh2o (#573) Mattia Righi
Restructure documentation (#575) Bouwe Andela
Fix mask in land variables for CCSM4 (#579) Klaus Zimmermann
Fix derive scripts wrt required method (#585) Klaus Zimmermann
Check coordinates do not have repeated standard names (#558) Javier Vegas-Regidor
Added derivation script for co2s (#587) Manuel Schlund
Adapted custom co2s table to match CMIP6 version (#588) Manuel Schlund
Increase version to v2.0.0b9 (#593) Bouwe Andela
Add a method to save citation information (#402) SarahAlidoost
For older releases, see the release notes on ESMValGroup/ESMValCore.