Changelog¶
v2.2.0¶
Highlights¶
ESMValCore is now using the recently released Iris 3. We acknowledge that this change may impact your work, as Iris 3 introduces several changes that are not backward-compatible, but we think that moving forward is the best decision for the tool in the long term.
This release is also the first one including support for downloading CMIP6 data using Synda and we have also started supporting Python 3.9. Give it a try!
This release includes
Bug fixes¶
Fix path settings for DKRZ/Mistral (#852) Bouwe Andela
Change logic for calling the diagnostic script to avoid problems with scripts where the executable bit is accidentally set (#877) Bouwe Andela
Add double quotes to script args in rerun screen message when using vprof profiling (#897) Valeriu Predoi
Simplify time handling in multi-model statistics preprocessor (#685) Peter Kalverla
Fix links to Iris documentation (#966) Javier Vegas-Regidor
Bugfix: Fix units for MSWEP data (#986) Stef Smeets
Deprecations¶
Deprecate defining write_plots and write_netcdf in config-user file (#808) Bouwe Andela
Documentation¶
Fix numbering of steps in release instructions (#838) Bouwe Andela
Add labels to changelogs of individual versions for easy reference (#899) Klaus Zimmermann
Make CircleCI badge specific to master branch (#902) Bouwe Andela
Fix docker build badge url (#906) Stef Smeets
Update github PR template (#909) Stef Smeets
Refer to ESMValTool GitHub discussions page in the error message (#900) Bouwe Andela
Support automatically closing issues (#922) Bouwe Andela
Fix checkboxes in PR template (#931) Stef Smeets
Change in config-user defaults and documentation with new location for esmeval OBS data on JASMIN (#958) Valeriu Predoi
Update Core Team info (#942) Axel Lauer
Update iris documentation URL for sphinx (#964) Bouwe Andela
Set version to 2.2.0 (#977) Javier Vegas-Regidor
Add first draft of v2.2.0 changelog (#983) Javier Vegas-Regidor
Add checkbox in PR template to assign labels (#985) Javier Vegas-Regidor
Change the order of the publication steps (#984) Javier Vegas-Regidor
Add instructions how to use esmvaltool from HPC central installations (#841) Valeriu Predoi
Fixes for datasets¶
Fixing unit for derived variable rsnstcsnorm to prevent overcorrection2 (#846) katjaweigel
Changed latitude longitude fix from Tas to AllVars. (#916) katjaweigel
Fix for precipitation (pr) to use ERA5-Land cmorizer (#879) katjaweigel
Improve ACCESS-CM2 and ACCESS-ESM1-5 fixes and add CIESM and CESM2-WACCM-FV2 fixes for cl, clw and cli (#635) Axel Lauer
Add fixes for cl, cli, clw and tas for several CMIP6 models (#955) Manuel Schlund
Dataset fixes for MSWEP (#969) Stef Smeets
Dataset fixes for: ACCESS-ESM1-5, CanESM5, CanESM5 for carbon cycle (#947) Bettina Gier
Fixes for KIOST-ESM (CMIP6) (#904) Rémi Kazeroni
Fixes for AWI-ESM-1-1-LR (CMIP6, piControl) (#911) Rémi Kazeroni
CMOR standard¶
CMOR check generic level coordinates in CMIP6 (#598) sloosvel
Update CMIP6 tables to 6.9.33 (#919) Javier Vegas-Regidor
Adding custom variables for tas uncertainty (#924) Lisa Bock
Remove monotonicity coordinate check for unstructured grids (#965) Javier Vegas-Regidor
Preprocessor¶
Added clip_start_end_year preprocessor (#796) Manuel Schlund
Add support for downloading CMIP6 data with Synda (#699) Bouwe Andela
Add multimodel tests using real data (#856) Stef Smeets
Add plev/altitude conversion to extract_levels (#892) Axel Lauer
Adding the ability to derive xch4 (#783) Birgit Hassler
Add preprocessor function to resample time and compute x-hourly statistics (#696) Javier Vegas-Regidor
Fix duplication in preprocessors DEFAULT_ORDER introduced in #696 (#973) Javier Vegas-Regidor
Use consistent precision in multi-model statistics calculation and update reference data for tests (#941) Peter Kalverla
Refactor multi-model statistics code to facilitate ensemble stats and lazy evaluation (#949) Peter Kalverla
Add option to exclude input cubes in output of multimodel statistics to solve an issue introduced by #949 (#978) Peter Kalverla
Automatic testing¶
Pin cftime>=1.3.0 to have newer string formatting in and fix two tests (#878) Valeriu Predoi
Switched miniconda conda setup hooks for Github Actions workflows (#873) Valeriu Predoi
Add test for latest version resolver (#874) Stef Smeets
Update codacy coverage reporter to fix coverage (#905) Niels Drost
Avoid hardcoded year in tests and add improvement to plev test case (#921) Bouwe Andela
Pin scipy to less than 1.6.0 until ESMValGroup/ESMValCore/issues/927 gets resolved (#928) Valeriu Predoi
Github Actions: change time when conda install test runs (#930) Valeriu Predoi
Remove redundant test line from test_utils.py (#935) Valeriu Predoi
Removed netCDF4 package from integration tests of fixes (#938) Manuel Schlund
Use new conda environment for installing ESMValCore in Docker containers (#951) Bouwe Andela
Notebook API (experimental)¶
Implement importable config object in experimental API submodule (#868) Stef Smeets
Add loading and running recipes to the notebook API (#907) Stef Smeets
Add displaying and loading of recipe output to the notebook API (#957) Stef Smeets
Add functionality to run single diagnostic task to notebook API (#962) Stef Smeets
Improvements¶
Create CODEOWNERS file (#809) Javier Vegas-Regidor
Remove code needed for Python <3.6 (#844) Bouwe Andela
Add requests as a dependency (#850) Bouwe Andela
Pin Python to less than 3.9 (#870) Valeriu Predoi
Remove numba dependency (#880) Manuel Schlund
Add Listing and finding recipes to the experimental notebook API (#901) Stef Smeets
Skip variables that don’t have dataset or additional_dataset keys (#860) Valeriu Predoi
Refactor logging configuration (#933) Stef Smeets
Xco2 derivation (#913) Bettina Gier
Working environment for Python 3.9 (pin to !=3.9.0) (#885) Valeriu Predoi
Print source file when using config get_config_user command (#960) Valeriu Predoi
Switch to Iris 3 (#819) Stef Smeets
Refactor tasks (#959) Stef Smeets
Restore task summary in debug log after #959 (#981) Bouwe Andela
Pin pre-commit hooks (#974) Stef Smeets
Improve error messages when data is missing (#917) Javier Vegas-Regidor
Set remove_preproc_dir to false in default config-user (#979) Valeriu Predoi
Move fiona to be installed from conda forge (#987) Valeriu Predoi
Re-added fiona in setup.py (#990) Valeriu Predoi
v2.1.0¶
This release includes
Bug fixes¶
Fix crash for FGOALS-g2 variables without longitude coordinate (#729) Bouwe Andela
Improve variable alias management (#595) Javier Vegas-Regidor
Fix area_statistics fx files loading (#798) Javier Vegas-Regidor
Fix units after derivation (#754) Manuel Schlund
Documentation¶
Update v2.0.0 release notes with final additions (#722) Bouwe Andela
Update package description in setup.py (#725) Mattia Righi
Add installation instructions for pip installation (#735) Bouwe Andela
Improve config-user documentation (#740) Bouwe Andela
Update the zenodo file with contributors (#807) Valeriu Predoi
Improve command line run documentation (#721) Javier Vegas-Regidor
Update the zenodo file with contributors (continued) (#810) Valeriu Predoi
Improvements¶
Reduce size of docker image (#723) Javier Vegas-Regidor
Add ‘test’ extra to installation, used by docker development tag (#733) Bouwe Andela
Correct dockerhub link (#736) Bouwe Andela
Create action-install-from-pypi.yml (#734) Valeriu Predoi
Add pre-commit for linting/formatting (#766) Stef Smeets
Run tests in parallel and when building conda package (#745) Bouwe Andela
Readable exclude pattern for pre-commit (#770) Stef Smeets
Github Actions Tests (#732) Valeriu Predoi
Remove isort setup to fix formatting conflict with yapf (#778) Stef Smeets
Fix yapf-isort import formatting conflict (Fixes #777) (#784) Stef Smeets
Sorted output for esmvaltool recipes list (#790) Stef Smeets
Replace vmprof with vprof (#780) Valeriu Predoi
Update CMIP6 tables to 6.9.32 (#706) Javier Vegas-Regidor
Default config-user path now set in config-user read function (#791) Javier Vegas-Regidor
Create Github Actions workflow to build and deploy on Test PyPi and PyPi (#820) Valeriu Predoi
Build and publish the esmvalcore package to conda via Github Actions workflow (#825) Valeriu Predoi
Fixes for datasets¶
Fix siconca variable in EC-Earth3 and EC-Earth3-Veg models in amip simulation (#702) Evgenia Galytska
Preprocessor¶
Move cmor_check_data to early in preprocessing chain (#743) Bouwe Andela
Add RMS iris analysis operator to statistics preprocessor functions (#747) Pep Cos
Add surface chlorophyll concentration as a derived variable (#720) sloosvel
Use dask to reduce memory consumption of extract_levels for masked data (#776) Valeriu Predoi
v2.0.0¶
This release includes
Bug fixes¶
Fixed derivation of co2s (#594) Manuel Schlund
Padding while cropping needs to stay within sane bounds for shapefiles that span the whole Earth (#626) Valeriu Predoi
Fix concatenation of a single cube (#655) Bouwe Andela
Fix mask fx dict handling not to fail if empty list in values (#661) Valeriu Predoi
Preserve metadata during anomalies computation when using iris cubes difference (#652) Valeriu Predoi
Avoid crashing when there is directory ‘esmvaltool’ in the current working directory (#672) Valeriu Predoi
Solve bug in ACCESS1 dataset fix for calendar. (#671) Peter Kalverla
Fix the syntax for adding multiple ensemble members from the same dataset (#678) SarahAlidoost
Fix bug that made preprocessor with fx files fail in rare cases (#670) Manuel Schlund
Add support for string coordinates (#657) Javier Vegas-Regidor
Fixed the shape extraction to account for wraparound shapefile coords (#319) Valeriu Predoi
Fixed bug in time weights calculation (#695) Manuel Schlund
Fix diagnostic filter (#713) Javier Vegas-Regidor
Documentation¶
Add pandas as a requirement for building the documentation (#607) Bouwe Andela
Document default order in which preprocessor functions are applied (#633) Bouwe Andela
Add pointers about data loading and CF standards to documentation (#571) Valeriu Predoi
Config file populated with site-specific data paths examples (#619) Valeriu Predoi
Update Codacy badges (#643) Bouwe Andela
Update copyright info on readthedocs (#668) Bouwe Andela
Updated references to documentation (now docs.esmvaltool.org) (#675) Axel Lauer
Add all European grants to Zenodo (#680) Bouwe Andela
Update Sphinx to v3 or later (#683) Bouwe Andela
Increase version to 2.0.0 and add release notes (#691) Bouwe Andela
Update setup.py and README.md for use on PyPI (#693) Bouwe Andela
Suggested Documentation changes (#690) Steve Smith
Improvements¶
Reduce the size of conda package (#606) Bouwe Andela
Add a few unit tests for DiagnosticTask (#613) Bouwe Andela
Make ncl or R tests not fail if package not installed (#610) Valeriu Predoi
Pin flake8<3.8.0 (#623) Valeriu Predoi
Log warnings for likely errors in provenance record (#592) Bouwe Andela
Unpin flake8 (#646) Bouwe Andela
More flexible native6 default DRS (#645) Bouwe Andela
Try to use the same python for running diagnostics as for esmvaltool (#656) Bouwe Andela
Fix test for lower python version and add note on lxml (#659) Valeriu Predoi
Added 1m deep average soil moisture variable (#664) bascrezee
Update docker recipe (#603) Javier Vegas-Regidor
Improve command line interface (#605) Javier Vegas-Regidor
Remove utils directory (#697) Bouwe Andela
Avoid pytest version that crashes (#707) Bouwe Andela
Options arg in read_config_user_file now optional (#716) Javier Vegas-Regidor
Produce a readable warning if ancestors are a string instead of a list. (#711) katjaweigel
Pin Yamale to v2 (#718) Bouwe Andela
Expanded cmor public API (#714) Manuel Schlund
Fixes for datasets¶
Added various fixes for hybrid height coordinates (#562) Manuel Schlund
Extended fix for cl-like variables of CESM2 models (#604) Manuel Schlund
Added fix to convert “geopotential” to “geopotential height” for ERA5 (#640) Evgenia Galytska
Do not fix longitude values if they are too far from valid range (#636) Javier Vegas-Regidor
Preprocessor¶
Implemented concatenation of cubes with derived coordinates (#546) Manuel Schlund
Fix derived variable ctotal calculation depending on project and standard name (#620) Valeriu Predoi
State of the art FX variables handling without preprocessing (#557) Valeriu Predoi
Add max, min and std operators to multimodel (#602) Javier Vegas-Regidor
Added preprocessor to extract amplitude of cycles (#597) Manuel Schlund
Overhaul concatenation and allow for correct concatenation of multiple overlapping datasets (#615) Valeriu Predoi
Change volume stats to handle and output masked array result (#618) Valeriu Predoi
Make multimodel work correctly with yearly data (#677) Valeriu Predoi
Optimize time weights in time preprocessor for climate statistics (#684) Valeriu Predoi
Add percentiles to multi-model stats (#679) Peter Kalverla
v2.0.0b9¶
This release includes
Bug fixes¶
Cast dtype float32 to output from zonal and meridional area preprocessors (#581) Valeriu Predoi
Improvements¶
Unpin on Python<3.8 for conda package (run) (#570) Valeriu Predoi
Update pytest installation marker (#572) Bouwe Andela
Remove vmrh2o (#573) Mattia Righi
Restructure documentation (#575) Bouwe Andela
Fix mask in land variables for CCSM4 (#579) Klaus Zimmermann
Fix derive scripts wrt required method (#585) Klaus Zimmermann
Check coordinates do not have repeated standard names (#558) Javier Vegas-Regidor
Added derivation script for co2s (#587) Manuel Schlund
Adapted custom co2s table to match CMIP6 version (#588) Manuel Schlund
Increase version to v2.0.0b9 (#593) Bouwe Andela
Add a method to save citation information (#402) SarahAlidoost
For older releases, see the release notes on https://github.com/ESMValGroup/ESMValCore/releases.