Find files on the local filesystem#

Find files on the local filesystem.

Classes:

`DataSource`(rootpath, dirname_template, ...)	Class for storing a data source and finding the associated files.
`LocalFile`(args, *kwargs)	File on the local filesystem.

Functions:

find_files(*[, debug])

Find files on the local filesystem.

class esmvalcore.local.DataSource(rootpath: Path, dirname_template: str, filename_template: str)[source]#

Bases: object

Class for storing a data source and finding the associated files.

Attributes:

`dirname_template`
`filename_template`
`rootpath`

Methods:

`find_files`(**facets)	Find files.
`get_glob_patterns`(**facets)	Compose the globs that will be used to look for files.

Parameters:

rootpath (Path)
dirname_template (str)
filename_template (str)

dirname_template: str#

filename_template: str#

find_files(**facets) → list[LocalFile][source]#

Find files.

Return type:: list[LocalFile]

get_glob_patterns(**facets) → list[Path][source]#

Compose the globs that will be used to look for files.

Return type:: list[Path]

rootpath: Path#

class esmvalcore.local.LocalFile(*args, **kwargs)[source]#

Bases: PosixPath

File on the local filesystem.

Attributes:

facets

Facets describing the file.

property facets: Dict[str, str | Sequence[str] | Number]#: Facets describing the file.

Note

When using find_files(), facets are read from the directory structure. Facets stored in filenames are not yet supported.

esmvalcore.local.find_files(*, debug: bool = False, **facets: str | Sequence[str] | Number) → list[LocalFile] | tuple[list[LocalFile], list[Path]][source]#

Find files on the local filesystem.

The directories that are searched for files are defined in esmvalcore.config.CFG under the 'rootpath' key using the directory structure defined under the 'drs' key. If esmvalcore.config.CFG['rootpath'] contains a key that matches the value of the project facet, those paths will be used. If there is no project specific key, the directories in esmvalcore.config.CFG['rootpath']['default'] will be searched.

See Input data for extensive instructions on configuring ESMValCore so it can find files locally.

Parameters:

debug (bool) – When debug is set to True, the function will return a tuple with the first element containing the files that were found and the second element containing the glob.glob() patterns that were used to search for files.
**facets (str | Sequence[str] | Number) – Facets used to search for files. An '*' can be used to match any value. By default, only the latest version of a file will be returned. To select all versions use version='*'. It is also possible to specify multiple values for a facet, e.g. exp=['historical', 'ssp585'] will match any file that belongs to either the historical or ssp585 experiment. The timerange facet can be specified in ISO 8601 format.

Return type:

list[LocalFile] | tuple[list[LocalFile], list[Path]]

Note

A value of timerange='*' is supported, but combining a '*' with a time or period as supported in the recipe is currently not supported and will return all found files.

Examples

Search for files containing surface air temperature from any CMIP6 model for the historical experiment:

>>> esmvalcore.local.find_files(
...     project='CMIP6',
...     activity='CMIP',
...     mip='Amon',
...     short_name='tas',
...     exp='historical',
...     dataset='*',
...     ensemble='*',
...     grid='*',
...     institute='*',
... )  
[LocalFile('/home/bandela/climate_data/CMIP6/CMIP/BCC/BCC-ESM1/historical/r1i1p1f1/Amon/tas/gn/v20181214/tas_Amon_BCC-ESM1_historical_r1i1p1f1_gn_185001-201412.nc')]

Returns:

The files that were found.

Return type:

list[LocalFile]

Parameters:

debug (bool)
facets (str | Sequence[str] | Number)