xarray.backends.CachingFileManager#
- class xarray.backends.CachingFileManager(opener, *args, mode=<unused>, kwargs=None, lock=None, cache=None, manager_id=None, ref_counts=None)[source]#
Wrapper for automatically opening and closing file objects.
Unlike files, CachingFileManager objects can be safely pickled and passed between processes. They should be explicitly closed to release resources, but a per-process least-recently-used cache for open files ensures that you can safely create arbitrarily large numbers of FileManager objects.
Don’t directly close files acquired from a FileManager. Instead, call FileManager.close(), which ensures that closed files are removed from the cache as well.
Example usage:
manager = FileManager(open, "example.txt", mode="w") f = manager.acquire() f.write(...) manager.close() # ensures file is closed
Note that as long as previous files are still cached, acquiring a file multiple times from the same FileManager is essentially free:
f1 = manager.acquire() f2 = manager.acquire() assert f1 is f2
- __init__(opener, *args, mode=<unused>, kwargs=None, lock=None, cache=None, manager_id=None, ref_counts=None)[source]#
Initialize a CachingFileManager.
The cache, manager_id and ref_counts arguments exist solely to facilitate dependency injection, and should only be set for tests.
- Parameters
opener (
callable()
) – Function that when called likeopener(*args, **kwargs)
returns an open file object. The file object must implement aclose()
method.*args – Positional arguments for opener. A
mode
argument should be provided as a keyword argument (see below). All arguments must be hashable.mode (optional) – If provided, passed as a keyword argument to
opener
along with**kwargs
.mode='w' `` has special treatment: after the first call it is replaced by ``mode='a'
in all subsequent function to avoid overriding the newly created file.kwargs (
dict
, optional) – Keyword arguments for opener, excludingmode
. All values must be hashable.lock (
duck-compatible threading.Lock
, optional) – Lock to use when modifying the cache inside acquire() and close(). By default, uses a new threading.Lock() object. If set, this object should be pickleable.cache (
MutableMapping
, optional) – Mapping to use as a cache for open files. By default, uses xarray’s global LRU file cache. Becausecache
typically points to a global variable and contains non-picklable file objects, an unpickled FileManager objects will be restored with the default cache.manager_id (hashable, optional) – Identifier for this CachingFileManager.
ref_counts (
dict
, optional) – Optional dict to use for keeping track the number of references to the same file.
Methods
__init__
(opener, *args[, mode, kwargs, ...])Initialize a CachingFileManager.
acquire
([needs_lock])Acquire a file object from the manager.
acquire_context
([needs_lock])Context manager for acquiring a file.
close
([needs_lock])Explicitly close any associated file object (if necessary).