Skip to content

Add CacheDict.filepath and observe_access - #314

Open
teonbrooks wants to merge 1 commit into
facebookresearch:mainfrom
teonbrooks:feat/cachedict-observe-filepath
Open

teonbrooks wants to merge 1 commit into
facebookresearch:mainfrom
teonbrooks:feat/cachedict-observe-filepath

Conversation

@teonbrooks

Copy link
Copy Markdown
Contributor

While working on our internal reporting tool, I've needed to patch these two features. They seem useful to upstream.

The first is to return the filepath from the cache so that it can be used to load parquets, etc.
The second is to build a graph of files used for the report. This allows us to export to minimally export a s3 bucket and it lets us do some treeshaking to remove cache files no longer needed for the reports.

  • CacheDict.filepath(key): public accessor returning the absolute on-disk path backing a key (parquet/npy/pickle dumps, directory entries), where indexing returns the loaded value instead.
  • observe_access(callback): context manager notified with the folder of every CacheDict constructed or accessed within its scope. Thread-safe; observer errors are caught and logged rather than propagated.

Adds unit tests for both.

- CacheDict.filepath(key): public accessor returning the absolute on-disk
  path backing a key (parquet/npy/pickle dumps, directory entries), where
  indexing returns the loaded value instead.
- observe_access(callback): context manager notified with the folder of every
  CacheDict constructed or accessed within its scope. Thread-safe; observer
  errors are caught and logged rather than propagated.

Adds unit tests for both.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant