diff --git a/.dialyzer_ignore.exs b/.dialyzer_ignore.exs new file mode 100644 index 0000000..58dadf9 --- /dev/null +++ b/.dialyzer_ignore.exs @@ -0,0 +1,6 @@ +[ + # ElixirCache `use Cache` macro generates code with unreachable pattern matches. + # These are test-only support modules and not under our control. + {"test/support/elixir_cache_ets.ex", :pattern_match}, + {"test/support/elixir_cache_redis.ex", :pattern_match_cov} +] diff --git a/README.md b/README.md index 44d8af6..8d58a4f 100644 --- a/README.md +++ b/README.md @@ -5,13 +5,12 @@ [![CI](https://github.com/BobbieBarker/schema_cache/actions/workflows/ci.yml/badge.svg)](https://github.com/BobbieBarker/schema_cache/actions/workflows/ci.yml) [![codecov](https://codecov.io/gh/BobbieBarker/schema_cache/branch/main/graph/badge.svg)](https://codecov.io/gh/BobbieBarker/schema_cache) -An Ecto-aware caching library implementing **Read Through**, **Write Through**, -and **Schema Mutation Key Eviction Strategy (SMKES)**. +An Ecto-aware caching library providing **cache-aside** and **write-through** +abstractions with automatic invalidation. SchemaCache understands the relationship between your Ecto schemas and cached query results. When a schema is mutated, it knows exactly which cached values -are affected and can evict or update them automatically, requiring no manual -cache invalidation. +are affected and can evict or update them automatically. ## Installation @@ -25,13 +24,6 @@ def deps do end ``` -Then configure your cache adapter: - -```elixir -# config/config.exs -config :schema_cache, adapter: SchemaCache.Adapters.ETS -``` - Add `SchemaCache.Supervisor` to your application supervision tree: ```elixir @@ -46,7 +38,7 @@ children = [ SchemaCache provides four core operations: **read**, **create**, **update**, and **delete**. -### Read Through +### Cache-Aside On a cache miss, SchemaCache invokes your callback to fetch from the source, caches the result, and records which schemas appear in it for later eviction. @@ -59,9 +51,9 @@ caches the result, and records which schemas appear in it for later eviction. end) # Cache a collection (prefix with "all_" for write-through support) -users = +{:ok, users} = SchemaCache.read("all_active_users", %{active: true}, :timer.minutes(5), fn -> - MyApp.Users.all(%{active: true}) + {:ok, MyApp.Users.all(%{active: true})} end) ``` @@ -91,10 +83,10 @@ instance. The next read will fetch fresh data from the source. end) ``` -### Update with Write Through +### Update with Write-Through Update all cached values referencing the schema in place, avoiding cache misses -entirely. Ideal for updates where you want zero-latency reads after the write. +entirely. Use when your application can't serve stale results after a write. ```elixir {:ok, updated_user} = @@ -114,220 +106,98 @@ instance. SchemaCache.delete(fn -> Repo.delete(user) end) ``` -## How SMKES Works - -Schema Mutation Key Eviction Strategy (SMKES) is the core innovation of this -library. It maintains a reverse index from Ecto schemas to every cache key -where they appear. When a schema is mutated, SchemaCache looks up the reverse -index and takes action on every affected cache entry. +## How It Works -### Key Reference Tracking +SchemaCache maintains a reverse index from Ecto schemas to cache keys (SMKES). +When a query result is cached, it records which schemas appear in that result. +On mutation, it looks up all affected cache keys and evicts or updates them. -When a query result is cached via `read/4`, SchemaCache inspects the result to -find Ecto structs and records two types of references: +For a detailed explanation, see the +[Architecture Guide](guides/explanation/architecture.md). -1. **Instance references**: maps a specific schema instance (e.g. `User#5`) - to every cache key containing it. -2. **Type references**: maps a schema module (e.g. `User`) to every cache key - holding a collection of that type. Used by `create/1` to evict collections - that need to include the newly created record. +## Using with ElixirCache -References are stored as compact integer IDs via `SchemaCache.KeyRegistry`, -reducing memory usage by approximately 10x compared to storing full cache key -strings. +[ElixirCache](https://github.com/MikaAK/elixir_cache) modules work out of the +box. No wrapper module needed. SchemaCache auto-detects ElixirCache modules and +translates API signatures automatically: -``` - SMKES: Key Reference Tracking - ========================================================================= - - SchemaCache.read("find_user", %{id: 5}, ttl, fn -> ... end) - SchemaCache.read("all_users", %{active: true}, ttl, fn -> ... end) - - Cache Storage Key Reference Sets - +------------------+ +-------------------------+ - | find_user:{...} | | User#5 | - | => %User{id:5} | .----->| => [1, 2] | - +------------------+ / | (integer IDs from | - | / | KeyRegistry) | - cache_key ----+--- sadd ---' +-------------------------+ - +-------------------------+ - +------------------+ .----->| User#8 | - | all_users:{...} | / | => [2] | - | => [%User{id:5}| / +-------------------------+ - | %User{id:8}|---' +-------------------------+ - | ] |- - sadd - ->| User (type) | - +------------------+ | => [2] | - +-------------------------+ +```elixir +children = [ + MyApp.Cache, + {SchemaCache.Supervisor, adapter: MyApp.Cache} +] ``` -### Read Through Flow +For Redis-backed ElixirCache modules, SchemaCache also auto-detects native set +operations via `command/1` with zero configuration. See the +[ElixirCache Integration Guide](guides/how-to/using_with_elixir_cache.md) +for details. -``` - SchemaCache.read("find_user", %{id: 5}, ttl, fn -> Repo.get(User, 5) end) - - +--------+ +-----------+ +----------+ - | Caller | -- 1 --> |SchemaCache| -- 2 -> | Adapter | - +--------+ +-----------+ +----------+ - ^ | | - | | cache miss (nil) | - | | <------- 3 ---------| - | | | - | | -- 4 -> callback() --> +------+ - | | | DB | - | | <- 5 -- {:ok, user} <-- +------+ - | | | - | | -- 6 -> put(key, | - | | user, ttl) | - | | | - | | -- 7 -> sadd( | - | | "User#5", | - | | key_id) | - | | | - | <-- 8 -- {:ok, user} | | - | | | - - Subsequent calls with the same key and params return - the cached value at step 3 without invoking the callback. -``` +## Documentation -### Write Through Flow +- [Introduction](guides/introduction.md) +- **Tutorials**: [Installation](guides/tutorials/installation.md) | [Basic Operations](guides/tutorials/basic_operations.md) +- **How-to**: [Writing Adapters](guides/how-to/writing_adapters.md) | [Using with ElixirCache](guides/how-to/using_with_elixir_cache.md) +- **Explanation**: [Architecture (SMKES)](guides/explanation/architecture.md) +- [HexDocs](https://hexdocs.pm/schema_cache) -``` - SchemaCache.update(fn -> update_user(user, params) end, strategy: :write_through) - - +--------+ +-----------+ +----------+ - | Caller | -- 1 --> |SchemaCache| | Adapter | - +--------+ +-----------+ +----------+ - ^ | | - | | -- 2 -> callback() --> +------+ - | | | DB | - | | <- 3 -- {:ok, updated} +------+ - | | | - | | -- 4 -> smembers( | - | | "User#5") | - | | | - | | <- 5 -- [id1, id2] | - | | | - | | -- 6 -> resolve IDs, | - | | for each key: | - | | put(key, updated) | - | | | - | <- 7 -- {:ok, updated} | | - | | | - - For collection keys (prefixed with "all_"), SchemaCache - finds the specific item within the list by primary key - and replaces it in place. -``` +## Contributing -### Eviction Flow (default and create) +### Prerequisites -``` - SchemaCache.update(fn -> update_user(user, params) end) - - +--------+ +-----------+ +----------+ - | Caller | -- 1 --> |SchemaCache| | Adapter | - +--------+ +-----------+ +----------+ - ^ | | - | | -- 2 -> callback() --> +------+ - | | | DB | - | | <- 3 -- {:ok, updated} +------+ - | | | - | | -- 4 -> smembers( | - | | "User#5") | - | | | - | | <- 5 -- [id1, id2] | - | | | - | | -- 6 -> resolve IDs, | - | | for each key: | - | | delete(key) | - | | srem("User#5",id) | - | | | - | <- 7 -- {:ok, updated} | | - | | | - - create/1 behaves similarly but looks up keys by the - schema module name (e.g. "User") instead of the instance - identity (e.g. "User#5"), evicting all collections of - that type. -``` +- Elixir ~> 1.14 +- PostgreSQL (for integration tests) +- Redis (optional, for Redis adapter tests) +- Docker (optional, for running services via docker-compose) -## Adapter Configuration +### Local Setup -SchemaCache is adapter-agnostic. Any module implementing the -`SchemaCache.Adapter` behaviour can be used as a backend. +```bash +# Clone the repository +git clone https://github.com/BobbieBarker/schema_cache.git +cd schema_cache -### Built-in: ETS Adapter +# Install dependencies +mix deps.get -A simple single-node adapter backed by ETS. Does not support TTL. -Suitable for development, testing, and single-node deployments. +# Create and migrate the test database +MIX_ENV=test mix ecto.setup -```elixir -config :schema_cache, adapter: SchemaCache.Adapters.ETS +# Run the test suite +mix test ``` -### Custom Adapters - -Implement the `SchemaCache.Adapter` behaviour with three required callbacks: -`get/1`, `put/3`, and `delete/1`. SchemaCache builds its key reference -tracking on top of these primitives, so your adapter only needs basic -key-value operations. +### Running Redis Tests -Adapters can optionally implement `sadd/2`, `srem/2`, `smembers/1`, and -`mget/1` for native set operations and batch reads. Adapters without these -callbacks automatically fall back to a partitioned lock mechanism via -`SchemaCache.SetLock`. +Redis adapter tests require a running Redis instance. You can start one with +docker-compose: -```elixir -defmodule MyApp.SchemaCacheAdapter do - @behaviour SchemaCache.Adapter - - @impl true - def get(key) do - case MyApp.Cache.get(key) do - nil -> {:ok, nil} - value -> {:ok, value} - end - end - - @impl true - def put(key, value, opts) do - ttl = Keyword.get(opts, :ttl) - MyApp.Cache.put(key, value, ttl: ttl) - :ok - end - - @impl true - def delete(key) do - MyApp.Cache.delete(key) - :ok - end -end +```bash +docker-compose up -d redis +mix test ``` -Then configure it: +Tests that require Redis will automatically skip if Redis is not available. -```elixir -config :schema_cache, adapter: MyApp.SchemaCacheAdapter -``` +### Code Quality -## Key Conventions +```bash +# Linting +mix credo -When caching collections, prefix the key with `all_`. This convention is -required for write-through to correctly identify and update collections in -the cache versus singular values. - -```elixir -# Singular, no prefix -SchemaCache.read("find_user", %{id: 5}, ttl, fn -> ... end) +# Type checking +mix dialyzer -# Collection, "all_" prefix -SchemaCache.read("all_users", %{active: true}, ttl, fn -> ... end) +# Test coverage +mix coveralls ``` -## API Reference +### Pull Requests -Full API documentation is available on [HexDocs](https://hexdocs.pm/schema_cache). +1. Fork the repository and create your branch from `main`. +2. Write tests for any new functionality. +3. Ensure `mix test`, `mix credo`, and `mix dialyzer` pass. +4. Open a pull request with a clear description of the change. ## License diff --git a/config/config.exs b/config/config.exs index 6756d05..d1186fe 100644 --- a/config/config.exs +++ b/config/config.exs @@ -1,5 +1,3 @@ import Config -config :schema_cache, adapter: SchemaCache.Adapters.ETS - import_config "#{config_env()}.exs" diff --git a/config/test.exs b/config/test.exs index 9e37b85..7a0f61c 100644 --- a/config/test.exs +++ b/config/test.exs @@ -9,9 +9,10 @@ config :schema_cache, SchemaCache.Test.Repo, pool_size: 10 config :schema_cache, - ecto_repos: [SchemaCache.Test.Repo], - adapter: SchemaCache.Adapters.ETS + ecto_repos: [SchemaCache.Test.Repo] config :schema_cache, :redis_url, "redis://localhost:6379" +config :schema_cache, :test_elixir_cache_redis_opts, uri: "redis://localhost:6379" + config :logger, level: :warning diff --git a/guides/explanation/architecture.md b/guides/explanation/architecture.md new file mode 100644 index 0000000..351fa4b --- /dev/null +++ b/guides/explanation/architecture.md @@ -0,0 +1,177 @@ +# Architecture: How SMKES Works + +Schema Mutation Key Eviction Strategy (SMKES) is the core innovation of SchemaCache. It maintains a reverse index from Ecto schemas to every cache key where they appear. When a schema is mutated, SchemaCache looks up the reverse index and takes action on every affected cache entry. + +## Key Reference Tracking + +When a query result is cached via `read/4`, SchemaCache inspects the result to find Ecto structs and records two types of references: + +1. **Instance references**: Maps a specific schema instance (e.g., `User#5`) to every cache key containing it. +2. **Type references**: Maps a schema module (e.g., `User`) to every cache key holding a collection of that type. Used by `create/1` to evict collections that need to include the newly created record. + +References are stored as compact integer IDs via `SchemaCache.KeyRegistry`, reducing memory usage by approximately 10x compared to storing full cache key strings. + +Given these two cached queries: + +```elixir +SchemaCache.read("find_user", %{id: 5}, ttl, fn -> Repo.get(User, 5) end) +SchemaCache.read("all_users", %{active: true}, ttl, fn -> Repo.all(User) end) +``` + +SchemaCache stores the results and builds these reverse-index sets: + +``` +Cache Entries (adapter key-value store) +─────────────────────────────────────────────────────────── + Key ID Value + find_user:{id:5} 1 %User{id: 5, name: "Alice"} + all_users:{active:true} 2 [%User{id: 5}, %User{id: 8}] + +Reverse-Index Sets (adapter set store) +─────────────────────────────────────────────────────────── + Set Key Members What it means + __set:User#5 [1, 2] User#5 appears in cache entries 1 and 2 + __set:User#8 [2] User#8 appears in cache entry 2 + __set:User [2] Entry 2 is a User collection (for create/1) +``` + +When User#5 is mutated, SchemaCache reads `__set:User#5`, resolves +IDs `[1, 2]` back to cache keys via KeyRegistry, and evicts or +updates both entries. + +## Cache-Aside Flow + +On a cache miss, SchemaCache invokes the callback, caches the result, and records schema references. + +``` +SchemaCache.read("find_user", %{id: 5}, ttl, fn -> Repo.get(User, 5) end) + + +--------+ +-----------+ +----------+ + | Caller | -- 1 --> |SchemaCache| -- 2 -> | Adapter | + +--------+ +-----------+ +----------+ + ^ | | + | | cache miss (nil) | + | | <------- 3 ---------| + | | | + | | -- 4 -> callback() --> +------+ + | | | DB | + | | <- 5 -- {:ok, user} <-- +------+ + | | | + | | -- 6 -> put(key, | + | | user, ttl) | + | | | + | | -- 7 -> sadd( | + | | "User#5", | + | | key_id) | + | | | + | <-- 8 -- {:ok, user} | | + | | | + +Subsequent calls with the same key and params return +the cached value at step 3 without invoking the callback. +``` + +## Write-Through Flow + +Write-through updates cached values in place, avoiding cache misses after mutations. + +``` +SchemaCache.update(fn -> update_user(user, params) end, strategy: :write_through) + + +--------+ +-----------+ +----------+ + | Caller | -- 1 --> |SchemaCache| | Adapter | + +--------+ +-----------+ +----------+ + ^ | | + | | -- 2 -> callback() --> +------+ + | | | DB | + | | <- 3 -- {:ok, updated} +------+ + | | | + | | -- 4 -> smembers( | + | | "User#5") | + | | | + | | <- 5 -- [id1, id2] | + | | | + | | -- 6 -> resolve IDs, | + | | for each key: | + | | put(key, updated) | + | | | + | <- 7 -- {:ok, updated} | | + | | | + +For collection keys (prefixed with "all_"), SchemaCache +finds the specific item within the list by primary key +and replaces it in place. +``` + +## Eviction Flow + +Default update and create use eviction. Affected cache entries are deleted and the next read re-populates from the source. + +``` +SchemaCache.update(fn -> update_user(user, params) end) + + +--------+ +-----------+ +----------+ + | Caller | -- 1 --> |SchemaCache| | Adapter | + +--------+ +-----------+ +----------+ + ^ | | + | | -- 2 -> callback() --> +------+ + | | | DB | + | | <- 3 -- {:ok, updated} +------+ + | | | + | | -- 4 -> smembers( | + | | "User#5") | + | | | + | | <- 5 -- [id1, id2] | + | | | + | | -- 6 -> resolve IDs, | + | | for each key: | + | | delete(key) | + | | srem("User#5",id) | + | | | + | <- 7 -- {:ok, updated} | | + | | | + +create/1 behaves similarly but looks up keys by the +schema module name (e.g., "User") instead of the instance +identity (e.g., "User#5"), evicting all collections of +that type. +``` + +## KeyRegistry: Compact ID Storage + +`SchemaCache.KeyRegistry` maintains a bidirectional mapping between cache key strings and integer IDs. SMKES reference sets store these integer IDs instead of full strings. + +A cache key like `"all_users:{\"active\":true,\"order_by\":{\"name\":\"asc\"}}"` consumes ~80 bytes per set reference; an integer ID consumes ~8 bytes. + +The registry uses two ETS tables for O(1) lookups in both directions: + +- `:schema_cache_key_to_id`: `{cache_key, id}` +- `:schema_cache_id_to_key`: `{id, cache_key}` + +ID generation uses `:atomics` for lock-free monotonic assignment. + +## SetLock: Partitioned Lock Fallback + +Adapters that don't implement native set operations (`sadd/2`, `srem/2`, `smembers/1`) fall back to `SchemaCache.SetLock`. This module serializes read-modify-write cycles using a partitioned lock pool backed by `Registry`, storing sets as `MapSet` values in the adapter's key-value store. + +The lock is partitioned by `System.schedulers_online/0 * 4` to reduce contention. Concurrent writes to different set keys rarely contend, while writes to the same key are serialized by hashing to the same partition. + +Adapters with native set operations (e.g., Redis SADD/SREM/SMEMBERS) bypass SetLock entirely. + +## Capability Resolution + +At startup, `SchemaCache.Supervisor` resolves the adapter's capabilities by checking which optional callbacks are implemented. This result is stored in a persistent term for O(1) access at runtime. + +The resolved capabilities determine routing for all adapter operations: + +- **Native set operations** (`sadd/2`, `srem/2`, `smembers/1`): Route directly to the adapter. +- **ElixirCache Redis** (`command/1`): Route set operations and `mget` through raw Redis commands. +- **Neither**: Set operations fall back to `SchemaCache.SetLock`. +- **Native `mget/1`**: Batch reads use the adapter's implementation. +- **Without `mget/1`**: Falls back to sequential `get/1` calls. + +### ElixirCache Auto-Detection + +Modules created with `use Cache` from [ElixirCache](https://github.com/MikaAK/elixir_cache) are detected automatically via `function_exported?(adapter, :cache_adapter, 0)`. SchemaCache translates API signatures at the dispatch layer. ElixirCache uses positional TTL (`put(key, ttl, value)`) while SchemaCache's behaviour uses keyword opts (`put(key, value, ttl: ttl)`). This translation is invisible to callers. + +For Redis-backed ElixirCache modules, SchemaCache further detects `command/1` and uses it for native set operations and batch reads, bypassing the SetLock fallback entirely. diff --git a/guides/how-to/using_with_elixir_cache.md b/guides/how-to/using_with_elixir_cache.md new file mode 100644 index 0000000..9800c52 --- /dev/null +++ b/guides/how-to/using_with_elixir_cache.md @@ -0,0 +1,95 @@ +# How to Use SchemaCache with ElixirCache + +[ElixirCache](https://github.com/MikaAK/elixir_cache) provides a standardized caching interface with adapters for ETS, Redis, DETS, ConCache, and more. SchemaCache detects ElixirCache modules automatically and translates API signatures, so you can use them directly. No wrapper module needed. + +## Why Use ElixirCache as a Backend? + +- **Production-ready Redis support** with connection pooling, hash operations, and JSON commands. +- **Telemetry integration** for cache hit/miss metrics and monitoring. +- **Sandbox mode** for isolated testing with concurrent test support. +- **Consistent interface** if you're already using ElixirCache elsewhere in your application. + +## Setup + +### 1. Add Dependencies + +```elixir +# mix.exs +def deps do + [ + {:schema_cache, "~> 0.1.0"}, + {:elixir_cache, "~> 0.3.8"} + ] +end +``` + +### 2. Define Your ElixirCache Module + +```elixir +defmodule MyApp.Cache do + use Cache, + adapter: Cache.Redis, + name: :my_app_cache, + opts: [ + uri: System.get_env("REDIS_URL", "redis://localhost:6379"), + pool_size: 10 + ], + sandbox?: Mix.env() == :test +end +``` + +### 3. Add Both to Your Supervision Tree + +```elixir +def start(_type, _args) do + children = [ + MyApp.Cache, + {SchemaCache.Supervisor, adapter: MyApp.Cache}, + # ... other children + ] + + opts = [strategy: :one_for_one, name: MyApp.Supervisor] + Supervisor.start_link(children, opts) +end +``` + +Start `MyApp.Cache` before `SchemaCache.Supervisor` so the backing store is available when SchemaCache initializes. + +That's it. No config files, no wrapper modules. + +## How Auto-Detection Works + +At startup, `SchemaCache.Supervisor` resolves the adapter's capabilities. The detection checks: + +1. **ElixirCache module?** Checks `function_exported?(adapter, :cache_adapter, 0)`, which returns `true` for any module created with `use Cache`. +2. **Redis-backed?** If the module is an ElixirCache module AND `adapter.cache_adapter()` returns `Cache.Redis` AND the module exports `command/1`. + +### What This Enables + +- **API translation**: ElixirCache uses `put(key, value)` and `put(key, ttl, value)` with positional TTL. SchemaCache's adapter behaviour uses `put(key, value, opts)` with keyword TTL. The dispatch layer handles this translation automatically. +- **Native Redis set operations**: For Redis-backed modules, SchemaCache routes `sadd`, `srem`, `smembers`, and `mget` through `command/1` using raw Redis commands. This gives you full SMKES performance without the partitioned lock fallback. + +### Capability Summary + +| Capability | Local ElixirCache | Redis ElixirCache | +|---|---|---| +| get/put/delete | Auto-translated | Auto-translated | +| sadd/srem/smembers | SetLock fallback | Native via `command/1` | +| mget | Sequential fallback | Native via `command/1` | + +## Testing + +If your ElixirCache module has `sandbox?: true`, use the sandbox registry in your test setup: + +```elixir +# test/test_helper.exs +{:ok, _pid} = Cache.SandboxRegistry.start_link() + +# test/support/data_case.ex +setup do + Cache.SandboxRegistry.start(MyApp.Cache) + :ok +end +``` + +This gives each test an isolated cache namespace, preventing interference between concurrent tests. diff --git a/guides/how-to/writing_adapters.md b/guides/how-to/writing_adapters.md new file mode 100644 index 0000000..b82f77d --- /dev/null +++ b/guides/how-to/writing_adapters.md @@ -0,0 +1,137 @@ +# How to Write a Custom Adapter + +SchemaCache is adapter-agnostic. Any module implementing the `SchemaCache.Adapter` behaviour can serve as the cache backend. + +## Required Callbacks + +Every adapter must implement three callbacks: + +```elixir +defmodule MyApp.SchemaCacheAdapter do + @behaviour SchemaCache.Adapter + + @impl true + def get(key) do + case MyApp.Cache.get(key) do + nil -> {:ok, nil} + value -> {:ok, value} + end + end + + @impl true + def put(key, value, opts) do + ttl = Keyword.get(opts, :ttl) + MyApp.Cache.put(key, value, ttl: ttl) + :ok + end + + @impl true + def delete(key) do + MyApp.Cache.delete(key) + :ok + end +end +``` + +### Return Conventions + +- `get/1`: `{:ok, value}` on hit, `{:ok, nil}` on miss, `{:error, reason}` on failure. +- `put/3`: `:ok` or `{:error, reason}`. +- `delete/1`: `:ok` or `{:error, reason}`. + +### TTL Handling + +TTL is passed as `:ttl` in the `opts` keyword list of `put/3`. If your backend supports TTL, extract it with `Keyword.get(opts, :ttl)` and use it. If not, ignore it. SchemaCache does not implement its own TTL mechanism. + +## Optional Callbacks + +### `init/0` + +Called by `SchemaCache.Supervisor` during startup. Use it to create ETS tables, open connections, or perform other initialization. + +```elixir +@impl true +def init do + :ets.new(:my_cache_table, [:set, :public, :named_table]) + :ok +end +``` + +### Native Set Operations + +Implementing `sadd/2`, `srem/2`, and `smembers/1` lets your adapter handle SMKES reference sets natively instead of falling back to the partitioned lock mechanism in `SchemaCache.SetLock`. + +This is especially valuable for backends like Redis that have built-in set operations: + +```elixir +@impl true +def sadd(key, member) do + case Redix.command(:redis, ["SADD", key, to_string(member)]) do + {:ok, _} -> :ok + {:error, reason} -> {:error, reason} + end +end + +@impl true +def srem(key, member) do + case Redix.command(:redis, ["SREM", key, to_string(member)]) do + {:ok, _} -> :ok + {:error, reason} -> {:error, reason} + end +end + +@impl true +def smembers(key) do + case Redix.command(:redis, ["SMEMBERS", key]) do + {:ok, []} -> {:ok, nil} + {:ok, members} -> {:ok, Enum.map(members, &String.to_integer/1)} + {:error, reason} -> {:error, reason} + end +end +``` + +Members are always integer IDs from `SchemaCache.KeyRegistry`. Return `{:ok, nil}` for empty or non-existent sets. + +### Batch Reads with `mget/1` + +Implementing `mget/1` enables efficient batch reads during write-through operations: + +```elixir +@impl true +def mget(keys) do + case Redix.command(:redis, ["MGET" | keys]) do + {:ok, values} -> + {:ok, Enum.map(values, fn + nil -> nil + binary -> :erlang.binary_to_term(binary) + end)} + + {:error, reason} -> + {:error, reason} + end +end +``` + +Without `mget/1`, SchemaCache falls back to sequential `get/1` calls. + +## Configuration + +Pass your adapter to `SchemaCache.Supervisor` in your supervision tree: + +```elixir +children = [ + {SchemaCache.Supervisor, adapter: MyApp.SchemaCacheAdapter}, + # ... other children +] +``` + +## Capability Resolution + +At startup, SchemaCache inspects which optional callbacks your adapter implements and stores the result in a persistent term. This determines routing at runtime: + +- **With** `sadd/2`, `srem/2`, `smembers/1`: Set operations go directly to your adapter. +- **Without**: Set operations route through `SchemaCache.SetLock`. +- **With** `mget/1`: Batch reads use your implementation. +- **Without**: Falls back to sequential `get/1` calls. + +No configuration is needed. Capability detection is automatic. diff --git a/guides/introduction.md b/guides/introduction.md new file mode 100644 index 0000000..b361f60 --- /dev/null +++ b/guides/introduction.md @@ -0,0 +1,22 @@ +# Introduction to SchemaCache + +SchemaCache is an Ecto-aware caching library for Elixir that provides **cache-aside** and **write-through** abstractions with automatic invalidation. When an Ecto schema is mutated, every affected cache entry is evicted or updated. No invalidation logic required. + +## Key Features + +- **Cache-aside**: On a cache miss, SchemaCache calls your function to fetch from the source and caches the result. When underlying schemas change, affected entries are evicted automatically. +- **Write-through**: When your application can't tolerate stale results, write-through updates cached values in place after mutations. +- **Adapter-agnostic**: Plug in any caching backend (ETS, Redis, Nebulex, Cachex, or your own). Adapters only need `get/1`, `put/3`, and `delete/1`. + +## When to Use SchemaCache + +SchemaCache is a good fit when: + +- Your application caches Ecto query results and can't serve stale data after mutations. +- You want cache-aside with invalidation driven by schema changes. +- You need write-through caching to keep cached values fresh after writes. +- You need a backend-agnostic caching layer that works with any storage engine. + +## Getting Started + +Check out the [installation guide](tutorials/installation.md) to add SchemaCache to your project. diff --git a/guides/tutorials/basic_operations.md b/guides/tutorials/basic_operations.md new file mode 100644 index 0000000..3ea1385 --- /dev/null +++ b/guides/tutorials/basic_operations.md @@ -0,0 +1,124 @@ +# Basic Operations + +SchemaCache provides four core operations: **read**, **create**, **update**, and **delete**. Each operation is Ecto-aware. SchemaCache inspects query results to find Ecto structs and automatically maintains a reverse index from schemas to cache keys. + +## Cache-Aside + +`read/4` checks the cache first. On a miss, it invokes your callback, caches the result, and records which schemas appear in it for later eviction. + +### Caching a Single Record + +```elixir +{:ok, user} = + SchemaCache.read("find_user", %{id: 5}, :timer.minutes(15), fn -> + MyApp.Users.find(%{id: 5}) + end) +``` + +The first call executes the callback and caches the result. Subsequent calls with the same key and params return the cached value directly. + +### Caching a Collection + +Prefix collection keys with `"all_"`. This convention is required for write-through to distinguish collections from singular values. + +```elixir +{:ok, users} = + SchemaCache.read("all_active_users", %{active: true}, :timer.minutes(5), fn -> + {:ok, MyApp.Users.all(%{active: true})} + end) +``` + +### How the Cache Key Works + +The cache key is derived from the first two arguments: `key` and `params`. Params are deterministically serialized, so `%{a: 1, b: 2}` and `%{b: 2, a: 1}` produce the same cache key. + +## Create + +After inserting a new record, `create/1` evicts all cached collections for that schema type. This ensures the next read includes the new record. + +```elixir +{:ok, new_user} = + SchemaCache.create(fn -> + %User{} + |> User.changeset(%{name: "Bob", email: "bob@example.com"}) + |> Repo.insert() + end) +``` + +On success, SchemaCache looks up all cache keys associated with the `User` schema type (collection references) and evicts them. + +## Update + +### Eviction (Default) + +By default, `update/2` evicts all cache keys that reference the mutated schema instance. The next read will fetch fresh data. + +```elixir +{:ok, updated_user} = + SchemaCache.update(fn -> + MyApp.Users.update_user(user, %{name: "Alice"}) + end) +``` + +### Write Through + +With `:write_through`, SchemaCache updates cached values in place instead of evicting them. This avoids cache misses after writes. + +```elixir +{:ok, updated_user} = + SchemaCache.update( + fn -> MyApp.Users.update_user(user, %{name: "Alice"}) end, + strategy: :write_through + ) +``` + +Write-through handles both singular cached values and collections. For collections, SchemaCache finds the specific item by primary key and replaces it in the list. + +## Delete + +After a deletion, `delete/1` evicts all cache keys that reference the deleted schema instance. + +```elixir +{:ok, deleted_user} = + SchemaCache.delete(fn -> Repo.delete(user) end) +``` + +## Flush + +`flush/2` evicts all cache entries associated with a specific schema instance. Use this when you need to invalidate cache entries for a schema outside of a create/update/delete flow. + +```elixir +# Evict all cached queries that include User#5 +SchemaCache.flush(user, :timer.minutes(15)) +``` + +The TTL argument is passed to the adapter when re-storing reference sets (if applicable). + +## Key Conventions + +Cache keys are composed from two parts: a name and params map. The name determines the cache key prefix, and the params are deterministically serialized into the key. + +### The `"all_"` Prefix + +Keys prefixed with `"all_"` are treated as collection keys. This distinction matters for: + +- **`create/1`**: Only evicts collection keys (prefixed with `"all_"`) for the schema type. +- **Write-through**: Updates items within lists for collection keys, replaces the entire value for singular keys. + +```elixir +# Singular (no prefix) +SchemaCache.read("find_user", %{id: 5}, ttl, fn -> ... end) + +# Collection ("all_" prefix required) +SchemaCache.read("all_users", %{active: true}, ttl, fn -> ... end) +``` + +### Params Determinism + +Params maps are serialized deterministically using sorted keys and Jason encoding. This means `%{a: 1, b: 2}` and `%{b: 2, a: 1}` produce the same cache key. + +## Next Steps + +- Understand [how invalidation works](../explanation/architecture.md) under the hood. +- Learn how to [write a custom adapter](../how-to/writing_adapters.md). +- Integrate with [ElixirCache](../how-to/using_with_elixir_cache.md) for a production-ready backend. diff --git a/guides/tutorials/installation.md b/guides/tutorials/installation.md new file mode 100644 index 0000000..b800861 --- /dev/null +++ b/guides/tutorials/installation.md @@ -0,0 +1,58 @@ +# Installation + +## Adding SchemaCache to Your Project + +Add `schema_cache` to your dependencies in `mix.exs`: + +```elixir +def deps do + [ + {:schema_cache, "~> 0.1.0"} + ] +end +``` + +Then fetch dependencies: + +```bash +mix deps.get +``` + +## Adding to Your Supervision Tree + +Add `SchemaCache.Supervisor` to your application's supervision tree with the +adapter you want to use: + +```elixir +# lib/my_app/application.ex +def start(_type, _args) do + children = [ + {SchemaCache.Supervisor, adapter: SchemaCache.Adapters.ETS}, + # ... other children + ] + + opts = [strategy: :one_for_one, name: MyApp.Supervisor] + Supervisor.start_link(children, opts) +end +``` + +The supervisor initializes the adapter (creating ETS tables, opening connections, etc.) and starts the `SchemaCache.KeyRegistry` for compact key ID management. + +## Verifying Your Installation + +Start an IEx session and verify the cache is working: + +```elixir +iex> SchemaCache.read("test_key", %{}, :timer.minutes(5), fn -> {:ok, "hello"} end) +{:ok, "hello"} + +iex> SchemaCache.read("test_key", %{}, :timer.minutes(5), fn -> {:ok, "this won't run"} end) +{:ok, "hello"} +``` + +The second call returns the cached value without invoking the callback. + +## Next Steps + +- Learn the [core operations](basic_operations.md): read, create, update, delete, flush, and write-through. +- Integrate with [ElixirCache](../how-to/using_with_elixir_cache.md) for a full-featured backend. diff --git a/lib/schema_cache.ex b/lib/schema_cache.ex index 14b8afd..595c939 100644 --- a/lib/schema_cache.ex +++ b/lib/schema_cache.ex @@ -1,40 +1,37 @@ defmodule SchemaCache do @moduledoc """ - An Ecto-aware caching library implementing Read Through, Write Through, - and Schema Mutation Key Eviction Strategy (SMKES). + An Ecto-aware caching library providing cache-aside and write-through + abstractions with automatic invalidation. - ## Schema Mutation Key Eviction Strategy (SMKES) + ## How Invalidation Works - SMKES is the core value of this library. It maintains a reverse index - from your Ecto schemas to every cache key where they appear. When a - schema is mutated (created, updated, or deleted), SchemaCache knows - exactly which cached values are affected and evicts or updates them - automatically, requiring no manual cache invalidation. - - SMKES works by storing cache key references in sets, keyed by schema - identity. When a query result is cached via `read/4`, SchemaCache - records which schemas appear in that result. On mutation, it looks up - all cache keys referencing the mutated schema and takes action: + SchemaCache maintains a reverse index from Ecto schemas to cache keys + (SMKES). When a query result is cached via `read/4`, it records which + schemas appear in that result. On mutation, it looks up all cache keys + referencing the mutated schema and takes action: * **`create/1`**: evicts every cached collection for the schema type, so the next read includes the new record. * **`update/2`** with `:evict` (default): deletes every cached entry referencing the mutated schema instance. * **`update/2`** with `:write_through`: updates every cached entry - in place with the new schema value, avoiding cache misses entirely. + in place with the new schema value. Use when your application + can't serve stale results. * **`delete/1`**: deletes every cached entry referencing the deleted schema instance. ## Adapter - SchemaCache is adapter-agnostic. Configure your cache backend: + SchemaCache is adapter-agnostic. Pass your adapter to the supervisor: - config :schema_cache, adapter: SchemaCache.Adapters.ETS + children = [ + {SchemaCache.Supervisor, adapter: SchemaCache.Adapters.ETS} + ] - Any module implementing `SchemaCache.Adapter` can be used. This makes - it easy to integrate with Nebulex, ConCache, Cachex, or any other - caching library. See `SchemaCache.Adapter` for the behaviour - specification and an example implementation. + Any module implementing `SchemaCache.Adapter` can be used. Modules + built with [ElixirCache](https://github.com/MikaAK/elixir_cache) + (`use Cache`) are detected automatically and work without a wrapper. + See `SchemaCache.Adapter` for details. ## TTL @@ -42,9 +39,10 @@ defmodule SchemaCache do it will be used. If not, it is ignored. SchemaCache does not implement its own TTL mechanism. - ## Read Through + ## Cache-Aside - The cache is responsible for fetching data from the source on a miss: + On a cache miss, SchemaCache calls your function to fetch from the + source and caches the result: SchemaCache.read("find_user", %{id: 5}, :timer.minutes(15), fn -> MyApp.Users.find(%{id: 5}) @@ -54,9 +52,10 @@ defmodule SchemaCache do callback. On a miss, the callback is invoked, the result is cached, and schema key references are recorded for SMKES. - ## Write Through + ## Write-Through - Updates go through the cache, which updates cached values in place: + When your application can't serve stale results, write-through + updates cached values in place after mutations: SchemaCache.update( fn -> MyApp.Users.update_user(user, params) end, @@ -90,11 +89,10 @@ defmodule SchemaCache do defp set_key(key), do: "__set:#{key}" defp adapter do - with nil <- :persistent_term.get(:schema_cache_adapter, nil), - nil <- Application.get_env(:schema_cache, :adapter) do + with nil <- :persistent_term.get(:schema_cache_adapter, nil) do raise """ SchemaCache adapter not configured. - Set config :schema_cache, adapter: YourAdapter + Start SchemaCache.Supervisor with adapter: YourAdapter """ end end @@ -287,7 +285,7 @@ defmodule SchemaCache do ttl, %_{} = value ) do - case adapter().get(key_ref) do + case Adapter.get(adapter(), key_ref) do {:ok, nil} -> :ok @@ -300,7 +298,7 @@ defmodule SchemaCache do end defp update_key_ref(key_ref, ttl, value) do - adapter().put(key_ref, value, ttl: ttl) + Adapter.put(adapter(), key_ref, value, ttl) end defp maybe_update_cached_collection( @@ -327,7 +325,7 @@ defmodule SchemaCache do idx -> cached_collection |> List.replace_at(idx, value) - |> then(&adapter().put(key_ref, &1, ttl: ttl)) + |> then(&Adapter.put(adapter(), key_ref, &1, ttl)) end end @@ -391,7 +389,7 @@ defmodule SchemaCache do def read(key, params, ttl \\ nil, fnc) do cache_key = KeyGenerator.cache_key(key, params) - case adapter().get(cache_key) do + case Adapter.get(adapter(), cache_key) do {:ok, nil} -> get_set_value(cache_key, ttl, fnc) @@ -409,7 +407,7 @@ defmodule SchemaCache do defp get_set_value(cache_key, ttl, fnc) do case fnc.() do {:ok, value} -> - adapter().put(cache_key, value, ttl: ttl) + Adapter.put(adapter(), cache_key, value, ttl) associate_key_reference_with_schema(cache_key, value) {:ok, value} @@ -417,7 +415,7 @@ defmodule SchemaCache do [] value when is_list(value) -> - adapter().put(cache_key, value, ttl: ttl) + Adapter.put(adapter(), cache_key, value, ttl) associate_key_reference_with_schema(cache_key, value) associate_key_reference_with_schema_type(cache_key, value) value @@ -508,7 +506,7 @@ defmodule SchemaCache do end) maybe_async_each(live, fn {{id, key_ref}, _} -> - a.delete(key_ref) + Adapter.delete(a, key_ref) Adapter.srem(a, set_k, id) KeyRegistry.unregister_id(id) end) diff --git a/lib/schema_cache/adapter.ex b/lib/schema_cache/adapter.ex index c7ec5ad..a50d9b4 100644 --- a/lib/schema_cache/adapter.ex +++ b/lib/schema_cache/adapter.ex @@ -2,12 +2,30 @@ defmodule SchemaCache.Adapter do @moduledoc """ Behaviour for cache adapter implementations. - SchemaCache is adapter-agnostic. Implement this behaviour to plug in - any caching backend: Nebulex, ConCache, Cachex, Redis, ETS, etc. + SchemaCache is adapter-agnostic. Any module implementing this behaviour + can serve as the cache backend: Nebulex, ConCache, Cachex, Redis, ETS, etc. - The behaviour requires three core operations: `get/1`, `put/3`, and - `delete/1`. SchemaCache builds its key reference tracking (SMKES) on - top of these primitives. + ## ElixirCache Integration + + Modules created with `use Cache` from + [ElixirCache](https://github.com/MikaAK/elixir_cache) are detected + automatically and work without a wrapper module. Just pass the module + directly: + + children = [ + MyApp.Cache, + {SchemaCache.Supervisor, adapter: MyApp.Cache} + ] + + For Redis-backed ElixirCache modules, SchemaCache also auto-detects + native set operations via `command/1`, giving you full SMKES performance + with zero configuration. + + ## Custom Adapters + + Implement three required callbacks: `get/1`, `put/3`, and `delete/1`. + SchemaCache builds its key reference tracking (SMKES) on top of these + primitives. Adapters can optionally implement `init/0`, `sadd/2`, `srem/2`, `smembers/1`, and `mget/1`. `init/0` is called by @@ -33,12 +51,6 @@ defmodule SchemaCache.Adapter do and use it. If not, ignore it. SchemaCache does not implement its own TTL mechanism. - ## Configuration - - Set the adapter in your application config: - - config :schema_cache, adapter: MyApp.SchemaCacheAdapter - ## Example: Nebulex Adapter defmodule MyApp.SchemaCacheAdapter do @@ -149,58 +161,135 @@ defmodule SchemaCache.Adapter do @doc false def init(adapter) do - if function_exported?(adapter, :init, 0) do - adapter.init() - else - :ok - end + if function_exported?(adapter, :init, 0), + do: adapter.init(), + else: :ok end @doc false def resolve_capabilities(adapter) do + Code.ensure_loaded(adapter) + elixir_cache? = elixir_cache?(adapter) + redis_backed? = elixir_cache? and detect_redis(adapter) + :persistent_term.put(:schema_cache_adapter_caps, %{ - sadd: function_exported?(adapter, :sadd, 2), - srem: function_exported?(adapter, :srem, 2), - smembers: function_exported?(adapter, :smembers, 1), - mget: function_exported?(adapter, :mget, 1) + # ElixirCache modules export sadd/2 etc. with incompatible signatures + # (key prefixing, TermEncoder encoding, different return values). + # Only mark as native for non-ElixirCache adapters. + native_sadd: not elixir_cache? and function_exported?(adapter, :sadd, 2), + native_srem: not elixir_cache? and function_exported?(adapter, :srem, 2), + native_smembers: not elixir_cache? and function_exported?(adapter, :smembers, 1), + native_mget: not elixir_cache? and function_exported?(adapter, :mget, 1), + elixir_cache: elixir_cache?, + redis_backed: redis_backed? }) end - # --- Runtime dispatch --- + defp elixir_cache?(adapter) do + with true <- function_exported?(adapter, :cache_adapter, 0), + mod = adapter.cache_adapter(), + {:module, ^mod} <- Code.ensure_loaded(mod) do + Cache in (mod.__info__(:attributes)[:behaviour] || []) + else + _ -> false + end + rescue + _ -> false + end + + defp detect_redis(adapter) do + function_exported?(adapter, :command, 1) and + adapter.cache_adapter() == Cache.Redis + rescue + _ -> false + end + + # --- Runtime dispatch: core operations --- + + @doc false + def get(adapter, key), do: adapter.get(key) + + @doc false + def put(adapter, key, value, ttl) do + if capability(:elixir_cache), + do: elixir_cache_put(adapter, key, value, ttl), + else: adapter.put(key, value, ttl: ttl) + end + + @doc false + def put_no_ttl(adapter, key, value) do + if capability(:elixir_cache), + do: adapter.put(key, value), + else: adapter.put(key, value, []) + end + + @doc false + def delete(adapter, key), do: adapter.delete(key) + + defp elixir_cache_put(adapter, key, value, nil), do: adapter.put(key, value) + defp elixir_cache_put(adapter, key, value, ttl), do: adapter.put(key, ttl, value) + + # --- Runtime dispatch: set operations --- @doc false def sadd(adapter, key, member) do - if capability(:sadd) do - adapter.sadd(key, member) - else - SchemaCache.SetLock.sadd(key, member, adapter) + cond do + capability(:native_sadd) -> adapter.sadd(key, member) + capability(:redis_backed) -> redis_sadd(adapter, key, member) + true -> SchemaCache.SetLock.sadd(key, member, adapter) end end @doc false def srem(adapter, key, member) do - if capability(:srem) do - adapter.srem(key, member) - else - SchemaCache.SetLock.srem(key, member, adapter) + cond do + capability(:native_srem) -> adapter.srem(key, member) + capability(:redis_backed) -> redis_srem(adapter, key, member) + true -> SchemaCache.SetLock.srem(key, member, adapter) end end @doc false def smembers(adapter, key) do - if capability(:smembers) do - adapter.smembers(key) - else - SchemaCache.SetLock.smembers(key, adapter) + cond do + capability(:native_smembers) -> adapter.smembers(key) + capability(:redis_backed) -> redis_smembers(adapter, key) + true -> SchemaCache.SetLock.smembers(key, adapter) end end @doc false def mget(adapter, keys) do - if capability(:mget) do - adapter.mget(keys) - else - SchemaCache.SetLock.mget(keys, adapter) + # No redis_backed path. ElixirCache's key prefixing and TermEncoder + # encoding make raw MGET via command/1 unreliable. SetLock uses + # individual adapter.get/1 calls which handle both correctly. + if capability(:native_mget), + do: adapter.mget(keys), + else: SchemaCache.SetLock.mget(keys, adapter) + end + + # --- ElixirCache Redis command helpers --- + + defp redis_sadd(adapter, key, member) do + with {:ok, _} <- adapter.command(["SADD", key, to_string(member)]), do: :ok + end + + defp redis_srem(adapter, key, member) do + with {:ok, _} <- adapter.command(["SREM", key, to_string(member)]), do: :ok + end + + defp redis_smembers(adapter, key) do + case adapter.command(["SMEMBERS", key]) do + {:ok, []} -> + {:ok, nil} + + {:ok, members} -> + members + |> Enum.map(&String.to_integer/1) + |> then(&{:ok, &1}) + + {:error, reason} -> + {:error, reason} end end diff --git a/lib/schema_cache/adapters/ets.ex b/lib/schema_cache/adapters/ets.ex index 9a5d60d..77afc8b 100644 --- a/lib/schema_cache/adapters/ets.ex +++ b/lib/schema_cache/adapters/ets.ex @@ -28,9 +28,11 @@ defmodule SchemaCache.Adapters.ETS do * Single-node only, not shared across a cluster * No memory limits; the table grows without bound - ## Configuration + ## Usage - config :schema_cache, adapter: SchemaCache.Adapters.ETS + children = [ + {SchemaCache.Supervisor, adapter: SchemaCache.Adapters.ETS} + ] """ @behaviour SchemaCache.Adapter @@ -38,6 +40,21 @@ defmodule SchemaCache.Adapters.ETS do @data_table :schema_cache_ets @set_table :schema_cache_ets_sets + @doc """ + Returns the list of ETS tables managed by this adapter and SchemaCache internals. + + Useful for test cleanup. Call this instead of hardcoding table names. + """ + @spec managed_tables() :: [atom()] + def managed_tables do + [ + @data_table, + @set_table, + :schema_cache_key_to_id, + :schema_cache_id_to_key + ] + end + @impl true def init do :ets.new(@data_table, [:set, :public, :named_table]) diff --git a/lib/schema_cache/set_lock.ex b/lib/schema_cache/set_lock.ex index 12f30dd..af5c619 100644 --- a/lib/schema_cache/set_lock.ex +++ b/lib/schema_cache/set_lock.ex @@ -8,7 +8,7 @@ defmodule SchemaCache.SetLock do `Registry` as a lock pool, storing sets as `MapSet` values in the adapter's key-value store. - The lock is partitioned by `System.schedulers_online/0` to reduce + The lock is partitioned by `System.schedulers_online() * 4` to reduce contention. Concurrent writes to different set keys rarely contend, while writes to the same key are serialized by hashing to the same partition. @@ -34,11 +34,11 @@ defmodule SchemaCache.SetLock do case Registry.register(@registry, lock_key, nil) do {:ok, _} -> try do - set_key - |> adapter.get() + adapter + |> SchemaCache.Adapter.get(set_key) |> current_set() |> update_fn.() - |> then(&adapter.put(set_key, &1, [])) + |> then(&SchemaCache.Adapter.put_no_ttl(adapter, set_key, &1)) :ok after @@ -66,8 +66,8 @@ defmodule SchemaCache.SetLock do @spec smembers(String.t(), module()) :: {:ok, list() | nil} | {:error, any()} def smembers(set_key, adapter) do - set_key - |> adapter.get() + adapter + |> SchemaCache.Adapter.get(set_key) |> format_smembers_result() end @@ -87,7 +87,7 @@ defmodule SchemaCache.SetLock do def mget(keys, adapter) do keys |> Enum.map(fn key -> - case adapter.get(key) do + case SchemaCache.Adapter.get(adapter, key) do {:ok, value} -> value _ -> nil end diff --git a/lib/schema_cache/supervisor.ex b/lib/schema_cache/supervisor.ex index 0afb0f1..c675e09 100644 --- a/lib/schema_cache/supervisor.ex +++ b/lib/schema_cache/supervisor.ex @@ -1,5 +1,26 @@ defmodule SchemaCache.Supervisor do - @moduledoc false + @moduledoc """ + Supervisor for the SchemaCache system. + + Add this to your application's supervision tree to initialize the cache + adapter, resolve adapter capabilities, and start internal processes. + + ## Usage + + children = [ + {SchemaCache.Supervisor, adapter: SchemaCache.Adapters.ETS}, + # ... other children + ] + + Any module implementing `SchemaCache.Adapter` works, as well as any + ElixirCache module (detected automatically): + + children = [ + MyApp.Cache, + {SchemaCache.Supervisor, adapter: MyApp.Cache} + ] + + """ use Supervisor @@ -12,10 +33,12 @@ defmodule SchemaCache.Supervisor do @impl true def init(opts) do adapter = - with nil <- Keyword.get(opts, :adapter), - nil <- Application.get_env(:schema_cache, :adapter) do - raise "SchemaCache adapter not configured. Set config :schema_cache, adapter: YourAdapter" - end + Keyword.get(opts, :adapter) || + raise ArgumentError, + """ + SchemaCache adapter not configured. + Pass adapter: YourAdapter to SchemaCache.Supervisor + """ :persistent_term.put(:schema_cache_adapter, adapter) diff --git a/mix.exs b/mix.exs index 2af9f5c..3af65cb 100644 --- a/mix.exs +++ b/mix.exs @@ -18,8 +18,8 @@ defmodule SchemaCache.MixProject do name: "SchemaCache", source_url: @source_url, description: - "An Ecto-aware caching library implementing Read Through, Write Through, and Schema Mutation Key Eviction Strategy (SMKES).", - dialyzer: [plt_add_apps: [:ecto, :ex_unit]], + "An Ecto-aware caching library providing cache-aside and write-through abstractions with automatic invalidation.", + dialyzer: [plt_add_apps: [:ecto, :ex_unit], ignore_warnings: ".dialyzer_ignore.exs"], test_coverage: [tool: ExCoveralls] ] end @@ -56,7 +56,8 @@ defmodule SchemaCache.MixProject do {:credo, "~> 1.7", only: [:dev, :test], runtime: false}, {:dialyxir, "~> 1.4", only: [:dev, :test], runtime: false}, {:excoveralls, "~> 0.18", only: :test}, - {:redix, "~> 1.5", only: :test} + {:redix, "~> 1.5", only: :test}, + {:elixir_cache, "~> 0.3", only: :test} ] end @@ -79,12 +80,31 @@ defmodule SchemaCache.MixProject do main: "readme", source_ref: "v#{@version}", source_url: @source_url, - extras: ["README.md"], + extras: [ + "README.md", + "guides/introduction.md", + "guides/tutorials/installation.md", + "guides/tutorials/basic_operations.md", + "guides/how-to/writing_adapters.md", + "guides/how-to/using_with_elixir_cache.md", + "guides/explanation/architecture.md", + "LICENSE" + ], + groups_for_extras: [ + "Getting Started": ["guides/introduction.md"], + Tutorials: [~r{guides/tutorials/.?}], + "How-to Guides": [~r{guides/how-to/.?}], + Explanation: [~r{guides/explanation/.?}] + ], groups_for_modules: [ - Core: [SchemaCache], + Core: [SchemaCache, SchemaCache.Supervisor], "Adapter Behaviour": [SchemaCache.Adapter], "Built-in Adapters": [SchemaCache.Adapters.ETS], - Internals: [SchemaCache.KeyGenerator] + Internals: [ + SchemaCache.KeyGenerator, + SchemaCache.KeyRegistry, + SchemaCache.SetLock + ] ] ] end diff --git a/mix.lock b/mix.lock index 93f2fb9..ace7d7c 100644 --- a/mix.lock +++ b/mix.lock @@ -1,5 +1,6 @@ %{ "bunt": {:hex, :bunt, "1.0.0", "081c2c665f086849e6d57900292b3a161727ab40431219529f13c4ddcf3e7a44", [:mix], [], "hexpm", "dc5f86aa08a5f6fa6b8096f0735c4e76d54ae5c9fa2c143e5a1fc7c1cd9bb6b5"}, + "con_cache": {:hex, :con_cache, "1.1.1", "9f47a68dfef5ac3bbff8ce2c499869dbc5ba889dadde6ac4aff8eb78ddaf6d82", [:mix], [{:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "1def4d1bec296564c75b5bbc60a19f2b5649d81bfa345a2febcc6ae380e8ae15"}, "credo": {:hex, :credo, "1.7.16", "a9f1389d13d19c631cb123c77a813dbf16449a2aebf602f590defa08953309d4", [:mix], [{:bunt, "~> 0.2.1 or ~> 1.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2 or ~> 1.0", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "d0562af33756b21f248f066a9119e3890722031b6d199f22e3cf95550e4f1579"}, "db_connection": {:hex, :db_connection, "2.9.0", "a6a97c5c958a2d7091a58a9be40caf41ab496b0701d21e1d1abff3fa27a7f371", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "17d502eacaf61829db98facf6f20808ed33da6ccf495354a41e64fe42f9c509c"}, "decimal": {:hex, :decimal, "2.3.0", "3ad6255aa77b4a3c4f818171b12d237500e63525c2fd056699967a3e7ea20f62", [:mix], [], "hexpm", "a4d66355cb29cb47c3cf30e71329e58361cfcb37c34235ef3bf1d7bf3773aeac"}, @@ -7,7 +8,9 @@ "earmark_parser": {:hex, :earmark_parser, "1.4.44", "f20830dd6b5c77afe2b063777ddbbff09f9759396500cdbe7523efd58d7a339c", [:mix], [], "hexpm", "4778ac752b4701a5599215f7030989c989ffdc4f6df457c5f36938cc2d2a2750"}, "ecto": {:hex, :ecto, "3.13.5", "9d4a69700183f33bf97208294768e561f5c7f1ecf417e0fa1006e4a91713a834", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "df9efebf70cf94142739ba357499661ef5dbb559ef902b68ea1f3c1fabce36de"}, "ecto_sql": {:hex, :ecto_sql, "3.13.4", "b6e9d07557ddba62508a9ce4a484989a5bb5e9a048ae0e695f6d93f095c25d60", [:mix], [{:db_connection, "~> 2.4.1 or ~> 2.5", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.13.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.7", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.19 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "2b38cf0749ca4d1c5a8bcbff79bbe15446861ca12a61f9fba604486cb6b62a14"}, + "elixir_cache": {:hex, :elixir_cache, "0.3.12", "dbfa92d82bf46022f7b3586f881e7a41bc6d25cbae145956de98479487170df0", [:mix], [{:con_cache, "~> 1.0", [hex: :con_cache, repo: "hexpm", optional: false]}, {:error_message, "~> 0.3", [hex: :error_message, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:nimble_options, "~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, repo: "hexpm", optional: false]}, {:prometheus_telemetry, "~> 0.3", [hex: :prometheus_telemetry, repo: "hexpm", optional: true]}, {:redix, "~> 1.2", [hex: :redix, repo: "hexpm", optional: false]}, {:sandbox_registry, "~> 0.1", [hex: :sandbox_registry, repo: "hexpm", optional: false]}, {:telemetry, "~> 1.1", [hex: :telemetry, repo: "hexpm", optional: false]}, {:telemetry_metrics, "~> 1.0", [hex: :telemetry_metrics, repo: "hexpm", optional: false]}], "hexpm", "0ad074bd687c199b938bacbcf5775879076df442bf21485508013a3a27134c4b"}, "erlex": {:hex, :erlex, "0.2.8", "cd8116f20f3c0afe376d1e8d1f0ae2452337729f68be016ea544a72f767d9c12", [:mix], [], "hexpm", "9d66ff9fedf69e49dc3fd12831e12a8a37b76f8651dd21cd45fcf5561a8a7590"}, + "error_message": {:hex, :error_message, "0.3.3", "493b6edae73a438647db843a4ba3e2a45baeff4c0c94236fca7f77e2ada2806a", [:mix], [{:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:plug, "~> 1.13", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "671a0a78708b299d119acdadc5f771c8c3cc4b171a9562745936bfe117674b73"}, "ex_doc": {:hex, :ex_doc, "0.40.1", "67542e4b6dde74811cfd580e2c0149b78010fd13001fda7cfeb2b2c2ffb1344d", [:mix], [{:earmark_parser, "~> 1.4.44", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_c, ">= 0.1.0", [hex: :makeup_c, repo: "hexpm", optional: true]}, {:makeup_elixir, "~> 0.14 or ~> 1.0", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1 or ~> 1.0", [hex: :makeup_erlang, repo: "hexpm", optional: false]}, {:makeup_html, ">= 0.1.0", [hex: :makeup_html, repo: "hexpm", optional: true]}], "hexpm", "bcef0e2d360d93ac19f01a85d58f91752d930c0a30e2681145feea6bd3516e00"}, "excoveralls": {:hex, :excoveralls, "0.18.5", "e229d0a65982613332ec30f07940038fe451a2e5b29bce2a5022165f0c9b157e", [:mix], [{:castore, "~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "523fe8a15603f86d64852aab2abe8ddbd78e68579c8525ae765facc5eae01562"}, "file_system": {:hex, :file_system, "1.1.1", "31864f4685b0148f25bd3fbef2b1228457c0c89024ad67f7a81a3ffbc0bbad3a", [:mix], [], "hexpm", "7a15ff97dfe526aeefb090a7a9d3d03aa907e100e262a0f8f7746b78f8f87a5d"}, @@ -15,9 +18,15 @@ "makeup": {:hex, :makeup, "1.2.1", "e90ac1c65589ef354378def3ba19d401e739ee7ee06fb47f94c687016e3713d1", [:mix], [{:nimble_parsec, "~> 1.4", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "d36484867b0bae0fea568d10131197a4c2e47056a6fbe84922bf6ba71c8d17ce"}, "makeup_elixir": {:hex, :makeup_elixir, "1.0.1", "e928a4f984e795e41e3abd27bfc09f51db16ab8ba1aebdba2b3a575437efafc2", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "7284900d412a3e5cfd97fdaed4f5ed389b8f2b4cb49efc0eb3bd10e2febf9507"}, "makeup_erlang": {:hex, :makeup_erlang, "1.0.3", "4252d5d4098da7415c390e847c814bad3764c94a814a0b4245176215615e1035", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "953297c02582a33411ac6208f2c6e55f0e870df7f80da724ed613f10e6706afd"}, + "mime": {:hex, :mime, "2.0.7", "b8d739037be7cd402aee1ba0306edfdef982687ee7e9859bee6198c1e7e2f128", [:mix], [], "hexpm", "6171188e399ee16023ffc5b76ce445eb6d9672e2e241d2df6050f3c771e80ccd"}, "nimble_options": {:hex, :nimble_options, "1.1.1", "e3a492d54d85fc3fd7c5baf411d9d2852922f66e69476317787a7b2bb000a61b", [:mix], [], "hexpm", "821b2470ca9442c4b6984882fe9bb0389371b8ddec4d45a9504f00a66f650b44"}, "nimble_parsec": {:hex, :nimble_parsec, "1.4.2", "8efba0122db06df95bfaa78f791344a89352ba04baedd3849593bfce4d0dc1c6", [:mix], [], "hexpm", "4b21398942dda052b403bbe1da991ccd03a053668d147d53fb8c4e0efe09c973"}, + "plug": {:hex, :plug, "1.19.1", "09bac17ae7a001a68ae393658aa23c7e38782be5c5c00c80be82901262c394c0", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "560a0017a8f6d5d30146916862aaf9300b7280063651dd7e532b8be168511e62"}, + "plug_crypto": {:hex, :plug_crypto, "2.1.1", "19bda8184399cb24afa10be734f84a16ea0a2bc65054e23a62bb10f06bc89491", [:mix], [], "hexpm", "6470bce6ffe41c8bd497612ffde1a7e4af67f36a15eea5f921af71cf3e11247c"}, + "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm", "dad79704ce5440f3d5a3681c8590b9dc25d1a561e8f5a9c995281012860901e3"}, "postgrex": {:hex, :postgrex, "0.22.0", "fb027b58b6eab1f6de5396a2abcdaaeb168f9ed4eccbb594e6ac393b02078cbd", [:mix], [{:db_connection, "~> 2.9", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:table, "~> 0.1.0", [hex: :table, repo: "hexpm", optional: true]}], "hexpm", "a68c4261e299597909e03e6f8ff5a13876f5caadaddd0d23af0d0a61afcc5d84"}, "redix": {:hex, :redix, "1.5.3", "4eaae29c75e3285c0ff9957046b7c209aa7f72a023a17f0a9ea51c2a50ab5b0f", [:mix], [{:castore, "~> 0.1.0 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:nimble_options, "~> 0.5.0 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "7b06fb5246373af41f5826b03334dfa3f636347d4d5d98b4d455b699d425ae7e"}, + "sandbox_registry": {:hex, :sandbox_registry, "0.1.1", "db6a116bf8e9553111820274701e48d1971185e89b53044f47c2a8f8e74956d7", [:mix], [], "hexpm", "88e12dc6be1bb98a05439e2b1de87db4bc0c043222b4fe4b46b5da7a0cc44948"}, "telemetry": {:hex, :telemetry, "1.3.0", "fedebbae410d715cf8e7062c96a1ef32ec22e764197f70cda73d82778d61e7a2", [:rebar3], [], "hexpm", "7015fc8919dbe63764f4b4b87a95b7c0996bd539e0d499be6ec9d7f3875b79e6"}, + "telemetry_metrics": {:hex, :telemetry_metrics, "1.1.0", "5bd5f3b5637e0abea0426b947e3ce5dd304f8b3bc6617039e2b5a008adc02f8f", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "e7b79e8ddfde70adb6db8a6623d1778ec66401f366e9a8f5dd0955c56bc8ce67"}, } diff --git a/test/schema_cache/adapter_dispatch_test.exs b/test/schema_cache/adapter_dispatch_test.exs index b4e22a3..afaa885 100644 --- a/test/schema_cache/adapter_dispatch_test.exs +++ b/test/schema_cache/adapter_dispatch_test.exs @@ -4,16 +4,10 @@ defmodule SchemaCache.AdapterDispatchTest do use ExUnit.Case, async: false alias SchemaCache.Adapter - - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] + alias SchemaCache.Adapters.ETS setup do - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end @@ -69,7 +63,7 @@ defmodule SchemaCache.AdapterDispatchTest do end test "goes native with ETS adapter" do - adapter = SchemaCache.Adapters.ETS + adapter = ETS Adapter.resolve_capabilities(adapter) assert :ok = Adapter.sadd(adapter, "native_set", "member_1") @@ -106,7 +100,7 @@ defmodule SchemaCache.AdapterDispatchTest do end test "goes native with ETS adapter" do - adapter = SchemaCache.Adapters.ETS + adapter = ETS Adapter.resolve_capabilities(adapter) Adapter.sadd(adapter, "native_set", "member_1") @@ -144,7 +138,7 @@ defmodule SchemaCache.AdapterDispatchTest do end test "goes native with ETS adapter" do - adapter = SchemaCache.Adapters.ETS + adapter = ETS Adapter.resolve_capabilities(adapter) adapter.sadd("native_set", "x") @@ -179,7 +173,7 @@ defmodule SchemaCache.AdapterDispatchTest do end test "goes native with ETS adapter" do - adapter = SchemaCache.Adapters.ETS + adapter = ETS Adapter.resolve_capabilities(adapter) adapter.put("k1", "v1", []) @@ -189,4 +183,198 @@ defmodule SchemaCache.AdapterDispatchTest do assert results == ["v1", nil, "v2"] end end + + describe "resolve_capabilities/1" do + setup do + original_caps = :persistent_term.get(:schema_cache_adapter_caps) + + on_exit(fn -> + :persistent_term.put(:schema_cache_adapter_caps, original_caps) + end) + + :ok + end + + test "detects ElixirCache ETS module via @behaviour Cache" do + Adapter.resolve_capabilities(SchemaCache.Test.ElixirCacheETS) + + assert %{ + elixir_cache: true, + redis_backed: false, + native_sadd: false, + native_mget: false + } = :persistent_term.get(:schema_cache_adapter_caps) + end + + test "detects ElixirCache Redis module" do + Adapter.resolve_capabilities(SchemaCache.Test.ElixirCacheRedis) + + assert %{ + elixir_cache: true, + redis_backed: true, + native_sadd: false, + native_mget: false + } = :persistent_term.get(:schema_cache_adapter_caps) + end + + test "native adapter has no elixir_cache flags" do + Adapter.resolve_capabilities(ETS) + + assert %{ + elixir_cache: false, + redis_backed: false, + native_sadd: true, + native_mget: true + } = :persistent_term.get(:schema_cache_adapter_caps) + end + end + + describe "ElixirCache ETS dispatch" do + setup do + original_caps = :persistent_term.get(:schema_cache_adapter_caps) + Adapter.resolve_capabilities(SchemaCache.Test.ElixirCacheETS) + + on_exit(fn -> + :persistent_term.put(:schema_cache_adapter_caps, original_caps) + + if :ets.whereis(:schema_cache_test_ec_ets) != :undefined do + :ets.delete_all_objects(:schema_cache_test_ec_ets) + end + end) + + :ok + end + + test "put/4 with nil TTL" do + adapter = SchemaCache.Test.ElixirCacheETS + + assert :ok = Adapter.put(adapter, "key", "value", nil) + assert {:ok, "value"} = adapter.get("key") + end + + test "put/4 with TTL" do + adapter = SchemaCache.Test.ElixirCacheETS + + assert :ok = Adapter.put(adapter, "key", "value", 60_000) + assert {:ok, "value"} = adapter.get("key") + end + + test "put_no_ttl/3" do + adapter = SchemaCache.Test.ElixirCacheETS + + assert :ok = Adapter.put_no_ttl(adapter, "key", "value") + assert {:ok, "value"} = adapter.get("key") + end + + test "get/2" do + adapter = SchemaCache.Test.ElixirCacheETS + + adapter.put("key", "value") + assert {:ok, "value"} = Adapter.get(adapter, "key") + end + + test "delete/2" do + adapter = SchemaCache.Test.ElixirCacheETS + + adapter.put("key", "value") + assert :ok = Adapter.delete(adapter, "key") + assert {:ok, nil} = adapter.get("key") + end + + test "round-trips complex Elixir terms" do + adapter = SchemaCache.Test.ElixirCacheETS + value = %{nested: [1, :atom, {"tuple"}], binary: <<0, 255>>} + + Adapter.put(adapter, "complex", value, nil) + assert {:ok, ^value} = Adapter.get(adapter, "complex") + end + + test "set operations fall through to SetLock" do + adapter = SchemaCache.Test.ElixirCacheETS + + Adapter.sadd(adapter, "my_set", "member_1") + Adapter.sadd(adapter, "my_set", "member_2") + + assert {:ok, members} = Adapter.smembers(adapter, "my_set") + assert "member_1" in members + assert "member_2" in members + end + + test "mget falls through to SetLock" do + adapter = SchemaCache.Test.ElixirCacheETS + + Adapter.put(adapter, "k1", "v1", nil) + Adapter.put(adapter, "k2", "v2", nil) + + assert {:ok, ["v1", nil, "v2"]} = + Adapter.mget(adapter, ["k1", "missing", "k2"]) + end + end + + describe "ElixirCache Redis dispatch" do + setup do + redis_url = + Application.get_env( + :schema_cache, + :redis_url, + "redis://localhost:6379" + ) + + with {:ok, conn} <- Redix.start_link(redis_url), + {:ok, "OK"} <- Redix.command(conn, ["FLUSHDB"]) do + GenServer.stop(conn) + + start_supervised!(SchemaCache.Test.ElixirCacheRedis) + + original_caps = :persistent_term.get(:schema_cache_adapter_caps) + Adapter.resolve_capabilities(SchemaCache.Test.ElixirCacheRedis) + + on_exit(fn -> + :persistent_term.put(:schema_cache_adapter_caps, original_caps) + end) + + :ok + else + _ -> :skip + end + end + + @tag :redis + test "sadd/srem/smembers route through command/1" do + adapter = SchemaCache.Test.ElixirCacheRedis + + assert :ok = Adapter.sadd(adapter, "my_set", 42) + assert :ok = Adapter.sadd(adapter, "my_set", 99) + + assert {:ok, members} = Adapter.smembers(adapter, "my_set") + assert 42 in members + assert 99 in members + + assert :ok = Adapter.srem(adapter, "my_set", 42) + + assert {:ok, members} = Adapter.smembers(adapter, "my_set") + refute 42 in members + assert 99 in members + end + + @tag :redis + test "mget falls through to SetLock (individual gets)" do + adapter = SchemaCache.Test.ElixirCacheRedis + + Adapter.put(adapter, "k1", "v1", nil) + Adapter.put(adapter, "k2", "v2", nil) + + assert {:ok, ["v1", nil, "v2"]} = + Adapter.mget(adapter, ["k1", "missing", "k2"]) + end + + @tag :redis + test "smembers returns {:ok, nil} for empty set" do + assert {:ok, nil} = + Adapter.smembers( + SchemaCache.Test.ElixirCacheRedis, + "nonexistent" + ) + end + end end diff --git a/test/schema_cache/adapters/ets_set_ops_test.exs b/test/schema_cache/adapters/ets_set_ops_test.exs index 58346d6..3d20646 100644 --- a/test/schema_cache/adapters/ets_set_ops_test.exs +++ b/test/schema_cache/adapters/ets_set_ops_test.exs @@ -5,15 +5,8 @@ defmodule SchemaCache.Adapters.ETSSetOpsTest do alias SchemaCache.Adapters.ETS - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] - setup do - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end @@ -71,7 +64,7 @@ defmodule SchemaCache.Adapters.ETSSetOpsTest do assert {:ok, members} = ETS.smembers("my_set") assert is_list(members) - assert length(members) == 3 + assert 3 = length(members) assert Enum.sort(members) == ["a", "b", "c"] end diff --git a/test/schema_cache/adapters/redis_adapter_set_ops_test.exs b/test/schema_cache/adapters/redis_adapter_set_ops_test.exs index f94e429..7b3fcd3 100644 --- a/test/schema_cache/adapters/redis_adapter_set_ops_test.exs +++ b/test/schema_cache/adapters/redis_adapter_set_ops_test.exs @@ -52,7 +52,7 @@ defmodule SchemaCache.Adapters.RedisAdapterSetOpsTest do assert {:ok, members} = RedisAdapter.smembers("my_set") assert is_list(members) - assert length(members) == 3 + assert 3 = length(members) assert Enum.sort(members) == [1, 2, 3] end diff --git a/test/schema_cache/key_registry_test.exs b/test/schema_cache/key_registry_test.exs index 662fc59..1cb3cbd 100644 --- a/test/schema_cache/key_registry_test.exs +++ b/test/schema_cache/key_registry_test.exs @@ -17,13 +17,13 @@ defmodule SchemaCache.KeyRegistryTest do test "returns the same ID for the same key" do id1 = KeyRegistry.register("find_user:{\"id\":1}") id2 = KeyRegistry.register("find_user:{\"id\":1}") - assert id1 == id2 + assert ^id1 = id2 end test "returns different IDs for different keys" do id1 = KeyRegistry.register("find_user:{\"id\":1}") id2 = KeyRegistry.register("find_user:{\"id\":2}") - assert id1 != id2 + refute id1 == id2 end test "handles concurrent registrations of the same key" do @@ -34,7 +34,7 @@ defmodule SchemaCache.KeyRegistryTest do ids = Task.await_many(tasks) # All should get the same ID - assert Enum.uniq(ids) |> length() == 1 + assert 1 = ids |> Enum.uniq() |> length() end test "handles concurrent registrations of different keys" do @@ -45,7 +45,7 @@ defmodule SchemaCache.KeyRegistryTest do ids = Task.await_many(tasks) # All should be unique - assert Enum.uniq(ids) |> length() == 100 + assert 100 = ids |> Enum.uniq() |> length() end end @@ -75,7 +75,7 @@ defmodule SchemaCache.KeyRegistryTest do test "filters out stale/unregistered IDs" do id1 = KeyRegistry.register("key_a") result = KeyRegistry.resolve([id1, 999_999_999]) - assert length(result) == 1 + assert 1 = length(result) assert {id1, "key_a"} in result end end diff --git a/test/schema_cache/set_lock_test.exs b/test/schema_cache/set_lock_test.exs index 65c45ff..949d943 100644 --- a/test/schema_cache/set_lock_test.exs +++ b/test/schema_cache/set_lock_test.exs @@ -14,7 +14,6 @@ defmodule SchemaCache.SetLockTest do :ets.delete_all_objects(:schema_cache_ets) end - Application.put_env(:schema_cache, :adapter, SchemaCache.Adapters.ETS) :ok end @@ -33,7 +32,7 @@ defmodule SchemaCache.SetLockTest do SetLock.sadd("my_set", "member_1", adapter) assert {:ok, members} = SetLock.smembers("my_set", adapter) - assert length(members) == 1 + assert 1 = length(members) end test "adds multiple different members" do @@ -43,7 +42,7 @@ defmodule SchemaCache.SetLockTest do SetLock.sadd("my_set", "member_3", adapter) assert {:ok, members} = SetLock.smembers("my_set", adapter) - assert length(members) == 3 + assert 3 = length(members) assert "member_1" in members assert "member_2" in members assert "member_3" in members @@ -64,7 +63,7 @@ defmodule SchemaCache.SetLockTest do assert {:ok, members} = SetLock.smembers("concurrent_set", adapter) # ALL 50 members should be present -- no lost writes - assert length(members) == 50 + assert 50 = length(members) end end @@ -97,7 +96,7 @@ defmodule SchemaCache.SetLockTest do SetLock.srem("my_set", "nonexistent", adapter) assert {:ok, members} = SetLock.smembers("my_set", adapter) - assert length(members) == 1 + assert 1 = length(members) assert "member_1" in members end @@ -122,7 +121,7 @@ defmodule SchemaCache.SetLockTest do assert {:ok, members} = SetLock.smembers("race_set", adapter) # Should have: 25 odd originals + 50 new = 75 - assert length(members) == 75 + assert 75 = length(members) end end @@ -139,7 +138,7 @@ defmodule SchemaCache.SetLockTest do assert {:ok, members} = SetLock.smembers("my_set", adapter) assert is_list(members) - assert length(members) == 2 + assert 2 = length(members) end end diff --git a/test/schema_cache/supervisor_test.exs b/test/schema_cache/supervisor_test.exs index a8c30e2..4d9edf2 100644 --- a/test/schema_cache/supervisor_test.exs +++ b/test/schema_cache/supervisor_test.exs @@ -3,15 +3,10 @@ defmodule SchemaCache.SupervisorTest do use ExUnit.Case, async: false - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] + alias SchemaCache.Adapters.ETS setup do - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end @@ -22,13 +17,16 @@ defmodule SchemaCache.SupervisorTest do describe "start_link/1" do test "adapter is set in persistent_term after startup" do - assert SchemaCache.Adapters.ETS = :persistent_term.get(:schema_cache_adapter) + assert ETS = :persistent_term.get(:schema_cache_adapter) end test "adapter capabilities are set with correct flags for ETS adapter" do - caps = :persistent_term.get(:schema_cache_adapter_caps) - - assert %{sadd: true, srem: true, smembers: true, mget: true} = caps + assert %{ + native_sadd: true, + native_srem: true, + native_smembers: true, + native_mget: true + } = :persistent_term.get(:schema_cache_adapter_caps) end test "KeyRegistry ETS tables exist after startup" do @@ -39,5 +37,18 @@ defmodule SchemaCache.SupervisorTest do test "SetLock.Registry is running" do assert Process.whereis(SchemaCache.SetLock.Registry) != nil end + + test "raises ArgumentError when adapter is not provided" do + assert_raise ArgumentError, ~r/adapter not configured/, fn -> + SchemaCache.Supervisor.init([]) + end + end + + test "capabilities include elixir_cache flags" do + assert %{ + elixir_cache: false, + redis_backed: false + } = :persistent_term.get(:schema_cache_adapter_caps) + end end end diff --git a/test/schema_cache_test.exs b/test/schema_cache_test.exs index e675c2e..d99c709 100644 --- a/test/schema_cache_test.exs +++ b/test/schema_cache_test.exs @@ -10,21 +10,13 @@ defmodule SchemaCacheTest do alias SchemaCache.KeyRegistry alias SchemaCache.Test.FakeSchema - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] - setup do - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end end - Application.put_env(:schema_cache, :adapter, SchemaCache.Adapters.ETS) :ok end @@ -77,7 +69,7 @@ defmodule SchemaCacheTest do assert [] = SchemaCache.read("all_users", %{}, nil, fetch) # Function was called twice (not cached) - assert :counters.get(call_count, 1) == 2 + assert 2 = :counters.get(call_count, 1) end test "passes through error results without caching" do @@ -168,12 +160,12 @@ defmodule SchemaCacheTest do } } = SchemaCache.read("find_user", %{id: user.id}, nil, fetch) - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) assert {:ok, %{id: ^user_id}} = SchemaCache.read("find_user", %{id: user.id}, nil, fetch) - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) end test "caches Repo.all collection, second call serves from cache" do @@ -187,13 +179,13 @@ defmodule SchemaCacheTest do end assert [_, _] = SchemaCache.read("all_users", %{}, nil, fetch) - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) result = SchemaCache.read("all_users", %{}, nil, fetch) ids = Enum.map(result, & &1.id) assert user1.id in ids assert user2.id in ids - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) end test "concurrent cache misses preserve all key references" do @@ -212,7 +204,7 @@ defmodule SchemaCacheTest do schema_key = KeyGenerator.schema_cache_key(user) {:ok, refs} = adapter.smembers("__set:#{schema_key}") - assert length(refs) == 20 + assert 20 = length(refs) end test "concurrent reads for different schemas build correct key references" do @@ -256,7 +248,7 @@ defmodule SchemaCacheTest do end results = Task.await_many(tasks) - assert length(results) == 15 + assert 15 = length(results) for i <- 1..10 do cache_key = KeyGenerator.cache_key("find_user", %{id: i}) @@ -282,7 +274,7 @@ defmodule SchemaCacheTest do end results = Task.await_many(tasks) - assert length(results) == 25 + assert 25 = length(results) end test "concurrent reads and write_through updates don't corrupt state" do @@ -303,7 +295,7 @@ defmodule SchemaCacheTest do end results = Task.await_many(tasks) - assert length(results) == 25 + assert 25 = length(results) end end @@ -365,7 +357,7 @@ defmodule SchemaCacheTest do ids = Enum.map(result, & &1.id) assert user1.id in ids assert user2.id in ids - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) end test "concurrent creates evict collection caches" do @@ -485,7 +477,7 @@ defmodule SchemaCacheTest do {:ok, Repo.get(User, user.id)} end) - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) end test "write_through singular updates cache in place without re-fetching" do @@ -514,7 +506,7 @@ defmodule SchemaCacheTest do {:ok, Repo.get(User, user.id)} end) - assert :counters.get(call_count, 1) == 0 + assert 0 = :counters.get(call_count, 1) end test "write_through collection updates cache in place" do @@ -540,7 +532,7 @@ defmodule SchemaCacheTest do Repo.all(User) end) - assert :counters.get(call_count, 1) == 0 + assert 0 = :counters.get(call_count, 1) end test "concurrent updates don't lose evictions" do @@ -614,7 +606,7 @@ defmodule SchemaCacheTest do {:ok, Repo.get(User, user.id)} end) - assert :counters.get(call_count, 1) == 1 + assert 1 = :counters.get(call_count, 1) end test "concurrent deletes evict all referenced keys" do @@ -776,10 +768,12 @@ defmodule SchemaCacheTest do :persistent_term.put(:schema_cache_adapter, SchemaCache.Test.FailMgetAdapter) :persistent_term.put(:schema_cache_adapter_caps, %{ - sadd: true, - srem: true, - smembers: true, - mget: true + native_sadd: true, + native_srem: true, + native_smembers: true, + native_mget: true, + elixir_cache: false, + redis_backed: false }) try do @@ -820,7 +814,7 @@ defmodule SchemaCacheTest do Repo.all(User) end) - assert :counters.get(call_count, 1) == 2 + assert 2 = :counters.get(call_count, 1) end end diff --git a/test/support/data_case.ex b/test/support/data_case.ex index bfab0a7..8fdbbc6 100644 --- a/test/support/data_case.ex +++ b/test/support/data_case.ex @@ -4,15 +4,9 @@ defmodule SchemaCache.Test.DataCase do use ExUnit.CaseTemplate alias Ecto.Adapters.SQL.Sandbox + alias SchemaCache.Adapters.ETS alias SchemaCache.Test.Repo - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] - using do quote do alias SchemaCache.Test.Repo @@ -27,7 +21,7 @@ defmodule SchemaCache.Test.DataCase do Sandbox.mode(Repo, {:shared, self()}) end - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end diff --git a/test/support/elixir_cache_ets.ex b/test/support/elixir_cache_ets.ex new file mode 100644 index 0000000..e930283 --- /dev/null +++ b/test/support/elixir_cache_ets.ex @@ -0,0 +1,8 @@ +defmodule SchemaCache.Test.ElixirCacheETS do + @moduledoc false + + use Cache, + adapter: Cache.ETS, + name: :schema_cache_test_ec_ets, + opts: [] +end diff --git a/test/support/elixir_cache_redis.ex b/test/support/elixir_cache_redis.ex new file mode 100644 index 0000000..3fc9eb3 --- /dev/null +++ b/test/support/elixir_cache_redis.ex @@ -0,0 +1,8 @@ +defmodule SchemaCache.Test.ElixirCacheRedis do + @moduledoc false + + use Cache, + adapter: Cache.Redis, + name: :schema_cache_test_ec_redis, + opts: {:schema_cache, :test_elixir_cache_redis_opts} +end diff --git a/test/support/redis_case.ex b/test/support/redis_case.ex index 96d5aef..ca3c619 100644 --- a/test/support/redis_case.ex +++ b/test/support/redis_case.ex @@ -4,13 +4,7 @@ defmodule SchemaCache.Test.RedisCase do use ExUnit.CaseTemplate alias SchemaCache.Adapter - - @ets_tables [ - :schema_cache_ets, - :schema_cache_ets_sets, - :schema_cache_key_to_id, - :schema_cache_id_to_key - ] + alias SchemaCache.Adapters.ETS setup do redis_url = Application.get_env(:schema_cache, :redis_url, "redis://localhost:6379") @@ -19,7 +13,7 @@ defmodule SchemaCache.Test.RedisCase do {:ok, conn} -> {:ok, "OK"} = Redix.command(conn, ["FLUSHDB"]) - for table <- @ets_tables do + for table <- ETS.managed_tables() do if :ets.whereis(table) != :undefined do :ets.delete_all_objects(table) end diff --git a/test/test_helper.exs b/test/test_helper.exs index cc5a61e..d65fa53 100644 --- a/test/test_helper.exs +++ b/test/test_helper.exs @@ -10,6 +10,13 @@ SchemaCache.Adapters.ETS.sadd("__init__", "__init__") {:ok, _} = SchemaCache.Test.Repo.start_link() Ecto.Adapters.SQL.Sandbox.mode(SchemaCache.Test.Repo, :manual) +# Start the ElixirCache ETS module so its table exists for the entire test run. +{:ok, _} = + Supervisor.start_link( + [SchemaCache.Test.ElixirCacheETS], + strategy: :one_for_one + ) + # Redis connections are started per-test in RedisCase, not globally. -# This provides better isolation — each test gets its own connection +# This provides better isolation. Each test gets its own connection # and a fresh FLUSHDB before running.