Pull from upstream - #1
Open
deepakverma wants to merge 513 commits into
Open
Conversation
* honor select disposition in transactions * rearranging if/switch * using server instead of multiplxer * release notes
Co-authored-by: Nick Craver <nrcraver@gmail.com>
This adds a bit of null ref handling (few ifs). Fixes #2576. Overall, this is biting people in the shutdown race, more likely under load, so let's eat the if checks here to prevent it. I decided to go with the specific approach here as to not affect inlining.
* fix #2642 1: don't treat trivial clusters as clusters - Alibaba uses this config 2: report synchronous failures immidiately and accurately * instead of using node count, use explicit tracking of the DB count * release notes
* fix #2653 * Update src/StackExchange.Redis/ServerEndPoint.cs Co-authored-by: Philo <philon@microsoft.com> * release notes * merge fail --------- Co-authored-by: Philo <philon@microsoft.com>
This is a new feature for allowing keepalive commands to be sent every heartbeat regardless of if they're needed for actual keepalives. I've gotten a ping about a network drop that went undetected for ~8 minutes because of usage only in strings. We see this use case not uncommonly, and because of that string type being the only thing used, we don't detect a protocol fault from subsequent string commands. The symptoms here are partial/unexpected string payloads but ultimately the wrong answers to the wrong query. Given we don't know at what layer this drop happens (this appears to be extremely rare, 1 in quadrillions as far as we know), the best we can currently do is react to it ASAP. There may be a better way to accomplish what this is after - discussing with Marc soon as we're both online but prepping this PR in case it's best path.
-Fix spelling
#2659) * add new AddLibraryNameSuffix API for annotating connections with usage * fixup test * new partial for lib-name bits * use hashing rather than array shenanigans * move to shipped; fix comment typo * comment example * reverse comment owner
* Provide new LoggingTunnel API; this * words * fix PR number * fix file location * save the sln * identify smessage as out-of-band * add .ForAwait() throughout LoggingTunnel * clarify meaning of path parameter
This issue was brought to my attention last night (thanks to Badrish Chandramouli): dotnet/dotnet-api-docs#6660 This changeset ensures that we do not honor self-signed certs or partial/broken chains as a result of `X509VerificationFlags.AllowUnknownCertificateAuthority` downstream and adds a few tests and utilities to generate test certificates (currently valid for ~9000 days). Instead we are checking that the certificate we're being told to trust is explicitly in the chain, given that the result of `.Build()` cannot be trusted for this case. This also resolves an issue where `TrustIssuer` could be called but we'd error when _no errors_ were detected (due to requiring chain errors in our validator), this means users couldn't temporarily trust a cert while getting it installed on the machine for instance and migrating between the 2 setups was difficult. This needs careful eyes, please scrutinize heavily. It's possible this breaks an existing user, but...it should be broken if so unless there's a case I'm not seeing.
Further hardening following #2665. This is an additional check to match the .NET implementation for TLS cert checks so that we don't treat a cert flagged as non-TLS-server effectively. This ensures that a certificate either doesn't have OIDs here (valid, backwards compatible) or has the server-certificate OID indicating it's valid for consumption over TLS for us. Cheers @bartonjs for the report and info here.
Also fix a couple typos
…ss ambiguous alternatives (#2702) Closes #2697. All of the replacements were empirically tested to be correct via simple programs in combination with a local redis instance. Notably, there is one worrying nit; in testing it turns out that the `IDatabase.List{Left,Right}Pop(RedisKey, long, CommandFlags)` overload which I talked about in the issue _can_ actually return null, contrary to its nullability annotations. This occurs on missing key; in that case redis replies Nil reply: if the key does not exist. as per https://redis.io/docs/latest/commands/lpop/, which then at https://github.com/StackExchange/StackExchange.Redis/blob/cb8b20df0e2975717bde97ce95ac20e8e8353572/src/StackExchange.Redis/ResultProcessor.cs#L1546-L1547 and later at https://github.com/StackExchange/StackExchange.Redis/blob/cb8b20df0e2975717bde97ce95ac20e8e8353572/src/StackExchange.Redis/ExtensionMethods.cs#L339-L341 turns into a `null`. I briefly attempted to rectify this, but the `RedisValueArrayProcessor` poses a problem here, as changing it to derive `ResultProcessor<RedisValue[]?>` causes the solution to light up in red, and I'd rather not mess with that as a first contribution without at least prior discussion concerning direction there.
* support reading from last message from stream with xread * recover previous formatting * add redisversion and use it in integration tests
* initial implementation of #2706 * build: net8 compat * Docs: add HighIntegrity initial docs * PR fixups * Options fixes * Update tests/StackExchange.Redis.Tests/TestBase.cs * add tests that result boxes / continuations work correctly for high-integrity-mode * - switch to counter rather than entropy - add transaction work to basic tests, to ensure handled * naming is hard * - add explicit connection failure type - burn the connection on failure - add initial metrics * benchmark impact of high performance mode * be more flexible in HeartbeatConsistencyCheckPingsAsync * add config tests --------- Co-authored-by: Nick Craver <nickcraver@microsoft.com> Co-authored-by: Nick Craver <nrcraver@gmail.com>
Closes/Fixes #2726 added two overloads and use a [filter object](https://github.com/StackExchange/StackExchange.Redis/compare/main...atakavci:StackExchange.Redis:ali/maxage?expand=1#diff-6c39ce347d72497c65974f0d78ab9a986be3198e7b2555975ff06410a9831bd8) to avoid a breaking change. Co-authored-by: Nick Craver <nickcraver@microsoft.com>
* - introduce CycleBufferPool as a concept that encapsulates MemoryPool<byte> with the growth logic - support `-d` in the benchmark * - add ConfigurationOptions support - add [SkipLocalsInit] to RESPite * add ResponseArrayPool (#3104) * add ResponseArrayPool * Lease * IMemoryOwner<byte>? --------- Co-authored-by: ITikhonov <ITikhonov@lanit.ru> * refactor API so that MemoryPool<T> can be used throughout, and shared between lease and cycle-buffers * we don't actually need a default CycleBufferPool<T> impl; * fix typo * avoid `Failed` nomenclature in tests - makes it hard to isolate CI problems --------- Co-authored-by: Ivan Tikhonov <pairbit@gmail.com> Co-authored-by: ITikhonov <ITikhonov@lanit.ru>
Updated release notes for version 2.13.17 with fixes and enhancements.
* move KeyIdle[Async]Tests to a server without replication * stabilize Windows CI * don't run the `OBJECT IDLETIME` tests in WSL * skip sentinel tests on Windows CI
* Merge pull request #4 from pairbit/RedisValueSequence Redis value sequence * fix order * fix * PublicAPI.Unshipped * GetCharCount and GetMaxCharCount * add checking OverflowException * comment SequenceCompareTo * implement StartsWith * fix StartsWith * refact * GetHashCode * add Equals, CompareTo, StartsWith for ByteArray or MemoryManager * add ReadOnlySequenceIterator * refac * - GetHashCode - GetChars/GetCharCount - Simplify - normalize .First.Span * TryParse * mark WriteUnifiedSequence(ROS) as redundant for now (and fix 64-bit length) * CompareTo * add integration test (and fix CI) * polishing * avoid the string alloc in GetHashCode (when possible) --------- Co-authored-by: ITikhonov <ITikhonov@lanit.ru> Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
…ShortBlob storage kind (#3114) * spike better RedisValue parsing logic * Introduce new storage kind for small payloads (<=8 bytes) - avoids array alloc * fix CI * CI: tweak failing test * tidying (support ulong as canonical)
3.0 is GA now
* investigate and fix #3123 * F+F the test setup * nit, nuke a using direction * in debug: make all string -> RedisValue *explicit* (incomplete, needs a fix to the ship-file) * fix shipfiles for DEBUG hack * docs * fix exp link * I knew we had a Write...(RedisKey) method somewhere! * move NO ONE to RedisLiterals
* propose agents guidelines and skills * add command source notes * ensure `SORT_RO` parse correctly; impact: Execute would not apply command-map rename correctly otherwise * Update AGENTS.md
* Throw a clear error when an Execute command contains whitespace
Execute("ACL SETUSER x") passes a whole command line as the single command
token, which gets sent as one unknown command and comes back as an opaque
server error. Since a redis command token never contains internal whitespace,
this is always a caller mistake, so the ExecuteMessage constructor now fails
fast with an ArgumentException-style RedisCommandException that names the
offending command and shows the correct token-per-argument form. Resolves #2689.
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
* test for simple space only
limit to simple space; if people are being *that* creative, that's on them
* Remove test case for 'echo\thello' command
Removed test case for command with tab character.
---------
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
* initial cut * transactions * iteration/evolution * docs * refactored API based on internal conversations * more docs * WIP * clarify MULTI/EXEC logic * avoid value-tuple
* fix: Queue batch commands to backlog during MOVED reconnection When a MOVED-to-same-endpoint triggers reconnection (PR #3003), batch commands issued during the ~50-100ms reconnection window fail immediately with NoConnectionAvailable. This is because: 1. RedisBatch.Execute() calls SelectServer() which returns null for disconnected/reconnecting servers. 2. PhysicalBridge.TryEnqueue() returns false when !IsConnected, unlike TryWriteAsync/TryWriteSync which queue to the backlog. This creates an inconsistency where individual async commands succeed (queued via BacklogPolicy) but batch commands throw during the same reconnection window. Fix: - RedisBatch.Execute(): When SelectServer returns null and BacklogPolicy.QueueWhileDisconnected is enabled, retry with allowDisconnected=true to find the server endpoint. - PhysicalBridge.TryEnqueue(): When disconnected/reconnecting and BacklogPolicy.QueueWhileDisconnected is enabled, queue messages to the backlog instead of returning false. This matches the existing behavior of individual command paths and ensures batch commands are transparently queued and sent after reconnection completes. Tested with proxy redirect scenario (MOVED-to-same-endpoint + disconnect): - Before: 1-3 NoConnectionAvailable errors per MOVED redirect - After: 0 errors, batch waits for reconnection via backlog * test: Add batch backlog tests for MOVED-triggered reconnection Add two integration tests verifying batch commands are queued to the backlog during MOVED-to-same-endpoint reconnection: - MovedToSameEndpoint_BatchCommands_QueuedDuringReconnect: Verifies that a batch containing a MOVED-triggering command completes successfully after reconnection (commands queued to backlog). - MovedToSameEndpoint_SubsequentBatch_QueuedDuringReconnect: Verifies that a second batch issued during the reconnection window is queued rather than throwing NoConnectionAvailable (fire-and-forget scenario). --------- Co-authored-by: advaMosh <advaMosh@users.noreply.github.com>
…es (#3130) SwitchPrimary only reconfigured when the new primary was an unknown endpoint. On an in-place failover (old master demoted, replica promoted), the new master is already in `servers`, so it no-op'd and the client kept writing to the demoted node until restart. Reconfigure with reconfigureAll:true when a known primary's cached role is stale, so connected nodes re-read their role before re-election. Also move the switch reconfigure out of the sentinelConnectionChildren lock. Refs #1891
* LMOVEM * test CI image available * SUNIONCARD/SDIFFCARD * XREAD/XREADGROUP MAXCOUNT/MAXSIZE * cleanup unshipped * fixup test versions
We hand-start every server instance the tests need, so drop a policy-rc.d that tells invoke-rc.d/deb-systemd-invoke not to start services at all. The packaged redis-server unit would otherwise race our own primary for 127.0.0.1:6379 on any run where WSL came up with systemd. That also sidesteps an install hang we saw with the 8.10.0 package: its redis.conf lost "supervised auto", so redis never sd_notify()s, the Type=notify unit never leaves "activating", and the postinst's "systemctl start" blocks (indefinitely, since redis then can't save its RDB on SIGTERM and the unit disables the stop timeout). That is an upstream packaging slip which they are already fixing, so this isn't a permanent change in behaviour from 8.10 - but not starting the service is the right thing for this job regardless. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR is best summarised by docs/Failover.md
Point directly to GitHub releases to save a click.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.