Delete pit service layer changes by bharath-techie · Pull Request #8 · bharath-techie/OpenSearch

bharath-techie · 2022-07-15T12:18:57Z

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-15T12:19:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: f20dbdc

github-actions · 2022-07-15T12:22:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: bba02294e075579aa7c2167911a3cc5c31a530f6

…ch into deletepitservice

github-actions · 2022-07-15T12:34:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 8f673c5

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-15T12:39:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 97c4f77

…ch into deletepitservice

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-19T10:02:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 9497923

github-actions · 2022-07-19T10:08:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 5c452d46bbb5c6cc0629a55cec8792ad7de9c6f7

…ch into deletepitservice

github-actions · 2022-07-19T10:22:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: ab3f0d4

* Create pit service layer changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-19T15:07:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: a30911349377483d8935f01309035f7dfc261a21

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-19T15:19:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 41d342d

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-20T05:00:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 25bb645

…rdAlreadyReplicating() (opensearch-project#3943) * Fixing flaky test failure happening for testShardAlreadyReplicating() Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

github-actions · 2022-07-21T05:50:48Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: e36c60a80a67169a195d96248bdc03a724a3634f

github-actions · 2022-07-21T06:37:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 595afcd92328bd2ef58985c8856af08eaf19a797

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-21T08:07:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 717cd66

…Replications() (opensearch-project#3963) * Fixing flaky test failure happening for testShardAlreadyReplicating() Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

…project#3658) * Adding Latest Recevied checkpoint, replay checkpoint logic along with tests Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

…#3837) * Make HybridDirectory MMAP Extensions Configurable Make the HybridDirectory's list of mmap extensions configurable via index settings instead of being hard-coded to a specfic set. Set defaults to the list of currently hard-coded values for backwards compatibility. Signed-off-by: Matt Weber <matt@mattweber.org> * Add javadoc to INDEX_STORE_HYBRID_MMAP_EXTENSIONS. Signed-off-by: Matt Weber <matt@mattweber.org>

…ogy in 'client' directory (opensearch-project#3966) * Deprecate public methods and variables that contain 'master' terminology in 'client' directory Signed-off-by: Tianli Feng <ftianli@amazon.com> * Change 3 String type usages with 'master' to 'cluster manager' Signed-off-by: Tianli Feng <ftianli@amazon.com>

github-actions · 2022-07-22T10:48:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 31668b2bbfce2b44ffc279f84892eab1f387bbfb

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-22T13:20:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: f4db20b

…and add ITs. (opensearch-project#3743) * Add null check when computing max segment version. With segment replication enabled it is possible Lucene does not set the SegmentInfos min segment version, leaving the default value as null. Signed-off-by: Marc Handalian <handalm@amazon.com> * Update peer recovery to set the translogUUID of replicas to the UUID generated on the primary. This change updates the UUID when the translog is created to the value stored in the passed segment userdata. This is to ensure during failover scenarios that the replica can be promoted and not have a uuid mismatch with the value stored in user data. Signed-off-by: Marc Handalian <handalm@amazon.com> * Wire up Segment Replication under the feature flag. This PR wires up segment replication and adds some initial integration tests. Signed-off-by: Marc Handalian <handalm@amazon.com> * Add test to ensure replicas use primary translog uuid with segrep. Signed-off-by: Marc Handalian <handalm@amazon.com> * Update SegmentReplicationIT to assert previous commit points are valid and SegmentInfos can be built. Fix nitpicks in PR feedback. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix test with Assert.fail to include a message. Signed-off-by: Marc Handalian <handalm@amazon.com>

@reta

…earch-project#3893) * opensearch-project#3842 adds a new parameter to the highlighter, the max_analyzer_offset. When this parameter is provided the highlight stops in its value. This prevents the highlighter to go beyond the index maxAnalyzedOffset. Signed-off-by: Hauck <joaoh14@gmail.com> * Adds a test for the new parameter Signed-off-by: Hauck <joaoh14@gmail.com> * Fix the test add in the previous commit; Signed-off-by: Hauck <joaoh14@gmail.com> * This was checking against the wrong field Signed-off-by: Hauck <joaoh14@gmail.com> * Only runs the test for the correct version Signed-off-by: Hauck <joaoh14@gmail.com> * Skips the test in Elasticsearch as well; Signed-off-by: Hauck <joaoh14@gmail.com> * Remove elastic 3.0 to test Signed-off-by: Hauck <joaoh14@gmail.com> * Skips all versions Signed-off-by: Hauck <joaoh14@gmail.com> * Remove unnecessary fields as pointed by @reta Signed-off-by: Hauck <joaoh14@gmail.com> * Compute if fieldMaxAnalyzedIsNotValid in the constructor as suggest by @reta Signed-off-by: Hauck <joaoh14@gmail.com> * As discussed, it is better to throws different exceptions for when the fieldMaxAnalyzed is not valid and for when it is disabled; Signed-off-by: Hauck <joaoh14@gmail.com> * hint what to do to allow highlight of bigger documents Signed-off-by: Hauck <joaoh14@gmail.com> * Let the user define the new parameter globally for all fields highlighted Signed-off-by: Hauck <joaoh14@gmail.com> * Change the fieldMaxAnalyzedOffset Integer in order to use null when it is absent in highlight. This allows the error messages to much more precise, showing invalid for all negative numbers; Signed-off-by: Hauck <joaoh14@gmail.com> * Update javadocs and implements the stream methods for the new fields; Signed-off-by: Hauck <joaoh14@gmail.com> * builder.field do not accept null, so check before calling the method is necessary Signed-off-by: Hauck <joaoh14@gmail.com> * Only send and read the new fields if the version supports it Signed-off-by: Hauck <joaoh14@gmail.com> * the previous commit was checking the wrong field Signed-off-by: Hauck <joaoh14@gmail.com> * Check for version 3.0.0 instead of current version Signed-off-by: Hauck <joaoh14@gmail.com> * Update server/src/main/java/org/apache/lucene/search/uhighlight/CustomUnifiedHighlighter.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Hauck <joaoh14@gmail.com> * Execute the test after version 3.0.0 Signed-off-by: Hauck <joaoh14@gmail.com> Co-authored-by: Andriy Redko <drreta@gmail.com>

…ogy in 'test/framework' directory (opensearch-project#3978) * Deprecate methods with 'master' name mainly in class 'InternalTestCluster', 'NodeRoles', and 'AbstractSnapshotIntegTestCase' * Deprecate variables contain 'master' in the name in class 'InternalTestCluster' Signed-off-by: Tianli Feng <ftianli@amazon.com>

…roject#3796) * Parallelize stale blobs deletion during snapshot delete Signed-off-by: Piyush Daftary <pdaftary@amazon.com> * Adding test which throws exception Signed-off-by: Piyush Daftary <pdaftary@amazon.com> * Adusting identation for spotlessJavaCheck Signed-off-by: Piyush Daftary <pdaftary@amazon.com> * Adding more description to MAX_SHARD_BLOB_DELETE_BATCH_SIZE Signed-off-by: Piyush Daftary <pdaftary@amazon.com> * Renaming max_shard_blob_delete_batch_size to max_snapshot_shard_blob_delete_batch_size Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

A set of guidelines to decide whether a feature should be included in OpenSearch. Signed-off-by: Rabi Panda <adnapibar@gmail.com>

github-actions · 2022-07-25T18:47:25Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 38313f4db8cf8380163f128981b3fd0d9af456a2

Uses a shell environment where `-o pipefail` is available. Closes opensearch-project#4005 Signed-off-by: Cole White <cwhite@wikimedia.org>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

github-actions · 2022-07-26T04:08:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 3f08013

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

… into deletepitservice

github-actions · 2022-07-26T05:15:42Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: 61923493514304da08c8de580ffce26bbd5961d3

…ch into deletepitservice

github-actions · 2022-07-26T05:58:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: null ❌
URL:
CommitID: ae618ff

bharath-techie added 3 commits July 15, 2022 17:02

Create pit service layer changes

a8be024

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Create pit service layer changes

8a8f708

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Delete pit service layer changes

f20dbdc

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Merge branch 'createpitservice' of github.com:bharath-techie/OpenSear…

8f673c5

…ch into deletepitservice

bharath-techie force-pushed the deletepitservice branch from bba0229 to 8f673c5 Compare July 15, 2022 12:34

changing to consistent action names

97c4f77

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

bharath-techie force-pushed the createpitservice branch 3 times, most recently from 6963b68 to 987cb57 Compare July 19, 2022 08:11

bharath-techie added 2 commits July 19, 2022 15:12

Merge branch 'createpitservice' of github.com:bharath-techie/OpenSear…

9497923

…ch into deletepitservice

Addressing review comments

8fcc25c

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Merge branch 'createpitservice' of github.com:bharath-techie/OpenSear…

ab3f0d4

…ch into deletepitservice

bharath-techie force-pushed the deletepitservice branch from 5c452d4 to ab3f0d4 Compare July 19, 2022 10:21

bharath-techie force-pushed the createpitservice branch 2 times, most recently from ed20a79 to 8067e7b Compare July 19, 2022 11:16

Create pit service layer changes (opensearch-project#3921)

55eb86d

* Create pit service layer changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Resolving conflicts

41d342d

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

bharath-techie force-pushed the deletepitservice branch from a309113 to 41d342d Compare July 19, 2022 15:18

Addressing comments

25bb645

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

[Segment Replication] Fixing flaky test failure happening for testSha…

4466a1f

…rdAlreadyReplicating() (opensearch-project#3943) * Fixing flaky test failure happening for testShardAlreadyReplicating() Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

bharath-techie force-pushed the deletepitservice branch from e36c60a to 595afcd Compare July 21, 2022 06:36

Addressing comments

717cd66

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

bharath-techie force-pushed the deletepitservice branch from 595afcd to 717cd66 Compare July 21, 2022 08:06

Rishikesh1159 and others added 4 commits July 21, 2022 06:59

Fix possible flaky test for testBeforeIndexShardClosed_CancelsOngoing…

8b5a10c

…Replications() (opensearch-project#3963) * Fixing flaky test failure happening for testShardAlreadyReplicating() Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

[Segment Replication] Checkpoint Replay on Replica Shard (opensearch-…

bb593d6

…project#3658) * Adding Latest Recevied checkpoint, replay checkpoint logic along with tests Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

Addressing review comment

f4db20b

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

bharath-techie force-pushed the deletepitservice branch from 31668b2 to f4db20b Compare July 22, 2022 13:19

mch2 and others added 5 commits July 22, 2022 09:31

Add guidelines for code contributions (opensearch-project#3976)

efde8c5

A set of guidelines to decide whether a feature should be included in OpenSearch. Signed-off-by: Rabi Panda <adnapibar@gmail.com>

shdubsh and others added 2 commits July 25, 2022 19:51

Use bash in systemd-entrypoint shebang (opensearch-project#4008)

b24b02f

Uses a shell environment where `-o pipefail` is available. Closes opensearch-project#4005 Signed-off-by: Cole White <cwhite@wikimedia.org>

Addressing review comment - adding tests to test concurrency

3f08013

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

bharath-techie force-pushed the deletepitservice branch from 38313f4 to 3f08013 Compare July 26, 2022 04:07

bharath-techie added 2 commits July 26, 2022 10:39

Addressing review comment - adding tests to test concurrency

f76fa24

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Merge branch 'main' of https://github.com/opensearch-project/OpenSearch…

d870dd5

… into deletepitservice

Merge branch 'deletepitservice' of github.com:bharath-techie/OpenSear…

ae618ff

…ch into deletepitservice

bharath-techie force-pushed the deletepitservice branch from 6192349 to ae618ff Compare July 26, 2022 05:57

Conversation

bharath-techie commented Jul 15, 2022

Description

Issues Resolved

Check List

Uh oh!

github-actions bot commented Jul 15, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 15, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 15, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 15, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 19, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 19, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 19, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 19, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 19, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 20, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 21, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 22, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 22, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 25, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 26, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 26, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 26, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants