From 7c46f8f14e1beefdd24eb2fe61792c6737fe9023 Mon Sep 17 00:00:00 2001 From: Andriy Redko Date: Wed, 22 Jan 2025 16:12:41 -0500 Subject: [PATCH] Update to Apache Lucene 10 for 3.0.0 (#16366) * Update to Apache Lucene 10 for 3.0.0 Signed-off-by: Andriy Redko * More Lucene 10 build fixes Core compiles now. (Plugins/modules don't yet.) Major areas covered: 1. MultiTermQuery now takes RewriteMethod via constructor instead of setter. 2. Scorable no longer has docId. 3. ScoreCachingWrappingScorer now takes + returns LeafCollector. 4. MMapDirectory allows configuring preload per file (so we don't have to). 5. Concurrent search can now operate at sub-segment level (but we don't yet). Signed-off-by: Michael Froh * Fix more compilation failures Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andrew Ross * Replace DocValuesFieldExistsQuery with FieldExistsQuery Signed-off-by: Andrew Ross * Fix signature changes in o.a.l.search.BooleanQuery Signed-off-by: Andrew Ross * Fix more compilation failures, update shadow Gradle plugin and fix loggerUsageCheck Gradle task Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fix analysis-common compilation issues Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * More compilation fixes Signed-off-by: Andrew Ross * More compilation fixes Signed-off-by: Andrew Ross * Fix assemble tasks Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fix more compilation failures, add CHANGELOG.md Signed-off-by: Andriy Redko * Fix more compilation failures Signed-off-by: Andriy Redko * Fixed Lucene latest codec name Signed-off-by: Andriy Redko * Refactor preload tests in FsDirectoryFactoryTests Also more compilation fixes. Signed-off-by: Andrew Ross * Fix more compilation failures Signed-off-by: Andrew Ross * Fix precommit failures Signed-off-by: Andrew Ross * Fix analysis test cases Signed-off-by: Andriy Redko * Fix FieldData & ParentJoinAggregator ordinal traversal Signed-off-by: Andriy Redko * Fix IpFieldMapper & CollapsingDocValuesSource ordinal traversal Signed-off-by: Andriy Redko * Fix MultiValueMode, GlobalOrdinalValuesSource, BinaryRangeAggregator, DiversifiedOrdinalsSamplerAggregator, GlobalOrdinalsStringTermsAggregator & CardinalityAggregator ordinal traversal Signed-off-by: Andriy Redko * Fix MultiValueModeTests, BinaryRangeAggregatorTests, IncludeExcludeTests & MissingValuesTests ordinal traversal Signed-off-by: Andriy Redko * Fix ICUCollationKeywordFieldMapper & VectorFieldTests tests Signed-off-by: Andriy Redko * Fix org.opensearch.common.xcontent.support.XContentMapValuesTests test cases Signed-off-by: Andriy Redko * Fix FunctionScoreQuery::CustomBoostFactorWeight::scorerSupplier that may throw NPE Signed-off-by: Andriy Redko * Fix CodecTests and non-concurrent ContextIndexSearcher::search implementation for aggregations Signed-off-by: Andriy Redko * Fixed GlobalOrdinalMapping ordinals iteration and SortedWiderNumericSortField implementation of missingValueAsComparableLong Signed-off-by: Andriy Redko * Fix FieldData and SortedSetDocValuesWriterWrapperTests Signed-off-by: Andriy Redko * Fix org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests test cases Signed-off-by: Andriy Redko * Fix org.opensearch.search.aggregations.bucket.composite.CompositeAggregatorTests test cases Signed-off-by: Andriy Redko * Fix MultiValueMode ordinal traversal Signed-off-by: Andriy Redko * Fix GlobalOrdinalMapping and GlobalOrdinalsStringTermsAggregator ordinal traversals Signed-off-by: Andriy Redko * Fix QueryStringQueryBuilderTests and QueryStringQueryBuilder fuzzy query default rewrite method detection logic Signed-off-by: Andriy Redko * Fix StarTree*DocValueFormatTests The Composite912DocValuesReader was not honoring the IOContext correctly. The metadata file should be opened with READONCE and immediately be closed (on the same thread). The data file needs to be opened with ReadAdvice.NORMAL to ensure that it's closeable on another thread if necessary. Signed-off-by: Michael Froh * Fix various MultiTermQuery-related tests Now that we use CONSTANT_SCORE_BLENDED_REWRITE by default for most MTQs, we need to update the tests. Also FuzzyQuery should default to TopTermsBlendedFreqScoringRewrite (taken from Lucene). Signed-off-by: Michael Froh * Fix Spotless Signed-off-by: Michael Froh * Fix RegEx automaton processing using Operations.determinize(...) Signed-off-by: Andriy Redko * Fix compilation issues (post merge) Signed-off-by: Andriy Redko * Fix UnsignedLongComparator::sortableBytesToLong encoding Signed-off-by: Andriy Redko * Fix HalfFloatComparator::sortableBytesToLong and HalfFloatComparator::missingValueAsComparableLong Signed-off-by: Andriy Redko * Fix infinite recursion in CollapsingTopDocsCollectorTests Signed-off-by: Michael Froh * Fix SingleDimensionValuesSourceTests Can no longer mock IndexReader as it's sealed. Signed-off-by: Michael Froh * Fix a few more MultiTermQuery-related test failures Signed-off-by: Michael Froh * Fixed QueryPhaseTests#minScoreOptimisation by not delegating setWeight to underlying delegate Signed-off-by: expani * Fix DerivedFieldMapperQueryTests Needed to add doc values to date and IP fields. Signed-off-by: Michael Froh * Fix SortedNumericDocValuesWriterWrapperTests Lucene's SortedNumericDocValuesWriter no longer throws an exception if you try to iterate past a document's last value. (Instead, it happily leaks the next document's value.) Signed-off-by: Michael Froh * Fix IpRangeAggregatorTests Rewrote a for-loop in BinaryRangeAggregator to be less confusing. As a side-effect, it fixed the test. Signed-off-by: Michael Froh * Fix IpFieldMapperTests I broke these tests as a side-effect for my fix to DerivedFieldMapperQueryTests. Signed-off-by: Michael Froh * Fix MissingValues ordinal traversal and MultiOrdinalsTests tests Signed-off-by: Andriy Redko * Prevent cascading test failures in SearchServiceTests I'm not really fixing the failing tests, but ensuring that the two tests that fail don't cause a bunch of other tests to fail by leaving transient settings around. (Clean up the transient settings even if those tests fail.) Signed-off-by: Michael Froh * Fix DerivedFieldTypeTests I broke these when fixing DerivedFieldMapperQueryTests. Signed-off-by: Michael Froh * Fix TransferManagerRemoteDirectoryReaderTests Needed to update mocked calls. Signed-off-by: Michael Froh * Add missing javadoc Signed-off-by: Andriy Redko * Uploaded Index generated from Lucene 8.x ( OpenSearch 1.3.0 ) to be used by UnitTests Signed-off-by: expani * Moved path to a constant Signed-off-by: expani * Update Apache Lucene to 10.1.0 Signed-off-by: Andriy Redko * Fixed unit tests for lucene 10 upgrade Signed-off-by: expani * Fix posting format for CompletionFieldMapper Signed-off-by: Andriy Redko * Fix org.opensearch.search.SearchServiceTests test cases Signed-off-by: Andriy Redko * Removed extra iteration of DISI and added stored field option to bypass modified check in lucene Field Signed-off-by: expani * Changed to use advance and init to use docId() instead of nextDoc() Signed-off-by: expani * Increased number of indexed docs to match segment count Signed-off-by: expani * Reduced segment count to match assertion instead of increasing docs Signed-off-by: expani * Revert to make 90_search_after integ pass as hits are not shown Signed-off-by: expani * Making Half Float failures work with it's conversions Signed-off-by: expani * Changing max long to signed from unsigned Signed-off-by: expani * Addressed PR Comments Signed-off-by: expani * Trying another approach for gracefully handling unsigned long conversions Signed-off-by: expani * Simplified unsigned to signed long conversion Signed-off-by: expani * Reverted changes as need to think more on this Signed-off-by: expani * Fix azure-fixture & s3-fixture Docker images Signed-off-by: Andriy Redko * Fix 'search/90_search_after/numeric skipping logic with competitive missing value' test cases Signed-off-by: Andriy Redko * Fix gcs-fixture Docker images Signed-off-by: Andriy Redko * Updated to use modern Romanian Unicode Signed-off-by: expani * Fix org.opensearch.wildfly.WildflyIT test cases Signed-off-by: Andriy Redko * Fix org.opensearch.search.aggregations.bucket.MinDocCountIT test cases Signed-off-by: Andriy Redko * Fix CardinalityAggregator.OrdinalsCollector ordinals traversal Signed-off-by: Andriy Redko * Fix org.opensearch.search.query.ScriptScoreQueryIT test cases Signed-off-by: Andriy Redko * Fix org.opensearch.search.scriptfilter.ScriptQuerySearchIT test cases Signed-off-by: Andriy Redko * Fix UnsignedLongComparator top/bottom as comparable long conversions Signed-off-by: Andriy Redko * Fix post-merge (with main) compilation issues Signed-off-by: Andriy Redko * Fix UnsignedLongComparator with respect to sortable bytes to long conversion Signed-off-by: Andriy Redko * Fix FlatObjectFieldTypeTests test cases Signed-off-by: Andriy Redko --------- Signed-off-by: Andriy Redko Signed-off-by: Michael Froh Signed-off-by: Andrew Ross Signed-off-by: expani Co-authored-by: Michael Froh Co-authored-by: Andrew Ross Co-authored-by: expani --- .github/workflows/assemble.yml | 17 +- .github/workflows/precommit.yml | 15 +- CHANGELOG-3.0.md | 1 + benchmarks/build.gradle | 43 +-- .../common/round/RoundableSupplier.java | 5 +- buildSrc/build.gradle | 10 +- buildSrc/reaper/build.gradle | 4 +- .../remote/.ci/java-versions.properties | 4 +- .../opensearch-server-signatures.txt | 4 - .../src/main/resources/minimumCompilerVersion | 2 +- .../src/main/resources/minimumRuntimeVersion | 2 +- .../src/testKit/thirdPartyAudit/build.gradle | 4 +- .../HighLevelRestClientCompressionIT.java | 2 +- .../java/org/opensearch/client/ReindexIT.java | 2 +- .../client/RestHighLevelClientTests.java | 2 +- .../java/org/opensearch/client/SearchIT.java | 50 +-- .../documentation/SearchDocumentationIT.java | 22 +- gradle/libs.versions.toml | 2 +- gradle/missing-javadoc.gradle | 3 +- libs/common/build.gradle | 73 +--- .../common/round/BtreeSearcher.java | 0 .../common/round/RoundableFactory.java | 38 +- .../common/round/RoundableFactory.java | 75 ---- .../core/licenses/lucene-core-10.1.0.jar.sha1 | 1 + .../core/licenses/lucene-core-9.12.1.jar.sha1 | 1 - .../java/org/opensearch/LegacyESVersion.java | 2 +- .../src/main/java/org/opensearch/Version.java | 2 +- .../common/StemmerTokenFilterFactory.java | 17 +- .../common/HighlighterWithAnalyzersTests.java | 2 +- .../test/analysis-common/20_analyzers.yml | 4 +- ...ractGeoAggregatorModulePluginTestCase.java | 2 +- .../metrics/GeoBoundsITTestCase.java | 4 +- ...idAggregationCompositeAggregatorTests.java | 14 +- .../lucene-expressions-10.1.0.jar.sha1 | 1 + .../lucene-expressions-9.12.1.jar.sha1 | 1 - .../script/expression/MoreExpressionIT.java | 64 ++-- .../expression/ExpressionScriptEngine.java | 11 +- .../mustache/SearchTemplateResponseTests.java | 2 +- .../opensearch/painless/SimplePainlessIT.java | 2 +- .../org/opensearch/painless/ArrayTests.java | 2 - .../ScriptedMetricAggContextsTests.java | 5 - .../painless/SimilarityScriptTests.java | 4 +- .../org/opensearch/painless/StringTests.java | 11 +- .../TokenCountFieldMapperIntegrationIT.java | 2 +- .../SearchAsYouTypeFieldMapperTests.java | 4 +- .../join/aggregations/ChildrenIT.java | 8 +- .../join/query/ChildQuerySearchIT.java | 170 ++++----- .../opensearch/join/query/InnerHitsIT.java | 40 +-- .../aggregations/ParentJoinAggregator.java | 11 +- .../ParentChildInnerHitContextBuilder.java | 8 +- .../ChildrenToParentAggregatorTests.java | 28 +- .../ParentToChildrenAggregatorTests.java | 4 +- .../join/query/HasChildQueryBuilderTests.java | 10 +- .../percolator/PercolatorQuerySearchIT.java | 6 +- .../opensearch/percolator/PercolateQuery.java | 12 +- .../percolator/PercolatorFieldMapper.java | 2 +- .../PercolatorMatchedSlotSubFetchPhase.java | 2 +- .../opensearch/percolator/QueryAnalyzer.java | 24 +- .../percolator/CandidateQueryTests.java | 57 ++- .../percolator/PercolateQueryTests.java | 8 +- .../PercolatorFieldMapperTests.java | 16 +- .../percolator/QueryAnalyzerTests.java | 2 +- .../reindex/remote/RemoteResponseParsers.java | 4 +- .../reindex/DeleteByQueryBasicTests.java | 2 +- .../url/URLSnapshotRestoreIT.java | 4 +- .../common/SearchPipelineCommonIT.java | 14 +- .../lucene-analysis-icu-10.1.0.jar.sha1 | 1 + .../lucene-analysis-icu-9.12.1.jar.sha1 | 1 - .../ICUCollationKeywordFieldMapper.java | 1 + .../index/mapper/CollationFieldTypeTests.java | 11 +- .../lucene-analysis-kuromoji-10.1.0.jar.sha1 | 1 + .../lucene-analysis-kuromoji-9.12.1.jar.sha1 | 1 - .../lucene-analysis-nori-10.1.0.jar.sha1 | 1 + .../lucene-analysis-nori-9.12.1.jar.sha1 | 1 - .../lucene-analysis-phonetic-10.1.0.jar.sha1 | 1 + .../lucene-analysis-phonetic-9.12.1.jar.sha1 | 1 - .../lucene-analysis-smartcn-10.1.0.jar.sha1 | 1 + .../lucene-analysis-smartcn-9.12.1.jar.sha1 | 1 - .../lucene-analysis-stempel-10.1.0.jar.sha1 | 1 + .../lucene-analysis-stempel-9.12.1.jar.sha1 | 1 - ...lucene-analysis-morfologik-10.1.0.jar.sha1 | 1 + ...lucene-analysis-morfologik-9.12.1.jar.sha1 | 1 - .../cache/store/disk/EhcacheDiskCache.java | 3 +- .../AnnotatedTextFieldMapperTests.java | 2 +- .../AnnotatedTextHighlighterTests.java | 2 +- .../hdfs/HdfsBlobStoreContainerTests.java | 1 + .../repositories/hdfs/HdfsTests.java | 2 +- .../index/store/AbstractAzureFsTestCase.java | 2 +- .../smbmmapfs/SmbMmapFsDirectoryFactory.java | 1 - .../telemetry/OTelTelemetrySettings.java | 2 +- .../metrics/OTelMetricsTelemetry.java | 3 + .../tracing/sampler/OTelSamplerFactory.java | 1 + ...rossClusterSearchUnavailableClusterIT.java | 18 +- .../java/org/opensearch/search/CCSDuelIT.java | 10 +- .../cluster/remote/test/RemoteClustersIT.java | 8 +- qa/wildfly/docker-compose.yml | 2 +- .../lucene-analysis-common-10.1.0.jar.sha1 | 1 + .../lucene-analysis-common-9.12.1.jar.sha1 | 1 - .../lucene-backward-codecs-10.1.0.jar.sha1 | 1 + .../lucene-backward-codecs-9.12.1.jar.sha1 | 1 - server/licenses/lucene-core-10.1.0.jar.sha1 | 1 + server/licenses/lucene-core-9.12.1.jar.sha1 | 1 - .../licenses/lucene-grouping-10.1.0.jar.sha1 | 1 + .../licenses/lucene-grouping-9.12.1.jar.sha1 | 1 - .../lucene-highlighter-10.1.0.jar.sha1 | 1 + .../lucene-highlighter-9.12.1.jar.sha1 | 1 - server/licenses/lucene-join-10.1.0.jar.sha1 | 1 + server/licenses/lucene-join-9.12.1.jar.sha1 | 1 - server/licenses/lucene-memory-10.1.0.jar.sha1 | 1 + server/licenses/lucene-memory-9.12.1.jar.sha1 | 1 - server/licenses/lucene-misc-10.1.0.jar.sha1 | 1 + server/licenses/lucene-misc-9.12.1.jar.sha1 | 1 - .../licenses/lucene-queries-10.1.0.jar.sha1 | 1 + .../licenses/lucene-queries-9.12.1.jar.sha1 | 1 - .../lucene-queryparser-10.1.0.jar.sha1 | 1 + .../lucene-queryparser-9.12.1.jar.sha1 | 1 - .../licenses/lucene-sandbox-10.1.0.jar.sha1 | 1 + .../licenses/lucene-sandbox-9.12.1.jar.sha1 | 1 - .../lucene-spatial-extras-10.1.0.jar.sha1 | 1 + .../lucene-spatial-extras-9.12.1.jar.sha1 | 1 - .../licenses/lucene-spatial3d-10.1.0.jar.sha1 | 1 + .../licenses/lucene-spatial3d-9.12.1.jar.sha1 | 1 - .../licenses/lucene-suggest-10.1.0.jar.sha1 | 1 + .../licenses/lucene-suggest-9.12.1.jar.sha1 | 1 - .../opensearch/action/IndicesRequestIT.java | 4 +- .../admin/cluster/node/tasks/TasksIT.java | 4 +- .../admin/indices/create/CreateIndexIT.java | 4 +- .../indices/create/RemoteSplitIndexIT.java | 2 +- .../admin/indices/create/SplitIndexIT.java | 2 +- .../action/bulk/BulkProcessorRetryIT.java | 6 +- .../action/search/TransportSearchIT.java | 18 +- .../IndexingClusterManagerFailoverIT.java | 2 +- .../opensearch/aliases/IndexAliasesIT.java | 45 ++- .../broadcast/BroadcastActionsIT.java | 2 +- .../cluster/MinimumClusterManagerNodesIT.java | 3 +- .../allocation/FilteringAllocationIT.java | 9 +- .../ClusterDisruptionCleanSettingsIT.java | 2 +- .../document/DocumentActionsIT.java | 4 +- .../org/opensearch/index/FinalPipelineIT.java | 6 +- .../index/engine/MaxDocsLimitIT.java | 8 +- .../mapper/CopyToMapperIntegrationIT.java | 4 +- .../ExternalValuesMapperIntegrationIT.java | 16 +- .../mapper/MultiFieldsIntegrationIT.java | 14 +- .../index/mapper/StarTreeMapperIT.java | 6 +- .../opensearch/index/shard/IndexShardIT.java | 4 +- .../opensearch/index/shard/SearchIdleIT.java | 2 +- .../index/store/ExceptionRetryIT.java | 2 +- .../opensearch/indexing/IndexActionIT.java | 8 +- .../indices/IndicesRequestCacheCleanupIT.java | 2 +- .../indices/IndicesRequestCacheIT.java | 50 +-- .../mapping/UpdateMappingIntegrationIT.java | 2 +- .../indices/recovery/IndexRecoveryIT.java | 2 +- .../replication/SegmentReplicationBaseIT.java | 2 +- .../replication/SegmentReplicationIT.java | 2 +- .../SegmentReplicationRelocationIT.java | 4 +- .../state/CloseWhileRelocatingShardsIT.java | 2 +- .../indices/stats/IndexStatsIT.java | 15 +- .../recovery/RecoveryWhileUnderLoadIT.java | 6 +- .../org/opensearch/recovery/RelocationIT.java | 16 +- .../RestoreShallowSnapshotV2IT.java | 6 +- .../multipart/RemoteStoreMultipartIT.java | 2 +- .../opensearch/routing/AliasRoutingIT.java | 126 ++++--- .../routing/PartitionedRoutingIT.java | 6 +- .../opensearch/routing/SimpleRoutingIT.java | 43 ++- .../opensearch/search/SearchTimeoutIT.java | 2 +- .../search/SearchWeightedRoutingIT.java | 2 +- .../search/aggregations/CombiIT.java | 2 +- .../search/aggregations/EquivalenceIT.java | 4 +- .../aggregations/FiltersAggsRewriteIT.java | 2 +- .../bucket/AdjacencyMatrixIT.java | 2 +- .../aggregations/bucket/DateHistogramIT.java | 6 +- .../bucket/DateHistogramOffsetIT.java | 6 +- .../aggregations/bucket/DateRangeIT.java | 18 +- .../search/aggregations/bucket/FilterIT.java | 2 +- .../search/aggregations/bucket/FiltersIT.java | 4 +- .../aggregations/bucket/GeoDistanceIT.java | 2 +- .../aggregations/bucket/HistogramIT.java | 2 +- .../search/aggregations/bucket/NestedIT.java | 2 +- .../search/aggregations/bucket/RangeIT.java | 2 +- .../aggregations/metrics/ExtendedStatsIT.java | 4 +- .../aggregations/metrics/GeoCentroidIT.java | 2 +- .../metrics/HDRPercentileRanksIT.java | 4 +- .../metrics/HDRPercentilesIT.java | 4 +- .../metrics/ScriptedMetricIT.java | 28 +- .../search/aggregations/metrics/StatsIT.java | 2 +- .../search/aggregations/metrics/SumIT.java | 2 +- .../metrics/TDigestPercentileRanksIT.java | 4 +- .../metrics/TDigestPercentilesIT.java | 4 +- .../aggregations/metrics/TopHitsIT.java | 40 +-- .../aggregations/metrics/ValueCountIT.java | 2 +- .../aggregations/pipeline/DerivativeIT.java | 12 +- .../basic/SearchWhileCreatingIndexIT.java | 6 +- .../search/basic/SearchWhileRelocatingIT.java | 6 +- .../basic/TransportTwoNodesSearchIT.java | 14 +- .../search/fetch/FetchSubPhasePluginIT.java | 2 +- .../search/fetch/subphase/InnerHitsIT.java | 48 +-- .../highlight/HighlighterSearchIT.java | 8 +- .../search/fields/SearchFieldsIT.java | 42 +-- .../search/fields/SearchIpFieldTermsIT.java | 2 +- .../functionscore/DecayFunctionScoreIT.java | 50 +-- .../functionscore/ExplainableScriptIT.java | 4 +- .../search/functionscore/FunctionScoreIT.java | 18 +- .../search/functionscore/QueryRescorerIT.java | 6 +- .../functionscore/RandomScoreFunctionIT.java | 8 +- .../geo/AbstractGeoBoundingBoxQueryIT.java | 28 +- .../search/geo/AbstractGeoDistanceIT.java | 2 +- .../search/geo/GeoShapeIntegrationIT.java | 20 +- .../geo/LegacyGeoShapeIntegrationIT.java | 10 +- .../search/morelikethis/MoreLikeThisIT.java | 2 +- .../search/nested/SimpleNestedExplainIT.java | 4 +- .../search/nested/SimpleNestedIT.java | 42 +-- .../search/preference/SearchPreferenceIT.java | 14 +- .../search/profile/query/QueryProfilerIT.java | 4 +- .../org/opensearch/search/query/ExistsIT.java | 2 +- .../search/query/MultiMatchQueryIT.java | 26 +- .../search/query/QueryStringIT.java | 2 +- .../search/query/SearchQueryIT.java | 16 +- .../search/query/SimpleQueryStringIT.java | 2 +- .../scriptfilter/ScriptQuerySearchIT.java | 34 +- .../search/scroll/DuelScrollIT.java | 10 +- .../search/scroll/SearchScrollIT.java | 54 +-- .../search/searchafter/SearchAfterIT.java | 2 +- .../search/simple/SimpleSearchIT.java | 28 +- .../search/slice/SearchSliceIT.java | 10 +- .../opensearch/search/sort/FieldSortIT.java | 98 +++--- .../opensearch/search/sort/SimpleSortIT.java | 8 +- .../search/source/MetadataFetchingIT.java | 4 +- .../suggest/CompletionSuggestSearchIT.java | 2 +- .../opensearch/similarity/SimilarityIT.java | 4 +- .../snapshots/ConcurrentSnapshotsIT.java | 2 +- .../Lucene90DocValuesConsumerWrapper.java | 10 +- .../queries/BinaryDocValuesRangeQuery.java | 7 +- .../lucene/queries/BlendedTermQuery.java | 18 +- .../search/grouping/CollapseTopFieldDocs.java | 4 +- .../grouping/CollapsingDocValuesSource.java | 2 +- .../automaton/MinimizationOperations.java | 331 ++++++++++++++++++ .../lucene/util/packed/XPackedInts.java | 10 +- .../search/BottomSortValuesCollector.java | 2 +- .../action/search/SearchPhaseController.java | 4 +- .../opensearch/bootstrap/BootstrapChecks.java | 2 +- .../org/opensearch/common/lucene/Lucene.java | 6 +- .../common/lucene/MinimumScoreCollector.java | 48 +-- .../lucene/index/FilterableTermsEnum.java | 6 + .../lucene/search/AutomatonQueries.java | 8 +- .../common/lucene/search/Queries.java | 10 +- .../SpanBooleanQueryRewriteWithMaxClause.java | 14 +- .../common/lucene/search/XMoreLikeThis.java | 5 +- .../search/function/FunctionScoreQuery.java | 11 +- .../search/function/MinScoreScorer.java | 2 +- .../search/function/ScriptScoreFunction.java | 8 - .../search/function/ScriptScoreQuery.java | 111 ++++-- .../common/util/concurrent/ThreadContext.java | 3 + .../xcontent/support/XContentMapValues.java | 6 +- .../org/opensearch/index/IndexModule.java | 3 +- .../opensearch/index/codec/CodecService.java | 12 +- .../PerFieldMappingPostingFormatCodec.java | 4 +- .../composite/CompositeCodecFactory.java | 14 +- .../LuceneDocValuesConsumerFactory.java | 19 + .../composite101/Composite101Codec.java | 58 +++ .../composite/composite101/package-info.java | 12 + .../composite912/Composite912Codec.java | 8 +- .../Composite912DocValuesReader.java | 35 +- .../Composite912DocValuesWriter.java | 1 + .../startree/utils/StarTreeUtils.java | 3 + ...SortedUnsignedLongDocValuesRangeQuery.java | 6 +- .../SortedUnsignedLongDocValuesSetQuery.java | 6 +- .../org/opensearch/index/engine/Engine.java | 5 +- .../index/engine/LuceneChangesSnapshot.java | 10 +- .../RecoverySourcePruneMergePolicy.java | 6 + .../index/engine/TranslogLeafReader.java | 21 +- .../opensearch/index/fielddata/FieldData.java | 11 +- .../ordinals/GlobalOrdinalMapping.java | 21 +- .../fielddata/ordinals/MultiOrdinals.java | 6 +- .../opensearch/index/get/ShardGetService.java | 2 + .../index/mapper/CompletionFieldMapper.java | 4 +- .../mapper/ConstantKeywordFieldMapper.java | 8 +- .../index/mapper/DateFieldMapper.java | 5 +- .../mapper/DerivedFieldSupportedTypes.java | 6 +- .../index/mapper/DerivedFieldType.java | 28 -- .../index/mapper/DocCountFieldMapper.java | 4 +- .../index/mapper/FlatObjectFieldMapper.java | 15 +- .../index/mapper/IdFieldMapper.java | 11 +- .../index/mapper/IpFieldMapper.java | 40 ++- .../index/mapper/KeywordFieldMapper.java | 22 +- .../index/mapper/MappedFieldType.java | 33 +- .../index/mapper/StringFieldType.java | 30 +- .../index/mapper/TermBasedFieldType.java | 11 +- .../index/mapper/TextFieldMapper.java | 1 + .../index/mapper/WildcardFieldMapper.java | 20 +- .../index/query/DerivedFieldQuery.java | 6 +- .../index/query/FuzzyQueryBuilder.java | 12 +- .../index/query/IntervalBuilder.java | 6 +- .../index/query/IntervalsSourceProvider.java | 13 +- .../index/query/MultiMatchQueryBuilder.java | 8 +- .../index/query/NestedQueryBuilder.java | 8 +- .../index/query/QueryStringQueryBuilder.java | 8 +- .../opensearch/index/query/RegexpFlag.java | 2 +- .../index/query/ScriptQueryBuilder.java | 6 +- .../index/query/SourceFieldMatchQuery.java | 5 +- .../index/query/TermsSetQueryBuilder.java | 5 +- .../index/query/support/QueryParsers.java | 13 +- .../reindex/ClientScrollableHitSource.java | 2 +- .../opensearch/index/search/MatchQuery.java | 30 +- .../index/search/MultiMatchQuery.java | 2 +- .../opensearch/index/search/NestedHelper.java | 29 +- .../index/search/QueryStringQueryParser.java | 9 +- .../search/SimpleQueryStringQueryParser.java | 1 + .../comparators/HalfFloatComparator.java | 36 +- .../comparators/UnsignedLongComparator.java | 37 +- .../RemoveCorruptedLuceneSegmentsAction.java | 5 +- .../index/shard/SearchOperationListener.java | 7 +- .../index/shard/ShardSplittingQuery.java | 9 +- .../index/store/CompositeDirectory.java | 3 +- .../index/store/FsDirectoryFactory.java | 80 +---- .../index/store/RemoteDirectory.java | 46 ++- .../store/RemoteSegmentStoreDirectory.java | 23 ++ .../org/opensearch/index/store/Store.java | 2 +- .../directory/RemoteSnapshotDirectory.java | 2 +- .../store/remote/utils/BlockIOContext.java | 53 --- .../store/remote/utils/TransferManager.java | 4 +- .../opensearch/indices/IndicesQueryCache.java | 14 - .../indices/SystemIndexDescriptor.java | 5 +- .../SegmentFileTransferHandler.java | 2 +- .../lucene/queries/MinDocQuery.java | 6 +- .../queries/SearchAfterSortedDocQuery.java | 8 +- .../main/java/org/opensearch/node/Node.java | 4 +- .../blobstore/BlobStoreRepository.java | 2 +- .../rest/action/cat/RestCountAction.java | 4 +- .../rest/action/search/RestCountAction.java | 2 +- .../org/opensearch/search/MultiValueMode.java | 4 +- .../org/opensearch/search/SearchHits.java | 6 +- .../org/opensearch/search/SearchModule.java | 4 +- .../aggregations/MultiBucketCollector.java | 53 ++- .../bucket/composite/CompositeAggregator.java | 2 +- .../composite/GlobalOrdinalValuesSource.java | 8 +- .../bucket/composite/LongValuesSource.java | 6 +- .../bucket/nested/NestedAggregator.java | 8 - .../bucket/range/BinaryRangeAggregator.java | 6 +- .../sampler/BestDocsDeferringCollector.java | 12 +- .../DiversifiedOrdinalsSamplerAggregator.java | 2 +- .../GlobalOrdinalsStringTermsAggregator.java | 15 +- .../bucket/terms/IncludeExclude.java | 7 +- .../metrics/CardinalityAggregator.java | 8 +- .../aggregations/metrics/InternalTopHits.java | 10 +- .../metrics/MetricInspectionHelper.java | 2 +- .../metrics/TopHitsAggregator.java | 10 +- .../aggregations/support/MissingValues.java | 10 +- .../ApproximatePointRangeQuery.java | 13 +- .../highlight/FragmentBuilderHelper.java | 6 +- .../search/internal/ContextIndexSearcher.java | 89 +++-- .../internal/MaxTargetSliceSupplier.java | 4 +- .../search/profile/query/ProfileScorer.java | 11 +- .../search/profile/query/ProfileWeight.java | 25 +- .../search/query/BitmapDocValuesQuery.java | 6 +- .../search/query/BitmapIndexQuery.java | 12 +- .../opensearch/search/query/QueryPhase.java | 2 +- .../search/query/TopDocsCollectorContext.java | 26 +- .../search/slice/DocValuesSliceQuery.java | 6 +- .../search/slice/TermsSliceQuery.java | 6 +- .../search/sort/ScoreSortBuilder.java | 1 - .../sort/SortedWiderNumericSortField.java | 16 + .../completion/CompletionSuggester.java | 3 +- .../phrase/DirectCandidateGenerator.java | 6 +- .../search/suggest/phrase/LaplaceScorer.java | 2 +- .../phrase/LinearInterpolatingScorer.java | 2 +- .../phrase/NoisyChannelSpellChecker.java | 2 +- .../suggest/phrase/StupidBackoffScorer.java | 4 +- ...ceableSearchRequestOperationsListener.java | 2 +- .../services/org.apache.lucene.codecs.Codec | 1 + .../java/org/opensearch/VersionTests.java | 6 +- .../action/search/DfsQueryPhaseTests.java | 6 +- .../action/search/FetchSearchPhaseTests.java | 10 +- .../search/SearchPhaseControllerTests.java | 24 +- .../SearchQueryThenFetchAsyncActionTests.java | 8 +- .../search/SearchResponseMergerTests.java | 20 +- .../action/search/SearchResponseTests.java | 12 +- .../AbstractTermVectorsTestCase.java | 2 +- .../termvectors/TermVectorsUnitTests.java | 4 +- .../opensearch/bootstrap/SecurityTests.java | 1 + .../opensearch/common/lucene/LuceneTests.java | 35 +- .../common/lucene/search/QueriesTests.java | 4 +- .../search/function/MinScoreScorerTests.java | 5 +- .../morelikethis/XMoreLikeThisTests.java | 2 +- .../deps/lucene/SimpleLuceneTests.java | 6 +- .../deps/lucene/VectorHighlighterTests.java | 8 +- .../gateway/MetadataStateFormatTests.java | 2 +- .../opensearch/index/IndexServiceTests.java | 6 +- .../opensearch/index/codec/CodecTests.java | 40 +-- .../LuceneDocValuesConsumerFactoryTests.java | 10 +- .../LuceneDocValuesProducerFactoryTests.java | 6 +- ...tedNumericDocValuesWriterWrapperTests.java | 9 +- .../SortedSetDocValuesWriterWrapperTests.java | 6 +- .../AbstractStarTreeDVFormatTests.java | 6 +- .../builder/BaseStarTreeBuilderTests.java | 8 +- .../startree/builder/BuilderTestsUtils.java | 13 +- .../builder/StarTreeBuildMetricTests.java | 13 +- .../StarTreeBuilderFlushFlowTests.java | 5 + .../builder/StarTreeBuilderTestCase.java | 2 + .../builder/StarTreesBuilderTests.java | 6 +- .../meta/StarTreeMetadataTests.java | 8 +- .../SequentialDocValuesIteratorTests.java | 2 + .../startree/utils/StarTreeUtilsTests.java | 2 +- .../engine/CompletionStatsCacheTests.java | 8 +- .../index/engine/InternalEngineTests.java | 42 +-- .../index/engine/LiveVersionMapTests.java | 25 +- .../index/engine/ReadOnlyEngineTests.java | 7 +- .../opensearch/index/engine/SegmentTests.java | 2 +- .../AbstractFieldDataImplTestCase.java | 12 +- .../AbstractStringFieldDataTestCase.java | 29 +- .../ordinals/MultiOrdinalsTests.java | 4 +- .../FieldStatsProviderRefreshTests.java | 6 +- .../index/mapper/BooleanFieldTypeTests.java | 8 +- .../index/mapper/DateFieldTypeTests.java | 10 +- .../mapper/DerivedFieldMapperQueryTests.java | 78 ++--- .../index/mapper/DerivedFieldTypeTests.java | 5 +- .../index/mapper/DocCountFieldTypeTests.java | 4 +- .../index/mapper/DoubleIndexingDocTests.java | 14 +- .../mapper/FlatObjectFieldTypeTests.java | 46 +-- .../index/mapper/IdFieldTypeTests.java | 6 +- .../index/mapper/IgnoredFieldTypeTests.java | 4 +- .../index/mapper/IpFieldMapperTests.java | 45 +-- .../index/mapper/IpFieldTypeTests.java | 2 +- .../index/mapper/KeywordFieldTypeTests.java | 24 +- .../mapper/StoredNumericValuesTests.java | 2 +- .../index/mapper/TextFieldTypeTests.java | 41 ++- .../index/query/BoolQueryBuilderTests.java | 10 +- .../index/query/DerivedFieldQueryTests.java | 4 +- .../index/query/DisMaxQueryBuilderTests.java | 2 +- .../DistanceFeatureQueryBuilderTests.java | 4 +- .../index/query/ExistsQueryBuilderTests.java | 15 +- .../query/IntervalQueryBuilderTests.java | 6 +- .../MatchBoolPrefixQueryBuilderTests.java | 24 +- .../index/query/MatchQueryBuilderTests.java | 11 +- .../query/MoreLikeThisQueryBuilderTests.java | 4 +- .../index/query/NestedQueryBuilderTests.java | 6 +- .../query/QueryStringQueryBuilderTests.java | 38 +- .../index/query/RangeQueryBuilderTests.java | 9 +- .../query/SimpleQueryStringBuilderTests.java | 12 +- .../query/SourceFieldMatchQueryTests.java | 12 +- .../query/SpanMultiTermQueryBuilderTests.java | 9 +- .../query/TermsSetQueryBuilderTests.java | 4 +- .../IndexLevelReplicationTests.java | 2 +- .../AbstractNumberNestedSortingTestCase.java | 14 +- .../nested/DoubleNestedSortingTests.java | 2 +- .../nested/FloatNestedSortingTests.java | 2 +- .../search/nested/NestedSortingTests.java | 102 +++--- .../index/shard/IndexReaderWrapperTests.java | 8 +- .../index/shard/IndexShardTests.java | 8 +- .../similarity/ScriptedSimilarityTests.java | 4 +- .../index/store/FsDirectoryFactoryTests.java | 66 +--- .../index/store/RemoteDirectoryTests.java | 17 +- .../opensearch/index/store/StoreTests.java | 7 +- .../OnDemandBlockSnapshotIndexInputTests.java | 2 +- ...sferManagerRemoteDirectoryReaderTests.java | 12 +- .../indices/IndicesQueryCacheTests.java | 20 +- .../indices/IndicesRequestCacheTests.java | 2 +- .../indices/IndicesServiceCloseTests.java | 15 +- .../CollapsingTopDocsCollectorTests.java | 15 +- .../lucene/queries/BlendedTermQueryTests.java | 18 +- .../CustomUnifiedHighlighterTests.java | 2 +- .../search/CreatePitSingleNodeTests.java | 37 +- .../search/GenericSearchExtBuilderTests.java | 12 +- .../search/MultiValueModeTests.java | 11 +- .../opensearch/search/SearchServiceTests.java | 97 +++-- .../MultiBucketCollectorTests.java | 6 - .../bucket/ShardSizeTestCase.java | 4 +- .../composite/CompositeAggregatorTests.java | 185 +++++----- .../SingleDimensionValuesSourceTests.java | 30 +- .../range/BinaryRangeAggregatorTests.java | 2 +- .../bucket/terms/IncludeExcludeTests.java | 2 +- .../terms/RareTermsAggregatorTests.java | 10 +- .../bucket/terms/TermsAggregatorTests.java | 10 +- .../metrics/AbstractGeoTestCase.java | 2 +- .../metrics/AvgAggregatorTests.java | 10 +- .../metrics/CardinalityAggregatorTests.java | 6 +- .../HDRPercentilesAggregatorTests.java | 10 +- .../metrics/InternalTopHitsTests.java | 12 +- .../metrics/MaxAggregatorTests.java | 12 +- ...edianAbsoluteDeviationAggregatorTests.java | 8 +- .../metrics/MinAggregatorTests.java | 6 +- .../metrics/SumAggregatorTests.java | 4 +- .../TDigestPercentilesAggregatorTests.java | 6 +- .../metrics/TopHitsAggregatorTests.java | 14 +- .../metrics/ValueCountAggregatorTests.java | 6 +- .../metrics/WeightedAvgAggregatorTests.java | 4 +- .../startree/MetricAggregatorTests.java | 6 +- .../startree/StarTreeFilterTests.java | 6 +- .../support/MissingValuesTests.java | 6 +- .../ApproximatePointRangeQueryTests.java | 8 +- .../search/geo/GeoShapeQueryTests.java | 8 +- .../internal/ContextIndexSearcherTests.java | 26 +- .../internal/MaxTargetSliceSupplierTests.java | 10 +- .../search/lookup/LeafFieldsLookupTests.java | 2 + .../profile/query/ProfileScorerTests.java | 8 +- .../profile/query/QueryProfilerTests.java | 5 - .../search/query/QueryPhaseTests.java | 105 +++--- .../search/query/QueryProfilePhaseTests.java | 69 ++-- .../sort/BucketedSortForFloatsTests.java | 6 - .../CategoryContextMappingTests.java | 4 +- .../phrase/DirectCandidateGeneratorTests.java | 8 +- .../snapshots/RestoreServiceIntegTests.java | 4 +- .../snapshots/SnapshotResiliencyTests.java | 8 +- .../bwc/es-6.3.0/testIndex-es-6.3.0.zip | Bin 2467 -> 0 bytes .../bwc/{es-6.3.0 => os-1.3.0}/README.md | 11 +- .../bwc/os-1.3.0/testIndex-os-1.3.0.zip | Bin 0 -> 2899 bytes test/fixtures/azure-fixture/Dockerfile | 4 +- test/fixtures/gcs-fixture/Dockerfile | 4 +- test/fixtures/s3-fixture/Dockerfile.eks | 4 +- .../index/mapper/MapperTestCase.java | 11 +- .../analysis/AnalysisFactoryTestCase.java | 1 + ...earchBlobStoreRepositoryIntegTestCase.java | 4 +- .../aggregations/AggregatorTestCase.java | 17 +- .../AbstractSnapshotIntegTestCase.java | 2 +- .../org/opensearch/test/CorruptionUtils.java | 2 +- .../test/OpenSearchIntegTestCase.java | 9 +- .../opensearch/test/TestSearchContext.java | 2 +- .../engine/ThrowingLeafReaderWrapper.java | 7 - .../test/hamcrest/OpenSearchAssertions.java | 12 +- .../OpenSearchLoggerUsageChecker.java | 2 +- 519 files changed, 3466 insertions(+), 3050 deletions(-) rename libs/common/src/main/{java20 => java}/org/opensearch/common/round/BtreeSearcher.java (100%) delete mode 100644 libs/common/src/main/java20/org/opensearch/common/round/RoundableFactory.java create mode 100644 libs/core/licenses/lucene-core-10.1.0.jar.sha1 delete mode 100644 libs/core/licenses/lucene-core-9.12.1.jar.sha1 create mode 100644 modules/lang-expression/licenses/lucene-expressions-10.1.0.jar.sha1 delete mode 100644 modules/lang-expression/licenses/lucene-expressions-9.12.1.jar.sha1 create mode 100644 plugins/analysis-icu/licenses/lucene-analysis-icu-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-icu/licenses/lucene-analysis-icu-9.12.1.jar.sha1 create mode 100644 plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.12.1.jar.sha1 create mode 100644 plugins/analysis-nori/licenses/lucene-analysis-nori-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-nori/licenses/lucene-analysis-nori-9.12.1.jar.sha1 create mode 100644 plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.12.1.jar.sha1 create mode 100644 plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.12.1.jar.sha1 create mode 100644 plugins/analysis-stempel/licenses/lucene-analysis-stempel-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.12.1.jar.sha1 create mode 100644 plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-10.1.0.jar.sha1 delete mode 100644 plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-analysis-common-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-analysis-common-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-backward-codecs-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-backward-codecs-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-core-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-core-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-grouping-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-grouping-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-highlighter-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-highlighter-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-join-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-join-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-memory-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-memory-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-misc-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-misc-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-queries-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-queries-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-queryparser-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-queryparser-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-sandbox-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-sandbox-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-spatial-extras-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-spatial-extras-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-spatial3d-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-spatial3d-9.12.1.jar.sha1 create mode 100644 server/licenses/lucene-suggest-10.1.0.jar.sha1 delete mode 100644 server/licenses/lucene-suggest-9.12.1.jar.sha1 create mode 100644 server/src/main/java/org/apache/lucene/util/automaton/MinimizationOperations.java create mode 100644 server/src/main/java/org/opensearch/index/codec/composite/composite101/Composite101Codec.java create mode 100644 server/src/main/java/org/opensearch/index/codec/composite/composite101/package-info.java delete mode 100644 server/src/main/java/org/opensearch/index/store/remote/utils/BlockIOContext.java delete mode 100644 server/src/test/resources/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip rename server/src/test/resources/indices/bwc/{es-6.3.0 => os-1.3.0}/README.md (73%) create mode 100644 server/src/test/resources/indices/bwc/os-1.3.0/testIndex-os-1.3.0.zip diff --git a/.github/workflows/assemble.yml b/.github/workflows/assemble.yml index 02da18859de63..6a68b58db5492 100644 --- a/.github/workflows/assemble.yml +++ b/.github/workflows/assemble.yml @@ -7,7 +7,7 @@ jobs: runs-on: ${{ matrix.os }} strategy: matrix: - java: [ 11, 17, 21 ] + java: [ 21, 23 ] os: [ubuntu-latest, windows-latest, macos-13, ubuntu-24.04-arm] steps: - uses: actions/checkout@v4 @@ -16,17 +16,6 @@ jobs: with: java-version: ${{ matrix.java }} distribution: temurin - - name: Set up JDK 17 - # See please https://docs.gradle.org/8.10/userguide/upgrading_version_8.html#minimum_daemon_jvm_version - if: matrix.java == 11 - uses: actions/setup-java@v4 - with: - java-version: 17 - distribution: temurin - - name: Set JAVA${{ matrix.java }}_HOME - shell: bash - run: | - echo "JAVA${{ matrix.java }}_HOME=$JAVA_HOME_${{ matrix.java }}_${{ runner.arch }}" >> $GITHUB_ENV - name: Setup docker (missing on MacOS) id: setup_docker if: runner.os == 'macos' @@ -44,8 +33,8 @@ jobs: shell: bash if: runner.os != 'macos' run: | - ./gradlew assemble --parallel --no-build-cache -PDISABLE_BUILD_CACHE -Druntime.java=${{ matrix.java }} + ./gradlew assemble --parallel --no-build-cache -PDISABLE_BUILD_CACHE - name: Run Gradle (assemble) if: runner.os == 'macos' && steps.setup_docker.outcome == 'success' run: | - ./gradlew assemble --parallel --no-build-cache -PDISABLE_BUILD_CACHE -Druntime.java=${{ matrix.java }} + ./gradlew assemble --parallel --no-build-cache -PDISABLE_BUILD_CACHE diff --git a/.github/workflows/precommit.yml b/.github/workflows/precommit.yml index e4903d27d97a5..7e20912fe1f60 100644 --- a/.github/workflows/precommit.yml +++ b/.github/workflows/precommit.yml @@ -7,7 +7,7 @@ jobs: runs-on: ${{ matrix.os }} strategy: matrix: - java: [ 11, 17, 21, 23 ] + java: [ 21, 23 ] os: [ubuntu-latest, windows-latest, macos-latest, macos-13, ubuntu-24.04-arm] steps: - uses: actions/checkout@v4 @@ -17,18 +17,7 @@ jobs: java-version: ${{ matrix.java }} distribution: temurin cache: gradle - - name: Set up JDK 17 - # See please https://docs.gradle.org/8.10/userguide/upgrading_version_8.html#minimum_daemon_jvm_version - if: matrix.java == 11 - uses: actions/setup-java@v4 - with: - java-version: 17 - distribution: temurin - - name: Set JAVA${{ matrix.java }}_HOME - shell: bash - run: | - echo "JAVA${{ matrix.java }}_HOME=$JAVA_HOME_${{ matrix.java }}_${{ runner.arch }}" >> $GITHUB_ENV - name: Run Gradle (precommit) shell: bash run: | - ./gradlew javadoc precommit --parallel -Druntime.java=${{ matrix.java }} + ./gradlew javadoc precommit --parallel diff --git a/CHANGELOG-3.0.md b/CHANGELOG-3.0.md index 8d7ee3c860318..6ca1543d85c7d 100644 --- a/CHANGELOG-3.0.md +++ b/CHANGELOG-3.0.md @@ -15,6 +15,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - Views, simplify data access and manipulation by providing a virtual layer over one or more indices ([#11957](https://github.com/opensearch-project/OpenSearch/pull/11957)) ### Dependencies +- Update Apache Lucene to 10.1.0 ([#16366](https://github.com/opensearch-project/OpenSearch/pull/16366)) - Bump Apache HttpCore5/HttpClient5 dependencies from 5.2.5/5.3.1 to 5.3.1/5.4.1 to support ExtendedSocketOption in HttpAsyncClient ([#16757](https://github.com/opensearch-project/OpenSearch/pull/16757)) ### Changed diff --git a/benchmarks/build.gradle b/benchmarks/build.gradle index be4579b4e5324..732e77934b427 100644 --- a/benchmarks/build.gradle +++ b/benchmarks/build.gradle @@ -85,44 +85,5 @@ spotless { } } -if (BuildParams.runtimeJavaVersion >= JavaVersion.VERSION_20) { - // Add support for incubator modules on supported Java versions. - run.jvmArgs += ['--add-modules=jdk.incubator.vector'] - run.classpath += files(jar.archiveFile) - run.classpath -= sourceSets.main.output - evaluationDependsOn(':libs:opensearch-common') - - sourceSets { - java20 { - java { - srcDirs = ['src/main/java20'] - } - } - } - - configurations { - java20Implementation.extendsFrom(implementation) - } - - dependencies { - java20Implementation sourceSets.main.output - java20Implementation project(':libs:opensearch-common').sourceSets.java20.output - java20AnnotationProcessor "org.openjdk.jmh:jmh-generator-annprocess:$versions.jmh" - } - - compileJava20Java { - targetCompatibility = JavaVersion.VERSION_20 - options.compilerArgs.addAll(["-processor", "org.openjdk.jmh.generators.BenchmarkProcessor"]) - } - - jar { - metaInf { - into 'versions/20' - from sourceSets.java20.output - } - manifest.attributes('Multi-Release': 'true') - } - - // classes generated by JMH can use all sorts of forbidden APIs but we have no influence at all and cannot exclude these classes - disableTasks('forbiddenApisJava20') -} +// Add support for incubator modules on supported Java versions. +run.jvmArgs += ['--add-modules=jdk.incubator.vector'] diff --git a/benchmarks/src/main/java/org/opensearch/common/round/RoundableSupplier.java b/benchmarks/src/main/java/org/opensearch/common/round/RoundableSupplier.java index 44ac42810996f..e81c1b137bd30 100644 --- a/benchmarks/src/main/java/org/opensearch/common/round/RoundableSupplier.java +++ b/benchmarks/src/main/java/org/opensearch/common/round/RoundableSupplier.java @@ -13,7 +13,7 @@ public class RoundableSupplier implements Supplier { private final Supplier delegate; - RoundableSupplier(String type, long[] values, int size) throws ClassNotFoundException { + RoundableSupplier(String type, long[] values, int size) { switch (type) { case "binary": delegate = () -> new BinarySearcher(values, size); @@ -22,7 +22,8 @@ public class RoundableSupplier implements Supplier { delegate = () -> new BidirectionalLinearSearcher(values, size); break; case "btree": - throw new ClassNotFoundException("BtreeSearcher is not supported below JDK 20"); + delegate = () -> new BtreeSearcher(values, size); + break; default: throw new IllegalArgumentException("invalid type: " + type); } diff --git a/buildSrc/build.gradle b/buildSrc/build.gradle index f7fc0d7760993..4f3c5de49dbc6 100644 --- a/buildSrc/build.gradle +++ b/buildSrc/build.gradle @@ -74,8 +74,8 @@ processResources { * Java version * *****************************************************************************/ -if (JavaVersion.current() < JavaVersion.VERSION_11) { - throw new GradleException('At least Java 11 is required to build opensearch gradle tools') +if (JavaVersion.current() < JavaVersion.VERSION_21) { + throw new GradleException('At least Java 21 is required to build opensearch gradle tools') } sourceSets { @@ -111,7 +111,7 @@ dependencies { api 'org.apache.rat:apache-rat:0.15' api "commons-io:commons-io:${props.getProperty('commonsio')}" api "net.java.dev.jna:jna:5.14.0" - api 'com.github.johnrengelman:shadow:8.1.1' + api 'com.gradleup.shadow:shadow-gradle-plugin:8.3.5' api 'org.jdom:jdom2:2.0.6.1' api "org.jetbrains.kotlin:kotlin-stdlib-jdk8:${props.getProperty('kotlin')}" api 'de.thetaphi:forbiddenapis:3.8' @@ -171,8 +171,8 @@ if (project != rootProject) { allprojects { java { - targetCompatibility = JavaVersion.VERSION_11 - sourceCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_21 + sourceCompatibility = JavaVersion.VERSION_21 } } diff --git a/buildSrc/reaper/build.gradle b/buildSrc/reaper/build.gradle index 58d06b02e9f4b..55efcf5d5dfff 100644 --- a/buildSrc/reaper/build.gradle +++ b/buildSrc/reaper/build.gradle @@ -12,8 +12,8 @@ apply plugin: 'java' java { - targetCompatibility = JavaVersion.VERSION_11 - sourceCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_21 + sourceCompatibility = JavaVersion.VERSION_21 } jar { diff --git a/buildSrc/src/integTest/resources/org/opensearch/gradle/internal/fake_git/remote/.ci/java-versions.properties b/buildSrc/src/integTest/resources/org/opensearch/gradle/internal/fake_git/remote/.ci/java-versions.properties index 97dbc8f7ced11..4c9c2554f5d9a 100644 --- a/buildSrc/src/integTest/resources/org/opensearch/gradle/internal/fake_git/remote/.ci/java-versions.properties +++ b/buildSrc/src/integTest/resources/org/opensearch/gradle/internal/fake_git/remote/.ci/java-versions.properties @@ -27,6 +27,6 @@ # specific language governing permissions and limitations # under the License. # -OPENSEARCH_BUILD_JAVA=openjdk11 -OPENSEARCH_RUNTIME_JAVA=java11 +OPENSEARCH_BUILD_JAVA=openjdk21 +OPENSEARCH_RUNTIME_JAVA=java21 GRADLE_TASK=build diff --git a/buildSrc/src/main/resources/forbidden/opensearch-server-signatures.txt b/buildSrc/src/main/resources/forbidden/opensearch-server-signatures.txt index 27fba8069125d..b57f4ff8e72ae 100644 --- a/buildSrc/src/main/resources/forbidden/opensearch-server-signatures.txt +++ b/buildSrc/src/main/resources/forbidden/opensearch-server-signatures.txt @@ -61,10 +61,6 @@ java.nio.channels.FileChannel#read(java.nio.ByteBuffer, long) @defaultMessage Use Lucene.parseLenient instead it strips off minor version org.apache.lucene.util.Version#parseLeniently(java.lang.String) -@defaultMessage Spawns a new thread which is solely under lucenes control use ThreadPool#relativeTimeInMillis instead -org.apache.lucene.search.TimeLimitingCollector#getGlobalTimerThread() -org.apache.lucene.search.TimeLimitingCollector#getGlobalCounter() - @defaultMessage Don't interrupt threads use FutureUtils#cancel(Future) instead java.util.concurrent.Future#cancel(boolean) diff --git a/buildSrc/src/main/resources/minimumCompilerVersion b/buildSrc/src/main/resources/minimumCompilerVersion index 98d9bcb75a685..aabe6ec3909c9 100644 --- a/buildSrc/src/main/resources/minimumCompilerVersion +++ b/buildSrc/src/main/resources/minimumCompilerVersion @@ -1 +1 @@ -17 +21 diff --git a/buildSrc/src/main/resources/minimumRuntimeVersion b/buildSrc/src/main/resources/minimumRuntimeVersion index b4de394767536..aabe6ec3909c9 100644 --- a/buildSrc/src/main/resources/minimumRuntimeVersion +++ b/buildSrc/src/main/resources/minimumRuntimeVersion @@ -1 +1 @@ -11 +21 diff --git a/buildSrc/src/testKit/thirdPartyAudit/build.gradle b/buildSrc/src/testKit/thirdPartyAudit/build.gradle index 553ff5d8e6ed2..caaaa6bd1aceb 100644 --- a/buildSrc/src/testKit/thirdPartyAudit/build.gradle +++ b/buildSrc/src/testKit/thirdPartyAudit/build.gradle @@ -46,11 +46,11 @@ dependencies { } tasks.register("empty", ThirdPartyAuditTask) { - targetCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_21 signatureFile = file('third-party-audit-empty.txt') } tasks.register("absurd", ThirdPartyAuditTask) { - targetCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_21 signatureFile = file('third-party-audit-absurd.txt') } diff --git a/client/rest-high-level/src/test/java/org/opensearch/client/HighLevelRestClientCompressionIT.java b/client/rest-high-level/src/test/java/org/opensearch/client/HighLevelRestClientCompressionIT.java index 6985353806a01..245c1729f567d 100644 --- a/client/rest-high-level/src/test/java/org/opensearch/client/HighLevelRestClientCompressionIT.java +++ b/client/rest-high-level/src/test/java/org/opensearch/client/HighLevelRestClientCompressionIT.java @@ -67,7 +67,7 @@ public void testCompressesResponseIfRequested() throws IOException { SearchResponse searchResponse = execute(searchRequest, highLevelClient()::search, highLevelClient()::searchAsync, requestOptions); assertThat(searchResponse.status().getStatus(), equalTo(200)); - assertEquals(1L, searchResponse.getHits().getTotalHits().value); + assertEquals(1L, searchResponse.getHits().getTotalHits().value()); assertEquals(SAMPLE_DOCUMENT, searchResponse.getHits().getHits()[0].getSourceAsString()); } diff --git a/client/rest-high-level/src/test/java/org/opensearch/client/ReindexIT.java b/client/rest-high-level/src/test/java/org/opensearch/client/ReindexIT.java index 2457bafdc6a22..0652eb86c0979 100644 --- a/client/rest-high-level/src/test/java/org/opensearch/client/ReindexIT.java +++ b/client/rest-high-level/src/test/java/org/opensearch/client/ReindexIT.java @@ -236,7 +236,7 @@ public void testDeleteByQuery() throws Exception { assertEquals(0, bulkResponse.getSearchFailures().size()); assertEquals( 2, - highLevelClient().search(new SearchRequest(sourceIndex), RequestOptions.DEFAULT).getHits().getTotalHits().value + highLevelClient().search(new SearchRequest(sourceIndex), RequestOptions.DEFAULT).getHits().getTotalHits().value() ); } { diff --git a/client/rest-high-level/src/test/java/org/opensearch/client/RestHighLevelClientTests.java b/client/rest-high-level/src/test/java/org/opensearch/client/RestHighLevelClientTests.java index b0964a22786f0..ade6188275c64 100644 --- a/client/rest-high-level/src/test/java/org/opensearch/client/RestHighLevelClientTests.java +++ b/client/rest-high-level/src/test/java/org/opensearch/client/RestHighLevelClientTests.java @@ -237,7 +237,7 @@ public void testSearchScroll() throws IOException { RequestOptions.DEFAULT ); assertEquals(mockSearchResponse.getScrollId(), searchResponse.getScrollId()); - assertEquals(0, searchResponse.getHits().getTotalHits().value); + assertEquals(0, searchResponse.getHits().getTotalHits().value()); assertEquals(5, searchResponse.getTotalShards()); assertEquals(5, searchResponse.getSuccessfulShards()); assertEquals(100, searchResponse.getTook().getMillis()); diff --git a/client/rest-high-level/src/test/java/org/opensearch/client/SearchIT.java b/client/rest-high-level/src/test/java/org/opensearch/client/SearchIT.java index d10dc3df43ee5..06fca965d6cb8 100644 --- a/client/rest-high-level/src/test/java/org/opensearch/client/SearchIT.java +++ b/client/rest-high-level/src/test/java/org/opensearch/client/SearchIT.java @@ -247,7 +247,7 @@ public void testSearchNoQuery() throws IOException { assertNull(searchResponse.getAggregations()); assertNull(searchResponse.getSuggest()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); assertEquals(5, searchResponse.getHits().getHits().length); for (SearchHit searchHit : searchResponse.getHits().getHits()) { assertEquals("index", searchHit.getIndex()); @@ -269,7 +269,7 @@ public void testSearchMatchQuery() throws IOException { assertNull(searchResponse.getAggregations()); assertNull(searchResponse.getSuggest()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); assertEquals(1, searchResponse.getHits().getHits().length); assertThat(searchResponse.getHits().getMaxScore(), greaterThan(0f)); SearchHit searchHit = searchResponse.getHits().getHits()[0]; @@ -443,7 +443,7 @@ public void testSearchWithRangeAgg() throws IOException { assertSearchHeader(searchResponse); assertNull(searchResponse.getSuggest()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); assertEquals(0, searchResponse.getHits().getHits().length); assertEquals(Float.NaN, searchResponse.getHits().getMaxScore(), 0f); Range rangeAgg = searchResponse.getAggregations().get("agg1"); @@ -560,7 +560,7 @@ public void testSearchWithMatrixStats() throws IOException { assertSearchHeader(searchResponse); assertNull(searchResponse.getSuggest()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); assertEquals(0, searchResponse.getHits().getHits().length); assertEquals(Float.NaN, searchResponse.getHits().getMaxScore(), 0f); assertEquals(1, searchResponse.getAggregations().asList().size()); @@ -664,7 +664,7 @@ public void testSearchWithParentJoin() throws IOException { assertSearchHeader(searchResponse); assertNull(searchResponse.getSuggest()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(3, searchResponse.getHits().getTotalHits().value); + assertEquals(3, searchResponse.getHits().getTotalHits().value()); assertEquals(0, searchResponse.getHits().getHits().length); assertEquals(Float.NaN, searchResponse.getHits().getMaxScore(), 0f); assertEquals(1, searchResponse.getAggregations().asList().size()); @@ -705,7 +705,7 @@ public void testSearchWithSuggest() throws IOException { assertSearchHeader(searchResponse); assertNull(searchResponse.getAggregations()); assertEquals(Collections.emptyMap(), searchResponse.getProfileResults()); - assertEquals(0, searchResponse.getHits().getTotalHits().value); + assertEquals(0, searchResponse.getHits().getTotalHits().value()); assertEquals(Float.NaN, searchResponse.getHits().getMaxScore(), 0f); assertEquals(0, searchResponse.getHits().getHits().length); assertEquals(1, searchResponse.getSuggest().size()); @@ -1009,7 +1009,7 @@ public void testSearchScroll() throws Exception { try { long counter = 0; assertSearchHeader(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -1021,7 +1021,7 @@ public void testSearchScroll() throws Exception { highLevelClient()::scrollAsync ); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertEquals(counter++, ((Number) hit.getSortValues()[0]).longValue()); @@ -1033,7 +1033,7 @@ public void testSearchScroll() throws Exception { highLevelClient()::scrollAsync ); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(30)); for (SearchHit hit : searchResponse.getHits()) { assertEquals(counter++, ((Number) hit.getSortValues()[0]).longValue()); @@ -1082,7 +1082,7 @@ public void testSearchWithPit() throws Exception { try { long counter = 0; assertSearchHeader(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -1120,21 +1120,21 @@ public void testMultiSearch() throws Exception { assertThat(multiSearchResponse.getResponses()[0].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[0].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("1")); assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getAt(1).getId(), Matchers.equalTo("2")); assertThat(multiSearchResponse.getResponses()[1].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[1].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[1].getResponse()); - assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("3")); assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getAt(1).getId(), Matchers.equalTo("4")); assertThat(multiSearchResponse.getResponses()[2].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[2].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[2].getResponse()); - assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("5")); assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getAt(1).getId(), Matchers.equalTo("6")); } @@ -1176,7 +1176,7 @@ public void testSearchWithSort() throws Exception { SearchResponse searchResponse = execute(searchRequest, highLevelClient()::search, highLevelClient()::searchAsync); assertThat(searchResponse.getTook().millis(), Matchers.greaterThanOrEqualTo(0L)); - assertThat(searchResponse.getHits().getTotalHits().value, Matchers.equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), Matchers.equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), Matchers.equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), Matchers.equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), Matchers.equalTo("3")); @@ -1220,7 +1220,7 @@ public void testMultiSearch_withAgg() throws Exception { assertThat(multiSearchResponse.getResponses()[0].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[0].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getHits().length, Matchers.equalTo(0)); Terms terms = multiSearchResponse.getResponses()[0].getResponse().getAggregations().get("name"); assertThat(terms.getBuckets().size(), Matchers.equalTo(2)); @@ -1230,7 +1230,7 @@ public void testMultiSearch_withAgg() throws Exception { assertThat(multiSearchResponse.getResponses()[1].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[1].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getHits().length, Matchers.equalTo(0)); terms = multiSearchResponse.getResponses()[1].getResponse().getAggregations().get("name"); assertThat(terms.getBuckets().size(), Matchers.equalTo(2)); @@ -1240,7 +1240,7 @@ public void testMultiSearch_withAgg() throws Exception { assertThat(multiSearchResponse.getResponses()[2].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[2].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getHits().length, Matchers.equalTo(0)); terms = multiSearchResponse.getResponses()[2].getResponse().getAggregations().get("name"); assertThat(terms.getBuckets().size(), Matchers.equalTo(2)); @@ -1267,19 +1267,19 @@ public void testMultiSearch_withQuery() throws Exception { assertThat(multiSearchResponse.getResponses()[0].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[0].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("2")); assertThat(multiSearchResponse.getResponses()[1].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[1].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[1].getResponse()); - assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("4")); assertThat(multiSearchResponse.getResponses()[2].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[2].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[2].getResponse()); - assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("6")); searchRequest1.source().highlighter(new HighlightBuilder().field("field")); @@ -1292,7 +1292,7 @@ public void testMultiSearch_withQuery() throws Exception { assertThat(multiSearchResponse.getResponses()[0].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[0].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[0].getResponse()); - assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[0].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat( multiSearchResponse.getResponses()[0].getResponse().getHits().getAt(0).getHighlightFields().get("field").fragments()[0] .string(), @@ -1302,7 +1302,7 @@ public void testMultiSearch_withQuery() throws Exception { assertThat(multiSearchResponse.getResponses()[1].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[1].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[1].getResponse()); - assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat(multiSearchResponse.getResponses()[1].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("4")); assertThat( multiSearchResponse.getResponses()[1].getResponse().getHits().getAt(0).getHighlightFields().get("field").fragments()[0] @@ -1313,7 +1313,7 @@ public void testMultiSearch_withQuery() throws Exception { assertThat(multiSearchResponse.getResponses()[2].getFailure(), Matchers.nullValue()); assertThat(multiSearchResponse.getResponses()[2].isFailure(), Matchers.is(false)); SearchIT.assertSearchHeader(multiSearchResponse.getResponses()[2].getResponse()); - assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value, Matchers.equalTo(1L)); + assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getTotalHits().value(), Matchers.equalTo(1L)); assertThat(multiSearchResponse.getResponses()[2].getResponse().getHits().getAt(0).getId(), Matchers.equalTo("6")); assertThat( multiSearchResponse.getResponses()[2].getResponse().getHits().getAt(0).getHighlightFields().get("field").fragments()[0] @@ -1369,7 +1369,7 @@ public void testSearchTemplate() throws IOException { SearchResponse searchResponse = searchTemplateResponse.getResponse(); assertNotNull(searchResponse); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); assertEquals(1, searchResponse.getHits().getHits().length); assertThat(searchResponse.getHits().getMaxScore(), greaterThan(0f)); @@ -1470,7 +1470,7 @@ public void testMultiSearchTemplate() throws Exception { SearchResponse goodResponse = responses[0].getResponse().getResponse(); assertNotNull(goodResponse); assertThat(responses[0].isFailure(), Matchers.is(false)); - assertEquals(1, goodResponse.getHits().getTotalHits().value); + assertEquals(1, goodResponse.getHits().getTotalHits().value()); assertEquals(1, goodResponse.getHits().getHits().length); assertThat(goodResponse.getHits().getMaxScore(), greaterThan(0f)); SearchHit hit = goodResponse.getHits().getHits()[0]; diff --git a/client/rest-high-level/src/test/java/org/opensearch/client/documentation/SearchDocumentationIT.java b/client/rest-high-level/src/test/java/org/opensearch/client/documentation/SearchDocumentationIT.java index 326dde54cfb61..14d3222a51083 100644 --- a/client/rest-high-level/src/test/java/org/opensearch/client/documentation/SearchDocumentationIT.java +++ b/client/rest-high-level/src/test/java/org/opensearch/client/documentation/SearchDocumentationIT.java @@ -252,9 +252,9 @@ public void onFailure(Exception e) { // tag::search-hits-info TotalHits totalHits = hits.getTotalHits(); // the total number of hits, must be interpreted in the context of totalHits.relation - long numHits = totalHits.value; + long numHits = totalHits.value(); // whether the number of hits is accurate (EQUAL_TO) or a lower bound of the total (GREATER_THAN_OR_EQUAL_TO) - TotalHits.Relation relation = totalHits.relation; + TotalHits.Relation relation = totalHits.relation(); float maxScore = hits.getMaxScore(); // end::search-hits-info // tag::search-hits-singleHit @@ -625,7 +625,7 @@ public void testScroll() throws Exception { String scrollId = searchResponse.getScrollId(); // <3> SearchHits hits = searchResponse.getHits(); // <4> // end::search-scroll-init - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(1, hits.getHits().length); assertNotNull(scrollId); @@ -635,7 +635,7 @@ public void testScroll() throws Exception { SearchResponse searchScrollResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT); scrollId = searchScrollResponse.getScrollId(); // <2> hits = searchScrollResponse.getHits(); // <3> - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(1, hits.getHits().length); assertNotNull(scrollId); // end::search-scroll2 @@ -665,7 +665,7 @@ public void testScroll() throws Exception { // end::search-scroll-execute-sync assertEquals(0, searchResponse.getFailedShards()); - assertEquals(3L, searchResponse.getHits().getTotalHits().value); + assertEquals(3L, searchResponse.getHits().getTotalHits().value()); // tag::search-scroll-execute-listener ActionListener scrollListener = @@ -802,7 +802,7 @@ public void testSearchTemplateWithInlineScript() throws Exception { // end::search-template-response assertNotNull(searchResponse); - assertTrue(searchResponse.getHits().getTotalHits().value > 0); + assertTrue(searchResponse.getHits().getTotalHits().value() > 0); // tag::render-search-template-request request.setSimulate(true); // <1> @@ -852,7 +852,7 @@ public void testSearchTemplateWithStoredScript() throws Exception { SearchResponse searchResponse = response.getResponse(); assertNotNull(searchResponse); - assertTrue(searchResponse.getHits().getTotalHits().value > 0); + assertTrue(searchResponse.getHits().getTotalHits().value() > 0); // tag::search-template-execute-listener ActionListener listener = new ActionListener() { @@ -929,7 +929,7 @@ public void testMultiSearchTemplateWithInlineScript() throws Exception { assertEquals(searchTerms.length, multiResponse.getResponses().length); assertNotNull(multiResponse.getResponses()[0]); SearchResponse searchResponse = multiResponse.getResponses()[0].getResponse().getResponse(); - assertTrue(searchResponse.getHits().getTotalHits().value > 0); + assertTrue(searchResponse.getHits().getTotalHits().value() > 0); } @@ -969,7 +969,7 @@ public void testMultiSearchTemplateWithStoredScript() throws Exception { assertEquals(searchTerms.length, multiResponse.getResponses().length); assertNotNull(multiResponse.getResponses()[0]); SearchResponse searchResponse = multiResponse.getResponses()[0].getResponse().getResponse(); - assertTrue(searchResponse.getHits().getTotalHits().value > 0); + assertTrue(searchResponse.getHits().getTotalHits().value() > 0); // tag::multi-search-template-execute-listener ActionListener listener = new ActionListener() { @@ -1250,11 +1250,11 @@ public void testMultiSearch() throws Exception { MultiSearchResponse.Item firstResponse = response.getResponses()[0]; // <1> assertNull(firstResponse.getFailure()); // <2> SearchResponse searchResponse = firstResponse.getResponse(); // <3> - assertEquals(4, searchResponse.getHits().getTotalHits().value); + assertEquals(4, searchResponse.getHits().getTotalHits().value()); MultiSearchResponse.Item secondResponse = response.getResponses()[1]; // <4> assertNull(secondResponse.getFailure()); searchResponse = secondResponse.getResponse(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); // end::multi-search-response // tag::multi-search-execute-listener diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 96f6178295f00..c2afc85f79078 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,6 +1,6 @@ [versions] opensearch = "3.0.0" -lucene = "9.12.1" +lucene = "10.1.0" bundled_jdk_vendor = "adoptium" bundled_jdk = "23.0.1+11" diff --git a/gradle/missing-javadoc.gradle b/gradle/missing-javadoc.gradle index 179c905c880b4..342417b3b40ab 100644 --- a/gradle/missing-javadoc.gradle +++ b/gradle/missing-javadoc.gradle @@ -303,7 +303,7 @@ class MissingJavadocTask extends DefaultTask { opts << [ '--missing-method', String.join(',', javadocMissingMethod) ] } opts << [ '-quiet' ] - opts << [ '--release', 11 ] + opts << [ '--release', 21 ] opts << '-Xdoclint:all,-missing' // Temporary file that holds all javadoc options for the current task. @@ -356,6 +356,7 @@ class MissingJavadocTask extends DefaultTask { args += [ "-J-Xmx512m" ] // force locale to be "en_US" (fix for: https://bugs.openjdk.java.net/browse/JDK-8222793) args += [ "-J-Duser.language=en", "-J-Duser.country=US" ] + args += ["--add-modules", "jdk.incubator.vector"] ignoreExitValue = true } diff --git a/libs/common/build.gradle b/libs/common/build.gradle index 2bf2dbb803d9f..576e78bbe19f4 100644 --- a/libs/common/build.gradle +++ b/libs/common/build.gradle @@ -13,6 +13,11 @@ import org.opensearch.gradle.info.BuildParams apply plugin: 'opensearch.publish' +ext { + // Do not fail on 'warning: using incubating module(s): jdk.incubator.vector' + failOnJavadocWarning = false +} + base { archivesName = 'opensearch-common' } @@ -42,65 +47,19 @@ tasks.named('forbiddenApisMain').configure { // :libs:opensearch-common does not depend on server // TODO: Need to decide how we want to handle for forbidden signatures with the changes to server replaceSignatureFiles 'jdk-signatures' + failOnMissingClasses = false + ignoreSignaturesOfMissingClasses = true } -// Add support for incubator modules on supported Java versions. -if (BuildParams.runtimeJavaVersion >= JavaVersion.VERSION_20) { - sourceSets { - java20 { - java { - srcDirs = ['src/main/java20'] - } - } - } - - configurations { - java20Implementation.extendsFrom(implementation) - } - - dependencies { - java20Implementation sourceSets.main.output - } - - compileJava20Java { - targetCompatibility = JavaVersion.VERSION_20 - options.compilerArgs += ['--add-modules', 'jdk.incubator.vector'] - options.compilerArgs -= '-Werror' // use of incubator modules is reported as a warning - } - - jar { - metaInf { - into 'versions/20' - from sourceSets.java20.output - } - manifest.attributes('Multi-Release': 'true') - } - - tasks.withType(Test).configureEach { - // Relying on the convention for Test.classpath in custom Test tasks has been deprecated - // and scheduled to be removed in Gradle 9.0. Below lines are added from the migration guide: - // https://docs.gradle.org/8.5/userguide/upgrading_version_8.html#test_task_default_classpath - testClassesDirs = testing.suites.test.sources.output.classesDirs - classpath = testing.suites.test.sources.runtimeClasspath - - // Adds the multi-release JAR to the classpath when executing tests. - // This allows newer sources to be picked up at test runtime (if supported). - classpath += files(jar.archiveFile) - // Removes the "main" sources from the classpath to avoid JarHell problems as - // the multi-release JAR already contains those classes. - classpath -= sourceSets.main.output - } - - tasks.register('roundableSimdTest', Test) { - group = 'verification' - include '**/RoundableTests.class' - systemProperty 'opensearch.experimental.feature.simd.rounding.enabled', 'forced' - } +compileJava { + options.compilerArgs += ['--add-modules', 'jdk.incubator.vector'] + options.compilerArgs -= '-Werror' // use of incubator modules is reported as a warning +} - check.dependsOn(roundableSimdTest) +test { + systemProperty 'opensearch.experimental.feature.simd.rounding.enabled', 'forced' +} - forbiddenApisJava20 { - failOnMissingClasses = false - ignoreSignaturesOfMissingClasses = true - } +javadoc { + options.addStringOption("-add-modules", "jdk.incubator.vector") } diff --git a/libs/common/src/main/java20/org/opensearch/common/round/BtreeSearcher.java b/libs/common/src/main/java/org/opensearch/common/round/BtreeSearcher.java similarity index 100% rename from libs/common/src/main/java20/org/opensearch/common/round/BtreeSearcher.java rename to libs/common/src/main/java/org/opensearch/common/round/BtreeSearcher.java diff --git a/libs/common/src/main/java/org/opensearch/common/round/RoundableFactory.java b/libs/common/src/main/java/org/opensearch/common/round/RoundableFactory.java index b7422694c3013..0709ed4374227 100644 --- a/libs/common/src/main/java/org/opensearch/common/round/RoundableFactory.java +++ b/libs/common/src/main/java/org/opensearch/common/round/RoundableFactory.java @@ -18,12 +18,46 @@ @InternalApi public final class RoundableFactory { /** - * The maximum limit up to which linear search is used, otherwise binary search is used. + * The maximum limit up to which linear search is used, otherwise binary or B-tree search is used. * This is because linear search is much faster on small arrays. * Benchmark results: PR #9727 */ private static final int LINEAR_SEARCH_MAX_SIZE = 64; + /** + * Indicates whether the vectorized (SIMD) B-tree search implementation is to be used. + * It is true when either: + * 1. The feature flag is set to "forced", or + * 2. The platform has a minimum of 4 long vector lanes and the feature flag is set to "true". + */ + private static final boolean USE_BTREE_SEARCHER; + + /** + * This class is initialized only when: + * - JDK-20+ + * - jdk.incubator.vector.LongVector is available (--add-modules=jdk.incubator.vector is passed) + */ + private static final class VectorCheck { + final static int SPECIES_PREFERRED = jdk.incubator.vector.LongVector.SPECIES_PREFERRED.length(); + } + + static { + String simdRoundingFeatureFlag = System.getProperty("opensearch.experimental.feature.simd.rounding.enabled"); + boolean useBtreeSearcher = false; + + try { + final Class incubator = Class.forName("jdk.incubator.vector.LongVector"); + + useBtreeSearcher = "forced".equalsIgnoreCase(simdRoundingFeatureFlag) + || (VectorCheck.SPECIES_PREFERRED >= 4 && "true".equalsIgnoreCase(simdRoundingFeatureFlag)); + + } catch (final ClassNotFoundException ex) { + /* do not use BtreeSearcher */ + } + + USE_BTREE_SEARCHER = useBtreeSearcher; + } + private RoundableFactory() {} /** @@ -32,6 +66,8 @@ private RoundableFactory() {} public static Roundable create(long[] values, int size) { if (size <= LINEAR_SEARCH_MAX_SIZE) { return new BidirectionalLinearSearcher(values, size); + } else if (USE_BTREE_SEARCHER) { + return new BtreeSearcher(values, size); } else { return new BinarySearcher(values, size); } diff --git a/libs/common/src/main/java20/org/opensearch/common/round/RoundableFactory.java b/libs/common/src/main/java20/org/opensearch/common/round/RoundableFactory.java deleted file mode 100644 index 0709ed4374227..0000000000000 --- a/libs/common/src/main/java20/org/opensearch/common/round/RoundableFactory.java +++ /dev/null @@ -1,75 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * - * The OpenSearch Contributors require contributions made to - * this file be licensed under the Apache-2.0 license or a - * compatible open source license. - */ - -package org.opensearch.common.round; - -import org.opensearch.common.annotation.InternalApi; - -/** - * Factory class to create and return the fastest implementation of {@link Roundable}. - * - * @opensearch.internal - */ -@InternalApi -public final class RoundableFactory { - /** - * The maximum limit up to which linear search is used, otherwise binary or B-tree search is used. - * This is because linear search is much faster on small arrays. - * Benchmark results: PR #9727 - */ - private static final int LINEAR_SEARCH_MAX_SIZE = 64; - - /** - * Indicates whether the vectorized (SIMD) B-tree search implementation is to be used. - * It is true when either: - * 1. The feature flag is set to "forced", or - * 2. The platform has a minimum of 4 long vector lanes and the feature flag is set to "true". - */ - private static final boolean USE_BTREE_SEARCHER; - - /** - * This class is initialized only when: - * - JDK-20+ - * - jdk.incubator.vector.LongVector is available (--add-modules=jdk.incubator.vector is passed) - */ - private static final class VectorCheck { - final static int SPECIES_PREFERRED = jdk.incubator.vector.LongVector.SPECIES_PREFERRED.length(); - } - - static { - String simdRoundingFeatureFlag = System.getProperty("opensearch.experimental.feature.simd.rounding.enabled"); - boolean useBtreeSearcher = false; - - try { - final Class incubator = Class.forName("jdk.incubator.vector.LongVector"); - - useBtreeSearcher = "forced".equalsIgnoreCase(simdRoundingFeatureFlag) - || (VectorCheck.SPECIES_PREFERRED >= 4 && "true".equalsIgnoreCase(simdRoundingFeatureFlag)); - - } catch (final ClassNotFoundException ex) { - /* do not use BtreeSearcher */ - } - - USE_BTREE_SEARCHER = useBtreeSearcher; - } - - private RoundableFactory() {} - - /** - * Creates and returns the fastest implementation of {@link Roundable}. - */ - public static Roundable create(long[] values, int size) { - if (size <= LINEAR_SEARCH_MAX_SIZE) { - return new BidirectionalLinearSearcher(values, size); - } else if (USE_BTREE_SEARCHER) { - return new BtreeSearcher(values, size); - } else { - return new BinarySearcher(values, size); - } - } -} diff --git a/libs/core/licenses/lucene-core-10.1.0.jar.sha1 b/libs/core/licenses/lucene-core-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..1727275619d1f --- /dev/null +++ b/libs/core/licenses/lucene-core-10.1.0.jar.sha1 @@ -0,0 +1 @@ +65d7670de89a72433ef374b332da679a484d3a1e \ No newline at end of file diff --git a/libs/core/licenses/lucene-core-9.12.1.jar.sha1 b/libs/core/licenses/lucene-core-9.12.1.jar.sha1 deleted file mode 100644 index 2521c91a81d64..0000000000000 --- a/libs/core/licenses/lucene-core-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -91447c90c1180122142773b5baddaf8547124794 \ No newline at end of file diff --git a/libs/core/src/main/java/org/opensearch/LegacyESVersion.java b/libs/core/src/main/java/org/opensearch/LegacyESVersion.java index 5d8e067a8fd8b..6de19c89053ad 100644 --- a/libs/core/src/main/java/org/opensearch/LegacyESVersion.java +++ b/libs/core/src/main/java/org/opensearch/LegacyESVersion.java @@ -50,7 +50,7 @@ public class LegacyESVersion extends Version { public static final LegacyESVersion V_6_0_0 = new LegacyESVersion(6000099, org.apache.lucene.util.Version.fromBits(7, 0, 0)); public static final LegacyESVersion V_6_5_0 = new LegacyESVersion(6050099, org.apache.lucene.util.Version.fromBits(7, 0, 0)); - public static final LegacyESVersion V_7_2_0 = new LegacyESVersion(7020099, org.apache.lucene.util.Version.LUCENE_8_0_0); + public static final LegacyESVersion V_7_2_0 = new LegacyESVersion(7020099, org.apache.lucene.util.Version.fromBits(8, 0, 0)); // todo move back to Version.java if retiring legacy version support protected static final Map idToVersion; diff --git a/libs/core/src/main/java/org/opensearch/Version.java b/libs/core/src/main/java/org/opensearch/Version.java index dd804fcc6db70..9db30ed47cb50 100644 --- a/libs/core/src/main/java/org/opensearch/Version.java +++ b/libs/core/src/main/java/org/opensearch/Version.java @@ -114,7 +114,7 @@ public class Version implements Comparable, ToXContentFragment { public static final Version V_2_18_0 = new Version(2180099, org.apache.lucene.util.Version.LUCENE_9_12_0); public static final Version V_2_18_1 = new Version(2180199, org.apache.lucene.util.Version.LUCENE_9_12_1); public static final Version V_2_19_0 = new Version(2190099, org.apache.lucene.util.Version.LUCENE_9_12_1); - public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_9_12_1); + public static final Version V_3_0_0 = new Version(3000099, org.apache.lucene.util.Version.LUCENE_10_1_0); public static final Version CURRENT = V_3_0_0; public static Version fromId(int id) { diff --git a/modules/analysis-common/src/main/java/org/opensearch/analysis/common/StemmerTokenFilterFactory.java b/modules/analysis-common/src/main/java/org/opensearch/analysis/common/StemmerTokenFilterFactory.java index e81f3c6cc09cc..22997e67e302f 100644 --- a/modules/analysis-common/src/main/java/org/opensearch/analysis/common/StemmerTokenFilterFactory.java +++ b/modules/analysis-common/src/main/java/org/opensearch/analysis/common/StemmerTokenFilterFactory.java @@ -60,7 +60,6 @@ import org.apache.lucene.analysis.lv.LatvianStemFilter; import org.apache.lucene.analysis.miscellaneous.EmptyTokenStream; import org.apache.lucene.analysis.no.NorwegianLightStemFilter; -import org.apache.lucene.analysis.no.NorwegianLightStemmer; import org.apache.lucene.analysis.no.NorwegianMinimalStemFilter; import org.apache.lucene.analysis.pt.PortugueseLightStemFilter; import org.apache.lucene.analysis.pt.PortugueseMinimalStemFilter; @@ -85,14 +84,11 @@ import org.tartarus.snowball.ext.EstonianStemmer; import org.tartarus.snowball.ext.FinnishStemmer; import org.tartarus.snowball.ext.FrenchStemmer; -import org.tartarus.snowball.ext.German2Stemmer; import org.tartarus.snowball.ext.GermanStemmer; import org.tartarus.snowball.ext.HungarianStemmer; import org.tartarus.snowball.ext.IrishStemmer; import org.tartarus.snowball.ext.ItalianStemmer; -import org.tartarus.snowball.ext.KpStemmer; import org.tartarus.snowball.ext.LithuanianStemmer; -import org.tartarus.snowball.ext.LovinsStemmer; import org.tartarus.snowball.ext.NorwegianStemmer; import org.tartarus.snowball.ext.PortugueseStemmer; import org.tartarus.snowball.ext.RomanianStemmer; @@ -138,18 +134,12 @@ public TokenStream create(TokenStream tokenStream) { // Dutch stemmers } else if ("dutch".equalsIgnoreCase(language)) { return new SnowballFilter(tokenStream, new DutchStemmer()); - } else if ("dutch_kp".equalsIgnoreCase(language) || "dutchKp".equalsIgnoreCase(language) || "kp".equalsIgnoreCase(language)) { - return new SnowballFilter(tokenStream, new KpStemmer()); - - // English stemmers } else if ("english".equalsIgnoreCase(language)) { return new PorterStemFilter(tokenStream); } else if ("light_english".equalsIgnoreCase(language) || "lightEnglish".equalsIgnoreCase(language) || "kstem".equalsIgnoreCase(language)) { return new KStemFilter(tokenStream); - } else if ("lovins".equalsIgnoreCase(language)) { - return new SnowballFilter(tokenStream, new LovinsStemmer()); } else if ("porter".equalsIgnoreCase(language)) { return new PorterStemFilter(tokenStream); } else if ("porter2".equalsIgnoreCase(language)) { @@ -191,7 +181,8 @@ public TokenStream create(TokenStream tokenStream) { } else if ("german".equalsIgnoreCase(language)) { return new SnowballFilter(tokenStream, new GermanStemmer()); } else if ("german2".equalsIgnoreCase(language)) { - return new SnowballFilter(tokenStream, new German2Stemmer()); + // Snowball has folded the "German2" stemmer into their "German" stemmer, so there's no "German2" anymore + return new SnowballFilter(tokenStream, new GermanStemmer()); } else if ("light_german".equalsIgnoreCase(language) || "lightGerman".equalsIgnoreCase(language)) { return new GermanLightStemFilter(tokenStream); } else if ("minimal_german".equalsIgnoreCase(language) || "minimalGerman".equalsIgnoreCase(language)) { @@ -237,9 +228,9 @@ public TokenStream create(TokenStream tokenStream) { // Norwegian (Nynorsk) stemmers } else if ("light_nynorsk".equalsIgnoreCase(language) || "lightNynorsk".equalsIgnoreCase(language)) { - return new NorwegianLightStemFilter(tokenStream, NorwegianLightStemmer.NYNORSK); + return new NorwegianLightStemFilter(tokenStream, 2 /* NorwegianLightStemmer.NYNORSK = 2 */); } else if ("minimal_nynorsk".equalsIgnoreCase(language) || "minimalNynorsk".equalsIgnoreCase(language)) { - return new NorwegianMinimalStemFilter(tokenStream, NorwegianLightStemmer.NYNORSK); + return new NorwegianMinimalStemFilter(tokenStream, 2 /* NorwegianLightStemmer.NYNORSK = 2 */); } else if ("persian".equalsIgnoreCase(language)) { return new PersianStemFilter(tokenStream); diff --git a/modules/analysis-common/src/test/java/org/opensearch/analysis/common/HighlighterWithAnalyzersTests.java b/modules/analysis-common/src/test/java/org/opensearch/analysis/common/HighlighterWithAnalyzersTests.java index e55c1c69b2e40..bd57b7e8c131a 100644 --- a/modules/analysis-common/src/test/java/org/opensearch/analysis/common/HighlighterWithAnalyzersTests.java +++ b/modules/analysis-common/src/test/java/org/opensearch/analysis/common/HighlighterWithAnalyzersTests.java @@ -310,7 +310,7 @@ public void testPhrasePrefix() throws IOException { ).highlighter(highlight().field("field1").order("score").preTags("").postTags("")); searchResponse = client().search(searchRequest("first_test_index").source(source)).actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); for (int i = 0; i < 2; i++) { assertHighlight( searchResponse, diff --git a/modules/analysis-common/src/yamlRestTest/resources/rest-api-spec/test/analysis-common/20_analyzers.yml b/modules/analysis-common/src/yamlRestTest/resources/rest-api-spec/test/analysis-common/20_analyzers.yml index dcec02729a44e..9cab92d26a435 100644 --- a/modules/analysis-common/src/yamlRestTest/resources/rest-api-spec/test/analysis-common/20_analyzers.yml +++ b/modules/analysis-common/src/yamlRestTest/resources/rest-api-spec/test/analysis-common/20_analyzers.yml @@ -948,7 +948,7 @@ text: absenţa analyzer: romanian - length: { tokens: 1 } - - match: { tokens.0.token: absenţ } + - match: { tokens.0.token: absenț } - do: indices.analyze: @@ -957,7 +957,7 @@ text: absenţa analyzer: my_analyzer - length: { tokens: 1 } - - match: { tokens.0.token: absenţ } + - match: { tokens.0.token: absenț } --- "russian": diff --git a/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/AbstractGeoAggregatorModulePluginTestCase.java b/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/AbstractGeoAggregatorModulePluginTestCase.java index 711744b944ce3..c05176c99d441 100644 --- a/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/AbstractGeoAggregatorModulePluginTestCase.java +++ b/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/AbstractGeoAggregatorModulePluginTestCase.java @@ -256,7 +256,7 @@ public void setupSuiteScopeCluster() throws Exception { .setSize(5000) .get(); assertSearchResponse(response); - long totalHits = response.getHits().getTotalHits().value; + long totalHits = response.getHits().getTotalHits().value(); XContentBuilder builder = XContentFactory.jsonBuilder(); response.toXContent(builder, ToXContent.EMPTY_PARAMS); logger.info("Full high_card_idx Response Content:\n{ {} }", builder.toString()); diff --git a/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/GeoBoundsITTestCase.java b/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/GeoBoundsITTestCase.java index 1c28df6bc4ea2..7a1bba5df37b1 100644 --- a/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/GeoBoundsITTestCase.java +++ b/modules/geo/src/internalClusterTest/java/org/opensearch/geo/search/aggregations/metrics/GeoBoundsITTestCase.java @@ -179,7 +179,7 @@ public void testEmptyAggregation() throws Exception { .addAggregation(geoBounds(aggName).field(SINGLE_VALUED_FIELD_NAME).wrapLongitude(false)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); GeoBounds geoBounds = searchResponse.getAggregations().get(aggName); assertThat(geoBounds, notNullValue()); assertThat(geoBounds.getName(), equalTo(aggName)); @@ -304,7 +304,7 @@ public void testEmptyAggregationOnGeoShapes() { .addAggregation(geoBounds(aggName).field(GEO_SHAPE_FIELD_NAME).wrapLongitude(false)) .get(); - MatcherAssert.assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + MatcherAssert.assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); final GeoBounds geoBounds = searchResponse.getAggregations().get(aggName); MatcherAssert.assertThat(geoBounds, notNullValue()); MatcherAssert.assertThat(geoBounds.getName(), equalTo(aggName)); diff --git a/modules/geo/src/test/java/org/opensearch/geo/search/aggregations/bucket/composite/GeoTileGridAggregationCompositeAggregatorTests.java b/modules/geo/src/test/java/org/opensearch/geo/search/aggregations/bucket/composite/GeoTileGridAggregationCompositeAggregatorTests.java index 0d25e4caa6e62..3f36168c79739 100644 --- a/modules/geo/src/test/java/org/opensearch/geo/search/aggregations/bucket/composite/GeoTileGridAggregationCompositeAggregatorTests.java +++ b/modules/geo/src/test/java/org/opensearch/geo/search/aggregations/bucket/composite/GeoTileGridAggregationCompositeAggregatorTests.java @@ -11,7 +11,7 @@ import org.apache.lucene.document.Document; import org.apache.lucene.document.LatLonPoint; import org.apache.lucene.document.SortedNumericDocValuesField; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.opensearch.common.geo.GeoPoint; import org.opensearch.geo.GeoModulePlugin; @@ -60,7 +60,7 @@ public void testUnmappedFieldWithGeopoint() throws Exception { // just unmapped = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder("name", Arrays.asList(new GeoTileGridValuesSourceBuilder("unmapped").field("unmapped"))), (result) -> assertEquals(0, result.getBuckets().size()) @@ -68,7 +68,7 @@ public void testUnmappedFieldWithGeopoint() throws Exception { // unmapped missing bucket = one result testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -84,7 +84,7 @@ public void testUnmappedFieldWithGeopoint() throws Exception { // field + unmapped, no missing bucket = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -98,7 +98,7 @@ public void testUnmappedFieldWithGeopoint() throws Exception { // field + unmapped with missing bucket = multiple results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -130,7 +130,7 @@ public void testWithGeoPoint() throws Exception { createDocument("geo_point", new GeoPoint(90.0, 0.0)) ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("geo_point")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("geo_point")), dataset, () -> { GeoTileGridValuesSourceBuilder geoTile = new GeoTileGridValuesSourceBuilder("geo_point").field("geo_point"); return new CompositeAggregationBuilder("name", Collections.singletonList(geoTile)); }, (result) -> { @@ -142,7 +142,7 @@ public void testWithGeoPoint() throws Exception { assertEquals(3L, result.getBuckets().get(1).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("geo_point")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("geo_point")), dataset, () -> { GeoTileGridValuesSourceBuilder geoTile = new GeoTileGridValuesSourceBuilder("geo_point").field("geo_point"); return new CompositeAggregationBuilder("name", Collections.singletonList(geoTile)).aggregateAfter( Collections.singletonMap("geo_point", "7/32/56") diff --git a/modules/lang-expression/licenses/lucene-expressions-10.1.0.jar.sha1 b/modules/lang-expression/licenses/lucene-expressions-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..54b10c7848627 --- /dev/null +++ b/modules/lang-expression/licenses/lucene-expressions-10.1.0.jar.sha1 @@ -0,0 +1 @@ +5b2af7e4940909058dde5c707be7ddb266590309 \ No newline at end of file diff --git a/modules/lang-expression/licenses/lucene-expressions-9.12.1.jar.sha1 b/modules/lang-expression/licenses/lucene-expressions-9.12.1.jar.sha1 deleted file mode 100644 index 9e0a5c2d7df21..0000000000000 --- a/modules/lang-expression/licenses/lucene-expressions-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -667ee99f31c8e42eac70b0adcf8deb4232935430 \ No newline at end of file diff --git a/modules/lang-expression/src/internalClusterTest/java/org/opensearch/script/expression/MoreExpressionIT.java b/modules/lang-expression/src/internalClusterTest/java/org/opensearch/script/expression/MoreExpressionIT.java index 8e15488900e5f..577725008c664 100644 --- a/modules/lang-expression/src/internalClusterTest/java/org/opensearch/script/expression/MoreExpressionIT.java +++ b/modules/lang-expression/src/internalClusterTest/java/org/opensearch/script/expression/MoreExpressionIT.java @@ -119,7 +119,7 @@ public void testBasic() throws Exception { ensureGreen("test"); client().prepareIndex("test").setId("1").setSource("foo", 4).setRefreshPolicy(IMMEDIATE).get(); SearchResponse rsp = buildRequest("doc['foo'] + 1").get(); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(5.0, rsp.getHits().getAt(0).field("foo").getValue(), 0.0D); } @@ -129,7 +129,7 @@ public void testFunction() throws Exception { client().prepareIndex("test").setId("1").setSource("foo", 4).setRefreshPolicy(IMMEDIATE).get(); SearchResponse rsp = buildRequest("doc['foo'] + abs(1)").get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(5.0, rsp.getHits().getAt(0).field("foo").getValue(), 0.0D); } @@ -138,7 +138,7 @@ public void testBasicUsingDotValue() throws Exception { ensureGreen("test"); client().prepareIndex("test").setId("1").setSource("foo", 4).setRefreshPolicy(IMMEDIATE).get(); SearchResponse rsp = buildRequest("doc['foo'].value + 1").get(); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(5.0, rsp.getHits().getAt(0).field("foo").getValue(), 0.0D); } @@ -160,7 +160,7 @@ public void testScore() throws Exception { SearchResponse rsp = req.get(); assertSearchResponse(rsp); SearchHits hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals("1", hits.getAt(0).getId()); assertEquals("3", hits.getAt(1).getId()); assertEquals("2", hits.getAt(2).getId()); @@ -183,22 +183,22 @@ public void testDateMethods() throws Exception { client().prepareIndex("test").setId("2").setSource("id", 2, "date0", "2013-12-25T11:56:45Z", "date1", "1983-10-13T23:15:00Z") ); SearchResponse rsp = buildRequest("doc['date0'].getSeconds() - doc['date0'].getMinutes()").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); SearchHits hits = rsp.getHits(); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(-11.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date0'].getHourOfDay() + doc['date1'].getDayOfMonth()").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(24.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date1'].getMonth() + 1").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(9.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(10.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date1'].getYear()").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(1985.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(1983.0, hits.getAt(1).field("foo").getValue(), 0.0D); @@ -213,22 +213,22 @@ public void testDateObjectMethods() throws Exception { client().prepareIndex("test").setId("2").setSource("id", 2, "date0", "2013-12-25T11:56:45Z", "date1", "1983-10-13T23:15:00Z") ); SearchResponse rsp = buildRequest("doc['date0'].date.secondOfMinute - doc['date0'].date.minuteOfHour").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); SearchHits hits = rsp.getHits(); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(-11.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date0'].date.getHourOfDay() + doc['date1'].date.dayOfMonth").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(24.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date1'].date.monthOfYear + 1").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(10.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(11.0, hits.getAt(1).field("foo").getValue(), 0.0D); rsp = buildRequest("doc['date1'].date.year").get(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); hits = rsp.getHits(); assertEquals(1985.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(1983.0, hits.getAt(1).field("foo").getValue(), 0.0D); @@ -266,7 +266,7 @@ public void testMultiValueMethods() throws Exception { SearchResponse rsp = buildRequest("doc['double0'].count() + doc['double1'].count()").get(); assertSearchResponse(rsp); SearchHits hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(2.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -274,7 +274,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].sum()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(7.5, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(6.0, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -282,7 +282,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].avg() + doc['double1'].avg()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(4.3, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(8.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(5.5, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -290,7 +290,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].median()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(1.5, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(1.25, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -298,7 +298,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].min()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(1.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(-1.5, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -306,7 +306,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].max()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -314,7 +314,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double0'].sum()/doc['double0'].count()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(2.5, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(1.5, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -323,7 +323,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double2'].count()").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(1.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(0.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(0.0, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -332,7 +332,7 @@ public void testMultiValueMethods() throws Exception { rsp = buildRequest("doc['double2'].empty ? 5.0 : 2.0").get(); assertSearchResponse(rsp); hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(2.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(5.0, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -370,7 +370,7 @@ public void testSparseField() throws Exception { SearchResponse rsp = buildRequest("doc['x'] + 1").get(); OpenSearchAssertions.assertSearchResponse(rsp); SearchHits hits = rsp.getHits(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); assertEquals(5.0, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(1.0, hits.getAt(1).field("foo").getValue(), 0.0D); } @@ -405,7 +405,7 @@ public void testParams() throws Exception { String script = "doc['x'] * a + b + ((c + doc['x']) > 5000000009 ? 1 : 0)"; SearchResponse rsp = buildRequest(script, "a", 2, "b", 3.5, "c", 5000000000L).get(); SearchHits hits = rsp.getHits(); - assertEquals(3, hits.getTotalHits().value); + assertEquals(3, hits.getTotalHits().value()); assertEquals(24.5, hits.getAt(0).field("foo").getValue(), 0.0D); assertEquals(9.5, hits.getAt(1).field("foo").getValue(), 0.0D); assertEquals(13.5, hits.getAt(2).field("foo").getValue(), 0.0D); @@ -527,7 +527,7 @@ public void testSpecialValueVariable() throws Exception { ); SearchResponse rsp = req.get(); - assertEquals(3, rsp.getHits().getTotalHits().value); + assertEquals(3, rsp.getHits().getTotalHits().value()); Stats stats = rsp.getAggregations().get("int_agg"); assertEquals(39.0, stats.getMax(), 0.0001); @@ -681,22 +681,22 @@ public void testGeo() throws Exception { // access .lat SearchResponse rsp = buildRequest("doc['location'].lat").get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(61.5240, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); // access .lon rsp = buildRequest("doc['location'].lon").get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(105.3188, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); // access .empty rsp = buildRequest("doc['location'].empty ? 1 : 0").get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(0, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); // call haversin rsp = buildRequest("haversin(38.9072, 77.0369, doc['location'].lat, doc['location'].lon)").get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(3170D, rsp.getHits().getAt(0).field("foo").getValue(), 50D); } @@ -718,14 +718,14 @@ public void testBoolean() throws Exception { // access .value SearchResponse rsp = buildRequest("doc['vip'].value").get(); assertSearchResponse(rsp); - assertEquals(3, rsp.getHits().getTotalHits().value); + assertEquals(3, rsp.getHits().getTotalHits().value()); assertEquals(1.0D, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); assertEquals(0.0D, rsp.getHits().getAt(1).field("foo").getValue(), 1.0D); assertEquals(0.0D, rsp.getHits().getAt(2).field("foo").getValue(), 1.0D); // access .empty rsp = buildRequest("doc['vip'].empty ? 1 : 0").get(); assertSearchResponse(rsp); - assertEquals(3, rsp.getHits().getTotalHits().value); + assertEquals(3, rsp.getHits().getTotalHits().value()); assertEquals(0.0D, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); assertEquals(0.0D, rsp.getHits().getAt(1).field("foo").getValue(), 1.0D); assertEquals(1.0D, rsp.getHits().getAt(2).field("foo").getValue(), 1.0D); @@ -733,7 +733,7 @@ public void testBoolean() throws Exception { // vip's have a 50% discount rsp = buildRequest("doc['vip'] ? doc['price']/2 : doc['price']").get(); assertSearchResponse(rsp); - assertEquals(3, rsp.getHits().getTotalHits().value); + assertEquals(3, rsp.getHits().getTotalHits().value()); assertEquals(0.5D, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D); assertEquals(2.0D, rsp.getHits().getAt(1).field("foo").getValue(), 1.0D); assertEquals(2.0D, rsp.getHits().getAt(2).field("foo").getValue(), 1.0D); @@ -752,7 +752,7 @@ public void testFilterScript() throws Exception { builder.setQuery(QueryBuilders.boolQuery().filter(QueryBuilders.scriptQuery(script))); SearchResponse rsp = builder.get(); assertSearchResponse(rsp); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); assertEquals(1.0D, rsp.getHits().getAt(0).field("foo").getValue(), 0.0D); } } diff --git a/modules/lang-expression/src/main/java/org/opensearch/script/expression/ExpressionScriptEngine.java b/modules/lang-expression/src/main/java/org/opensearch/script/expression/ExpressionScriptEngine.java index 0520177b72b62..b247ff2b3fd36 100644 --- a/modules/lang-expression/src/main/java/org/opensearch/script/expression/ExpressionScriptEngine.java +++ b/modules/lang-expression/src/main/java/org/opensearch/script/expression/ExpressionScriptEngine.java @@ -59,6 +59,8 @@ import org.opensearch.script.TermsSetQueryScript; import org.opensearch.search.lookup.SearchLookup; +import java.io.IOException; +import java.io.UncheckedIOException; import java.security.AccessControlContext; import java.security.AccessController; import java.security.PrivilegedAction; @@ -197,7 +199,7 @@ protected Class loadClass(String name, boolean resolve) throws ClassNotFoundE }; } // NOTE: validation is delayed to allow runtime vars, and we don't have access to per index stuff here - return JavascriptCompiler.compile(scriptSource, JavascriptCompiler.DEFAULT_FUNCTIONS, loader); + return JavascriptCompiler.compile(scriptSource, JavascriptCompiler.DEFAULT_FUNCTIONS); } catch (ParseException e) { throw convertToScriptException("compile error", scriptSource, scriptSource, e); } @@ -250,7 +252,12 @@ public Double execute() { placeholder.setValue(((Number) value).doubleValue()); } }); - return expr.evaluate(functionValuesArray); + + try { + return expr.evaluate(functionValuesArray); + } catch (final IOException ex) { + throw new UncheckedIOException(ex); + } } }; }; diff --git a/modules/lang-mustache/src/test/java/org/opensearch/script/mustache/SearchTemplateResponseTests.java b/modules/lang-mustache/src/test/java/org/opensearch/script/mustache/SearchTemplateResponseTests.java index c00751d8ef758..a83f893884188 100644 --- a/modules/lang-mustache/src/test/java/org/opensearch/script/mustache/SearchTemplateResponseTests.java +++ b/modules/lang-mustache/src/test/java/org/opensearch/script/mustache/SearchTemplateResponseTests.java @@ -141,7 +141,7 @@ protected void assertEqualInstances(SearchTemplateResponse expectedInstance, Sea SearchResponse expectedResponse = expectedInstance.getResponse(); SearchResponse newResponse = newInstance.getResponse(); - assertEquals(expectedResponse.getHits().getTotalHits().value, newResponse.getHits().getTotalHits().value); + assertEquals(expectedResponse.getHits().getTotalHits().value(), newResponse.getHits().getTotalHits().value()); assertEquals(expectedResponse.getHits().getMaxScore(), newResponse.getHits().getMaxScore(), 0.0001); } } diff --git a/modules/lang-painless/src/internalClusterTest/java/org/opensearch/painless/SimplePainlessIT.java b/modules/lang-painless/src/internalClusterTest/java/org/opensearch/painless/SimplePainlessIT.java index c9078fdeeea28..22e4c95c2f15e 100644 --- a/modules/lang-painless/src/internalClusterTest/java/org/opensearch/painless/SimplePainlessIT.java +++ b/modules/lang-painless/src/internalClusterTest/java/org/opensearch/painless/SimplePainlessIT.java @@ -196,7 +196,7 @@ public void testSimpleDerivedFieldsQuery() { ); SearchResponse response = client().search(searchRequest).actionGet(); assertSearchResponse(response); - assertEquals(2, Objects.requireNonNull(response.getHits().getTotalHits()).value); + assertEquals(2, Objects.requireNonNull(response.getHits().getTotalHits()).value()); } public void testSimpleDerivedFieldsAgg() { diff --git a/modules/lang-painless/src/test/java/org/opensearch/painless/ArrayTests.java b/modules/lang-painless/src/test/java/org/opensearch/painless/ArrayTests.java index 0b83a4c558ef6..9af70a62f37fb 100644 --- a/modules/lang-painless/src/test/java/org/opensearch/painless/ArrayTests.java +++ b/modules/lang-painless/src/test/java/org/opensearch/painless/ArrayTests.java @@ -32,7 +32,6 @@ package org.opensearch.painless; -import org.apache.lucene.util.Constants; import org.hamcrest.Matcher; import java.lang.invoke.MethodHandle; @@ -58,7 +57,6 @@ protected Matcher outOfBoundsExceptionMessageMatcher(int index, int size } public void testArrayLengthHelper() throws Throwable { - assertEquals(Constants.JRE_IS_MINIMUM_JAVA9, Def.JAVA9_ARRAY_LENGTH_MH_FACTORY != null); assertArrayLength(2, new int[2]); assertArrayLength(3, new long[3]); assertArrayLength(4, new byte[4]); diff --git a/modules/lang-painless/src/test/java/org/opensearch/painless/ScriptedMetricAggContextsTests.java b/modules/lang-painless/src/test/java/org/opensearch/painless/ScriptedMetricAggContextsTests.java index 72ab00c4f15cb..5064c82dcc392 100644 --- a/modules/lang-painless/src/test/java/org/opensearch/painless/ScriptedMetricAggContextsTests.java +++ b/modules/lang-painless/src/test/java/org/opensearch/painless/ScriptedMetricAggContextsTests.java @@ -110,11 +110,6 @@ public void testMapBasic() throws IOException { Map state = new HashMap<>(); Scorable scorer = new Scorable() { - @Override - public int docID() { - return 0; - } - @Override public float score() { return 0.5f; diff --git a/modules/lang-painless/src/test/java/org/opensearch/painless/SimilarityScriptTests.java b/modules/lang-painless/src/test/java/org/opensearch/painless/SimilarityScriptTests.java index 3b3f77d982450..22cee4fa41871 100644 --- a/modules/lang-painless/src/test/java/org/opensearch/painless/SimilarityScriptTests.java +++ b/modules/lang-painless/src/test/java/org/opensearch/painless/SimilarityScriptTests.java @@ -122,7 +122,7 @@ public void testBasics() throws IOException { 3.2f ); TopDocs topDocs = searcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals((float) (3.2 * 2 / 3), topDocs.scoreDocs[0].score, 0); w.close(); dir.close(); @@ -171,7 +171,7 @@ public void testWeightScript() throws IOException { 3.2f ); TopDocs topDocs = searcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals((float) (3.2 * 2 / 3), topDocs.scoreDocs[0].score, 0); w.close(); dir.close(); diff --git a/modules/lang-painless/src/test/java/org/opensearch/painless/StringTests.java b/modules/lang-painless/src/test/java/org/opensearch/painless/StringTests.java index b9586924d4fcd..49d4e717337fd 100644 --- a/modules/lang-painless/src/test/java/org/opensearch/painless/StringTests.java +++ b/modules/lang-painless/src/test/java/org/opensearch/painless/StringTests.java @@ -32,8 +32,6 @@ package org.opensearch.painless; -import org.apache.lucene.util.Constants; - import java.util.HashMap; import java.util.Locale; import java.util.Map; @@ -257,8 +255,7 @@ public void testBase64Augmentations() { assertEquals(rando, exec("params.rando.encodeBase64().decodeBase64()", singletonMap("rando", rando), true)); } - public void testJava9ConstantStringConcatBytecode() { - assumeTrue("Needs Java 9 to test indified String concat", Constants.JRE_IS_MINIMUM_JAVA9); + public void testConstantStringConcatBytecode() { assertNotNull(WriterConstants.INDY_STRING_CONCAT_BOOTSTRAP_HANDLE); assertBytecodeExists( "String s = \"cat\"; return s + true + 'abc' + null;", @@ -266,8 +263,7 @@ public void testJava9ConstantStringConcatBytecode() { ); } - public void testJava9StringConcatBytecode() { - assumeTrue("Needs Java 9 to test indified String concat", Constants.JRE_IS_MINIMUM_JAVA9); + public void testStringConcatBytecode() { assertNotNull(WriterConstants.INDY_STRING_CONCAT_BOOTSTRAP_HANDLE); assertBytecodeExists( "String s = \"cat\"; boolean t = true; Object u = null; return s + t + 'abc' + u;", @@ -284,8 +280,7 @@ public void testNullStringConcat() { assertEquals("" + null + 2, exec("null + '' + 2")); } - public void testJava9NullStringConcatBytecode() { - assumeTrue("Needs Java 9 to test indified String concat", Constants.JRE_IS_MINIMUM_JAVA9); + public void testNullStringConcatBytecode() { assertNotNull(WriterConstants.INDY_STRING_CONCAT_BOOTSTRAP_HANDLE); assertEquals("" + null + null, exec("'' + null + null")); } diff --git a/modules/mapper-extras/src/javaRestTest/java/org/opensearch/index/mapper/TokenCountFieldMapperIntegrationIT.java b/modules/mapper-extras/src/javaRestTest/java/org/opensearch/index/mapper/TokenCountFieldMapperIntegrationIT.java index bd0795f07139b..c885e0843f52d 100644 --- a/modules/mapper-extras/src/javaRestTest/java/org/opensearch/index/mapper/TokenCountFieldMapperIntegrationIT.java +++ b/modules/mapper-extras/src/javaRestTest/java/org/opensearch/index/mapper/TokenCountFieldMapperIntegrationIT.java @@ -211,7 +211,7 @@ private SearchRequestBuilder prepareSearch() { } private void assertSearchReturns(SearchResponse result, String... ids) { - assertThat(result.getHits().getTotalHits().value, equalTo((long) ids.length)); + assertThat(result.getHits().getTotalHits().value(), equalTo((long) ids.length)); assertThat(result.getHits().getHits().length, equalTo(ids.length)); List foundIds = new ArrayList<>(); for (SearchHit hit : result.getHits()) { diff --git a/modules/mapper-extras/src/test/java/org/opensearch/index/mapper/SearchAsYouTypeFieldMapperTests.java b/modules/mapper-extras/src/test/java/org/opensearch/index/mapper/SearchAsYouTypeFieldMapperTests.java index 7746cb714a019..0fee445e51f60 100644 --- a/modules/mapper-extras/src/test/java/org/opensearch/index/mapper/SearchAsYouTypeFieldMapperTests.java +++ b/modules/mapper-extras/src/test/java/org/opensearch/index/mapper/SearchAsYouTypeFieldMapperTests.java @@ -45,9 +45,9 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.ConstantScoreQuery; import org.apache.lucene.search.DisjunctionMaxQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.MultiPhraseQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.SynonymQuery; import org.apache.lucene.search.TermQuery; @@ -577,7 +577,7 @@ public void testNestedExistsQuery() throws IOException { QueryShardContext queryShardContext = createQueryShardContext(mapperService); Query actual = new QueryStringQueryBuilder("field:*").toQuery(queryShardContext); Query expected = new ConstantScoreQuery( - new BooleanQuery.Builder().add(new NormsFieldExistsQuery("field.nested_field"), BooleanClause.Occur.SHOULD).build() + new BooleanQuery.Builder().add(new FieldExistsQuery("field.nested_field"), BooleanClause.Occur.SHOULD).build() ); assertEquals(expected, actual); } diff --git a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/aggregations/ChildrenIT.java b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/aggregations/ChildrenIT.java index ab74463382aaa..a750e65d8dbda 100644 --- a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/aggregations/ChildrenIT.java +++ b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/aggregations/ChildrenIT.java @@ -149,7 +149,7 @@ public void testParentWithMultipleBuckets() throws Exception { logger.info("bucket={}", bucket.getKey()); Children childrenBucket = bucket.getAggregations().get("to_comment"); TopHits topHits = childrenBucket.getAggregations().get("top_comments"); - logger.info("total_hits={}", topHits.getHits().getTotalHits().value); + logger.info("total_hits={}", topHits.getHits().getTotalHits().value()); for (SearchHit searchHit : topHits.getHits()) { logger.info("hit= {} {}", searchHit.getSortValues()[0], searchHit.getId()); } @@ -163,7 +163,7 @@ public void testParentWithMultipleBuckets() throws Exception { assertThat(childrenBucket.getName(), equalTo("to_comment")); assertThat(childrenBucket.getDocCount(), equalTo(2L)); TopHits topHits = childrenBucket.getAggregations().get("top_comments"); - assertThat(topHits.getHits().getTotalHits().value, equalTo(2L)); + assertThat(topHits.getHits().getTotalHits().value(), equalTo(2L)); assertThat(topHits.getHits().getAt(0).getId(), equalTo("e")); assertThat(topHits.getHits().getAt(1).getId(), equalTo("f")); @@ -175,7 +175,7 @@ public void testParentWithMultipleBuckets() throws Exception { assertThat(childrenBucket.getName(), equalTo("to_comment")); assertThat(childrenBucket.getDocCount(), equalTo(1L)); topHits = childrenBucket.getAggregations().get("top_comments"); - assertThat(topHits.getHits().getTotalHits().value, equalTo(1L)); + assertThat(topHits.getHits().getTotalHits().value(), equalTo(1L)); assertThat(topHits.getHits().getAt(0).getId(), equalTo("f")); categoryBucket = categoryTerms.getBucketByKey("c"); @@ -186,7 +186,7 @@ public void testParentWithMultipleBuckets() throws Exception { assertThat(childrenBucket.getName(), equalTo("to_comment")); assertThat(childrenBucket.getDocCount(), equalTo(1L)); topHits = childrenBucket.getAggregations().get("top_comments"); - assertThat(topHits.getHits().getTotalHits().value, equalTo(1L)); + assertThat(topHits.getHits().getTotalHits().value(), equalTo(1L)); assertThat(topHits.getHits().getAt(0).getId(), equalTo("f")); } diff --git a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/ChildQuerySearchIT.java b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/ChildQuerySearchIT.java index 99527c3273c4b..1e2870a44662d 100644 --- a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/ChildQuerySearchIT.java +++ b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/ChildQuerySearchIT.java @@ -144,7 +144,7 @@ public void testMultiLevelChild() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); searchResponse = client().prepareSearch("test") @@ -152,7 +152,7 @@ public void testMultiLevelChild() throws Exception { .execute() .actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("c1")); searchResponse = client().prepareSearch("test") @@ -160,7 +160,7 @@ public void testMultiLevelChild() throws Exception { .execute() .actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("gc1")); searchResponse = client().prepareSearch("test") @@ -168,7 +168,7 @@ public void testMultiLevelChild() throws Exception { .execute() .actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("c1")); searchResponse = client().prepareSearch("test") @@ -176,7 +176,7 @@ public void testMultiLevelChild() throws Exception { .execute() .actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("gc1")); } @@ -193,7 +193,7 @@ public void test2744() throws IOException { .setQuery(hasChildQuery("test", matchQuery("foo", 1), ScoreMode.None)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); } @@ -215,7 +215,7 @@ public void testSimpleChildQuery() throws Exception { SearchResponse searchResponse; searchResponse = client().prepareSearch("test").setQuery(idsQuery().addIds("c1")).get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("c1")); assertThat(extractValue("join_field.name", searchResponse.getHits().getAt(0).getSourceAsMap()), equalTo("child")); assertThat(extractValue("join_field.parent", searchResponse.getHits().getAt(0).getSourceAsMap()), equalTo("p1")); @@ -225,7 +225,7 @@ public void testSimpleChildQuery() throws Exception { .setQuery(boolQuery().filter(termQuery("join_field#parent", "p1")).filter(termQuery("join_field", "child"))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), anyOf(equalTo("c1"), equalTo("c2"))); assertThat(extractValue("join_field.name", searchResponse.getHits().getAt(0).getSourceAsMap()), equalTo("child")); assertThat(extractValue("join_field.parent", searchResponse.getHits().getAt(0).getSourceAsMap()), equalTo("p1")); @@ -236,7 +236,7 @@ public void testSimpleChildQuery() throws Exception { // HAS CHILD searchResponse = client().prepareSearch("test").setQuery(randomHasChild("child", "c_field", "yellow")).get(); assertHitCount(searchResponse, 1L); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); searchResponse = client().prepareSearch("test").setQuery(randomHasChild("child", "c_field", "blue")).execute().actionGet(); @@ -338,8 +338,8 @@ public void testHasParentFilter() throws Exception { assertNoFailures(searchResponse); Set childIds = parentToChildrenEntry.getValue(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) childIds.size())); - for (int i = 0; i < searchResponse.getHits().getTotalHits().value; i++) { + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) childIds.size())); + for (int i = 0; i < searchResponse.getHits().getTotalHits().value(); i++) { assertThat(childIds.remove(searchResponse.getHits().getAt(i).getId()), is(true)); assertThat(searchResponse.getHits().getAt(i).getScore(), is(1.0f)); } @@ -372,19 +372,19 @@ public void testSimpleChildQueryWithFlush() throws Exception { .setQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.None)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); searchResponse = client().prepareSearch("test") .setQuery(hasChildQuery("child", termQuery("c_field", "blue"), ScoreMode.None)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p2")); searchResponse = client().prepareSearch("test").setQuery(hasChildQuery("child", termQuery("c_field", "red"), ScoreMode.None)).get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), anyOf(equalTo("p2"), equalTo("p1"))); assertThat(searchResponse.getHits().getAt(1).getId(), anyOf(equalTo("p2"), equalTo("p1"))); @@ -393,21 +393,21 @@ public void testSimpleChildQueryWithFlush() throws Exception { .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); searchResponse = client().prepareSearch("test") .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "blue"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p2")); searchResponse = client().prepareSearch("test") .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "red"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), anyOf(equalTo("p2"), equalTo("p1"))); assertThat(searchResponse.getHits().getAt(1).getId(), anyOf(equalTo("p2"), equalTo("p1"))); } @@ -449,7 +449,7 @@ public void testScopedFacet() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), anyOf(equalTo("p2"), equalTo("p1"))); assertThat(searchResponse.getHits().getAt(1).getId(), anyOf(equalTo("p2"), equalTo("p1"))); @@ -480,7 +480,7 @@ public void testDeletedParent() throws Exception { .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); assertThat(searchResponse.getHits().getAt(0).getSourceAsString(), containsString("\"p_value1\"")); @@ -493,7 +493,7 @@ public void testDeletedParent() throws Exception { .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); assertThat(searchResponse.getHits().getAt(0).getSourceAsString(), containsString("\"p_value1_updated\"")); } @@ -540,13 +540,13 @@ public void testHasChildAndHasParentFailWhenSomeSegmentsDontContainAnyParentOrCh .setQuery(boolQuery().must(matchAllQuery()).filter(hasChildQuery("child", matchAllQuery(), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery(boolQuery().must(matchAllQuery()).filter(hasParentQuery("parent", matchAllQuery(), false))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testCountApiUsage() throws Exception { @@ -672,7 +672,7 @@ public void testScoreForParentChildQueriesWithFunctionScore() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("1")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -691,7 +691,7 @@ public void testScoreForParentChildQueriesWithFunctionScore() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(4f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("2")); @@ -710,7 +710,7 @@ public void testScoreForParentChildQueriesWithFunctionScore() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(4f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("2")); @@ -731,7 +731,7 @@ public void testScoreForParentChildQueriesWithFunctionScore() throws Exception { .addSort(SortBuilders.scoreSort()) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(7L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(7L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("16")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(5f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("17")); @@ -757,7 +757,7 @@ public void testParentChildQueriesCanHandleNoRelevantTypesInIndex() throws Excep .setQuery(hasChildQuery("child", matchQuery("text", "value"), ScoreMode.None)) .get(); assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); client().prepareIndex("test") .setSource(jsonBuilder().startObject().field("text", "value").endObject()) @@ -766,19 +766,19 @@ public void testParentChildQueriesCanHandleNoRelevantTypesInIndex() throws Excep response = client().prepareSearch("test").setQuery(hasChildQuery("child", matchQuery("text", "value"), ScoreMode.None)).get(); assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = client().prepareSearch("test").setQuery(hasChildQuery("child", matchQuery("text", "value"), ScoreMode.Max)).get(); assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = client().prepareSearch("test").setQuery(hasParentQuery("parent", matchQuery("text", "value"), false)).get(); assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = client().prepareSearch("test").setQuery(hasParentQuery("parent", matchQuery("text", "value"), true)).get(); assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); } public void testHasChildAndHasParentFilter_withFilter() throws Exception { @@ -796,14 +796,14 @@ public void testHasChildAndHasParentFilter_withFilter() throws Exception { .setQuery(boolQuery().must(matchAllQuery()).filter(hasChildQuery("child", termQuery("c_field", 1), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("1")); searchResponse = client().prepareSearch("test") .setQuery(boolQuery().must(matchAllQuery()).filter(hasParentQuery("parent", termQuery("p_field", 1), false))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("2")); } @@ -825,7 +825,7 @@ public void testHasChildInnerHitsHighlighting() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("1")); SearchHit[] searchHits = searchResponse.getHits().getHits()[0].getInnerHits().get("child").getHits(); assertThat(searchHits.length, equalTo(1)); @@ -903,7 +903,7 @@ public void testSimpleQueryRewrite() throws Exception { .setSize(5) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(10L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(10L)); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("p000")); assertThat(searchResponse.getHits().getHits()[1].getId(), equalTo("p001")); assertThat(searchResponse.getHits().getHits()[2].getId(), equalTo("p002")); @@ -917,7 +917,7 @@ public void testSimpleQueryRewrite() throws Exception { .setSize(5) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(500L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(500L)); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("c000")); assertThat(searchResponse.getHits().getHits()[1].getId(), equalTo("c001")); assertThat(searchResponse.getHits().getHits()[2].getId(), equalTo("c002")); @@ -945,7 +945,7 @@ public void testReIndexingParentAndChildDocuments() throws Exception { .setQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.Total)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); assertThat(searchResponse.getHits().getAt(0).getSourceAsString(), containsString("\"p_value1\"")); @@ -953,7 +953,7 @@ public void testReIndexingParentAndChildDocuments() throws Exception { .setQuery(boolQuery().must(matchQuery("c_field", "x")).must(hasParentQuery("parent", termQuery("p_field", "p_value2"), true))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("c3")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("c4")); @@ -970,7 +970,7 @@ public void testReIndexingParentAndChildDocuments() throws Exception { .setQuery(hasChildQuery("child", termQuery("c_field", "yellow"), ScoreMode.Total)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p1")); assertThat(searchResponse.getHits().getAt(0).getSourceAsString(), containsString("\"p_value1\"")); @@ -978,7 +978,7 @@ public void testReIndexingParentAndChildDocuments() throws Exception { .setQuery(boolQuery().must(matchQuery("c_field", "x")).must(hasParentQuery("parent", termQuery("p_field", "p_value2"), true))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getId(), Matchers.anyOf(equalTo("c3"), equalTo("c4"))); assertThat(searchResponse.getHits().getAt(1).getId(), Matchers.anyOf(equalTo("c3"), equalTo("c4"))); } @@ -1002,7 +1002,7 @@ public void testHasChildQueryWithMinimumScore() throws Exception { .setMinScore(3) // Score needs to be 3 or above! .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("p2")); assertThat(searchResponse.getHits().getAt(0).getScore(), equalTo(3.0f)); } @@ -1084,7 +1084,7 @@ public void testHasChildNotBeingCached() throws IOException { .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "blue"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); createIndexRequest("test", "child", "c2", "p2", "c_field", "blue").get(); client().admin().indices().prepareRefresh("test").get(); @@ -1093,7 +1093,7 @@ public void testHasChildNotBeingCached() throws IOException { .setQuery(constantScoreQuery(hasChildQuery("child", termQuery("c_field", "blue"), ScoreMode.None))) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); } private QueryBuilder randomHasChild(String type, String field, String value) { @@ -1242,7 +1242,7 @@ public void testHasChildQueryWithNestedInnerObjects() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -1251,7 +1251,7 @@ public void testHasChildQueryWithNestedInnerObjects() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); } public void testNamedFilters() throws Exception { @@ -1367,7 +1367,7 @@ public void testParentChildCaching() throws Exception { ) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); } createIndexRequest("test", "child", "c3", "p2", "c_field", "blue").get(); @@ -1380,7 +1380,7 @@ public void testParentChildCaching() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testParentChildQueriesViaScrollApi() throws Exception { @@ -1409,10 +1409,10 @@ public void testParentChildQueriesViaScrollApi() throws Exception { .actionGet(); assertNoFailures(scrollResponse); - assertThat(scrollResponse.getHits().getTotalHits().value, equalTo(10L)); + assertThat(scrollResponse.getHits().getTotalHits().value(), equalTo(10L)); int scannedDocs = 0; do { - assertThat(scrollResponse.getHits().getTotalHits().value, equalTo(10L)); + assertThat(scrollResponse.getHits().getTotalHits().value(), equalTo(10L)); scannedDocs += scrollResponse.getHits().getHits().length; scrollResponse = client().prepareSearchScroll(scrollResponse.getScrollId()).setScroll(TimeValue.timeValueSeconds(30)).get(); } while (scrollResponse.getHits().getHits().length > 0); @@ -1474,7 +1474,7 @@ public void testMinMaxChildren() throws Exception { // Score mode = NONE response = minMaxQuery(ScoreMode.None, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("2")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1484,7 +1484,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("2")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1494,7 +1494,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 2, null); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("4")); @@ -1502,17 +1502,17 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 3, null); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); response = minMaxQuery(ScoreMode.None, 4, null); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = minMaxQuery(ScoreMode.None, 1, 4); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("2")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1522,7 +1522,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 1, 3); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("2")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1532,7 +1532,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 1, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("2")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1540,7 +1540,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.None, 2, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1f)); @@ -1550,7 +1550,7 @@ public void testMinMaxChildren() throws Exception { // Score mode = SUM response = minMaxQuery(ScoreMode.Total, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1560,7 +1560,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1570,7 +1570,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 2, null); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1578,17 +1578,17 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 3, null); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); response = minMaxQuery(ScoreMode.Total, 4, null); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = minMaxQuery(ScoreMode.Total, 1, 4); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1598,7 +1598,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 1, 3); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(6f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1608,7 +1608,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 1, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("2")); @@ -1616,7 +1616,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Total, 2, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); @@ -1626,7 +1626,7 @@ public void testMinMaxChildren() throws Exception { // Score mode = MAX response = minMaxQuery(ScoreMode.Max, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1636,7 +1636,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1646,7 +1646,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 2, null); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1654,17 +1654,17 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 3, null); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); response = minMaxQuery(ScoreMode.Max, 4, null); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = minMaxQuery(ScoreMode.Max, 1, 4); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1674,7 +1674,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 1, 3); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(3f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1684,7 +1684,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 1, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("2")); @@ -1692,7 +1692,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Max, 2, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); @@ -1702,7 +1702,7 @@ public void testMinMaxChildren() throws Exception { // Score mode = AVG response = minMaxQuery(ScoreMode.Avg, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1712,7 +1712,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 1, null); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1722,7 +1722,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 2, null); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1730,17 +1730,17 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 3, null); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); response = minMaxQuery(ScoreMode.Avg, 4, null); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); response = minMaxQuery(ScoreMode.Avg, 1, 4); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1750,7 +1750,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 1, 3); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("4")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(2f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("3")); @@ -1760,7 +1760,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 1, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1.5f)); assertThat(response.getHits().getHits()[1].getId(), equalTo("2")); @@ -1768,7 +1768,7 @@ public void testMinMaxChildren() throws Exception { response = minMaxQuery(ScoreMode.Avg, 2, 2); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getHits()[0].getId(), equalTo("3")); assertThat(response.getHits().getHits()[0].getScore(), equalTo(1.5f)); diff --git a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/InnerHitsIT.java b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/InnerHitsIT.java index 4b5470d17c100..03573dc7f5344 100644 --- a/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/InnerHitsIT.java +++ b/modules/parent-join/src/internalClusterTest/java/org/opensearch/join/query/InnerHitsIT.java @@ -162,7 +162,7 @@ public void testSimpleParentChild() throws Exception { assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); SearchHits innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(2L)); + assertThat(innerHits.getTotalHits().value(), equalTo(2L)); assertThat(innerHits.getAt(0).getId(), equalTo("c1")); assertThat(innerHits.getAt(1).getId(), equalTo("c2")); @@ -181,7 +181,7 @@ public void testSimpleParentChild() throws Exception { assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(3L)); + assertThat(innerHits.getTotalHits().value(), equalTo(3L)); assertThat(innerHits.getAt(0).getId(), equalTo("c4")); assertThat(innerHits.getAt(1).getId(), equalTo("c5")); @@ -311,7 +311,7 @@ public void testRandomParentChild() throws Exception { assertThat(searchHit.getShard(), notNullValue()); SearchHits inner = searchHit.getInnerHits().get("a"); - assertThat(inner.getTotalHits().value, equalTo((long) child1InnerObjects[parent])); + assertThat(inner.getTotalHits().value(), equalTo((long) child1InnerObjects[parent])); for (int child = 0; child < child1InnerObjects[parent] && child < size; child++) { SearchHit innerHit = inner.getAt(child); String childId = String.format(Locale.ENGLISH, "c1_%04d", offset1 + child); @@ -321,7 +321,7 @@ public void testRandomParentChild() throws Exception { offset1 += child1InnerObjects[parent]; inner = searchHit.getInnerHits().get("b"); - assertThat(inner.getTotalHits().value, equalTo((long) child2InnerObjects[parent])); + assertThat(inner.getTotalHits().value(), equalTo((long) child2InnerObjects[parent])); for (int child = 0; child < child2InnerObjects[parent] && child < size; child++) { SearchHit innerHit = inner.getAt(child); String childId = String.format(Locale.ENGLISH, "c2_%04d", offset2 + child); @@ -378,12 +378,12 @@ public void testInnerHitsOnHasParent() throws Exception { SearchHit searchHit = response.getHits().getAt(0); assertThat(searchHit.getId(), equalTo("3")); - assertThat(searchHit.getInnerHits().get("question").getTotalHits().value, equalTo(1L)); + assertThat(searchHit.getInnerHits().get("question").getTotalHits().value(), equalTo(1L)); assertThat(searchHit.getInnerHits().get("question").getAt(0).getId(), equalTo("1")); searchHit = response.getHits().getAt(1); assertThat(searchHit.getId(), equalTo("4")); - assertThat(searchHit.getInnerHits().get("question").getTotalHits().value, equalTo(1L)); + assertThat(searchHit.getInnerHits().get("question").getTotalHits().value(), equalTo(1L)); assertThat(searchHit.getInnerHits().get("question").getAt(0).getId(), equalTo("2")); } @@ -425,11 +425,11 @@ public void testParentChildMultipleLayers() throws Exception { assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); SearchHits innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getAt(0).getId(), equalTo("3")); innerHits = innerHits.getAt(0).getInnerHits().get("remark"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getAt(0).getId(), equalTo("5")); response = client().prepareSearch("articles") @@ -448,11 +448,11 @@ public void testParentChildMultipleLayers() throws Exception { assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getAt(0).getId(), equalTo("4")); innerHits = innerHits.getAt(0).getInnerHits().get("remark"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getAt(0).getId(), equalTo("6")); } @@ -512,34 +512,34 @@ public void testRoyals() throws Exception { assertThat(response.getHits().getAt(0).getId(), equalTo("duke")); SearchHits innerHits = response.getHits().getAt(0).getInnerHits().get("earls"); - assertThat(innerHits.getTotalHits().value, equalTo(4L)); + assertThat(innerHits.getTotalHits().value(), equalTo(4L)); assertThat(innerHits.getAt(0).getId(), equalTo("earl1")); assertThat(innerHits.getAt(1).getId(), equalTo("earl2")); assertThat(innerHits.getAt(2).getId(), equalTo("earl3")); assertThat(innerHits.getAt(3).getId(), equalTo("earl4")); SearchHits innerInnerHits = innerHits.getAt(0).getInnerHits().get("barons"); - assertThat(innerInnerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerInnerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerInnerHits.getAt(0).getId(), equalTo("baron1")); innerInnerHits = innerHits.getAt(1).getInnerHits().get("barons"); - assertThat(innerInnerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerInnerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerInnerHits.getAt(0).getId(), equalTo("baron2")); innerInnerHits = innerHits.getAt(2).getInnerHits().get("barons"); - assertThat(innerInnerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerInnerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerInnerHits.getAt(0).getId(), equalTo("baron3")); innerInnerHits = innerHits.getAt(3).getInnerHits().get("barons"); - assertThat(innerInnerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerInnerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerInnerHits.getAt(0).getId(), equalTo("baron4")); innerHits = response.getHits().getAt(0).getInnerHits().get("princes"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getAt(0).getId(), equalTo("prince")); innerInnerHits = innerHits.getAt(0).getInnerHits().get("kings"); - assertThat(innerInnerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerInnerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerInnerHits.getAt(0).getId(), equalTo("king")); } @@ -561,12 +561,12 @@ public void testMatchesQueriesParentChildInnerHits() throws Exception { .get(); assertHitCount(response, 2); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); - assertThat(response.getHits().getAt(0).getInnerHits().get("child").getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("child").getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getInnerHits().get("child").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(response.getHits().getAt(0).getInnerHits().get("child").getAt(0).getMatchedQueries()[0], equalTo("_name1")); assertThat(response.getHits().getAt(1).getId(), equalTo("2")); - assertThat(response.getHits().getAt(1).getInnerHits().get("child").getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getAt(1).getInnerHits().get("child").getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(1).getInnerHits().get("child").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(response.getHits().getAt(1).getInnerHits().get("child").getAt(0).getMatchedQueries()[0], equalTo("_name1")); @@ -576,7 +576,7 @@ public void testMatchesQueriesParentChildInnerHits() throws Exception { response = client().prepareSearch("index").setQuery(query).addSort("id", SortOrder.ASC).get(); assertHitCount(response, 1); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); - assertThat(response.getHits().getAt(0).getInnerHits().get("child").getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("child").getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getInnerHits().get("child").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(response.getHits().getAt(0).getInnerHits().get("child").getAt(0).getMatchedQueries()[0], equalTo("_name2")); } diff --git a/modules/parent-join/src/main/java/org/opensearch/join/aggregations/ParentJoinAggregator.java b/modules/parent-join/src/main/java/org/opensearch/join/aggregations/ParentJoinAggregator.java index 4e1016a596874..f41278100d35d 100644 --- a/modules/parent-join/src/main/java/org/opensearch/join/aggregations/ParentJoinAggregator.java +++ b/modules/parent-join/src/main/java/org/opensearch/join/aggregations/ParentJoinAggregator.java @@ -120,7 +120,7 @@ public final LeafBucketCollector getLeafCollector(LeafReaderContext ctx, final L public void collect(int docId, long owningBucketOrd) throws IOException { if (parentDocs.get(docId) && globalOrdinals.advanceExact(docId)) { int globalOrdinal = (int) globalOrdinals.nextOrd(); - assert globalOrdinal != -1 && globalOrdinals.nextOrd() == SortedSetDocValues.NO_MORE_ORDS; + assert globalOrdinal != -1 && globalOrdinals.docValueCount() == 1; collectionStrategy.add(owningBucketOrd, globalOrdinal); } } @@ -151,15 +151,10 @@ protected void beforeBuildingBuckets(long[] ordsToCollect) throws IOException { public float score() { return 1f; } - - @Override - public int docID() { - return childDocsIter.docID(); - } }); final Bits liveDocs = ctx.reader().getLiveDocs(); - for (int docId = childDocsIter.nextDoc(); docId != DocIdSetIterator.NO_MORE_DOCS; docId = childDocsIter.nextDoc()) { + for (int docId = childDocsIter.nextDoc(); docId != SortedSetDocValues.NO_MORE_DOCS; docId = childDocsIter.nextDoc()) { if (liveDocs != null && liveDocs.get(docId) == false) { continue; } @@ -167,7 +162,7 @@ public int docID() { continue; } int globalOrdinal = (int) globalOrdinals.nextOrd(); - assert globalOrdinal != -1 && globalOrdinals.nextOrd() == SortedSetDocValues.NO_MORE_ORDS; + assert globalOrdinal != -1 && globalOrdinals.docValueCount() == 1; /* * Check if we contain every ordinal. It's almost certainly be * faster to replay all the matching ordinals and filter them down diff --git a/modules/parent-join/src/main/java/org/opensearch/join/query/ParentChildInnerHitContextBuilder.java b/modules/parent-join/src/main/java/org/opensearch/join/query/ParentChildInnerHitContextBuilder.java index 61114cba3fa08..e134a516b32d0 100644 --- a/modules/parent-join/src/main/java/org/opensearch/join/query/ParentChildInnerHitContextBuilder.java +++ b/modules/parent-join/src/main/java/org/opensearch/join/query/ParentChildInnerHitContextBuilder.java @@ -41,8 +41,8 @@ import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopDocsCollector; -import org.apache.lucene.search.TopFieldCollector; -import org.apache.lucene.search.TopScoreDocCollector; +import org.apache.lucene.search.TopFieldCollectorManager; +import org.apache.lucene.search.TopScoreDocCollectorManager; import org.apache.lucene.search.TotalHitCountCollector; import org.apache.lucene.search.TotalHits; import org.apache.lucene.search.Weight; @@ -170,12 +170,12 @@ public TopDocsAndMaxScore topDocs(SearchHit hit) throws IOException { TopDocsCollector topDocsCollector; MaxScoreCollector maxScoreCollector = null; if (sort() != null) { - topDocsCollector = TopFieldCollector.create(sort().sort, topN, Integer.MAX_VALUE); + topDocsCollector = new TopFieldCollectorManager(sort().sort, topN, null, Integer.MAX_VALUE, false).newCollector(); if (trackScores()) { maxScoreCollector = new MaxScoreCollector(); } } else { - topDocsCollector = TopScoreDocCollector.create(topN, Integer.MAX_VALUE); + topDocsCollector = new TopScoreDocCollectorManager(topN, null, Integer.MAX_VALUE, false).newCollector(); maxScoreCollector = new MaxScoreCollector(); } for (LeafReaderContext ctx : context.searcher().getIndexReader().leaves()) { diff --git a/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ChildrenToParentAggregatorTests.java b/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ChildrenToParentAggregatorTests.java index cdc2764ae1ac9..1d729726b2602 100644 --- a/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ChildrenToParentAggregatorTests.java +++ b/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ChildrenToParentAggregatorTests.java @@ -142,18 +142,22 @@ public void testParentChild() throws IOException { // verify for each children for (String parent : expectedParentChildRelations.keySet()) { - testCase(new TermInSetQuery(IdFieldMapper.NAME, Uid.encodeId("child0_" + parent)), indexSearcher, aggregation -> { - assertEquals( - "Expected one result for min-aggregation for parent: " + parent + ", but had aggregation-results: " + aggregation, - 1, - aggregation.getDocCount() - ); - assertEquals( - expectedParentChildRelations.get(parent).v2(), - ((InternalMin) aggregation.getAggregations().get("in_parent")).getValue(), - Double.MIN_VALUE - ); - }); + testCase( + new TermInSetQuery(IdFieldMapper.NAME, Collections.singleton(Uid.encodeId("child0_" + parent))), + indexSearcher, + aggregation -> { + assertEquals( + "Expected one result for min-aggregation for parent: " + parent + ", but had aggregation-results: " + aggregation, + 1, + aggregation.getDocCount() + ); + assertEquals( + expectedParentChildRelations.get(parent).v2(), + ((InternalMin) aggregation.getAggregations().get("in_parent")).getValue(), + Double.MIN_VALUE + ); + } + ); } indexReader.close(); diff --git a/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ParentToChildrenAggregatorTests.java b/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ParentToChildrenAggregatorTests.java index 8a0857d287717..f31389d3fae5b 100644 --- a/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ParentToChildrenAggregatorTests.java +++ b/modules/parent-join/src/test/java/org/opensearch/join/aggregations/ParentToChildrenAggregatorTests.java @@ -99,7 +99,7 @@ public void testNoDocs() throws IOException { IndexReader indexReader = DirectoryReader.open(directory); testCase(new MatchAllDocsQuery(), newSearcher(indexReader, false, true), parentToChild -> { - assertEquals(0, parentToChild.getDocCount()); + assertEquals(0L, parentToChild.getDocCount()); assertEquals( Double.POSITIVE_INFINITY, ((InternalMin) parentToChild.getAggregations().get("in_child")).getValue(), @@ -134,7 +134,7 @@ public void testParentChild() throws IOException { }); for (String parent : expectedParentChildRelations.keySet()) { - testCase(new TermInSetQuery(IdFieldMapper.NAME, Uid.encodeId(parent)), indexSearcher, child -> { + testCase(new TermInSetQuery(IdFieldMapper.NAME, Collections.singleton(Uid.encodeId(parent))), indexSearcher, child -> { assertEquals((long) expectedParentChildRelations.get(parent).v1(), child.getDocCount()); assertEquals( expectedParentChildRelations.get(parent).v2(), diff --git a/modules/parent-join/src/test/java/org/opensearch/join/query/HasChildQueryBuilderTests.java b/modules/parent-join/src/test/java/org/opensearch/join/query/HasChildQueryBuilderTests.java index 96220c247d909..e265180aa0e3a 100644 --- a/modules/parent-join/src/test/java/org/opensearch/join/query/HasChildQueryBuilderTests.java +++ b/modules/parent-join/src/test/java/org/opensearch/join/query/HasChildQueryBuilderTests.java @@ -300,15 +300,15 @@ static void assertLateParsingQuery(Query query, String type, String id) throws I BooleanQuery booleanQuery = (BooleanQuery) lateParsingQuery.getInnerQuery(); assertThat(booleanQuery.clauses().size(), equalTo(2)); // check the inner ids query, we have to call rewrite to get to check the type it's executed against - assertThat(booleanQuery.clauses().get(0).getOccur(), equalTo(BooleanClause.Occur.MUST)); - assertThat(booleanQuery.clauses().get(0).getQuery(), instanceOf(TermInSetQuery.class)); - TermInSetQuery termsQuery = (TermInSetQuery) booleanQuery.clauses().get(0).getQuery(); + assertThat(booleanQuery.clauses().get(0).occur(), equalTo(BooleanClause.Occur.MUST)); + assertThat(booleanQuery.clauses().get(0).query(), instanceOf(TermInSetQuery.class)); + TermInSetQuery termsQuery = (TermInSetQuery) booleanQuery.clauses().get(0).query(); // The query is of type MultiTermQueryConstantScoreBlendedWrapper and is sealed inside Apache Lucene, // no access to inner queries without using the reflection, falling back to stringified query comparison assertThat(termsQuery.toString(), equalTo("_id:([ff 69 64])")); // check the type filter - assertThat(booleanQuery.clauses().get(1).getOccur(), equalTo(BooleanClause.Occur.FILTER)); - assertEquals(new TermQuery(new Term("join_field", type)), booleanQuery.clauses().get(1).getQuery()); + assertThat(booleanQuery.clauses().get(1).occur(), equalTo(BooleanClause.Occur.FILTER)); + assertEquals(new TermQuery(new Term("join_field", type)), booleanQuery.clauses().get(1).query()); } @Override diff --git a/modules/percolator/src/internalClusterTest/java/org/opensearch/percolator/PercolatorQuerySearchIT.java b/modules/percolator/src/internalClusterTest/java/org/opensearch/percolator/PercolatorQuerySearchIT.java index 01436404e8a85..d03173b6b37fe 100644 --- a/modules/percolator/src/internalClusterTest/java/org/opensearch/percolator/PercolatorQuerySearchIT.java +++ b/modules/percolator/src/internalClusterTest/java/org/opensearch/percolator/PercolatorQuerySearchIT.java @@ -1332,7 +1332,7 @@ public void testWrappedWithConstantScore() throws Exception { ) ) .get(); - assertEquals(1, response.getHits().getTotalHits().value); + assertEquals(1, response.getHits().getTotalHits().value()); response = client().prepareSearch("test") .setQuery( @@ -1344,7 +1344,7 @@ public void testWrappedWithConstantScore() throws Exception { ) .addSort("_doc", SortOrder.ASC) .get(); - assertEquals(1, response.getHits().getTotalHits().value); + assertEquals(1, response.getHits().getTotalHits().value()); response = client().prepareSearch("test") .setQuery( @@ -1357,7 +1357,7 @@ public void testWrappedWithConstantScore() throws Exception { ) ) .get(); - assertEquals(1, response.getHits().getTotalHits().value); + assertEquals(1, response.getHits().getTotalHits().value()); } } diff --git a/modules/percolator/src/main/java/org/opensearch/percolator/PercolateQuery.java b/modules/percolator/src/main/java/org/opensearch/percolator/PercolateQuery.java index 99907b48d4fdd..ebc55def01750 100644 --- a/modules/percolator/src/main/java/org/opensearch/percolator/PercolateQuery.java +++ b/modules/percolator/src/main/java/org/opensearch/percolator/PercolateQuery.java @@ -134,7 +134,7 @@ public Explanation explain(LeafReaderContext leafReaderContext, int docId) throw } @Override - public Scorer scorer(LeafReaderContext leafReaderContext) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext leafReaderContext) throws IOException { final Scorer approximation = candidateMatchesWeight.scorer(leafReaderContext); if (approximation == null) { return null; @@ -142,7 +142,7 @@ public Scorer scorer(LeafReaderContext leafReaderContext) throws IOException { final CheckedFunction percolatorQueries = queryStore.getQueries(leafReaderContext); if (scoreMode.needsScores()) { - return new BaseScorer(this, approximation) { + return new DefaultScorerSupplier(new BaseScorer(this, approximation) { float score; @@ -171,11 +171,11 @@ boolean matchDocId(int docId) throws IOException { public float score() throws IOException { return score; } - }; + }); } else { ScorerSupplier verifiedDocsScorer = verifiedMatchesWeight.scorerSupplier(leafReaderContext); Bits verifiedDocsBits = Lucene.asSequentialAccessBits(leafReaderContext.reader().maxDoc(), verifiedDocsScorer); - return new BaseScorer(this, approximation) { + return new DefaultScorerSupplier(new BaseScorer(this, approximation) { @Override public float score() throws IOException { @@ -200,7 +200,7 @@ boolean matchDocId(int docId) throws IOException { } return Lucene.exists(percolatorIndexSearcher, query); } - }; + }); } } @@ -290,7 +290,7 @@ abstract static class BaseScorer extends Scorer { final Scorer approximation; BaseScorer(Weight weight, Scorer approximation) { - super(weight); + super(); this.approximation = approximation; } diff --git a/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorFieldMapper.java b/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorFieldMapper.java index e30ce218ed5ff..8aa8fd19615c9 100644 --- a/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorFieldMapper.java +++ b/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorFieldMapper.java @@ -289,7 +289,7 @@ Tuple createCandidateQuery(IndexReader indexReader, Versi List extractedTerms = t.v1(); Map> encodedPointValuesByField = t.v2(); // `1 + ` is needed to take into account the EXTRACTION_FAILED should clause - boolean canUseMinimumShouldMatchField = 1 + extractedTerms.size() + encodedPointValuesByField.size() <= BooleanQuery + boolean canUseMinimumShouldMatchField = 1 + extractedTerms.size() + encodedPointValuesByField.size() <= IndexSearcher .getMaxClauseCount(); List subQueries = new ArrayList<>(); diff --git a/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorMatchedSlotSubFetchPhase.java b/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorMatchedSlotSubFetchPhase.java index 0884d534849da..7ae0d77150dcb 100644 --- a/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorMatchedSlotSubFetchPhase.java +++ b/modules/percolator/src/main/java/org/opensearch/percolator/PercolatorMatchedSlotSubFetchPhase.java @@ -103,7 +103,7 @@ public void process(HitContext hitContext) throws IOException { IndexSearcher percolatorIndexSearcher = pc.percolateQuery.getPercolatorIndexSearcher(); int memoryIndexMaxDoc = percolatorIndexSearcher.getIndexReader().maxDoc(); TopDocs topDocs = percolatorIndexSearcher.search(query, memoryIndexMaxDoc, new Sort(SortField.FIELD_DOC)); - if (topDocs.totalHits.value == 0) { + if (topDocs.totalHits.value() == 0) { // This hit didn't match with a percolate query, // likely to happen when percolating multiple documents continue; diff --git a/modules/percolator/src/main/java/org/opensearch/percolator/QueryAnalyzer.java b/modules/percolator/src/main/java/org/opensearch/percolator/QueryAnalyzer.java index 96c022f0145a8..84349ed6c4e91 100644 --- a/modules/percolator/src/main/java/org/opensearch/percolator/QueryAnalyzer.java +++ b/modules/percolator/src/main/java/org/opensearch/percolator/QueryAnalyzer.java @@ -32,7 +32,6 @@ package org.opensearch.percolator; import org.apache.lucene.document.BinaryRange; -import org.apache.lucene.index.PrefixCodedTerms; import org.apache.lucene.index.Term; import org.apache.lucene.queries.BlendedTermQuery; import org.apache.lucene.queries.spans.SpanOrQuery; @@ -51,12 +50,15 @@ import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.BytesRefIterator; import org.apache.lucene.util.NumericUtils; import org.apache.lucene.util.automaton.ByteRunAutomaton; import org.opensearch.Version; import org.opensearch.common.lucene.search.function.FunctionScoreQuery; import org.opensearch.index.query.DateRangeIncludingNowQuery; +import java.io.IOException; +import java.io.UncheckedIOException; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -197,7 +199,7 @@ public QueryVisitor getSubVisitor(Occur occur, Query parent) { if (parent instanceof BooleanQuery) { BooleanQuery bq = (BooleanQuery) parent; if (bq.getMinimumNumberShouldMatch() == 0 - && bq.clauses().stream().anyMatch(c -> c.getOccur() == Occur.MUST || c.getOccur() == Occur.FILTER)) { + && bq.clauses().stream().anyMatch(c -> c.occur() == Occur.MUST || c.occur() == Occur.FILTER)) { return QueryVisitor.EMPTY_VISITOR; } minimumShouldMatch = bq.getMinimumNumberShouldMatch(); @@ -233,14 +235,18 @@ public void consumeTerms(Query query, Term... terms) { @Override public void consumeTermsMatching(Query query, String field, Supplier automaton) { if (query instanceof TermInSetQuery) { - TermInSetQuery q = (TermInSetQuery) query; - PrefixCodedTerms.TermIterator ti = q.getTermData().iterator(); - BytesRef term; - Set qe = new HashSet<>(); - while ((term = ti.next()) != null) { - qe.add(new QueryExtraction(new Term(field, term))); + try { + TermInSetQuery q = (TermInSetQuery) query; + BytesRefIterator ti = q.getBytesRefIterator(); + BytesRef term; + Set qe = new HashSet<>(); + while ((term = ti.next()) != null) { + qe.add(new QueryExtraction(new Term(field, term))); + } + this.terms.add(new Result(true, qe, 1)); + } catch (final IOException ex) { + throw new UncheckedIOException(ex); } - this.terms.add(new Result(true, qe, 1)); } else { super.consumeTermsMatching(query, field, automaton); } diff --git a/modules/percolator/src/test/java/org/opensearch/percolator/CandidateQueryTests.java b/modules/percolator/src/test/java/org/opensearch/percolator/CandidateQueryTests.java index 143b0942deb75..8e0aca471ac68 100644 --- a/modules/percolator/src/test/java/org/opensearch/percolator/CandidateQueryTests.java +++ b/modules/percolator/src/test/java/org/opensearch/percolator/CandidateQueryTests.java @@ -81,6 +81,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Sort; import org.apache.lucene.search.SortField; import org.apache.lucene.search.TermInSetQuery; @@ -267,15 +268,13 @@ public void testDuel() throws Exception { queryFunctions.add( () -> new TermInSetQuery( field1, - new BytesRef(randomFrom(stringContent.get(field1))), - new BytesRef(randomFrom(stringContent.get(field1))) + List.of(new BytesRef(randomFrom(stringContent.get(field1))), new BytesRef(randomFrom(stringContent.get(field1)))) ) ); queryFunctions.add( () -> new TermInSetQuery( field2, - new BytesRef(randomFrom(stringContent.get(field1))), - new BytesRef(randomFrom(stringContent.get(field1))) + List.of(new BytesRef(randomFrom(stringContent.get(field1))), new BytesRef(randomFrom(stringContent.get(field1)))) ) ); // many iterations with boolean queries, which are the most complex queries to deal with when nested @@ -682,7 +681,7 @@ public void testRangeQueries() throws Exception { v ); TopDocs topDocs = shardSearcher.search(query, 1); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); @@ -690,7 +689,7 @@ public void testRangeQueries() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = fieldType.percolateQuery("_name", queryStore, Collections.singletonList(new BytesArray("{}")), percolateSearcher, false, v); topDocs = shardSearcher.search(query, 1); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(1, topDocs.scoreDocs[0].doc); @@ -698,7 +697,7 @@ public void testRangeQueries() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = fieldType.percolateQuery("_name", queryStore, Collections.singletonList(new BytesArray("{}")), percolateSearcher, false, v); topDocs = shardSearcher.search(query, 1); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(2, topDocs.scoreDocs[0].doc); @@ -706,7 +705,7 @@ public void testRangeQueries() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = fieldType.percolateQuery("_name", queryStore, Collections.singletonList(new BytesArray("{}")), percolateSearcher, false, v); topDocs = shardSearcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(3, topDocs.scoreDocs[0].doc); @@ -714,7 +713,7 @@ public void testRangeQueries() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = fieldType.percolateQuery("_name", queryStore, Collections.singletonList(new BytesArray("{}")), percolateSearcher, false, v); topDocs = shardSearcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(4, topDocs.scoreDocs[0].doc); @@ -725,7 +724,7 @@ public void testRangeQueries() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = fieldType.percolateQuery("_name", queryStore, Collections.singletonList(new BytesArray("{}")), percolateSearcher, false, v); topDocs = shardSearcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(5, topDocs.scoreDocs[0].doc); } @@ -871,14 +870,14 @@ public void testPercolateMatchAll() throws Exception { Version.CURRENT ); TopDocs topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(3L, topDocs.totalHits.value); + assertEquals(3L, topDocs.totalHits.value()); assertEquals(3, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(1, topDocs.scoreDocs[1].doc); assertEquals(4, topDocs.scoreDocs[2].doc); topDocs = shardSearcher.search(new ConstantScoreQuery(query), 10); - assertEquals(3L, topDocs.totalHits.value); + assertEquals(3L, topDocs.totalHits.value()); assertEquals(3, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(1, topDocs.scoreDocs[1].doc); @@ -910,7 +909,7 @@ public void testFunctionScoreQuery() throws Exception { Version.CURRENT ); TopDocs topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(2, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(2, topDocs.scoreDocs[1].doc); @@ -966,15 +965,15 @@ public void testPercolateSmallAndLargeDocument() throws Exception { v ); BooleanQuery candidateQuery = (BooleanQuery) query.getCandidateMatchesQuery(); - assertThat(candidateQuery.clauses().get(0).getQuery(), instanceOf(CoveringQuery.class)); + assertThat(candidateQuery.clauses().get(0).query(), instanceOf(CoveringQuery.class)); TopDocs topDocs = shardSearcher.search(query, 10); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(2, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(2, topDocs.scoreDocs[1].doc); topDocs = shardSearcher.search(new ConstantScoreQuery(query), 10); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(2, topDocs.scoreDocs.length); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(2, topDocs.scoreDocs[1].doc); @@ -1002,16 +1001,16 @@ public void testPercolateSmallAndLargeDocument() throws Exception { v ); BooleanQuery candidateQuery = (BooleanQuery) query.getCandidateMatchesQuery(); - assertThat(candidateQuery.clauses().get(0).getQuery(), instanceOf(TermInSetQuery.class)); + assertThat(candidateQuery.clauses().get(0).query(), instanceOf(TermInSetQuery.class)); TopDocs topDocs = shardSearcher.search(query, 10); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(2, topDocs.scoreDocs.length); assertEquals(1, topDocs.scoreDocs[0].doc); assertEquals(2, topDocs.scoreDocs[1].doc); topDocs = shardSearcher.search(new ConstantScoreQuery(query), 10); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(2, topDocs.scoreDocs.length); assertEquals(1, topDocs.scoreDocs[0].doc); assertEquals(2, topDocs.scoreDocs[1].doc); @@ -1062,7 +1061,7 @@ public void testDuplicatedClauses() throws Exception { IndexSearcher percolateSearcher = memoryIndex.createSearcher(); PercolateQuery query = (PercolateQuery) fieldType.percolateQuery("_name", queryStore, sources, percolateSearcher, false, v); TopDocs topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(2L, topDocs.totalHits.value); + assertEquals(2L, topDocs.totalHits.value()); assertEquals(0, topDocs.scoreDocs[0].doc); assertEquals(1, topDocs.scoreDocs[1].doc); } @@ -1096,7 +1095,7 @@ public void testDuplicatedClauses2() throws Exception { IndexSearcher percolateSearcher = memoryIndex.createSearcher(); PercolateQuery query = (PercolateQuery) fieldType.percolateQuery("_name", queryStore, sources, percolateSearcher, false, v); TopDocs topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(0, topDocs.scoreDocs[0].doc); memoryIndex = new MemoryIndex(); @@ -1104,7 +1103,7 @@ public void testDuplicatedClauses2() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = (PercolateQuery) fieldType.percolateQuery("_name", queryStore, sources, percolateSearcher, false, v); topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(0, topDocs.scoreDocs[0].doc); memoryIndex = new MemoryIndex(); @@ -1112,7 +1111,7 @@ public void testDuplicatedClauses2() throws Exception { percolateSearcher = memoryIndex.createSearcher(); query = (PercolateQuery) fieldType.percolateQuery("_name", queryStore, sources, percolateSearcher, false, v); topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(0, topDocs.scoreDocs[0].doc); } @@ -1147,7 +1146,7 @@ public void testMsmAndRanges_disjunction() throws Exception { IndexSearcher percolateSearcher = memoryIndex.createSearcher(); PercolateQuery query = (PercolateQuery) fieldType.percolateQuery("_name", queryStore, sources, percolateSearcher, false, v); TopDocs topDocs = shardSearcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertEquals(1L, topDocs.totalHits.value); + assertEquals(1L, topDocs.totalHits.value()); assertEquals(0, topDocs.scoreDocs[0].doc); } @@ -1170,7 +1169,7 @@ private void duelRun(PercolateQuery.QueryStore queryStore, MemoryIndex memoryInd TopDocs controlTopDocs = shardSearcher.search(controlQuery, 100); try { - assertThat(topDocs.totalHits.value, equalTo(controlTopDocs.totalHits.value)); + assertThat(topDocs.totalHits.value(), equalTo(controlTopDocs.totalHits.value())); assertThat(topDocs.scoreDocs.length, equalTo(controlTopDocs.scoreDocs.length)); for (int j = 0; j < topDocs.scoreDocs.length; j++) { assertThat(topDocs.scoreDocs[j].doc, equalTo(controlTopDocs.scoreDocs[j].doc)); @@ -1198,7 +1197,7 @@ private void duelRun(PercolateQuery.QueryStore queryStore, MemoryIndex memoryInd logger.error("controlTopDocs.scoreDocs[{}].score={}", i, controlTopDocs.scoreDocs[i].score); // Additional stored information that is useful when debugging: - String queryToString = shardSearcher.doc(controlTopDocs.scoreDocs[i].doc).get("query_to_string"); + String queryToString = shardSearcher.storedFields().document(controlTopDocs.scoreDocs[i].doc).get("query_to_string"); logger.error("controlTopDocs.scoreDocs[{}].query_to_string={}", i, queryToString); TermsEnum tenum = MultiTerms.getTerms(shardSearcher.getIndexReader(), fieldType.queryTermsField.name()).iterator(); @@ -1330,7 +1329,7 @@ public String toString() { } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { float _score[] = new float[] { boost }; DocIdSetIterator allDocs = DocIdSetIterator.all(context.reader().maxDoc()); CheckedFunction leaf = queryStore.getQueries(context); @@ -1354,7 +1353,7 @@ protected boolean match(int doc) { } } }; - return new Scorer(this) { + return new DefaultScorerSupplier(new Scorer() { @Override public int docID() { @@ -1375,7 +1374,7 @@ public float score() throws IOException { public float getMaxScore(int upTo) throws IOException { return _score[0]; } - }; + }); } @Override diff --git a/modules/percolator/src/test/java/org/opensearch/percolator/PercolateQueryTests.java b/modules/percolator/src/test/java/org/opensearch/percolator/PercolateQueryTests.java index f75367fc054c0..fb688ad88b884 100644 --- a/modules/percolator/src/test/java/org/opensearch/percolator/PercolateQueryTests.java +++ b/modules/percolator/src/test/java/org/opensearch/percolator/PercolateQueryTests.java @@ -141,7 +141,7 @@ public void testPercolateQuery() throws Exception { ) ); TopDocs topDocs = shardSearcher.search(query, 10); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); assertThat(topDocs.scoreDocs.length, equalTo(1)); assertThat(topDocs.scoreDocs[0].doc, equalTo(0)); Explanation explanation = shardSearcher.explain(query, 0); @@ -160,7 +160,7 @@ public void testPercolateQuery() throws Exception { ) ); topDocs = shardSearcher.search(query, 10); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs.length, equalTo(3)); assertThat(topDocs.scoreDocs[0].doc, equalTo(1)); explanation = shardSearcher.explain(query, 1); @@ -189,7 +189,7 @@ public void testPercolateQuery() throws Exception { ) ); topDocs = shardSearcher.search(query, 10); - assertThat(topDocs.totalHits.value, equalTo(4L)); + assertThat(topDocs.totalHits.value(), equalTo(4L)); query = new PercolateQuery( "_name", @@ -201,7 +201,7 @@ public void testPercolateQuery() throws Exception { new MatchNoDocsQuery("") ); topDocs = shardSearcher.search(query, 10); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs.length, equalTo(3)); assertThat(topDocs.scoreDocs[0].doc, equalTo(3)); explanation = shardSearcher.explain(query, 3); diff --git a/modules/percolator/src/test/java/org/opensearch/percolator/PercolatorFieldMapperTests.java b/modules/percolator/src/test/java/org/opensearch/percolator/PercolatorFieldMapperTests.java index ea04f21be4cee..3a5add974c439 100644 --- a/modules/percolator/src/test/java/org/opensearch/percolator/PercolatorFieldMapperTests.java +++ b/modules/percolator/src/test/java/org/opensearch/percolator/PercolatorFieldMapperTests.java @@ -462,8 +462,8 @@ public void testCreateCandidateQuery() throws Exception { Tuple t = fieldType.createCandidateQuery(indexReader, Version.CURRENT); assertTrue(t.v2()); assertEquals(2, t.v1().clauses().size()); - assertThat(t.v1().clauses().get(0).getQuery(), instanceOf(CoveringQuery.class)); - assertThat(t.v1().clauses().get(1).getQuery(), instanceOf(TermQuery.class)); + assertThat(t.v1().clauses().get(0).query(), instanceOf(CoveringQuery.class)); + assertThat(t.v1().clauses().get(1).query(), instanceOf(TermQuery.class)); // Now push it over the edge, so that it falls back using TermInSetQuery memoryIndex.addField("field2", "value", new WhitespaceAnalyzer()); @@ -471,10 +471,10 @@ public void testCreateCandidateQuery() throws Exception { t = fieldType.createCandidateQuery(indexReader, Version.CURRENT); assertFalse(t.v2()); assertEquals(3, t.v1().clauses().size()); - TermInSetQuery terms = (TermInSetQuery) t.v1().clauses().get(0).getQuery(); - assertEquals(1023, terms.getTermData().size()); - assertThat(t.v1().clauses().get(1).getQuery().toString(), containsString(fieldName + ".range_field: t = fieldType.createCandidateQuery(indexReader, Version.CURRENT); assertTrue(t.v2()); assertEquals(2, t.v1().clauses().size()); - assertThat(t.v1().clauses().get(0).getQuery(), instanceOf(CoveringQuery.class)); - assertThat(t.v1().clauses().get(1).getQuery(), instanceOf(TermQuery.class)); + assertThat(t.v1().clauses().get(0).query(), instanceOf(CoveringQuery.class)); + assertThat(t.v1().clauses().get(1).query(), instanceOf(TermQuery.class)); } public void testExtractTermsAndRanges_numberFields() throws Exception { diff --git a/modules/percolator/src/test/java/org/opensearch/percolator/QueryAnalyzerTests.java b/modules/percolator/src/test/java/org/opensearch/percolator/QueryAnalyzerTests.java index 9699fb741a678..d5c6e0865ef37 100644 --- a/modules/percolator/src/test/java/org/opensearch/percolator/QueryAnalyzerTests.java +++ b/modules/percolator/src/test/java/org/opensearch/percolator/QueryAnalyzerTests.java @@ -110,7 +110,7 @@ public void testExtractQueryMetadata_termQuery() { } public void testExtractQueryMetadata_termsQuery() { - TermInSetQuery termsQuery = new TermInSetQuery("_field", new BytesRef("_term1"), new BytesRef("_term2")); + TermInSetQuery termsQuery = new TermInSetQuery("_field", List.of(new BytesRef("_term1"), new BytesRef("_term2"))); Result result = analyze(termsQuery, Version.CURRENT); assertThat(result.verified, is(true)); assertThat(result.minimumShouldMatch, equalTo(1)); diff --git a/modules/reindex/src/main/java/org/opensearch/index/reindex/remote/RemoteResponseParsers.java b/modules/reindex/src/main/java/org/opensearch/index/reindex/remote/RemoteResponseParsers.java index 981a22d4e7945..a63511f606fc2 100644 --- a/modules/reindex/src/main/java/org/opensearch/index/reindex/remote/RemoteResponseParsers.java +++ b/modules/reindex/src/main/java/org/opensearch/index/reindex/remote/RemoteResponseParsers.java @@ -121,8 +121,8 @@ class Fields { HITS_PARSER.declareField(constructorArg(), (p, c) -> { if (p.currentToken() == XContentParser.Token.START_OBJECT) { final TotalHits totalHits = SearchHits.parseTotalHitsFragment(p); - assert totalHits.relation == TotalHits.Relation.EQUAL_TO; - return totalHits.value; + assert totalHits.relation() == TotalHits.Relation.EQUAL_TO; + return totalHits.value(); } else { // For BWC with nodes pre 7.0 return p.longValue(); diff --git a/modules/reindex/src/test/java/org/opensearch/index/reindex/DeleteByQueryBasicTests.java b/modules/reindex/src/test/java/org/opensearch/index/reindex/DeleteByQueryBasicTests.java index 6874f96628761..373745f9ad22a 100644 --- a/modules/reindex/src/test/java/org/opensearch/index/reindex/DeleteByQueryBasicTests.java +++ b/modules/reindex/src/test/java/org/opensearch/index/reindex/DeleteByQueryBasicTests.java @@ -181,7 +181,7 @@ public void testDeleteByQueryWithRouting() throws Exception { String routing = String.valueOf(randomIntBetween(2, docs)); logger.info("--> counting documents with routing [{}]", routing); - long expected = client().prepareSearch().setSize(0).setRouting(routing).get().getHits().getTotalHits().value; + long expected = client().prepareSearch().setSize(0).setRouting(routing).get().getHits().getTotalHits().value(); logger.info("--> delete all documents with routing [{}] with a delete-by-query", routing); DeleteByQueryRequestBuilder delete = deleteByQuery().source("test").filter(QueryBuilders.matchAllQuery()); diff --git a/modules/repository-url/src/internalClusterTest/java/org/opensearch/repositories/url/URLSnapshotRestoreIT.java b/modules/repository-url/src/internalClusterTest/java/org/opensearch/repositories/url/URLSnapshotRestoreIT.java index fd0c6b309c4fd..4c71f80a30926 100644 --- a/modules/repository-url/src/internalClusterTest/java/org/opensearch/repositories/url/URLSnapshotRestoreIT.java +++ b/modules/repository-url/src/internalClusterTest/java/org/opensearch/repositories/url/URLSnapshotRestoreIT.java @@ -80,7 +80,7 @@ public void testUrlRepository() throws Exception { index("test-idx", "doc", Integer.toString(i), "foo", "bar" + i); } refresh(); - assertThat(client.prepareSearch("test-idx").setSize(0).get().getHits().getTotalHits().value, equalTo(100L)); + assertThat(client.prepareSearch("test-idx").setSize(0).get().getHits().getTotalHits().value(), equalTo(100L)); logger.info("--> snapshot"); CreateSnapshotResponse createSnapshotResponse = client.admin() @@ -121,7 +121,7 @@ public void testUrlRepository() throws Exception { .actionGet(); assertThat(restoreSnapshotResponse.getRestoreInfo().totalShards(), greaterThan(0)); - assertThat(client.prepareSearch("test-idx").setSize(0).get().getHits().getTotalHits().value, equalTo(100L)); + assertThat(client.prepareSearch("test-idx").setSize(0).get().getHits().getTotalHits().value(), equalTo(100L)); logger.info("--> list available shapshots"); GetSnapshotsResponse getSnapshotsResponse = client.admin().cluster().prepareGetSnapshots("url-repo").get(); diff --git a/modules/search-pipeline-common/src/internalClusterTest/java/org/opensearch/search/pipeline/common/SearchPipelineCommonIT.java b/modules/search-pipeline-common/src/internalClusterTest/java/org/opensearch/search/pipeline/common/SearchPipelineCommonIT.java index 35d748bc2d06e..7e938623975fc 100644 --- a/modules/search-pipeline-common/src/internalClusterTest/java/org/opensearch/search/pipeline/common/SearchPipelineCommonIT.java +++ b/modules/search-pipeline-common/src/internalClusterTest/java/org/opensearch/search/pipeline/common/SearchPipelineCommonIT.java @@ -77,12 +77,12 @@ public void testFilterQuery() { // Search without the pipeline. Should see both documents. SearchRequest req = new SearchRequest(TEST_INDEX).source(new SearchSourceBuilder().query(new MatchAllQueryBuilder())); SearchResponse rsp = client().search(req).actionGet(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); // Search with the pipeline. Should only see document with "field":"value". req.pipeline(PIPELINE_NAME); rsp = client().search(req).actionGet(); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); // Clean up. deletePipeline(); @@ -93,7 +93,7 @@ public void testSearchWithTemporaryPipeline() throws Exception { // Search without the pipeline. Should see both documents. SearchRequest req = new SearchRequest(TEST_INDEX).source(new SearchSourceBuilder().query(new MatchAllQueryBuilder())); SearchResponse rsp = client().search(req).actionGet(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); // Search with temporary pipeline Map pipelineSourceMap = new HashMap<>(); @@ -109,7 +109,7 @@ public void testSearchWithTemporaryPipeline() throws Exception { ); SearchResponse rspWithTempPipeline = client().search(req).actionGet(); - assertEquals(1, rspWithTempPipeline.getHits().getTotalHits().value); + assertEquals(1, rspWithTempPipeline.getHits().getTotalHits().value()); } public void testSearchWithDefaultPipeline() throws Exception { @@ -119,7 +119,7 @@ public void testSearchWithDefaultPipeline() throws Exception { // Search without the pipeline. Should see both documents. SearchRequest req = new SearchRequest(TEST_INDEX).source(new SearchSourceBuilder().query(new MatchAllQueryBuilder())); SearchResponse rsp = client().search(req).actionGet(); - assertEquals(2, rsp.getHits().getTotalHits().value); + assertEquals(2, rsp.getHits().getTotalHits().value()); // Set pipeline as default for the index UpdateSettingsRequest updateSettingsRequest = new UpdateSettingsRequest(TEST_INDEX); @@ -129,7 +129,7 @@ public void testSearchWithDefaultPipeline() throws Exception { // Search with the default pipeline. Should only see document with "field":"value". rsp = client().search(req).actionGet(); - assertEquals(1, rsp.getHits().getTotalHits().value); + assertEquals(1, rsp.getHits().getTotalHits().value()); // Clean up: Remove default pipeline setting updateSettingsRequest = new UpdateSettingsRequest(TEST_INDEX); @@ -149,7 +149,7 @@ public void testUpdateSearchPipeline() throws Exception { SearchRequest req = new SearchRequest(TEST_INDEX).source(new SearchSourceBuilder().query(new MatchAllQueryBuilder())); req.pipeline(PIPELINE_NAME); SearchResponse initialRsp = client().search(req).actionGet(); - assertEquals(1, initialRsp.getHits().getTotalHits().value); + assertEquals(1, initialRsp.getHits().getTotalHits().value()); BytesReference pipelineConfig = new BytesArray( "{" diff --git a/plugins/analysis-icu/licenses/lucene-analysis-icu-10.1.0.jar.sha1 b/plugins/analysis-icu/licenses/lucene-analysis-icu-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..8de1c34f0c81f --- /dev/null +++ b/plugins/analysis-icu/licenses/lucene-analysis-icu-10.1.0.jar.sha1 @@ -0,0 +1 @@ +93c315f11e5f6901595a5bd41e35224e081f4c84 \ No newline at end of file diff --git a/plugins/analysis-icu/licenses/lucene-analysis-icu-9.12.1.jar.sha1 b/plugins/analysis-icu/licenses/lucene-analysis-icu-9.12.1.jar.sha1 deleted file mode 100644 index acb73de8b5dc9..0000000000000 --- a/plugins/analysis-icu/licenses/lucene-analysis-icu-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -abaef4767ad64289e62abdd4606bf6ed2ddea0fd \ No newline at end of file diff --git a/plugins/analysis-icu/src/main/java/org/opensearch/index/mapper/ICUCollationKeywordFieldMapper.java b/plugins/analysis-icu/src/main/java/org/opensearch/index/mapper/ICUCollationKeywordFieldMapper.java index bd9c164811093..2f6b3d5cb50bb 100644 --- a/plugins/analysis-icu/src/main/java/org/opensearch/index/mapper/ICUCollationKeywordFieldMapper.java +++ b/plugins/analysis-icu/src/main/java/org/opensearch/index/mapper/ICUCollationKeywordFieldMapper.java @@ -167,6 +167,7 @@ public Query fuzzyQuery( int prefixLength, int maxExpansions, boolean transpositions, + org.apache.lucene.search.MultiTermQuery.RewriteMethod method, QueryShardContext context ) { throw new UnsupportedOperationException("[fuzzy] queries are not supported on [" + CONTENT_TYPE + "] fields."); diff --git a/plugins/analysis-icu/src/test/java/org/opensearch/index/mapper/CollationFieldTypeTests.java b/plugins/analysis-icu/src/test/java/org/opensearch/index/mapper/CollationFieldTypeTests.java index 1e08000117f61..bf2bf3ffc52d6 100644 --- a/plugins/analysis-icu/src/test/java/org/opensearch/index/mapper/CollationFieldTypeTests.java +++ b/plugins/analysis-icu/src/test/java/org/opensearch/index/mapper/CollationFieldTypeTests.java @@ -37,6 +37,7 @@ import com.ibm.icu.text.RawCollationKey; import com.ibm.icu.util.ULocale; import org.apache.lucene.index.Term; +import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.TermRangeQuery; @@ -127,7 +128,15 @@ public void testFuzzyQuery() { MappedFieldType ft = createFieldType(); UnsupportedOperationException e = expectThrows( UnsupportedOperationException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, randomMockShardContext()) + () -> ft.fuzzyQuery( + "foo", + Fuzziness.fromEdits(2), + 1, + 50, + true, + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, + randomMockShardContext() + ) ); assertEquals("[fuzzy] queries are not supported on [icu_collation_keyword] fields.", e.getMessage()); } diff --git a/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-10.1.0.jar.sha1 b/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..deb1761045dc4 --- /dev/null +++ b/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-10.1.0.jar.sha1 @@ -0,0 +1 @@ +5cbe34565be95b4dfe4386e7b8dc62eb9e9020e6 \ No newline at end of file diff --git a/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.12.1.jar.sha1 b/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.12.1.jar.sha1 deleted file mode 100644 index 916778086a6bd..0000000000000 --- a/plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -635c41143b896f402589d29e33695dcfabae9cc5 \ No newline at end of file diff --git a/plugins/analysis-nori/licenses/lucene-analysis-nori-10.1.0.jar.sha1 b/plugins/analysis-nori/licenses/lucene-analysis-nori-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..875e0a2ab10b5 --- /dev/null +++ b/plugins/analysis-nori/licenses/lucene-analysis-nori-10.1.0.jar.sha1 @@ -0,0 +1 @@ +ae634b73ddd2e5de044993bf2a5ab36aeeadbf99 \ No newline at end of file diff --git a/plugins/analysis-nori/licenses/lucene-analysis-nori-9.12.1.jar.sha1 b/plugins/analysis-nori/licenses/lucene-analysis-nori-9.12.1.jar.sha1 deleted file mode 100644 index 9c057370df5d1..0000000000000 --- a/plugins/analysis-nori/licenses/lucene-analysis-nori-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -e265410a6a4d9cd23b2e9c73321e6bd307bc1422 \ No newline at end of file diff --git a/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-10.1.0.jar.sha1 b/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..8d2cf1a59a754 --- /dev/null +++ b/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-10.1.0.jar.sha1 @@ -0,0 +1 @@ +62f9351b6ba692e948f0e4f718cf97e16962f441 \ No newline at end of file diff --git a/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.12.1.jar.sha1 b/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.12.1.jar.sha1 deleted file mode 100644 index 30db9fc8d69e2..0000000000000 --- a/plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -3787b8edc0cfad21998abc6aeb9d2cbf152b4b26 \ No newline at end of file diff --git a/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-10.1.0.jar.sha1 b/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..67271157d3883 --- /dev/null +++ b/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-10.1.0.jar.sha1 @@ -0,0 +1 @@ +277221f0ddfb8b21053cd154cb973c61631ec669 \ No newline at end of file diff --git a/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.12.1.jar.sha1 b/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.12.1.jar.sha1 deleted file mode 100644 index 96f8d70e6ee53..0000000000000 --- a/plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -e935f600bf153c46f5725198ca9352c32025f274 \ No newline at end of file diff --git a/plugins/analysis-stempel/licenses/lucene-analysis-stempel-10.1.0.jar.sha1 b/plugins/analysis-stempel/licenses/lucene-analysis-stempel-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..ceb636de044e1 --- /dev/null +++ b/plugins/analysis-stempel/licenses/lucene-analysis-stempel-10.1.0.jar.sha1 @@ -0,0 +1 @@ +980b7136dac6b375d621237488930e8bc3a664c7 \ No newline at end of file diff --git a/plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.12.1.jar.sha1 b/plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.12.1.jar.sha1 deleted file mode 100644 index d6d5f1c2609ff..0000000000000 --- a/plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -c4e1c94b1adbd1cb9dbdc0d3c2d2c33beabfc777 \ No newline at end of file diff --git a/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-10.1.0.jar.sha1 b/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..127a8a25d2284 --- /dev/null +++ b/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-10.1.0.jar.sha1 @@ -0,0 +1 @@ +b641b4a674c4c89d6cee7b28018aa378b933398b \ No newline at end of file diff --git a/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.12.1.jar.sha1 b/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.12.1.jar.sha1 deleted file mode 100644 index 661f3062458e2..0000000000000 --- a/plugins/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -d8e4716dab6d829e7b37a8b185cbd242650aeb9e \ No newline at end of file diff --git a/plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java b/plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java index 33c27eb301ad1..96a323c00d8b4 100644 --- a/plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java +++ b/plugins/cache-ehcache/src/main/java/org/opensearch/cache/store/disk/EhcacheDiskCache.java @@ -194,7 +194,7 @@ PersistentCacheManager getCacheManager() { return this.cacheManager; } - @SuppressWarnings({ "rawtypes" }) + @SuppressWarnings({ "rawtypes", "removal" }) private Cache buildCache(Duration expireAfterAccess, Builder builder) { // Creating the cache requires permissions specified in plugin-security.policy return AccessController.doPrivileged((PrivilegedAction>) () -> { @@ -279,6 +279,7 @@ Map, CompletableFuture, V>>> getCompletableFutur return completableFutureMap; } + @SuppressWarnings("removal") @SuppressForbidden(reason = "Ehcache uses File.io") PersistentCacheManager buildCacheManager() { // In case we use multiple ehCaches, we can define this cache manager at a global level. diff --git a/plugins/mapper-annotated-text/src/internalClusterTest/java/org/opensearch/index/mapper/annotatedtext/AnnotatedTextFieldMapperTests.java b/plugins/mapper-annotated-text/src/internalClusterTest/java/org/opensearch/index/mapper/annotatedtext/AnnotatedTextFieldMapperTests.java index 9bfd5669e731b..f8912d7760949 100644 --- a/plugins/mapper-annotated-text/src/internalClusterTest/java/org/opensearch/index/mapper/annotatedtext/AnnotatedTextFieldMapperTests.java +++ b/plugins/mapper-annotated-text/src/internalClusterTest/java/org/opensearch/index/mapper/annotatedtext/AnnotatedTextFieldMapperTests.java @@ -264,7 +264,7 @@ public void testIndexedTermVectors() throws IOException { withLuceneIndex(mapperService, iw -> iw.addDocument(doc.rootDoc()), reader -> { LeafReader leaf = reader.leaves().get(0).reader(); - Terms terms = leaf.getTermVector(0, "field"); + Terms terms = leaf.terms("field"); TermsEnum iterator = terms.iterator(); BytesRef term; Set foundTerms = new HashSet<>(); diff --git a/plugins/mapper-annotated-text/src/test/java/org/opensearch/search/fetch/subphase/highlight/AnnotatedTextHighlighterTests.java b/plugins/mapper-annotated-text/src/test/java/org/opensearch/search/fetch/subphase/highlight/AnnotatedTextHighlighterTests.java index 34851cf9bcbae..ac0bc86b00a6a 100644 --- a/plugins/mapper-annotated-text/src/test/java/org/opensearch/search/fetch/subphase/highlight/AnnotatedTextHighlighterTests.java +++ b/plugins/mapper-annotated-text/src/test/java/org/opensearch/search/fetch/subphase/highlight/AnnotatedTextHighlighterTests.java @@ -121,7 +121,7 @@ private void assertHighlightOneDoc( } TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 1, Sort.INDEXORDER); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); String rawValue = Strings.collectionToDelimitedString(plainTextForHighlighter, String.valueOf(MULTIVAL_SEP_CHAR)); CustomUnifiedHighlighter highlighter = new CustomUnifiedHighlighter( searcher, diff --git a/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsBlobStoreContainerTests.java b/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsBlobStoreContainerTests.java index 5f7454df4ecfc..ace75dce9d7e9 100644 --- a/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsBlobStoreContainerTests.java +++ b/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsBlobStoreContainerTests.java @@ -75,6 +75,7 @@ private FileContext createTestContext() { return fileContext; } + @SuppressWarnings("removal") @SuppressForbidden(reason = "lesser of two evils (the other being a bunch of JNI/classloader nightmares)") private FileContext createContext(URI uri) { // mirrors HdfsRepository.java behaviour diff --git a/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsTests.java b/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsTests.java index 130bbbf1d2198..a13a9d55c4735 100644 --- a/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsTests.java +++ b/plugins/repository-hdfs/src/test/java/org/opensearch/repositories/hdfs/HdfsTests.java @@ -213,6 +213,6 @@ public void testMissingPath() { } private long count(Client client, String index) { - return client.prepareSearch(index).setSize(0).get().getHits().getTotalHits().value; + return client.prepareSearch(index).setSize(0).get().getHits().getTotalHits().value(); } } diff --git a/plugins/store-smb/src/internalClusterTest/java/org/opensearch/index/store/AbstractAzureFsTestCase.java b/plugins/store-smb/src/internalClusterTest/java/org/opensearch/index/store/AbstractAzureFsTestCase.java index ab3ffe97199ed..8d2bde09f9952 100644 --- a/plugins/store-smb/src/internalClusterTest/java/org/opensearch/index/store/AbstractAzureFsTestCase.java +++ b/plugins/store-smb/src/internalClusterTest/java/org/opensearch/index/store/AbstractAzureFsTestCase.java @@ -57,6 +57,6 @@ public void testAzureFs() { } refresh(); SearchResponse response = client().prepareSearch("test").get(); - assertThat(response.getHits().getTotalHits().value, is(nbDocs)); + assertThat(response.getHits().getTotalHits().value(), is(nbDocs)); } } diff --git a/plugins/store-smb/src/main/java/org/opensearch/index/store/smbmmapfs/SmbMmapFsDirectoryFactory.java b/plugins/store-smb/src/main/java/org/opensearch/index/store/smbmmapfs/SmbMmapFsDirectoryFactory.java index 286688f61e658..c38597720c5a9 100644 --- a/plugins/store-smb/src/main/java/org/opensearch/index/store/smbmmapfs/SmbMmapFsDirectoryFactory.java +++ b/plugins/store-smb/src/main/java/org/opensearch/index/store/smbmmapfs/SmbMmapFsDirectoryFactory.java @@ -51,7 +51,6 @@ protected Directory newFSDirectory(Path location, LockFactory lockFactory, Index return new SmbDirectoryWrapper( setPreload( new MMapDirectory(location, lockFactory), - lockFactory, new HashSet<>(indexSettings.getValue(IndexModule.INDEX_STORE_PRE_LOAD_SETTING)) ) ); diff --git a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/OTelTelemetrySettings.java b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/OTelTelemetrySettings.java index 95ce6918fcb70..91e8268622677 100644 --- a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/OTelTelemetrySettings.java +++ b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/OTelTelemetrySettings.java @@ -120,7 +120,7 @@ private OTelTelemetrySettings() {} /** * Samplers orders setting. */ - @SuppressWarnings("unchecked") + @SuppressWarnings({ "unchecked", "removal" }) public static final Setting>> OTEL_TRACER_SPAN_SAMPLER_CLASS_SETTINGS = Setting.listSetting( "telemetry.otel.tracer.span.sampler.classes", Arrays.asList(ProbabilisticTransportActionSampler.class.getName(), ProbabilisticSampler.class.getName()), diff --git a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/metrics/OTelMetricsTelemetry.java b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/metrics/OTelMetricsTelemetry.java index 3258e91738ba6..2964c8f63b081 100644 --- a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/metrics/OTelMetricsTelemetry.java +++ b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/metrics/OTelMetricsTelemetry.java @@ -82,6 +82,7 @@ public Counter createUpDownCounter(String name, String description, String unit) * @param unit unit of the metric. * @return histogram */ + @SuppressWarnings("removal") @Override public Histogram createHistogram(String name, String description, String unit) { DoubleHistogram doubleHistogram = AccessController.doPrivileged( @@ -90,6 +91,7 @@ public Histogram createHistogram(String name, String description, String unit) { return new OTelHistogram(doubleHistogram); } + @SuppressWarnings("removal") @Override public Closeable createGauge(String name, String description, String unit, Supplier valueProvider, Tags tags) { ObservableDoubleGauge doubleObservableGauge = AccessController.doPrivileged( @@ -101,6 +103,7 @@ public Closeable createGauge(String name, String description, String unit, Suppl return () -> doubleObservableGauge.close(); } + @SuppressWarnings("removal") @Override public Closeable createGauge(String name, String description, String unit, Supplier value) { ObservableDoubleGauge doubleObservableGauge = AccessController.doPrivileged( diff --git a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/tracing/sampler/OTelSamplerFactory.java b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/tracing/sampler/OTelSamplerFactory.java index b9d5c07a40cd8..da8887867a43f 100644 --- a/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/tracing/sampler/OTelSamplerFactory.java +++ b/plugins/telemetry-otel/src/main/java/org/opensearch/telemetry/tracing/sampler/OTelSamplerFactory.java @@ -63,6 +63,7 @@ public static Sampler create(TelemetrySettings telemetrySettings, Settings setti return fallbackSampler; } + @SuppressWarnings("removal") private static Sampler instantiateSampler( Class samplerClassName, TelemetrySettings telemetrySettings, diff --git a/qa/ccs-unavailable-clusters/src/test/java/org/opensearch/search/CrossClusterSearchUnavailableClusterIT.java b/qa/ccs-unavailable-clusters/src/test/java/org/opensearch/search/CrossClusterSearchUnavailableClusterIT.java index ea20d8000f640..83b421fd53a56 100644 --- a/qa/ccs-unavailable-clusters/src/test/java/org/opensearch/search/CrossClusterSearchUnavailableClusterIT.java +++ b/qa/ccs-unavailable-clusters/src/test/java/org/opensearch/search/CrossClusterSearchUnavailableClusterIT.java @@ -170,7 +170,7 @@ public void testSearchSkipUnavailable() throws IOException { { SearchResponse response = restHighLevelClient.search(new SearchRequest("index"), RequestOptions.DEFAULT); assertSame(SearchResponse.Clusters.EMPTY, response.getClusters()); - assertEquals(10, response.getHits().getTotalHits().value); + assertEquals(10, response.getHits().getTotalHits().value()); assertEquals(10, response.getHits().getHits().length); } { @@ -178,7 +178,7 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(2, response.getClusters().getTotal()); assertEquals(2, response.getClusters().getSuccessful()); assertEquals(0, response.getClusters().getSkipped()); - assertEquals(10, response.getHits().getTotalHits().value); + assertEquals(10, response.getHits().getTotalHits().value()); assertEquals(10, response.getHits().getHits().length); } { @@ -186,7 +186,7 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(1, response.getClusters().getTotal()); assertEquals(1, response.getClusters().getSuccessful()); assertEquals(0, response.getClusters().getSkipped()); - assertEquals(0, response.getHits().getTotalHits().value); + assertEquals(0, response.getHits().getTotalHits().value()); } { @@ -195,12 +195,12 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(2, response.getClusters().getTotal()); assertEquals(2, response.getClusters().getSuccessful()); assertEquals(0, response.getClusters().getSkipped()); - assertEquals(10, response.getHits().getTotalHits().value); + assertEquals(10, response.getHits().getTotalHits().value()); assertEquals(10, response.getHits().getHits().length); String scrollId = response.getScrollId(); SearchResponse scrollResponse = restHighLevelClient.scroll(new SearchScrollRequest(scrollId), RequestOptions.DEFAULT); assertSame(SearchResponse.Clusters.EMPTY, scrollResponse.getClusters()); - assertEquals(10, scrollResponse.getHits().getTotalHits().value); + assertEquals(10, scrollResponse.getHits().getTotalHits().value()); assertEquals(0, scrollResponse.getHits().getHits().length); } @@ -213,7 +213,7 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(2, response.getClusters().getTotal()); assertEquals(1, response.getClusters().getSuccessful()); assertEquals(1, response.getClusters().getSkipped()); - assertEquals(10, response.getHits().getTotalHits().value); + assertEquals(10, response.getHits().getTotalHits().value()); assertEquals(10, response.getHits().getHits().length); } { @@ -221,7 +221,7 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(1, response.getClusters().getTotal()); assertEquals(0, response.getClusters().getSuccessful()); assertEquals(1, response.getClusters().getSkipped()); - assertEquals(0, response.getHits().getTotalHits().value); + assertEquals(0, response.getHits().getTotalHits().value()); } { @@ -230,12 +230,12 @@ public void testSearchSkipUnavailable() throws IOException { assertEquals(2, response.getClusters().getTotal()); assertEquals(1, response.getClusters().getSuccessful()); assertEquals(1, response.getClusters().getSkipped()); - assertEquals(10, response.getHits().getTotalHits().value); + assertEquals(10, response.getHits().getTotalHits().value()); assertEquals(10, response.getHits().getHits().length); String scrollId = response.getScrollId(); SearchResponse scrollResponse = restHighLevelClient.scroll(new SearchScrollRequest(scrollId), RequestOptions.DEFAULT); assertSame(SearchResponse.Clusters.EMPTY, scrollResponse.getClusters()); - assertEquals(10, scrollResponse.getHits().getTotalHits().value); + assertEquals(10, scrollResponse.getHits().getTotalHits().value()); assertEquals(0, scrollResponse.getHits().getHits().length); } diff --git a/qa/multi-cluster-search/src/test/java/org/opensearch/search/CCSDuelIT.java b/qa/multi-cluster-search/src/test/java/org/opensearch/search/CCSDuelIT.java index 3dc2444d8a16e..d3460c616f454 100644 --- a/qa/multi-cluster-search/src/test/java/org/opensearch/search/CCSDuelIT.java +++ b/qa/multi-cluster-search/src/test/java/org/opensearch/search/CCSDuelIT.java @@ -444,7 +444,7 @@ public void testSortByField() throws Exception { searchRequest.source(sourceBuilder); duelSearch(searchRequest, response -> { assertHits(response, 30); - if (response.getHits().getTotalHits().value > 30) { + if (response.getHits().getTotalHits().value() > 30) { assertEquals(3, response.getHits().getHits()[0].getSortValues().length); } }); @@ -670,7 +670,7 @@ public void testShardFailures() throws Exception { searchRequest.source(sourceBuilder); duelSearch(searchRequest, response -> { assertMultiClusterSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, greaterThan(0L)); + assertThat(response.getHits().getTotalHits().value(), greaterThan(0L)); assertNull(response.getAggregations()); assertNull(response.getSuggest()); assertThat(response.getHits().getHits().length, greaterThan(0)); @@ -798,11 +798,11 @@ private static void assertHits(SearchResponse response) { private static void assertHits(SearchResponse response, int from) { assertMultiClusterSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, greaterThan(0L)); + assertThat(response.getHits().getTotalHits().value(), greaterThan(0L)); assertEquals(0, response.getFailedShards()); assertNull(response.getAggregations()); assertNull(response.getSuggest()); - if (response.getHits().getTotalHits().value > from) { + if (response.getHits().getTotalHits().value() > from) { assertThat(response.getHits().getHits().length, greaterThan(0)); } else { assertThat(response.getHits().getHits().length, equalTo(0)); @@ -811,7 +811,7 @@ private static void assertHits(SearchResponse response, int from) { private static void assertAggs(SearchResponse response) { assertMultiClusterSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, greaterThan(0L)); + assertThat(response.getHits().getTotalHits().value(), greaterThan(0L)); assertEquals(0, response.getHits().getHits().length); assertNull(response.getSuggest()); assertNotNull(response.getAggregations()); diff --git a/qa/remote-clusters/src/test/java/org/opensearch/cluster/remote/test/RemoteClustersIT.java b/qa/remote-clusters/src/test/java/org/opensearch/cluster/remote/test/RemoteClustersIT.java index c38fcc468c673..bb9db0c6d588d 100644 --- a/qa/remote-clusters/src/test/java/org/opensearch/cluster/remote/test/RemoteClustersIT.java +++ b/qa/remote-clusters/src/test/java/org/opensearch/cluster/remote/test/RemoteClustersIT.java @@ -65,8 +65,8 @@ public void setupIndices() throws IOException { .source(XContentFactory.jsonBuilder().startObject().field("foo", "bar").endObject()), RequestOptions.DEFAULT); cluster2Client().index(new IndexRequest("test2").id("id2").setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE) .source(XContentFactory.jsonBuilder().startObject().field("foo", "bar").endObject()), RequestOptions.DEFAULT); - assertEquals(1L, cluster1Client().search(new SearchRequest("test1"), RequestOptions.DEFAULT).getHits().getTotalHits().value); - assertEquals(2L, cluster2Client().search(new SearchRequest("test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value); + assertEquals(1L, cluster1Client().search(new SearchRequest("test1"), RequestOptions.DEFAULT).getHits().getTotalHits().value()); + assertEquals(2L, cluster2Client().search(new SearchRequest("test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value()); } @After @@ -97,7 +97,7 @@ public void testProxyModeConnectionWorks() throws IOException { assertTrue(rci.isConnected()); assertEquals(2L, cluster1Client().search( - new SearchRequest("cluster2:test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value); + new SearchRequest("cluster2:test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value()); } public void testSniffModeConnectionFails() throws IOException { @@ -133,6 +133,6 @@ public void testHAProxyModeConnectionWorks() throws Exception { }, 10, TimeUnit.SECONDS); assertEquals(2L, cluster1Client().search( - new SearchRequest("haproxynosn:test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value); + new SearchRequest("haproxynosn:test2"), RequestOptions.DEFAULT).getHits().getTotalHits().value()); } } diff --git a/qa/wildfly/docker-compose.yml b/qa/wildfly/docker-compose.yml index b0f1609f01e72..2a0779d999f25 100644 --- a/qa/wildfly/docker-compose.yml +++ b/qa/wildfly/docker-compose.yml @@ -2,7 +2,7 @@ version: '3.7' services: wildfly: - image: quay.io/wildfly/wildfly:28.0.1.Final-jdk11 + image: quay.io/wildfly/wildfly:34.0.1.Final-jdk21 environment: JAVA_OPTS: -Dopensearch.uri=opensearch:9200 -Djboss.http.port=8080 -Djava.net.preferIPv4Stack=true volumes: diff --git a/server/licenses/lucene-analysis-common-10.1.0.jar.sha1 b/server/licenses/lucene-analysis-common-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..779b248fcf495 --- /dev/null +++ b/server/licenses/lucene-analysis-common-10.1.0.jar.sha1 @@ -0,0 +1 @@ +ddbc824a311d49a54f5808d5a01d5c52424c48b8 \ No newline at end of file diff --git a/server/licenses/lucene-analysis-common-9.12.1.jar.sha1 b/server/licenses/lucene-analysis-common-9.12.1.jar.sha1 deleted file mode 100644 index 2b9a8cf6e43fd..0000000000000 --- a/server/licenses/lucene-analysis-common-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -86836497e35c1ab33259d9864ceb280c0016075e \ No newline at end of file diff --git a/server/licenses/lucene-backward-codecs-10.1.0.jar.sha1 b/server/licenses/lucene-backward-codecs-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..a212225c34cb9 --- /dev/null +++ b/server/licenses/lucene-backward-codecs-10.1.0.jar.sha1 @@ -0,0 +1 @@ +fbbebd58f1505cc70d73dbdbb8196bfc29b9cf08 \ No newline at end of file diff --git a/server/licenses/lucene-backward-codecs-9.12.1.jar.sha1 b/server/licenses/lucene-backward-codecs-9.12.1.jar.sha1 deleted file mode 100644 index 89d6ddbec3eec..0000000000000 --- a/server/licenses/lucene-backward-codecs-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -d0e79d06a0ed021663737e4df777ab7b80cd28c4 \ No newline at end of file diff --git a/server/licenses/lucene-core-10.1.0.jar.sha1 b/server/licenses/lucene-core-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..1727275619d1f --- /dev/null +++ b/server/licenses/lucene-core-10.1.0.jar.sha1 @@ -0,0 +1 @@ +65d7670de89a72433ef374b332da679a484d3a1e \ No newline at end of file diff --git a/server/licenses/lucene-core-9.12.1.jar.sha1 b/server/licenses/lucene-core-9.12.1.jar.sha1 deleted file mode 100644 index 2521c91a81d64..0000000000000 --- a/server/licenses/lucene-core-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -91447c90c1180122142773b5baddaf8547124794 \ No newline at end of file diff --git a/server/licenses/lucene-grouping-10.1.0.jar.sha1 b/server/licenses/lucene-grouping-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..9a0a50f0afca7 --- /dev/null +++ b/server/licenses/lucene-grouping-10.1.0.jar.sha1 @@ -0,0 +1 @@ +2f6409cbe456d5a9840969038173f0826f7ce4ff \ No newline at end of file diff --git a/server/licenses/lucene-grouping-9.12.1.jar.sha1 b/server/licenses/lucene-grouping-9.12.1.jar.sha1 deleted file mode 100644 index 61d7ff62ac3cc..0000000000000 --- a/server/licenses/lucene-grouping-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -e4bc3d0aa7eec4f41b4f350de0263a8d5625d2b3 \ No newline at end of file diff --git a/server/licenses/lucene-highlighter-10.1.0.jar.sha1 b/server/licenses/lucene-highlighter-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..173b24a96da90 --- /dev/null +++ b/server/licenses/lucene-highlighter-10.1.0.jar.sha1 @@ -0,0 +1 @@ +db9c72d8606788e5cd3a5c97e0a8b57648c8c6d4 \ No newline at end of file diff --git a/server/licenses/lucene-highlighter-9.12.1.jar.sha1 b/server/licenses/lucene-highlighter-9.12.1.jar.sha1 deleted file mode 100644 index 57fc10a58b806..0000000000000 --- a/server/licenses/lucene-highlighter-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -2eeedfcec47dd65969f36e88931ed452291dd43e \ No newline at end of file diff --git a/server/licenses/lucene-join-10.1.0.jar.sha1 b/server/licenses/lucene-join-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..833dfeadd4f81 --- /dev/null +++ b/server/licenses/lucene-join-10.1.0.jar.sha1 @@ -0,0 +1 @@ +8ae87dbc1ee0a044b72366eb750c40b269c52fd1 \ No newline at end of file diff --git a/server/licenses/lucene-join-9.12.1.jar.sha1 b/server/licenses/lucene-join-9.12.1.jar.sha1 deleted file mode 100644 index 8d46f20c39974..0000000000000 --- a/server/licenses/lucene-join-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -3c5e9ff2925a8373ae0d35c1d0a7b2465cebec9f \ No newline at end of file diff --git a/server/licenses/lucene-memory-10.1.0.jar.sha1 b/server/licenses/lucene-memory-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..836c18e425ce5 --- /dev/null +++ b/server/licenses/lucene-memory-10.1.0.jar.sha1 @@ -0,0 +1 @@ +8f6a402ff8c40f28fe2debe3ca01c441ccbe5b2d \ No newline at end of file diff --git a/server/licenses/lucene-memory-9.12.1.jar.sha1 b/server/licenses/lucene-memory-9.12.1.jar.sha1 deleted file mode 100644 index 55de1c9322aa3..0000000000000 --- a/server/licenses/lucene-memory-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -e80eecfb1dcc324140387c8357c81e12c2a01937 \ No newline at end of file diff --git a/server/licenses/lucene-misc-10.1.0.jar.sha1 b/server/licenses/lucene-misc-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..abf5baa3a8602 --- /dev/null +++ b/server/licenses/lucene-misc-10.1.0.jar.sha1 @@ -0,0 +1 @@ +ae1104521d00501e18e3c18c2b326f15589cc873 \ No newline at end of file diff --git a/server/licenses/lucene-misc-9.12.1.jar.sha1 b/server/licenses/lucene-misc-9.12.1.jar.sha1 deleted file mode 100644 index 86982eb1c900c..0000000000000 --- a/server/licenses/lucene-misc-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -4e65d01d1c23f3f49dc325d552701bbefafee7ee \ No newline at end of file diff --git a/server/licenses/lucene-queries-10.1.0.jar.sha1 b/server/licenses/lucene-queries-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..33697dcb6ce74 --- /dev/null +++ b/server/licenses/lucene-queries-10.1.0.jar.sha1 @@ -0,0 +1 @@ +48584c8a650cb4ce33a7e25864474e0da8c9eb53 \ No newline at end of file diff --git a/server/licenses/lucene-queries-9.12.1.jar.sha1 b/server/licenses/lucene-queries-9.12.1.jar.sha1 deleted file mode 100644 index f2087ec8eb623..0000000000000 --- a/server/licenses/lucene-queries-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -14f24315041b686683dba4bc679ca7dc6a505906 \ No newline at end of file diff --git a/server/licenses/lucene-queryparser-10.1.0.jar.sha1 b/server/licenses/lucene-queryparser-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..217b6e770740b --- /dev/null +++ b/server/licenses/lucene-queryparser-10.1.0.jar.sha1 @@ -0,0 +1 @@ +2774ab95128b0615568a4861b9a56f24511f774a \ No newline at end of file diff --git a/server/licenses/lucene-queryparser-9.12.1.jar.sha1 b/server/licenses/lucene-queryparser-9.12.1.jar.sha1 deleted file mode 100644 index 489e6719da342..0000000000000 --- a/server/licenses/lucene-queryparser-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -aa6df09a99f8881d843e9863aa1713dc9f3ed24f \ No newline at end of file diff --git a/server/licenses/lucene-sandbox-10.1.0.jar.sha1 b/server/licenses/lucene-sandbox-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..9b44d2dc75154 --- /dev/null +++ b/server/licenses/lucene-sandbox-10.1.0.jar.sha1 @@ -0,0 +1 @@ +1c10f36c5c2bb39ce5dd52af87834d5f0172b31d \ No newline at end of file diff --git a/server/licenses/lucene-sandbox-9.12.1.jar.sha1 b/server/licenses/lucene-sandbox-9.12.1.jar.sha1 deleted file mode 100644 index c1d613e23f1fe..0000000000000 --- a/server/licenses/lucene-sandbox-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -1a66485629d60779f039fc26360f4374ef1496e7 \ No newline at end of file diff --git a/server/licenses/lucene-spatial-extras-10.1.0.jar.sha1 b/server/licenses/lucene-spatial-extras-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..5af6c28bf8d91 --- /dev/null +++ b/server/licenses/lucene-spatial-extras-10.1.0.jar.sha1 @@ -0,0 +1 @@ +601a8e6f9be21ad99fbc43d8fe222553e55ccf80 \ No newline at end of file diff --git a/server/licenses/lucene-spatial-extras-9.12.1.jar.sha1 b/server/licenses/lucene-spatial-extras-9.12.1.jar.sha1 deleted file mode 100644 index c38b794ce9948..0000000000000 --- a/server/licenses/lucene-spatial-extras-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -0a7379410eff21676472adc8ea76a57891ec83c2 \ No newline at end of file diff --git a/server/licenses/lucene-spatial3d-10.1.0.jar.sha1 b/server/licenses/lucene-spatial3d-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..6259680f363a5 --- /dev/null +++ b/server/licenses/lucene-spatial3d-10.1.0.jar.sha1 @@ -0,0 +1 @@ +59819cdc2b1af71dcb215284a2f48076b6d4235a \ No newline at end of file diff --git a/server/licenses/lucene-spatial3d-9.12.1.jar.sha1 b/server/licenses/lucene-spatial3d-9.12.1.jar.sha1 deleted file mode 100644 index bc327a8cec830..0000000000000 --- a/server/licenses/lucene-spatial3d-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -d2fdea4edabb1f616f494999651c43abfd0aa124 \ No newline at end of file diff --git a/server/licenses/lucene-suggest-10.1.0.jar.sha1 b/server/licenses/lucene-suggest-10.1.0.jar.sha1 new file mode 100644 index 0000000000000..5d479ef0838ee --- /dev/null +++ b/server/licenses/lucene-suggest-10.1.0.jar.sha1 @@ -0,0 +1 @@ +84c212cf954bb48b022511d63b362d9a0a282ecf \ No newline at end of file diff --git a/server/licenses/lucene-suggest-9.12.1.jar.sha1 b/server/licenses/lucene-suggest-9.12.1.jar.sha1 deleted file mode 100644 index 961f6da619149..0000000000000 --- a/server/licenses/lucene-suggest-9.12.1.jar.sha1 +++ /dev/null @@ -1 +0,0 @@ -0660e0996ec7653fe0c13c608137e264645eecac \ No newline at end of file diff --git a/server/src/internalClusterTest/java/org/opensearch/action/IndicesRequestIT.java b/server/src/internalClusterTest/java/org/opensearch/action/IndicesRequestIT.java index 927a79d4884ef..26898b70b8ba2 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/IndicesRequestIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/IndicesRequestIT.java @@ -606,7 +606,7 @@ public void testSearchQueryThenFetch() throws Exception { SearchRequest searchRequest = new SearchRequest(randomIndicesOrAliases).searchType(SearchType.QUERY_THEN_FETCH); SearchResponse searchResponse = internalCluster().coordOnlyNodeClient().search(searchRequest).actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, greaterThan(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), greaterThan(0L)); clearInterceptedActions(); assertSameIndices(searchRequest, SearchTransportService.QUERY_ACTION_NAME, SearchTransportService.FETCH_ID_ACTION_NAME); @@ -631,7 +631,7 @@ public void testSearchDfsQueryThenFetch() throws Exception { SearchRequest searchRequest = new SearchRequest(randomIndicesOrAliases).searchType(SearchType.DFS_QUERY_THEN_FETCH); SearchResponse searchResponse = internalCluster().coordOnlyNodeClient().search(searchRequest).actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, greaterThan(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), greaterThan(0L)); clearInterceptedActions(); assertSameIndices( diff --git a/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/TasksIT.java b/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/TasksIT.java index 4d8c80954cd0a..b929919ddab0a 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/TasksIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/TasksIT.java @@ -847,13 +847,13 @@ public void testTaskStoringSuccessfulResult() throws Exception { .setSource(SearchSourceBuilder.searchSource().query(QueryBuilders.termQuery("task.action", taskInfo.getAction()))) .get(); - assertEquals(1L, searchResponse.getHits().getTotalHits().value); + assertEquals(1L, searchResponse.getHits().getTotalHits().value()); searchResponse = client().prepareSearch(TaskResultsService.TASK_INDEX) .setSource(SearchSourceBuilder.searchSource().query(QueryBuilders.termQuery("task.node", taskInfo.getTaskId().getNodeId()))) .get(); - assertEquals(1L, searchResponse.getHits().getTotalHits().value); + assertEquals(1L, searchResponse.getHits().getTotalHits().value()); GetTaskResponse getResponse = expectFinishedTask(taskId); assertEquals(result, getResponse.getTask().getResponseAsMap()); diff --git a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/CreateIndexIT.java b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/CreateIndexIT.java index bd3c9e1456074..c44215fc2f327 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/CreateIndexIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/CreateIndexIT.java @@ -317,8 +317,8 @@ public void onFailure(Exception e) { .setQuery(new RangeQueryBuilder("index_version").from(indexVersion.get(), true)) .get(); SearchResponse all = client().prepareSearch("test").setIndicesOptions(IndicesOptions.lenientExpandOpen()).get(); - assertEquals(expected + " vs. " + all, expected.getHits().getTotalHits().value, all.getHits().getTotalHits().value); - logger.info("total: {}", expected.getHits().getTotalHits().value); + assertEquals(expected + " vs. " + all, expected.getHits().getTotalHits().value(), all.getHits().getTotalHits().value()); + logger.info("total: {}", expected.getHits().getTotalHits().value()); } public void testRestartIndexCreationAfterFullClusterRestart() throws Exception { diff --git a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/RemoteSplitIndexIT.java b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/RemoteSplitIndexIT.java index 3038db4583cf4..dedabc7bafd00 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/RemoteSplitIndexIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/RemoteSplitIndexIT.java @@ -339,7 +339,7 @@ public void assertNested(String index, int numDocs) { .setQuery(nestedQuery("nested1", termQuery("nested1.n_field1", "n_value1_1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) numDocs)); } public void assertAllUniqueDocs(SearchResponse response, int numDocs) { diff --git a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/SplitIndexIT.java b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/SplitIndexIT.java index c8b151e24ce98..0cf0dff0fc378 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/SplitIndexIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/admin/indices/create/SplitIndexIT.java @@ -302,7 +302,7 @@ public void assertNested(String index, int numDocs) { .setQuery(nestedQuery("nested1", termQuery("nested1.n_field1", "n_value1_1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) numDocs)); } public void assertAllUniqueDocs(SearchResponse response, int numDocs) { diff --git a/server/src/internalClusterTest/java/org/opensearch/action/bulk/BulkProcessorRetryIT.java b/server/src/internalClusterTest/java/org/opensearch/action/bulk/BulkProcessorRetryIT.java index cd6cb0ca3b172..a020e5246b8fb 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/bulk/BulkProcessorRetryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/bulk/BulkProcessorRetryIT.java @@ -160,11 +160,11 @@ public void afterBulk(long executionId, BulkRequest request, Throwable failure) SearchResponse results = client().prepareSearch(INDEX_NAME).setQuery(QueryBuilders.matchAllQuery()).setSize(0).get(); if (rejectedExecutionExpected) { - assertThat((int) results.getHits().getTotalHits().value, lessThanOrEqualTo(numberOfAsyncOps)); + assertThat((int) results.getHits().getTotalHits().value(), lessThanOrEqualTo(numberOfAsyncOps)); } else if (rejectedAfterAllRetries) { - assertThat((int) results.getHits().getTotalHits().value, lessThan(numberOfAsyncOps)); + assertThat((int) results.getHits().getTotalHits().value(), lessThan(numberOfAsyncOps)); } else { - assertThat((int) results.getHits().getTotalHits().value, equalTo(numberOfAsyncOps)); + assertThat((int) results.getHits().getTotalHits().value(), equalTo(numberOfAsyncOps)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/action/search/TransportSearchIT.java b/server/src/internalClusterTest/java/org/opensearch/action/search/TransportSearchIT.java index b1934f901ac65..1e85f449eb9aa 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/search/TransportSearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/search/TransportSearchIT.java @@ -154,7 +154,7 @@ public void testLocalClusterAlias() { randomBoolean() ); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); SearchHit[] hits = searchResponse.getHits().getHits(); assertEquals(1, hits.length); SearchHit hit = hits[0]; @@ -171,7 +171,7 @@ public void testLocalClusterAlias() { randomBoolean() ); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); SearchHit[] hits = searchResponse.getHits().getHits(); assertEquals(1, hits.length); SearchHit hit = hits[0]; @@ -201,7 +201,7 @@ public void testAbsoluteStartMillis() { { SearchRequest searchRequest = new SearchRequest(); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); } { SearchRequest searchRequest = new SearchRequest(""); @@ -212,13 +212,13 @@ public void testAbsoluteStartMillis() { { SearchRequest searchRequest = SearchRequest.subSearchRequest(new SearchRequest(), Strings.EMPTY_ARRAY, "", 0, randomBoolean()); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); } { SearchRequest searchRequest = SearchRequest.subSearchRequest(new SearchRequest(), Strings.EMPTY_ARRAY, "", 0, randomBoolean()); searchRequest.indices(""); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); assertEquals("test-1970.01.01", searchResponse.getHits().getHits()[0].getIndex()); } { @@ -230,7 +230,7 @@ public void testAbsoluteStartMillis() { sourceBuilder.query(rangeQuery); searchRequest.source(sourceBuilder); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); assertEquals("test-1970.01.01", searchResponse.getHits().getHits()[0].getIndex()); } } @@ -267,7 +267,7 @@ public void testFinalReduce() { ? originalRequest : SearchRequest.subSearchRequest(originalRequest, Strings.EMPTY_ARRAY, "remote", nowInMillis, true); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); Aggregations aggregations = searchResponse.getAggregations(); LongTerms longTerms = aggregations.get("terms"); assertEquals(1, longTerms.getBuckets().size()); @@ -281,7 +281,7 @@ public void testFinalReduce() { false ); SearchResponse searchResponse = client().search(searchRequest).actionGet(); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); Aggregations aggregations = searchResponse.getAggregations(); LongTerms longTerms = aggregations.get("terms"); assertEquals(2, longTerms.getBuckets().size()); @@ -366,7 +366,7 @@ public void testSearchIdle() throws Exception { .setQuery(new RangeQueryBuilder("created_date").gte("2020-01-02").lte("2020-01-03")) .setPreFilterShardSize(randomIntBetween(1, 3)) .get(); - assertThat(resp.getHits().getTotalHits().value, equalTo(2L)); + assertThat(resp.getHits().getTotalHits().value(), equalTo(2L)); }); } diff --git a/server/src/internalClusterTest/java/org/opensearch/action/support/clustermanager/IndexingClusterManagerFailoverIT.java b/server/src/internalClusterTest/java/org/opensearch/action/support/clustermanager/IndexingClusterManagerFailoverIT.java index 959b16d4c4694..f3293e6d8f3f6 100644 --- a/server/src/internalClusterTest/java/org/opensearch/action/support/clustermanager/IndexingClusterManagerFailoverIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/action/support/clustermanager/IndexingClusterManagerFailoverIT.java @@ -126,7 +126,7 @@ public void run() { ensureGreen("myindex"); refresh(); - assertThat(client().prepareSearch("myindex").get().getHits().getTotalHits().value, equalTo(10L)); + assertThat(client().prepareSearch("myindex").get().getHits().getTotalHits().value(), equalTo(10L)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/aliases/IndexAliasesIT.java b/server/src/internalClusterTest/java/org/opensearch/aliases/IndexAliasesIT.java index f91df19232971..292a13bdf7f5c 100644 --- a/server/src/internalClusterTest/java/org/opensearch/aliases/IndexAliasesIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/aliases/IndexAliasesIT.java @@ -406,7 +406,7 @@ public void testSearchingFilteringAliasesTwoIndices() throws Exception { SearchResponse searchResponse = client().prepareSearch("foos").setQuery(QueryBuilders.matchAllQuery()).get(); assertHits(searchResponse.getHits(), "1", "5"); assertThat( - client().prepareSearch("foos").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value, + client().prepareSearch("foos").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value(), equalTo(2L) ); @@ -414,7 +414,7 @@ public void testSearchingFilteringAliasesTwoIndices() throws Exception { searchResponse = client().prepareSearch("bars").setQuery(QueryBuilders.matchAllQuery()).get(); assertHits(searchResponse.getHits(), "2"); assertThat( - client().prepareSearch("bars").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value, + client().prepareSearch("bars").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value(), equalTo(1L) ); @@ -422,7 +422,13 @@ public void testSearchingFilteringAliasesTwoIndices() throws Exception { searchResponse = client().prepareSearch("foos", "test1").setQuery(QueryBuilders.matchAllQuery()).get(); assertHits(searchResponse.getHits(), "1", "2", "3", "4", "5"); assertThat( - client().prepareSearch("foos", "test1").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value, + client().prepareSearch("foos", "test1") + .setSize(0) + .setQuery(QueryBuilders.matchAllQuery()) + .get() + .getHits() + .getTotalHits() + .value(), equalTo(5L) ); @@ -435,20 +441,22 @@ public void testSearchingFilteringAliasesTwoIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(5L) ); logger.info("--> checking filtering alias for two indices and non-filtering alias for both indices"); searchResponse = client().prepareSearch("foos", "aliasToTests").setQuery(QueryBuilders.matchAllQuery()).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(8L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(8L)); assertThat( client().prepareSearch("foos", "aliasToTests") .setSize(0) .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(8L) ); @@ -461,7 +469,8 @@ public void testSearchingFilteringAliasesTwoIndices() throws Exception { .setQuery(QueryBuilders.termQuery("name", "something")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -526,7 +535,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(4L) ); @@ -538,7 +548,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(5L) ); @@ -550,7 +561,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(4L) ); @@ -562,7 +574,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(6L) ); @@ -574,7 +587,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(6L) ); @@ -586,7 +600,8 @@ public void testSearchingFilteringAliasesMultipleIndices() throws Exception { .setQuery(QueryBuilders.matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(8L) ); } @@ -641,7 +656,7 @@ public void testDeletingByQueryFilteringAliases() throws Exception { logger.info("--> checking counts before delete"); assertThat( - client().prepareSearch("bars").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value, + client().prepareSearch("bars").setSize(0).setQuery(QueryBuilders.matchAllQuery()).get().getHits().getTotalHits().value(), equalTo(1L) ); } @@ -1457,7 +1472,7 @@ private void checkAliases() { } private void assertHits(SearchHits hits, String... ids) { - assertThat(hits.getTotalHits().value, equalTo((long) ids.length)); + assertThat(hits.getTotalHits().value(), equalTo((long) ids.length)); Set hitIds = new HashSet<>(); for (SearchHit hit : hits.getHits()) { hitIds.add(hit.getId()); diff --git a/server/src/internalClusterTest/java/org/opensearch/broadcast/BroadcastActionsIT.java b/server/src/internalClusterTest/java/org/opensearch/broadcast/BroadcastActionsIT.java index 96b8c63f6c2ba..cae778b73d16a 100644 --- a/server/src/internalClusterTest/java/org/opensearch/broadcast/BroadcastActionsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/broadcast/BroadcastActionsIT.java @@ -67,7 +67,7 @@ public void testBroadcastOperations() throws IOException { for (int i = 0; i < 5; i++) { // test successful SearchResponse countResponse = client().prepareSearch("test").setSize(0).setQuery(matchAllQuery()).get(); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(countResponse.getTotalShards(), equalTo(numShards.numPrimaries)); assertThat(countResponse.getSuccessfulShards(), equalTo(numShards.numPrimaries)); assertThat(countResponse.getFailedShards(), equalTo(0)); diff --git a/server/src/internalClusterTest/java/org/opensearch/cluster/MinimumClusterManagerNodesIT.java b/server/src/internalClusterTest/java/org/opensearch/cluster/MinimumClusterManagerNodesIT.java index 84648eda3d38c..6db2a02dbc846 100644 --- a/server/src/internalClusterTest/java/org/opensearch/cluster/MinimumClusterManagerNodesIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/cluster/MinimumClusterManagerNodesIT.java @@ -146,7 +146,8 @@ public void testTwoNodesNoClusterManagerBlock() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(100L) ); } diff --git a/server/src/internalClusterTest/java/org/opensearch/cluster/allocation/FilteringAllocationIT.java b/server/src/internalClusterTest/java/org/opensearch/cluster/allocation/FilteringAllocationIT.java index ff95cca5ffde9..3700f2d963577 100644 --- a/server/src/internalClusterTest/java/org/opensearch/cluster/allocation/FilteringAllocationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/cluster/allocation/FilteringAllocationIT.java @@ -82,7 +82,8 @@ public void testDecommissionNodeNoReplicas() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(100L) ); @@ -123,7 +124,8 @@ public void testDecommissionNodeNoReplicas() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(100L) ); } @@ -197,7 +199,8 @@ public void testDisablingAllocationFiltering() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(100L) ); diff --git a/server/src/internalClusterTest/java/org/opensearch/discovery/ClusterDisruptionCleanSettingsIT.java b/server/src/internalClusterTest/java/org/opensearch/discovery/ClusterDisruptionCleanSettingsIT.java index 39a4f2aa82828..b9251e61c4d2d 100644 --- a/server/src/internalClusterTest/java/org/opensearch/discovery/ClusterDisruptionCleanSettingsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/discovery/ClusterDisruptionCleanSettingsIT.java @@ -87,6 +87,6 @@ public void testSearchWithRelocationAndSlowClusterStateProcessing() throws Excep IndicesStoreIntegrationIT.relocateAndBlockCompletion(logger, "test", 0, node_1, node_2); // now search for the documents and see if we get a reply - assertThat(client().prepareSearch().setSize(0).get().getHits().getTotalHits().value, equalTo(100L)); + assertThat(client().prepareSearch().setSize(0).get().getHits().getTotalHits().value(), equalTo(100L)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/document/DocumentActionsIT.java b/server/src/internalClusterTest/java/org/opensearch/document/DocumentActionsIT.java index 0336ccf3f4647..b92d9c9142b2d 100644 --- a/server/src/internalClusterTest/java/org/opensearch/document/DocumentActionsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/document/DocumentActionsIT.java @@ -180,7 +180,7 @@ public void testIndexActions() throws Exception { // test successful SearchResponse countResponse = client().prepareSearch("test").setSize(0).setQuery(matchAllQuery()).execute().actionGet(); assertNoFailures(countResponse); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(countResponse.getSuccessfulShards(), equalTo(numShards.numPrimaries)); assertThat(countResponse.getFailedShards(), equalTo(0)); @@ -191,7 +191,7 @@ public void testIndexActions() throws Exception { countResponse.getShardFailures() == null ? 0 : countResponse.getShardFailures().length, equalTo(0) ); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(countResponse.getSuccessfulShards(), equalTo(numShards.numPrimaries)); assertThat(countResponse.getFailedShards(), equalTo(0)); } diff --git a/server/src/internalClusterTest/java/org/opensearch/index/FinalPipelineIT.java b/server/src/internalClusterTest/java/org/opensearch/index/FinalPipelineIT.java index 03b8fb5ff7afc..d9b5b68ba3548 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/FinalPipelineIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/FinalPipelineIT.java @@ -140,7 +140,7 @@ public void testFinalPipelineOfOldDestinationIsNotInvoked() { .get(); assertEquals(RestStatus.CREATED, indexResponse.status()); SearchResponse target = client().prepareSearch("target").get(); - assertEquals(1, target.getHits().getTotalHits().value); + assertEquals(1, target.getHits().getTotalHits().value()); assertFalse(target.getHits().getAt(0).getSourceAsMap().containsKey("final")); } @@ -170,7 +170,7 @@ public void testFinalPipelineOfNewDestinationIsInvoked() { .get(); assertEquals(RestStatus.CREATED, indexResponse.status()); SearchResponse target = client().prepareSearch("target").get(); - assertEquals(1, target.getHits().getTotalHits().value); + assertEquals(1, target.getHits().getTotalHits().value()); assertEquals(true, target.getHits().getAt(0).getSourceAsMap().get("final")); } @@ -200,7 +200,7 @@ public void testDefaultPipelineOfNewDestinationIsNotInvoked() { .get(); assertEquals(RestStatus.CREATED, indexResponse.status()); SearchResponse target = client().prepareSearch("target").get(); - assertEquals(1, target.getHits().getTotalHits().value); + assertEquals(1, target.getHits().getTotalHits().value()); assertFalse(target.getHits().getAt(0).getSourceAsMap().containsKey("final")); } diff --git a/server/src/internalClusterTest/java/org/opensearch/index/engine/MaxDocsLimitIT.java b/server/src/internalClusterTest/java/org/opensearch/index/engine/MaxDocsLimitIT.java index 8321630d34229..ac43f900f8613 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/engine/MaxDocsLimitIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/engine/MaxDocsLimitIT.java @@ -136,7 +136,7 @@ public void testMaxDocsLimit() throws Exception { .setSize(0) .get(); OpenSearchAssertions.assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) maxDocs.get())); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) maxDocs.get())); if (randomBoolean()) { client().admin().indices().prepareFlush("test").get(); } @@ -149,7 +149,7 @@ public void testMaxDocsLimit() throws Exception { .setSize(0) .get(); OpenSearchAssertions.assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) maxDocs.get())); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) maxDocs.get())); } public void testMaxDocsLimitConcurrently() throws Exception { @@ -167,7 +167,7 @@ public void testMaxDocsLimitConcurrently() throws Exception { .setSize(0) .get(); OpenSearchAssertions.assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) indexingResult.numSuccess)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) indexingResult.numSuccess)); int totalSuccess = indexingResult.numSuccess; while (totalSuccess < maxDocs.get()) { indexingResult = indexDocs(between(1, 10), between(1, 8)); @@ -185,7 +185,7 @@ public void testMaxDocsLimitConcurrently() throws Exception { .setSize(0) .get(); OpenSearchAssertions.assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) totalSuccess)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) totalSuccess)); } static final class IndexingResult { diff --git a/server/src/internalClusterTest/java/org/opensearch/index/mapper/CopyToMapperIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/index/mapper/CopyToMapperIntegrationIT.java index 0df84261ade63..636c84ad94dc2 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/mapper/CopyToMapperIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/mapper/CopyToMapperIntegrationIT.java @@ -69,7 +69,7 @@ public void testDynamicTemplateCopyTo() throws Exception { .execute() .actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) recordCount)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) recordCount)); assertThat(((Terms) response.getAggregations().get("test")).getBuckets().size(), equalTo(recordCount + 1)); assertThat(((Terms) response.getAggregations().get("test_raw")).getBuckets().size(), equalTo(recordCount)); @@ -90,7 +90,7 @@ public void testDynamicObjectCopyTo() throws Exception { client().prepareIndex("test-idx").setId("1").setSource("foo", "bar").get(); client().admin().indices().prepareRefresh("test-idx").execute().actionGet(); SearchResponse response = client().prepareSearch("test-idx").setQuery(QueryBuilders.termQuery("root.top.child", "bar")).get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); } private XContentBuilder createDynamicTemplateMapping() throws IOException { diff --git a/server/src/internalClusterTest/java/org/opensearch/index/mapper/ExternalValuesMapperIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/index/mapper/ExternalValuesMapperIntegrationIT.java index 3734bbbe8aa6c..3b8ea42efd253 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/mapper/ExternalValuesMapperIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/mapper/ExternalValuesMapperIntegrationIT.java @@ -83,7 +83,7 @@ public void testHighlightingOnCustomString() throws Exception { .execute() .actionGet(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getHighlightFields().size(), equalTo(0)); // make sure it is not excluded when we explicitly provide the fieldname @@ -93,7 +93,7 @@ public void testHighlightingOnCustomString() throws Exception { .execute() .actionGet(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getHighlightFields().size(), equalTo(1)); assertThat( response.getHits().getAt(0).getHighlightFields().get("field").fragments()[0].string(), @@ -107,7 +107,7 @@ public void testHighlightingOnCustomString() throws Exception { .execute() .actionGet(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getHighlightFields().size(), equalTo(1)); assertThat( response.getHits().getAt(0).getHighlightFields().get("field").fragments()[0].string(), @@ -136,14 +136,14 @@ public void testExternalValues() throws Exception { response = client().prepareSearch("test-idx").setPostFilter(QueryBuilders.termQuery("field.bool", "true")).execute().actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) 1)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) 1)); response = client().prepareSearch("test-idx") .setPostFilter(QueryBuilders.geoDistanceQuery("field.point").point(42.0, 51.0).distance("1km")) .execute() .actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) 1)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) 1)); response = client().prepareSearch("test-idx") .setPostFilter( @@ -153,11 +153,11 @@ public void testExternalValues() throws Exception { .execute() .actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) 1)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) 1)); response = client().prepareSearch("test-idx").setPostFilter(QueryBuilders.termQuery("field.field", "foo")).execute().actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) 1)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) 1)); } public void testExternalValuesWithMultifield() throws Exception { @@ -192,6 +192,6 @@ public void testExternalValuesWithMultifield() throws Exception { .execute() .actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) 1)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) 1)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/index/mapper/MultiFieldsIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/index/mapper/MultiFieldsIntegrationIT.java index 50440f9527a6a..fd11a7e0031f4 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/mapper/MultiFieldsIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/mapper/MultiFieldsIntegrationIT.java @@ -72,9 +72,9 @@ public void testMultiFields() throws Exception { client().prepareIndex("my-index").setId("1").setSource("title", "Multi fields").setRefreshPolicy(IMMEDIATE).get(); SearchResponse searchResponse = client().prepareSearch("my-index").setQuery(matchQuery("title", "multi")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("my-index").setQuery(matchQuery("title.not_analyzed", "Multi fields")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertAcked(client().admin().indices().preparePutMapping("my-index").setSource(createPutMappingSource())); @@ -93,7 +93,7 @@ public void testMultiFields() throws Exception { client().prepareIndex("my-index").setId("1").setSource("title", "Multi fields").setRefreshPolicy(IMMEDIATE).get(); searchResponse = client().prepareSearch("my-index").setQuery(matchQuery("title.uncased", "Multi")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } @SuppressWarnings("unchecked") @@ -120,9 +120,9 @@ public void testGeoPointMultiField() throws Exception { .setSize(0) .setQuery(constantScoreQuery(geoDistanceQuery("a").point(51, 19).distance(50, DistanceUnit.KILOMETERS))) .get(); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(1L)); countResponse = client().prepareSearch("my-index").setSize(0).setQuery(matchQuery("a.b", point.geohash())).get(); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(1L)); } @SuppressWarnings("unchecked") @@ -144,7 +144,7 @@ public void testCompletionMultiField() throws Exception { client().prepareIndex("my-index").setId("1").setSource("a", "complete me").setRefreshPolicy(IMMEDIATE).get(); SearchResponse countResponse = client().prepareSearch("my-index").setSize(0).setQuery(matchQuery("a.b", "complete me")).get(); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(1L)); } @SuppressWarnings("unchecked") @@ -166,7 +166,7 @@ public void testIpMultiField() throws Exception { client().prepareIndex("my-index").setId("1").setSource("a", "127.0.0.1").setRefreshPolicy(IMMEDIATE).get(); SearchResponse countResponse = client().prepareSearch("my-index").setSize(0).setQuery(matchQuery("a.b", "127.0.0.1")).get(); - assertThat(countResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(countResponse.getHits().getTotalHits().value(), equalTo(1L)); } private XContentBuilder createMappingSource(String fieldType) throws IOException { diff --git a/server/src/internalClusterTest/java/org/opensearch/index/mapper/StarTreeMapperIT.java b/server/src/internalClusterTest/java/org/opensearch/index/mapper/StarTreeMapperIT.java index 1d01f717aad1f..007fefa1499ab 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/mapper/StarTreeMapperIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/mapper/StarTreeMapperIT.java @@ -882,7 +882,7 @@ public void testCompositeIndexWithArraysInNestedCompositeFieldSameNameAsNormalFi // Verify the document was indexed SearchResponse searchResponse = client().prepareSearch(TEST_INDEX).setQuery(QueryBuilders.matchAllQuery()).get(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); // Verify the values in the indexed document SearchHit hit = searchResponse.getHits().getAt(0); @@ -921,7 +921,7 @@ public void testCompositeIndexWithNestedArraysInNonCompositeField() throws IOExc // Verify the document was indexed SearchResponse searchResponse = client().prepareSearch(TEST_INDEX).setQuery(QueryBuilders.matchAllQuery()).get(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); // Verify the values in the indexed document SearchHit hit = searchResponse.getHits().getAt(0); @@ -958,7 +958,7 @@ public void testCompositeIndexWithArraysInNonCompositeField() throws IOException // Verify the document was indexed SearchResponse searchResponse = client().prepareSearch(TEST_INDEX).setQuery(QueryBuilders.matchAllQuery()).get(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); // Verify the values in the indexed document SearchHit hit = searchResponse.getHits().getAt(0); diff --git a/server/src/internalClusterTest/java/org/opensearch/index/shard/IndexShardIT.java b/server/src/internalClusterTest/java/org/opensearch/index/shard/IndexShardIT.java index f97950f2652a3..7fd219a3dd9dc 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/shard/IndexShardIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/shard/IndexShardIT.java @@ -753,7 +753,7 @@ public void testInvalidateIndicesRequestCacheWhenRollbackEngine() throws Excepti } } shard.refresh("test"); - assertThat(client().search(countRequest).actionGet().getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(client().search(countRequest).actionGet().getHits().getTotalHits().value(), equalTo(numDocs)); assertThat(shard.getLocalCheckpoint(), equalTo(shard.seqNoStats().getMaxSeqNo())); final CountDownLatch engineResetLatch = new CountDownLatch(1); @@ -784,7 +784,7 @@ public void testInvalidateIndicesRequestCacheWhenRollbackEngine() throws Excepti } assertThat( "numDocs=" + numDocs + " moreDocs=" + moreDocs, - client().search(countRequest).actionGet().getHits().getTotalHits().value, + client().search(countRequest).actionGet().getHits().getTotalHits().value(), equalTo(numDocs + moreDocs) ); } diff --git a/server/src/internalClusterTest/java/org/opensearch/index/shard/SearchIdleIT.java b/server/src/internalClusterTest/java/org/opensearch/index/shard/SearchIdleIT.java index 43d86b232de77..d47716ee268fd 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/shard/SearchIdleIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/shard/SearchIdleIT.java @@ -58,7 +58,7 @@ public class SearchIdleIT extends OpenSearchSingleNodeTestCase { public void testAutomaticRefreshSearch() throws InterruptedException { - runTestAutomaticRefresh(numDocs -> client().prepareSearch("test").get().getHits().getTotalHits().value); + runTestAutomaticRefresh(numDocs -> client().prepareSearch("test").get().getHits().getTotalHits().value()); } public void testAutomaticRefreshGet() throws InterruptedException { diff --git a/server/src/internalClusterTest/java/org/opensearch/index/store/ExceptionRetryIT.java b/server/src/internalClusterTest/java/org/opensearch/index/store/ExceptionRetryIT.java index 1e3c22c85b0b2..6f6b840c92299 100644 --- a/server/src/internalClusterTest/java/org/opensearch/index/store/ExceptionRetryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/index/store/ExceptionRetryIT.java @@ -152,7 +152,7 @@ public void testRetryDueToExceptionOnNetworkLayer() throws ExecutionException, I .setQuery(termQuery("_id", searchResponse.getHits().getHits()[i].getId())) .setExplain(true) .get(); - assertThat(dupIdResponse.getHits().getTotalHits().value, greaterThan(1L)); + assertThat(dupIdResponse.getHits().getTotalHits().value(), greaterThan(1L)); logger.info("found a duplicate id:"); for (SearchHit hit : dupIdResponse.getHits()) { logger.info("Doc {} was found on shard {}", hit.getId(), hit.getShard().getShardId()); diff --git a/server/src/internalClusterTest/java/org/opensearch/indexing/IndexActionIT.java b/server/src/internalClusterTest/java/org/opensearch/indexing/IndexActionIT.java index 8cb54631b593f..c6718500f14fe 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indexing/IndexActionIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indexing/IndexActionIT.java @@ -95,9 +95,9 @@ public void testAutoGenerateIdNoDuplicates() throws Exception { try { logger.debug("running search with all types"); SearchResponse response = client().prepareSearch("test").get(); - if (response.getHits().getTotalHits().value != numOfDocs) { + if (response.getHits().getTotalHits().value() != numOfDocs) { final String message = "Count is " - + response.getHits().getTotalHits().value + + response.getHits().getTotalHits().value() + " but " + numOfDocs + " was expected. " @@ -114,9 +114,9 @@ public void testAutoGenerateIdNoDuplicates() throws Exception { try { logger.debug("running search with a specific type"); SearchResponse response = client().prepareSearch("test").get(); - if (response.getHits().getTotalHits().value != numOfDocs) { + if (response.getHits().getTotalHits().value() != numOfDocs) { final String message = "Count is " - + response.getHits().getTotalHits().value + + response.getHits().getTotalHits().value() + " but " + numOfDocs + " was expected. " diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheCleanupIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheCleanupIT.java index 988ea99130b13..e178ff063205a 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheCleanupIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheCleanupIT.java @@ -83,7 +83,7 @@ public void testCacheWithInvalidation() throws Exception { SearchResponse resp = client.prepareSearch(index).setRequestCache(true).setQuery(QueryBuilders.termQuery("k", "hello")).get(); assertSearchResponse(resp); OpenSearchAssertions.assertAllSuccessful(resp); - assertThat(resp.getHits().getTotalHits().value, equalTo(1L)); + assertThat(resp.getHits().getTotalHits().value(), equalTo(1L)); assertCacheState(client, index, 0, 1); // Index but don't refresh diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheIT.java index a16d2065598ba..1e46124115953 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/IndicesRequestCacheIT.java @@ -242,7 +242,7 @@ public void testQueryRewrite() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 5); final SearchResponse r2 = client.prepareSearch(index) @@ -252,7 +252,7 @@ public void testQueryRewrite() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 3, 7); final SearchResponse r3 = client.prepareSearch(index) @@ -262,7 +262,7 @@ public void testQueryRewrite() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 6, 9); } @@ -310,7 +310,7 @@ public void testQueryRewriteMissingValues() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-19").lte("2016-03-28")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, index, 0, 1); final SearchResponse r2 = client.prepareSearch(index) @@ -319,7 +319,7 @@ public void testQueryRewriteMissingValues() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-19").lte("2016-03-28")) .get(); OpenSearchAssertions.assertAllSuccessful(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, index, 1, 1); final SearchResponse r3 = client.prepareSearch(index) @@ -328,7 +328,7 @@ public void testQueryRewriteMissingValues() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-19").lte("2016-03-28")) .get(); OpenSearchAssertions.assertAllSuccessful(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, index, 2, 1); } @@ -378,7 +378,7 @@ public void testQueryRewriteDates() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(9L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(9L)); assertCacheState(client, index, 0, 1); final SearchResponse r2 = client.prepareSearch(index) @@ -388,7 +388,7 @@ public void testQueryRewriteDates() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(9L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(9L)); assertCacheState(client, index, 1, 1); final SearchResponse r3 = client.prepareSearch(index) @@ -398,7 +398,7 @@ public void testQueryRewriteDates() throws Exception { .addAggregation(new GlobalAggregationBuilder("global")) .get(); OpenSearchAssertions.assertAllSuccessful(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(9L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(9L)); assertCacheState(client, index, 2, 1); } @@ -453,7 +453,7 @@ public void testQueryRewriteDatesWithNow() throws Exception { .setQuery(QueryBuilders.rangeQuery("d").gte("now-7d/d").lte("now")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, "index-1", 0, 1); assertCacheState(client, "index-2", 0, 1); // Because the query will INTERSECT with the 3rd index it will not be @@ -467,7 +467,7 @@ public void testQueryRewriteDatesWithNow() throws Exception { .setQuery(QueryBuilders.rangeQuery("d").gte("now-7d/d").lte("now")) .get(); OpenSearchAssertions.assertAllSuccessful(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, "index-1", 1, 1); assertCacheState(client, "index-2", 1, 1); assertCacheState(client, "index-3", 0, 0); @@ -478,7 +478,7 @@ public void testQueryRewriteDatesWithNow() throws Exception { .setQuery(QueryBuilders.rangeQuery("d").gte("now-7d/d").lte("now")) .get(); OpenSearchAssertions.assertAllSuccessful(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(8L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(8L)); assertCacheState(client, "index-1", 2, 1); assertCacheState(client, "index-2", 2, 1); assertCacheState(client, "index-3", 0, 0); @@ -522,7 +522,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-19").lte("2016-03-25")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 0); // If search type is DFS_QUERY_THEN_FETCH we should not cache @@ -532,7 +532,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-20").lte("2016-03-26")) .get(); OpenSearchAssertions.assertAllSuccessful(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 0); // If search type is DFS_QUERY_THEN_FETCH we should not cache even if @@ -544,7 +544,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-20").lte("2016-03-26")) .get(); OpenSearchAssertions.assertAllSuccessful(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 0); // If the request has an non-filter aggregation containing now we should not cache @@ -556,7 +556,7 @@ public void testCanCache() throws Exception { .addAggregation(dateRange("foo").field("s").addRange("now-10y", "now")) .get(); OpenSearchAssertions.assertAllSuccessful(r5); - assertThat(r5.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r5.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 0); // If size > 1 and cache flag is set on the request we should cache @@ -567,7 +567,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-21").lte("2016-03-27")) .get(); OpenSearchAssertions.assertAllSuccessful(r6); - assertThat(r6.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r6.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 2); // If the request has a filter aggregation containing now we should cache since it gets rewritten @@ -579,7 +579,7 @@ public void testCanCache() throws Exception { .addAggregation(filter("foo", QueryBuilders.rangeQuery("s").from("now-10y").to("now"))) .get(); OpenSearchAssertions.assertAllSuccessful(r4); - assertThat(r4.getHits().getTotalHits().value, equalTo(7L)); + assertThat(r4.getHits().getTotalHits().value(), equalTo(7L)); assertCacheState(client, index, 0, 4); // Update max cacheable size for request cache from default value of 0 @@ -597,7 +597,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-22").lte("2016-03-26")) .get(); OpenSearchAssertions.assertAllSuccessful(r7); - assertThat(r7.getHits().getTotalHits().value, equalTo(5L)); + assertThat(r7.getHits().getTotalHits().value(), equalTo(5L)); assertCacheState(client, index, 0, 6); // Sizes > the cluster setting value should not be cached @@ -607,7 +607,7 @@ public void testCanCache() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("2016-03-22").lte("2016-03-26")) .get(); OpenSearchAssertions.assertAllSuccessful(r8); - assertThat(r8.getHits().getTotalHits().value, equalTo(5L)); + assertThat(r8.getHits().getTotalHits().value(), equalTo(5L)); assertCacheState(client, index, 0, 6); } @@ -647,7 +647,7 @@ public void testCacheWithFilteredAlias() throws InterruptedException { .setQuery(QueryBuilders.rangeQuery("created_at").gte("now-7d/d")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(1L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(1L)); assertCacheState(client, index, 0, 1); r1 = client.prepareSearch(index) @@ -656,17 +656,17 @@ public void testCacheWithFilteredAlias() throws InterruptedException { .setQuery(QueryBuilders.rangeQuery("created_at").gte("now-7d/d")) .get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(1L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(1L)); assertCacheState(client, index, 1, 1); r1 = client.prepareSearch("last_week").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(1L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(1L)); assertCacheState(client, index, 1, 2); r1 = client.prepareSearch("last_week").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get(); OpenSearchAssertions.assertAllSuccessful(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(1L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(1L)); assertCacheState(client, index, 2, 2); } @@ -704,7 +704,7 @@ public void testProfileDisableCache() throws Exception { .get(); assertSearchResponse(resp); OpenSearchAssertions.assertAllSuccessful(resp); - assertThat(resp.getHits().getTotalHits().value, equalTo(1L)); + assertThat(resp.getHits().getTotalHits().value(), equalTo(1L)); if (profile == false) { if (i == 1) { expectedMisses++; diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/mapping/UpdateMappingIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/mapping/UpdateMappingIntegrationIT.java index 575c1956d0fda..2c5d408b3b8f6 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/mapping/UpdateMappingIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/mapping/UpdateMappingIntegrationIT.java @@ -122,7 +122,7 @@ public void testDynamicUpdates() throws Exception { RefreshResponse refreshResponse = client().admin().indices().prepareRefresh().execute().actionGet(); assertThat(refreshResponse.getFailedShards(), equalTo(0)); SearchResponse response = client().prepareSearch("test").setSize(0).execute().actionGet(); - assertThat(response.getHits().getTotalHits().value, equalTo((long) recCount)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) recCount)); logger.info("checking all the fields are in the mappings"); diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/recovery/IndexRecoveryIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/recovery/IndexRecoveryIT.java index 68b29851c6c04..d30806b4325ac 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/recovery/IndexRecoveryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/recovery/IndexRecoveryIT.java @@ -841,7 +841,7 @@ private IndicesStatsResponse createAndPopulateIndex(String name, int nodeCount, indexRandom(true, docs); flush(); - assertThat(client().prepareSearch(name).setSize(0).get().getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(client().prepareSearch(name).setSize(0).get().getHits().getTotalHits().value(), equalTo((long) numDocs)); return client().admin().indices().prepareStats(name).execute().actionGet(); } diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationBaseIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationBaseIT.java index be849452c0f5e..8b4913fcb2704 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationBaseIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationBaseIT.java @@ -118,7 +118,7 @@ public static void waitForSearchableDocs(String indexName, long docCount, List { for (String node : nodes) { final SearchResponse response = client(node).prepareSearch(indexName).setSize(0).setPreference("_only_local").get(); - final long hits = response.getHits().getTotalHits().value; + final long hits = response.getHits().getTotalHits().value(); if (hits < docCount) { fail("Expected search hits on node: " + node + " to be at least " + docCount + " but was: " + hits); } diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java index 2421a1a507372..b3fa43bc99e2e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationIT.java @@ -155,7 +155,7 @@ public void testPrimaryStopped_ReplicaPromoted() throws Exception { assertTrue(replicaShardRouting + " should be promoted as a primary", replicaShardRouting.primary()); final SearchResponse response = client(replica).prepareSearch(INDEX_NAME).setSize(0).setPreference("_only_local").get(); // new primary should have at least the doc count from the first set of segments. - assertTrue(response.getHits().getTotalHits().value >= 1); + assertTrue(response.getHits().getTotalHits().value() >= 1); // assert we can index into the new primary. client().prepareIndex(INDEX_NAME).setId("3").setSource("bar", "baz").setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE).get(); diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationRelocationIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationRelocationIT.java index a7be63bc61bc2..a2372670bd16f 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationRelocationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/replication/SegmentReplicationRelocationIT.java @@ -419,7 +419,7 @@ public void testNewlyAddedReplicaIsUpdated() throws Exception { client().prepareIndex(INDEX_NAME).setId(Integer.toString(i)).setSource("field", "value" + i).execute().actionGet(); } refresh(INDEX_NAME); - assertEquals(client().prepareSearch(INDEX_NAME).setSize(0).execute().actionGet().getHits().getTotalHits().value, 20L); + assertEquals(client().prepareSearch(INDEX_NAME).setSize(0).execute().actionGet().getHits().getTotalHits().value(), 20L); logger.info("--> start empty node to add replica shard"); final String replica = internalCluster().startNode(); @@ -460,7 +460,7 @@ public void testAddNewReplicaFailure() throws Exception { } refresh(INDEX_NAME); logger.info("--> verifying count"); - assertEquals(client().prepareSearch(INDEX_NAME).setSize(0).execute().actionGet().getHits().getTotalHits().value, 20L); + assertEquals(client().prepareSearch(INDEX_NAME).setSize(0).execute().actionGet().getHits().getTotalHits().value(), 20L); logger.info("--> start empty node to add replica shard"); final String replica = internalCluster().startNode(); diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/state/CloseWhileRelocatingShardsIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/state/CloseWhileRelocatingShardsIT.java index d6dce78061a7a..d4df608547a9e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/state/CloseWhileRelocatingShardsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/state/CloseWhileRelocatingShardsIT.java @@ -288,7 +288,7 @@ public void testCloseWhileRelocatingShards() throws Exception { ensureGreen(indices); for (String index : acknowledgedCloses) { - long docsCount = client().prepareSearch(index).setSize(0).setTrackTotalHits(true).get().getHits().getTotalHits().value; + long docsCount = client().prepareSearch(index).setSize(0).setTrackTotalHits(true).get().getHits().getTotalHits().value(); assertEquals( "Expected " + docsPerIndex.get(index) diff --git a/server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java b/server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java index 1d5da9370cce3..3477f05097a83 100644 --- a/server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java @@ -444,7 +444,7 @@ public void testQueryCache() throws Exception { ); for (int i = 0; i < 10; i++) { assertThat( - client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value, + client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value(), equalTo((long) numDocs) ); assertThat( @@ -493,7 +493,7 @@ public void testQueryCache() throws Exception { for (int i = 0; i < 10; i++) { assertThat( - client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value, + client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value(), equalTo((long) numDocs) ); assertThat( @@ -524,7 +524,8 @@ public void testQueryCache() throws Exception { .setRequestCache(false) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo((long) numDocs) ); assertThat( @@ -539,7 +540,8 @@ public void testQueryCache() throws Exception { .setRequestCache(true) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo((long) numDocs) ); assertThat( @@ -558,7 +560,7 @@ public void testQueryCache() throws Exception { ); assertThat( - client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value, + client().prepareSearch("idx").setSearchType(SearchType.QUERY_THEN_FETCH).setSize(0).get().getHits().getTotalHits().value(), equalTo((long) numDocs) ); assertThat( @@ -573,7 +575,8 @@ public void testQueryCache() throws Exception { .setRequestCache(true) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo((long) numDocs) ); assertThat( diff --git a/server/src/internalClusterTest/java/org/opensearch/recovery/RecoveryWhileUnderLoadIT.java b/server/src/internalClusterTest/java/org/opensearch/recovery/RecoveryWhileUnderLoadIT.java index 988aeac7da541..5b7b8d9d17882 100644 --- a/server/src/internalClusterTest/java/org/opensearch/recovery/RecoveryWhileUnderLoadIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/recovery/RecoveryWhileUnderLoadIT.java @@ -406,7 +406,7 @@ private void iterateAssertCount(final int numberOfShards, final int iterations, .get(); logSearchResponse(numberOfShards, numberOfDocs, i, searchResponse); iterationResults[i] = searchResponse; - if (searchResponse.getHits().getTotalHits().value != numberOfDocs) { + if (searchResponse.getHits().getTotalHits().value() != numberOfDocs) { error = true; } } @@ -452,7 +452,7 @@ private void iterateAssertCount(final int numberOfShards, final int iterations, .setSize(0) .setQuery(matchAllQuery()) .get(); - if (searchResponse.getHits().getTotalHits().value != numberOfDocs) { + if (searchResponse.getHits().getTotalHits().value() != numberOfDocs) { errorOccurred = true; } } @@ -481,7 +481,7 @@ private void logSearchResponse(int numberOfShards, long numberOfDocs, int iterat logger.info( "iteration [{}] - returned documents: {} (expected {})", iteration, - searchResponse.getHits().getTotalHits().value, + searchResponse.getHits().getTotalHits().value(), numberOfDocs ); } diff --git a/server/src/internalClusterTest/java/org/opensearch/recovery/RelocationIT.java b/server/src/internalClusterTest/java/org/opensearch/recovery/RelocationIT.java index 8d8aea7fc1f3b..e20c00ac0a551 100644 --- a/server/src/internalClusterTest/java/org/opensearch/recovery/RelocationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/recovery/RelocationIT.java @@ -171,7 +171,7 @@ public void testSimpleRelocationNoIndexing() { logger.info("--> verifying count"); refreshAndWaitForReplication(); - assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value, equalTo(20L)); + assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value(), equalTo(20L)); logger.info("--> start another node"); final String node_2 = internalCluster().startNode(); @@ -199,7 +199,7 @@ public void testSimpleRelocationNoIndexing() { logger.info("--> verifying count again..."); refreshAndWaitForReplication(); - assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value, equalTo(20L)); + assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value(), equalTo(20L)); } public void testRelocationWhileIndexingRandom() throws Exception { @@ -290,7 +290,7 @@ public void testRelocationWhileIndexingRandom() throws Exception { .actionGet() .getHits(); ranOnce = true; - if (hits.getTotalHits().value != indexer.totalIndexedDocs()) { + if (hits.getTotalHits().value() != indexer.totalIndexedDocs()) { int[] hitIds = new int[(int) indexer.totalIndexedDocs()]; for (int hit = 0; hit < indexer.totalIndexedDocs(); hit++) { hitIds[hit] = hit + 1; @@ -304,7 +304,7 @@ public void testRelocationWhileIndexingRandom() throws Exception { } set.forEach(value -> logger.error("Missing id [{}]", value)); } - assertThat(hits.getTotalHits().value, equalTo(indexer.totalIndexedDocs())); + assertThat(hits.getTotalHits().value(), equalTo(indexer.totalIndexedDocs())); logger.info("--> DONE search test round {}", i + 1); } @@ -425,9 +425,9 @@ public void indexShardStateChanged( SearchResponse response = client.prepareSearch("test").setPreference("_local").setSize(0).get(); assertNoFailures(response); if (expectedCount < 0) { - expectedCount = response.getHits().getTotalHits().value; + expectedCount = response.getHits().getTotalHits().value(); } else { - assertEquals(expectedCount, response.getHits().getTotalHits().value); + assertEquals(expectedCount, response.getHits().getTotalHits().value()); } } @@ -663,7 +663,7 @@ public void testRelocateWhileWaitingForRefresh() { logger.info("--> verifying count"); refreshAndWaitForReplication(); - assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value, equalTo(20L)); + assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value(), equalTo(20L)); } public void testRelocateWhileContinuouslyIndexingAndWaitingForRefresh() throws Exception { @@ -742,7 +742,7 @@ public void testRelocateWhileContinuouslyIndexingAndWaitingForRefresh() throws E assertTrue(pendingIndexResponses.stream().allMatch(ActionFuture::isDone)); }, 1, TimeUnit.MINUTES); - assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value, equalTo(120L)); + assertThat(client().prepareSearch("test").setSize(0).execute().actionGet().getHits().getTotalHits().value(), equalTo(120L)); } public void testRelocationEstablishedPeerRecoveryRetentionLeases() throws Exception { diff --git a/server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java b/server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java index ecb97e79b348e..1493a1b259e13 100644 --- a/server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java +++ b/server/src/internalClusterTest/java/org/opensearch/remotestore/RestoreShallowSnapshotV2IT.java @@ -970,7 +970,8 @@ public void testContinuousIndexing() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value; + .getTotalHits() + .value(); assertEquals(snapshots.get(snapshot), hits); }); @@ -1078,7 +1079,8 @@ public void testHashedPrefixTranslogMetadataCombination() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value; + .getTotalHits() + .value(); assertEquals(snapshots.get(snapshot), hits); }); diff --git a/server/src/internalClusterTest/java/org/opensearch/remotestore/multipart/RemoteStoreMultipartIT.java b/server/src/internalClusterTest/java/org/opensearch/remotestore/multipart/RemoteStoreMultipartIT.java index 99cc58848a04a..eda70c0769d64 100644 --- a/server/src/internalClusterTest/java/org/opensearch/remotestore/multipart/RemoteStoreMultipartIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/remotestore/multipart/RemoteStoreMultipartIT.java @@ -142,6 +142,6 @@ public void testRateLimitedRemoteUploads() throws Exception { assertThat(uploadPauseTime, greaterThan(TimeValue.timeValueSeconds(randomIntBetween(5, 10)).nanos())); }, 30, TimeUnit.SECONDS); - assertThat(client.prepareSearch(INDEX_NAME).setSize(0).get().getHits().getTotalHits().value, equalTo(10L)); + assertThat(client.prepareSearch(INDEX_NAME).setSize(0).get().getHits().getTotalHits().value(), equalTo(10L)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/routing/AliasRoutingIT.java b/server/src/internalClusterTest/java/org/opensearch/routing/AliasRoutingIT.java index 299c2da21c222..b0e740b4c1678 100644 --- a/server/src/internalClusterTest/java/org/opensearch/routing/AliasRoutingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/routing/AliasRoutingIT.java @@ -146,7 +146,7 @@ public void testAliasSearchRouting() throws Exception { logger.info("--> search with no routing, should fine one"); for (int i = 0; i < 5; i++) { assertThat( - client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value, + client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value(), equalTo(1L) ); } @@ -160,7 +160,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); @@ -172,7 +173,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); @@ -182,7 +184,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); @@ -193,7 +196,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); } @@ -208,7 +212,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -219,7 +224,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -228,7 +234,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -238,7 +245,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -249,7 +257,7 @@ public void testAliasSearchRouting() throws Exception { logger.info("--> search with no routing, should fine two"); for (int i = 0; i < 5; i++) { assertThat( - client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value, + client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value(), equalTo(2L) ); assertThat( @@ -259,7 +267,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -273,7 +282,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -284,7 +294,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -293,7 +304,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -303,7 +315,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -317,7 +330,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -328,7 +342,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -337,7 +352,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -347,7 +363,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -361,7 +378,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -372,7 +390,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -381,7 +400,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -391,7 +411,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -404,7 +425,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -414,7 +436,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -427,7 +450,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -437,7 +461,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -450,7 +475,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -460,7 +486,8 @@ public void testAliasSearchRouting() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -519,7 +546,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); assertThat( @@ -529,7 +557,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); } @@ -542,7 +571,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -552,7 +582,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -565,7 +596,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -575,7 +607,8 @@ public void testAliasSearchRoutingWithTwoIndices() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -605,7 +638,8 @@ public void testAliasSearchRoutingWithConcreteAndAliasedIndices_issue2682() thro .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -636,7 +670,7 @@ public void testAliasSearchRoutingWithConcreteAndAliasedIndices_issue3268() thro .actionGet(); logger.info("--> search all on index_* should find two"); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); // Let's make sure that, even though 2 docs are available, only one is returned according to the size we set in the request // Therefore the reduce phase has taken place, which proves that the QUERY_AND_FETCH search type wasn't erroneously forced. assertThat(searchResponse.getHits().getHits().length, equalTo(1)); @@ -661,7 +695,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -671,7 +706,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -687,7 +723,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); assertThat( @@ -697,7 +734,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); } @@ -724,7 +762,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -734,7 +773,8 @@ public void testIndexingAliasesOverTime() throws Exception { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } diff --git a/server/src/internalClusterTest/java/org/opensearch/routing/PartitionedRoutingIT.java b/server/src/internalClusterTest/java/org/opensearch/routing/PartitionedRoutingIT.java index 64df858a18c9d..5255d4c044f47 100644 --- a/server/src/internalClusterTest/java/org/opensearch/routing/PartitionedRoutingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/routing/PartitionedRoutingIT.java @@ -187,7 +187,7 @@ private void verifyRoutedSearches(String index, Map> routing + "] shards for routing [" + routing + "] and got hits [" - + response.getHits().getTotalHits().value + + response.getHits().getTotalHits().value() + "]" ); @@ -195,7 +195,7 @@ private void verifyRoutedSearches(String index, Map> routing response.getTotalShards() + " was not in " + expectedShards + " for " + index, expectedShards.contains(response.getTotalShards()) ); - assertEquals(expectedDocuments, response.getHits().getTotalHits().value); + assertEquals(expectedDocuments, response.getHits().getTotalHits().value()); Set found = new HashSet<>(); response.getHits().forEach(h -> found.add(h.getId())); @@ -217,7 +217,7 @@ private void verifyBroadSearches(String index, Map> routingT .actionGet(); assertEquals(expectedShards, response.getTotalShards()); - assertEquals(expectedDocuments, response.getHits().getTotalHits().value); + assertEquals(expectedDocuments, response.getHits().getTotalHits().value()); Set found = new HashSet<>(); response.getHits().forEach(h -> found.add(h.getId())); diff --git a/server/src/internalClusterTest/java/org/opensearch/routing/SimpleRoutingIT.java b/server/src/internalClusterTest/java/org/opensearch/routing/SimpleRoutingIT.java index 80e82fa387c96..4fc68b27d8c9e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/routing/SimpleRoutingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/routing/SimpleRoutingIT.java @@ -163,7 +163,7 @@ public void testSimpleSearchRouting() { logger.info("--> search with no routing, should fine one"); for (int i = 0; i < 5; i++) { assertThat( - client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value, + client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value(), equalTo(1L) ); } @@ -177,7 +177,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); assertThat( @@ -188,7 +189,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(0L) ); } @@ -202,7 +204,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -213,7 +216,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -230,7 +234,7 @@ public void testSimpleSearchRouting() { logger.info("--> search with no routing, should fine two"); for (int i = 0; i < 5; i++) { assertThat( - client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value, + client().prepareSearch().setQuery(QueryBuilders.matchAllQuery()).execute().actionGet().getHits().getTotalHits().value(), equalTo(2L) ); assertThat( @@ -240,7 +244,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -254,7 +259,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -265,7 +271,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -279,7 +286,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); assertThat( @@ -290,7 +298,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(1L) ); } @@ -304,7 +313,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -315,7 +325,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } @@ -329,7 +340,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); assertThat( @@ -340,7 +352,8 @@ public void testSimpleSearchRouting() { .execute() .actionGet() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), equalTo(2L) ); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/SearchTimeoutIT.java b/server/src/internalClusterTest/java/org/opensearch/search/SearchTimeoutIT.java index 79caef1f45a26..b36ecfab57e58 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/SearchTimeoutIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/SearchTimeoutIT.java @@ -110,7 +110,7 @@ public void testSimpleDoesNotTimeout() throws Exception { .get(); assertFalse(searchResponse.isTimedOut()); assertEquals(0, searchResponse.getFailedShards()); - assertEquals(numDocs, searchResponse.getHits().getTotalHits().value); + assertEquals(numDocs, searchResponse.getHits().getTotalHits().value()); } public void testPartialResultsIntolerantTimeout() throws Exception { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/SearchWeightedRoutingIT.java b/server/src/internalClusterTest/java/org/opensearch/search/SearchWeightedRoutingIT.java index f08d3e871c579..7516b0090d2fa 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/SearchWeightedRoutingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/SearchWeightedRoutingIT.java @@ -823,7 +823,7 @@ public void testStrictWeightedRoutingWithShardPref() throws Exception { try { SearchResponse searchResponse = responses[i].get(); assertEquals(searchResponse.getFailedShards(), 0); - assertNotEquals(searchResponse.getHits().getTotalHits().value, 0); + assertNotEquals(searchResponse.getHits().getTotalHits().value(), 0); } catch (Exception t) { fail("search should not fail"); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/CombiIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/CombiIT.java index 4ce74f8195da9..24ad9f600f248 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/CombiIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/CombiIT.java @@ -162,7 +162,7 @@ public void testSubAggregationForTopAggregationOnUnmappedField() throws Exceptio ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, Matchers.equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), Matchers.equalTo(0L)); Histogram values = searchResponse.getAggregations().get("values"); assertThat(values, notNullValue()); assertThat(values.getBuckets().isEmpty(), is(true)); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/EquivalenceIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/EquivalenceIT.java index 2fcf09d6ebf50..e611b905dd68a 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/EquivalenceIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/EquivalenceIT.java @@ -346,7 +346,7 @@ public void testDuelTerms() throws Exception { ) .get(); assertAllSuccessful(resp); - assertEquals(numDocs, resp.getHits().getTotalHits().value); + assertEquals(numDocs, resp.getHits().getTotalHits().value()); final Terms longTerms = resp.getAggregations().get("long"); final Terms doubleTerms = resp.getAggregations().get("double"); @@ -464,7 +464,7 @@ public void testLargeNumbersOfPercentileBuckets() throws Exception { ) .get(); assertAllSuccessful(response); - assertEquals(numDocs, response.getHits().getTotalHits().value); + assertEquals(numDocs, response.getHits().getTotalHits().value()); } // https://github.com/elastic/elasticsearch/issues/6435 diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/FiltersAggsRewriteIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/FiltersAggsRewriteIT.java index 5926ff9f0cad1..b8d1d3cad77b4 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/FiltersAggsRewriteIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/FiltersAggsRewriteIT.java @@ -79,7 +79,7 @@ public void testWrapperQueryIsRewritten() throws IOException { metadata.put(randomAlphaOfLengthBetween(1, 20), randomAlphaOfLengthBetween(1, 20)); builder.setMetadata(metadata); SearchResponse searchResponse = client().prepareSearch("test").setSize(0).addAggregation(builder).get(); - assertEquals(3, searchResponse.getHits().getTotalHits().value); + assertEquals(3, searchResponse.getHits().getTotalHits().value()); InternalFilters filters = searchResponse.getAggregations().get("titles"); assertEquals(1, filters.getBuckets().size()); assertEquals(2, filters.getBuckets().get(0).getDocCount()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/AdjacencyMatrixIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/AdjacencyMatrixIT.java index 557ec9a37978d..ae70b251c5347 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/AdjacencyMatrixIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/AdjacencyMatrixIT.java @@ -397,7 +397,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramIT.java index 6a15490cbfe63..6ab9c8154cdc6 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramIT.java @@ -1031,7 +1031,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); List buckets = histo.getBuckets(); @@ -1070,7 +1070,7 @@ public void testSingleValueWithTimeZone() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(5L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(5L)); Histogram histo = response.getAggregations().get("date_histo"); List buckets = histo.getBuckets(); @@ -1241,7 +1241,7 @@ public void testSingleValueFieldWithExtendedBoundsTimezone() throws Exception { assertThat( "Expected 24 buckets for one day aggregation with hourly interval", - response.getHits().getTotalHits().value, + response.getHits().getTotalHits().value(), equalTo(2L) ); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramOffsetIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramOffsetIT.java index eea896e01afe1..677cd8e883c75 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramOffsetIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateHistogramOffsetIT.java @@ -122,7 +122,7 @@ public void testSingleValueWithPositiveOffset() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(5L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(5L)); Histogram histo = response.getAggregations().get("date_histo"); List buckets = histo.getBuckets(); @@ -142,7 +142,7 @@ public void testSingleValueWithNegativeOffset() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(5L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(5L)); Histogram histo = response.getAggregations().get("date_histo"); List buckets = histo.getBuckets(); @@ -170,7 +170,7 @@ public void testSingleValueWithOffsetMinDocCount() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(24L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(24L)); Histogram histo = response.getAggregations().get("date_histo"); List buckets = histo.getBuckets(); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateRangeIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateRangeIT.java index f00b601a54b80..eb2df1cc3c145 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateRangeIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/DateRangeIT.java @@ -894,7 +894,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -1101,7 +1101,7 @@ public void testRangeWithFormatStringValue() throws Exception { .setSize(0) .addAggregation(dateRange("date_range").field("date").addRange("00:16:40", "00:50:00").addRange("00:50:00", "01:06:40")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); List buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "00:16:40-00:50:00", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "00:50:00-01:06:40", 3000000L, 4000000L); @@ -1114,7 +1114,7 @@ public void testRangeWithFormatStringValue() throws Exception { dateRange("date_range").field("date").addRange("00.16.40", "00.50.00").addRange("00.50.00", "01.06.40").format("HH.mm.ss") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "00.16.40-00.50.00", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "00.50.00-01.06.40", 3000000L, 4000000L); @@ -1127,7 +1127,7 @@ public void testRangeWithFormatStringValue() throws Exception { dateRange("date_range").field("date").addRange(1000000, 3000000).addRange(3000000, 4000000).format("epoch_millis") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "1000000-3000000", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "3000000-4000000", 3000000L, 4000000L); @@ -1164,7 +1164,7 @@ public void testRangeWithFormatNumericValue() throws Exception { .setSize(0) .addAggregation(dateRange("date_range").field("date").addRange(1000, 3000).addRange(3000, 4000)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); List buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "1000-3000", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "3000-4000", 3000000L, 4000000L); @@ -1174,7 +1174,7 @@ public void testRangeWithFormatNumericValue() throws Exception { .setSize(0) .addAggregation(dateRange("date_range").field("date").addRange("1000", "3000").addRange("3000", "4000")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "1000-3000", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "3000-4000", 3000000L, 4000000L); @@ -1184,7 +1184,7 @@ public void testRangeWithFormatNumericValue() throws Exception { .setSize(0) .addAggregation(dateRange("date_range").field("date").addRange(1.0e3, 3000.8123).addRange(3000.8123, 4.0e3)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "1000-3000", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "3000-4000", 3000000L, 4000000L); @@ -1197,7 +1197,7 @@ public void testRangeWithFormatNumericValue() throws Exception { dateRange("date_range").field("date").addRange("00.16.40", "00.50.00").addRange("00.50.00", "01.06.40").format("HH.mm.ss") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "00.16.40-00.50.00", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "00.50.00-01.06.40", 3000000L, 4000000L); @@ -1210,7 +1210,7 @@ public void testRangeWithFormatNumericValue() throws Exception { dateRange("date_range").field("date").addRange(1000000, 3000000).addRange(3000000, 4000000).format("epoch_millis") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); buckets = checkBuckets(searchResponse.getAggregations().get("date_range"), "date_range", 2); assertBucket(buckets.get(0), 2L, "1000000-3000000", 1000000L, 3000000L); assertBucket(buckets.get(1), 1L, "3000000-4000000", 3000000L, 4000000L); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FilterIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FilterIT.java index 2863711d49580..af7491c463057 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FilterIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FilterIT.java @@ -208,7 +208,7 @@ public void testEmptyAggregation() throws Exception { .addAggregation(histogram("histo").field("value").interval(1L).minDocCount(0).subAggregation(filter("filter", matchAllQuery()))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FiltersIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FiltersIT.java index e64877a1d4030..f627259cf2f11 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FiltersIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/FiltersIT.java @@ -295,7 +295,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -507,7 +507,7 @@ public void testEmptyAggregationWithOtherBucket() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/GeoDistanceIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/GeoDistanceIT.java index ed0bd3aad5bab..0ce6fc69ac5c5 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/GeoDistanceIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/GeoDistanceIT.java @@ -454,7 +454,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/HistogramIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/HistogramIT.java index 4abd068d6fe37..e6562a5478c00 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/HistogramIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/HistogramIT.java @@ -961,7 +961,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); List buckets = histo.getBuckets(); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/NestedIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/NestedIT.java index 288d4d2c4e525..b93e748c8d736 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/NestedIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/NestedIT.java @@ -391,7 +391,7 @@ public void testEmptyAggregation() throws Exception { .addAggregation(histogram("histo").field("value").interval(1L).minDocCount(0).subAggregation(nested("nested", "nested"))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/RangeIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/RangeIT.java index 50cee4e9ecd92..5483db285dded 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/RangeIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/bucket/RangeIT.java @@ -923,7 +923,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, Matchers.notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ExtendedStatsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ExtendedStatsIT.java index 4a2c100690de4..c7d75b5595c6f 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ExtendedStatsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ExtendedStatsIT.java @@ -126,7 +126,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -159,7 +159,7 @@ public void testUnmapped() throws Exception { .addAggregation(extendedStats("stats").field("value")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); ExtendedStats stats = searchResponse.getAggregations().get("stats"); assertThat(stats, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/GeoCentroidIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/GeoCentroidIT.java index ed87fa6d8f5f6..3014487479a0e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/GeoCentroidIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/GeoCentroidIT.java @@ -67,7 +67,7 @@ public void testEmptyAggregation() throws Exception { assertSearchResponse(response); GeoCentroid geoCentroid = response.getAggregations().get(aggName); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); assertThat(geoCentroid, notNullValue()); assertThat(geoCentroid.getName(), equalTo(aggName)); GeoPoint centroid = geoCentroid.centroid(); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentileRanksIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentileRanksIT.java index ae67f0b1c0b66..ad7aa7f89fa37 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentileRanksIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentileRanksIT.java @@ -147,7 +147,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -172,7 +172,7 @@ public void testUnmapped() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); PercentileRanks reversePercentiles = searchResponse.getAggregations().get("percentile_ranks"); assertThat(reversePercentiles, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentilesIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentilesIT.java index ff1cab85c18e6..1463433ce1ec4 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentilesIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/HDRPercentilesIT.java @@ -151,7 +151,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -177,7 +177,7 @@ public void testUnmapped() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); Percentiles percentiles = searchResponse.getAggregations().get("percentiles"); assertThat(percentiles, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ScriptedMetricIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ScriptedMetricIT.java index 1725aa7847d72..2bdd8fe706221 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ScriptedMetricIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ScriptedMetricIT.java @@ -442,7 +442,7 @@ public void testMap() { .addAggregation(scriptedMetric("scripted").mapScript(mapScript).combineScript(combineScript).reduceScript(reduceScript)) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -492,7 +492,7 @@ public void testMapWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -546,7 +546,7 @@ public void testInitMutatesParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -601,7 +601,7 @@ public void testMapCombineWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -666,7 +666,7 @@ public void testInitMapCombineWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -736,7 +736,7 @@ public void testInitMapCombineReduceWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -795,7 +795,7 @@ public void testInitMapCombineReduceGetProperty() throws Exception { .get(); assertSearchResponse(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocs)); Global global = searchResponse.getAggregations().get("global"); assertThat(global, notNullValue()); @@ -853,7 +853,7 @@ public void testMapCombineReduceWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -903,7 +903,7 @@ public void testInitMapReduceWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -947,7 +947,7 @@ public void testMapReduceWithParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -1005,7 +1005,7 @@ public void testInitMapCombineReduceWithParamsAndReduceParams() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -1040,7 +1040,7 @@ public void testInitMapCombineReduceWithParamsStored() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("scripted"); assertThat(aggregation, notNullValue()); @@ -1100,7 +1100,7 @@ public void testInitMapCombineReduceWithParamsAsSubAgg() { ) .get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo(numDocs)); Aggregation aggregation = response.getAggregations().get("histo"); assertThat(aggregation, notNullValue()); assertThat(aggregation, instanceOf(Histogram.class)); @@ -1173,7 +1173,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/StatsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/StatsIT.java index 3708e1e6ab21b..c7df3efd6ac1e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/StatsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/StatsIT.java @@ -84,7 +84,7 @@ public void testEmptyAggregation() throws Exception { assertShardExecutionState(searchResponse, 0); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/SumIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/SumIT.java index b2aa3438b2306..8886713caa42b 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/SumIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/SumIT.java @@ -110,7 +110,7 @@ public void testEmptyAggregation() throws Exception { .addAggregation(histogram("histo").field("value").interval(1L).minDocCount(0).subAggregation(sum("sum").field("value"))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentileRanksIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentileRanksIT.java index 4225c027c4d96..49b7311e60ca6 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentileRanksIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentileRanksIT.java @@ -140,7 +140,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -184,7 +184,7 @@ public void testUnmapped() throws Exception { .addAggregation(randomCompression(percentileRanks("percentile_ranks", new double[] { 0, 10, 15, 100 })).field("value")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); PercentileRanks reversePercentiles = searchResponse.getAggregations().get("percentile_ranks"); assertThat(reversePercentiles, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesIT.java index 974e90fab16e8..f90b36f4d9e04 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesIT.java @@ -146,7 +146,7 @@ public void testEmptyAggregation() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); Histogram histo = searchResponse.getAggregations().get("histo"); assertThat(histo, notNullValue()); Histogram.Bucket bucket = histo.getBuckets().get(1); @@ -166,7 +166,7 @@ public void testUnmapped() throws Exception { .addAggregation(randomCompression(percentiles("percentiles")).field("value").percentiles(0, 10, 15, 100)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); Percentiles percentiles = searchResponse.getAggregations().get("percentiles"); assertThat(percentiles, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TopHitsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TopHitsIT.java index 5d84452998e40..0f2d709526657 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TopHitsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/TopHitsIT.java @@ -374,7 +374,7 @@ public void testBasics() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(3)); higestSortValue += 10; assertThat((Long) hits.getAt(0).getSortValues()[0], equalTo(higestSortValue)); @@ -395,7 +395,7 @@ public void testIssue11119() throws Exception { assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(8L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(8L)); assertThat(response.getHits().getHits().length, equalTo(0)); assertThat(response.getHits().getMaxScore(), equalTo(Float.NaN)); Terms terms = response.getAggregations().get("terms"); @@ -429,7 +429,7 @@ public void testIssue11119() throws Exception { assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(8L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(8L)); assertThat(response.getHits().getHits().length, equalTo(0)); assertThat(response.getHits().getMaxScore(), equalTo(Float.NaN)); terms = response.getAggregations().get("terms"); @@ -462,7 +462,7 @@ public void testBreadthFirstWithScoreNeeded() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(3)); assertThat(hits.getAt(0).getSourceAsMap().size(), equalTo(5)); @@ -494,7 +494,7 @@ public void testBreadthFirstWithAggOrderAndScoreNeeded() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(3)); assertThat(hits.getAt(0).getSourceAsMap().size(), equalTo(5)); @@ -554,7 +554,7 @@ public void testPagination() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(controlHits.getTotalHits().value)); + assertThat(hits.getTotalHits().value(), equalTo(controlHits.getTotalHits().value())); assertThat(hits.getHits().length, equalTo(controlHits.getHits().length)); for (int i = 0; i < hits.getHits().length; i++) { logger.info( @@ -594,7 +594,7 @@ public void testSortByBucket() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(3)); assertThat(hits.getAt(0).getSortValues()[0], equalTo(higestSortValue)); assertThat(hits.getAt(1).getSortValues()[0], equalTo(higestSortValue - 1)); @@ -629,7 +629,7 @@ public void testFieldCollapsing() throws Exception { assertThat(key(bucket), equalTo("b")); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(4L)); + assertThat(hits.getTotalHits().value(), equalTo(4L)); assertThat(hits.getHits().length, equalTo(1)); assertThat(hits.getAt(0).getId(), equalTo("6")); @@ -637,7 +637,7 @@ public void testFieldCollapsing() throws Exception { assertThat(key(bucket), equalTo("c")); topHits = bucket.getAggregations().get("hits"); hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(3L)); + assertThat(hits.getTotalHits().value(), equalTo(3L)); assertThat(hits.getHits().length, equalTo(1)); assertThat(hits.getAt(0).getId(), equalTo("9")); @@ -645,7 +645,7 @@ public void testFieldCollapsing() throws Exception { assertThat(key(bucket), equalTo("a")); topHits = bucket.getAggregations().get("hits"); hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(2L)); + assertThat(hits.getTotalHits().value(), equalTo(2L)); assertThat(hits.getHits().length, equalTo(1)); assertThat(hits.getAt(0).getId(), equalTo("2")); } @@ -681,7 +681,7 @@ public void testFetchFeatures() { for (Terms.Bucket bucket : terms.getBuckets()) { TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(1)); SearchHit hit = hits.getAt(0); @@ -743,7 +743,7 @@ public void testEmptyIndex() throws Exception { TopHits hits = response.getAggregations().get("hits"); assertThat(hits, notNullValue()); assertThat(hits.getName(), equalTo("hits")); - assertThat(hits.getHits().getTotalHits().value, equalTo(0L)); + assertThat(hits.getHits().getTotalHits().value(), equalTo(0L)); } public void testTrackScores() throws Exception { @@ -805,7 +805,7 @@ public void testTopHitsInNestedSimple() throws Exception { assertThat(bucket.getDocCount(), equalTo(1L)); TopHits topHits = bucket.getAggregations().get("top-comments"); SearchHits searchHits = topHits.getHits(); - assertThat(searchHits.getTotalHits().value, equalTo(1L)); + assertThat(searchHits.getTotalHits().value(), equalTo(1L)); assertThat(searchHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(searchHits.getAt(0).getNestedIdentity().getOffset(), equalTo(0)); assertThat(extractValue("date", searchHits.getAt(0).getSourceAsMap()), equalTo(1)); @@ -814,7 +814,7 @@ public void testTopHitsInNestedSimple() throws Exception { assertThat(bucket.getDocCount(), equalTo(2L)); topHits = bucket.getAggregations().get("top-comments"); searchHits = topHits.getHits(); - assertThat(searchHits.getTotalHits().value, equalTo(2L)); + assertThat(searchHits.getTotalHits().value(), equalTo(2L)); assertThat(searchHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(searchHits.getAt(0).getNestedIdentity().getOffset(), equalTo(1)); assertThat(extractValue("date", searchHits.getAt(0).getSourceAsMap()), equalTo(2)); @@ -826,7 +826,7 @@ public void testTopHitsInNestedSimple() throws Exception { assertThat(bucket.getDocCount(), equalTo(1L)); topHits = bucket.getAggregations().get("top-comments"); searchHits = topHits.getHits(); - assertThat(searchHits.getTotalHits().value, equalTo(1L)); + assertThat(searchHits.getTotalHits().value(), equalTo(1L)); assertThat(searchHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(searchHits.getAt(0).getNestedIdentity().getOffset(), equalTo(1)); assertThat(extractValue("date", searchHits.getAt(0).getSourceAsMap()), equalTo(4)); @@ -850,7 +850,7 @@ public void testTopHitsInSecondLayerNested() throws Exception { assertThat(toComments.getDocCount(), equalTo(4L)); TopHits topComments = toComments.getAggregations().get("top-comments"); - assertThat(topComments.getHits().getTotalHits().value, equalTo(4L)); + assertThat(topComments.getHits().getTotalHits().value(), equalTo(4L)); assertThat(topComments.getHits().getHits().length, equalTo(4)); assertThat(topComments.getHits().getAt(0).getId(), equalTo("2")); @@ -877,7 +877,7 @@ public void testTopHitsInSecondLayerNested() throws Exception { assertThat(toReviewers.getDocCount(), equalTo(7L)); TopHits topReviewers = toReviewers.getAggregations().get("top-reviewers"); - assertThat(topReviewers.getHits().getTotalHits().value, equalTo(7L)); + assertThat(topReviewers.getHits().getTotalHits().value(), equalTo(7L)); assertThat(topReviewers.getHits().getHits().length, equalTo(7)); assertThat(topReviewers.getHits().getAt(0).getId(), equalTo("1")); @@ -958,7 +958,7 @@ public void testNestedFetchFeatures() { assertThat(nested.getDocCount(), equalTo(4L)); SearchHits hits = ((TopHits) nested.getAggregations().get("top-comments")).getHits(); - assertThat(hits.getTotalHits().value, equalTo(4L)); + assertThat(hits.getTotalHits().value(), equalTo(4L)); SearchHit searchHit = hits.getAt(0); assertThat(searchHit.getId(), equalTo("1")); assertThat(searchHit.getNestedIdentity().getField().string(), equalTo("comments")); @@ -1018,7 +1018,7 @@ public void testTopHitsInNested() throws Exception { TopHits hits = nested.getAggregations().get("comments"); SearchHits searchHits = hits.getHits(); - assertThat(searchHits.getTotalHits().value, equalTo(numNestedDocs)); + assertThat(searchHits.getTotalHits().value(), equalTo(numNestedDocs)); for (int j = 0; j < 3; j++) { assertThat(searchHits.getAt(j).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(searchHits.getAt(j).getNestedIdentity().getOffset(), equalTo(0)); @@ -1144,7 +1144,7 @@ public void testNoStoredFields() throws Exception { assertThat(bucket.getDocCount(), equalTo(10L)); TopHits topHits = bucket.getAggregations().get("hits"); SearchHits hits = topHits.getHits(); - assertThat(hits.getTotalHits().value, equalTo(10L)); + assertThat(hits.getTotalHits().value(), equalTo(10L)); assertThat(hits.getHits().length, equalTo(3)); for (SearchHit hit : hits) { assertThat(hit.getSourceAsMap(), nullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ValueCountIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ValueCountIT.java index 4610281c4b8a8..c17b2108d4077 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ValueCountIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/metrics/ValueCountIT.java @@ -114,7 +114,7 @@ public void testUnmapped() throws Exception { .addAggregation(count("count").field("value")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); ValueCount valueCount = searchResponse.getAggregations().get("count"); assertThat(valueCount, notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/pipeline/DerivativeIT.java b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/pipeline/DerivativeIT.java index 6fabbb32a4d15..98fed92ecc42f 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/aggregations/pipeline/DerivativeIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/aggregations/pipeline/DerivativeIT.java @@ -441,7 +441,7 @@ public void testDocCountDerivativeWithGaps() throws Exception { .addAggregation(histogram("histo").field(SINGLE_VALUED_FIELD_NAME).interval(1).subAggregation(derivative("deriv", "_count"))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); @@ -472,7 +472,7 @@ public void testDocCountDerivativeWithGaps_random() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx_rnd)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx_rnd)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); @@ -502,7 +502,7 @@ public void testDocCountDerivativeWithGaps_insertZeros() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); @@ -533,7 +533,7 @@ public void testSingleValueAggDerivativeWithGaps() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); @@ -576,7 +576,7 @@ public void testSingleValueAggDerivativeWithGaps_insertZeros() throws Exception ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); @@ -617,7 +617,7 @@ public void testSingleValueAggDerivativeWithGaps_random() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(numDocsEmptyIdx_rnd)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(numDocsEmptyIdx_rnd)); Histogram deriv = searchResponse.getAggregations().get("histo"); assertThat(deriv, Matchers.notNullValue()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileCreatingIndexIT.java b/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileCreatingIndexIT.java index 681f7081fa2dc..b245c1a303cd3 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileCreatingIndexIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileCreatingIndexIT.java @@ -121,7 +121,7 @@ private void searchWhileCreatingIndex(boolean createIndex, int numberOfReplicas) .setPreference(preference + Integer.toString(counter++)) .setQuery(QueryBuilders.termQuery("field", "test")) .get(); - if (searchResponse.getHits().getTotalHits().value != 1) { + if (searchResponse.getHits().getTotalHits().value() != 1) { refresh(); SearchResponse searchResponseAfterRefresh = client.prepareSearch("test") .setPreference(preference) @@ -129,7 +129,7 @@ private void searchWhileCreatingIndex(boolean createIndex, int numberOfReplicas) .get(); logger.info( "hits count mismatch on any shard search failed, post explicit refresh hits are {}", - searchResponseAfterRefresh.getHits().getTotalHits().value + searchResponseAfterRefresh.getHits().getTotalHits().value() ); ensureGreen(); SearchResponse searchResponseAfterGreen = client.prepareSearch("test") @@ -138,7 +138,7 @@ private void searchWhileCreatingIndex(boolean createIndex, int numberOfReplicas) .get(); logger.info( "hits count mismatch on any shard search failed, post explicit wait for green hits are {}", - searchResponseAfterGreen.getHits().getTotalHits().value + searchResponseAfterGreen.getHits().getTotalHits().value() ); assertHitCount(searchResponse, 1); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileRelocatingIT.java b/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileRelocatingIT.java index f7b8b0df7dca7..68a57217d4c9e 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileRelocatingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/basic/SearchWhileRelocatingIT.java @@ -118,14 +118,14 @@ public void run() { try { while (!stop.get()) { SearchResponse sr = client().prepareSearch().setSize(numDocs).get(); - if (sr.getHits().getTotalHits().value != numDocs) { + if (sr.getHits().getTotalHits().value() != numDocs) { // if we did not search all shards but had no failures that is potentially fine // if only the hit-count is wrong. this can happen if the cluster-state is behind when the // request comes in. It's a small window but a known limitation. if (sr.getTotalShards() != sr.getSuccessfulShards() && sr.getFailedShards() == 0) { nonCriticalExceptions.add( "Count is " - + sr.getHits().getTotalHits().value + + sr.getHits().getTotalHits().value() + " but " + numDocs + " was expected. " @@ -139,7 +139,7 @@ public void run() { final SearchHits sh = sr.getHits(); assertThat( "Expected hits to be the same size the actual hits array", - sh.getTotalHits().value, + sh.getTotalHits().value(), equalTo((long) (sh.getHits().length)) ); // this is the more critical but that we hit the actual hit array has a different size than the diff --git a/server/src/internalClusterTest/java/org/opensearch/search/basic/TransportTwoNodesSearchIT.java b/server/src/internalClusterTest/java/org/opensearch/search/basic/TransportTwoNodesSearchIT.java index a82b6f12755ca..b2316d0b1bcf4 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/basic/TransportTwoNodesSearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/basic/TransportTwoNodesSearchIT.java @@ -170,7 +170,7 @@ public void testDfsQueryThenFetch() throws Exception { .get(); while (true) { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); SearchHit[] hits = searchResponse.getHits().getHits(); if (hits.length == 0) { break; // finished @@ -212,7 +212,7 @@ public void testDfsQueryThenFetchWithSort() throws Exception { .get(); while (true) { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); SearchHit[] hits = searchResponse.getHits().getHits(); if (hits.length == 0) { break; // finished @@ -250,7 +250,7 @@ public void testQueryThenFetch() throws Exception { .get(); while (true) { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); SearchHit[] hits = searchResponse.getHits().getHits(); if (hits.length == 0) { break; // finished @@ -277,7 +277,7 @@ public void testQueryThenFetchWithFrom() throws Exception { SearchResponse searchResponse = client().search(searchRequest("test").source(source.from(0).size(60)).searchType(QUERY_THEN_FETCH)) .actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(60)); for (int i = 0; i < 60; i++) { SearchHit hit = searchResponse.getHits().getHits()[i]; @@ -285,7 +285,7 @@ public void testQueryThenFetchWithFrom() throws Exception { } searchResponse = client().search(searchRequest("test").source(source.from(60).size(60)).searchType(QUERY_THEN_FETCH)).actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(40)); for (int i = 0; i < 40; i++) { SearchHit hit = searchResponse.getHits().getHits()[i]; @@ -307,7 +307,7 @@ public void testQueryThenFetchWithSort() throws Exception { .get(); while (true) { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); SearchHit[] hits = searchResponse.getHits().getHits(); if (hits.length == 0) { break; // finished @@ -336,7 +336,7 @@ public void testSimpleFacets() throws Exception { SearchResponse searchResponse = client().search(searchRequest("test").source(sourceBuilder)).actionGet(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); Global global = searchResponse.getAggregations().get("global"); Filter all = global.getAggregations().get("all"); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/fetch/FetchSubPhasePluginIT.java b/server/src/internalClusterTest/java/org/opensearch/search/fetch/FetchSubPhasePluginIT.java index 13b4abb58b4df..6fb45f06d8777 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/fetch/FetchSubPhasePluginIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/fetch/FetchSubPhasePluginIT.java @@ -176,7 +176,7 @@ private void hitExecute(FetchContext context, HitContext hitContext) throws IOEx hitField = new DocumentField(NAME, new ArrayList<>(1)); hitContext.hit().setDocumentField(NAME, hitField); } - Terms terms = hitContext.reader().getTermVector(hitContext.docId(), field); + Terms terms = hitContext.reader().termVectors().get(hitContext.docId(), field); if (terms != null) { TermsEnum te = terms.iterator(); Map tv = new HashMap<>(); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/InnerHitsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/InnerHitsIT.java index b743c00bf4549..14476ce0f69f3 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/InnerHitsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/InnerHitsIT.java @@ -188,7 +188,7 @@ public void testSimpleNested() throws Exception { assertSearchHit(response, 1, hasId("1")); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); SearchHits innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(2L)); + assertThat(innerHits.getTotalHits().value(), equalTo(2L)); assertThat(innerHits.getHits().length, equalTo(2)); assertThat(innerHits.getAt(0).getId(), equalTo("1")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -208,7 +208,7 @@ public void testSimpleNested() throws Exception { assertThat(response.getHits().getAt(0).getShard(), notNullValue()); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comment"); - assertThat(innerHits.getTotalHits().value, equalTo(3L)); + assertThat(innerHits.getTotalHits().value(), equalTo(3L)); assertThat(innerHits.getHits().length, equalTo(3)); assertThat(innerHits.getAt(0).getId(), equalTo("2")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -233,7 +233,7 @@ public void testSimpleNested() throws Exception { .get(); assertNoFailures(response); innerHits = response.getHits().getAt(0).getInnerHits().get("comments"); - assertThat(innerHits.getTotalHits().value, equalTo(2L)); + assertThat(innerHits.getTotalHits().value(), equalTo(2L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat( innerHits.getAt(0).getHighlightFields().get("comments.message").getFragments()[0].string(), @@ -304,7 +304,7 @@ public void testRandomNested() throws Exception { SearchHit searchHit = searchResponse.getHits().getAt(i); assertThat(searchHit.getShard(), notNullValue()); SearchHits inner = searchHit.getInnerHits().get("a"); - assertThat(inner.getTotalHits().value, equalTo((long) field1InnerObjects[i])); + assertThat(inner.getTotalHits().value(), equalTo((long) field1InnerObjects[i])); for (int j = 0; j < field1InnerObjects[i] && j < size; j++) { SearchHit innerHit = inner.getAt(j); assertThat(innerHit.getNestedIdentity().getField().string(), equalTo("field1")); @@ -313,7 +313,7 @@ public void testRandomNested() throws Exception { } inner = searchHit.getInnerHits().get("b"); - assertThat(inner.getTotalHits().value, equalTo((long) field2InnerObjects[i])); + assertThat(inner.getTotalHits().value(), equalTo((long) field2InnerObjects[i])); for (int j = 0; j < field2InnerObjects[i] && j < size; j++) { SearchHit innerHit = inner.getAt(j); assertThat(innerHit.getNestedIdentity().getField().string(), equalTo("field2")); @@ -418,13 +418,13 @@ public void testNestedMultipleLayers() throws Exception { assertSearchHit(response, 1, hasId("1")); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); SearchHits innerHits = response.getHits().getAt(0).getInnerHits().get("comments"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("1")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(innerHits.getAt(0).getNestedIdentity().getOffset(), equalTo(0)); innerHits = innerHits.getAt(0).getInnerHits().get("remark"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("1")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -449,13 +449,13 @@ public void testNestedMultipleLayers() throws Exception { assertSearchHit(response, 1, hasId("1")); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comments"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("1")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(innerHits.getAt(0).getNestedIdentity().getOffset(), equalTo(1)); innerHits = innerHits.getAt(0).getInnerHits().get("remark"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("1")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -476,7 +476,7 @@ public void testNestedMultipleLayers() throws Exception { assertSearchHit(response, 1, hasId("2")); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comments.remarks"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("2")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -500,13 +500,13 @@ public void testNestedMultipleLayers() throws Exception { assertSearchHit(response, 1, hasId("2")); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); innerHits = response.getHits().getAt(0).getInnerHits().get("comments"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("2")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); assertThat(innerHits.getAt(0).getNestedIdentity().getOffset(), equalTo(0)); innerHits = innerHits.getAt(0).getInnerHits().get("remark"); - assertThat(innerHits.getTotalHits().value, equalTo(1L)); + assertThat(innerHits.getTotalHits().value(), equalTo(1L)); assertThat(innerHits.getHits().length, equalTo(1)); assertThat(innerHits.getAt(0).getId(), equalTo("2")); assertThat(innerHits.getAt(0).getNestedIdentity().getField().string(), equalTo("comments")); @@ -576,7 +576,7 @@ public void testNestedDefinedAsObject() throws Exception { assertNoFailures(response); assertHitCount(response, 1); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); - assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getId(), equalTo("1")); assertThat( response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getNestedIdentity().getField().string(), @@ -683,7 +683,7 @@ public void testInnerHitsWithObjectFieldThatHasANestedField() throws Exception { SearchHit hit = response.getHits().getAt(0); assertThat(hit.getId(), equalTo("1")); SearchHits messages = hit.getInnerHits().get("comments.messages"); - assertThat(messages.getTotalHits().value, equalTo(2L)); + assertThat(messages.getTotalHits().value(), equalTo(2L)); assertThat(messages.getAt(0).getId(), equalTo("1")); assertThat(messages.getAt(0).getNestedIdentity().getField().string(), equalTo("comments.messages")); assertThat(messages.getAt(0).getNestedIdentity().getOffset(), equalTo(2)); @@ -705,7 +705,7 @@ public void testInnerHitsWithObjectFieldThatHasANestedField() throws Exception { hit = response.getHits().getAt(0); assertThat(hit.getId(), equalTo("1")); messages = hit.getInnerHits().get("comments.messages"); - assertThat(messages.getTotalHits().value, equalTo(1L)); + assertThat(messages.getTotalHits().value(), equalTo(1L)); assertThat(messages.getAt(0).getId(), equalTo("1")); assertThat(messages.getAt(0).getNestedIdentity().getField().string(), equalTo("comments.messages")); assertThat(messages.getAt(0).getNestedIdentity().getOffset(), equalTo(1)); @@ -740,7 +740,7 @@ public void testInnerHitsWithObjectFieldThatHasANestedField() throws Exception { hit = response.getHits().getAt(0); assertThat(hit.getId(), equalTo("1")); messages = hit.getInnerHits().get("comments.messages"); - assertThat(messages.getTotalHits().value, equalTo(1L)); + assertThat(messages.getTotalHits().value(), equalTo(1L)); assertThat(messages.getAt(0).getId(), equalTo("1")); assertThat(messages.getAt(0).getNestedIdentity().getField().string(), equalTo("comments.messages")); assertThat(messages.getAt(0).getNestedIdentity().getOffset(), equalTo(0)); @@ -845,22 +845,22 @@ public void testMatchesQueriesNestedInnerHits() throws Exception { .get(); assertNoFailures(searchResponse); assertAllSuccessful(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) numDocs)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("0")); - assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getAt(0).getMatchedQueries()[0], equalTo("test1")); assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getAt(1).getMatchedQueries().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getInnerHits().get("nested1").getAt(1).getMatchedQueries()[0], equalTo("test3")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); - assertThat(searchResponse.getHits().getAt(1).getInnerHits().get("nested1").getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getAt(1).getInnerHits().get("nested1").getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(1).getInnerHits().get("nested1").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(1).getInnerHits().get("nested1").getAt(0).getMatchedQueries()[0], equalTo("test2")); for (int i = 2; i < numDocs; i++) { assertThat(searchResponse.getHits().getAt(i).getId(), equalTo(String.valueOf(i))); - assertThat(searchResponse.getHits().getAt(i).getInnerHits().get("nested1").getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getAt(i).getInnerHits().get("nested1").getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(i).getInnerHits().get("nested1").getAt(0).getMatchedQueries().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(i).getInnerHits().get("nested1").getAt(0).getMatchedQueries()[0], equalTo("test3")); } @@ -905,7 +905,7 @@ public void testNestedSource() throws Exception { assertNoFailures(response); assertHitCount(response, 1); - assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().size(), equalTo(1)); assertThat( response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().get("message"), @@ -923,7 +923,7 @@ public void testNestedSource() throws Exception { assertNoFailures(response); assertHitCount(response, 1); - assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().size(), equalTo(2)); assertThat( response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().get("message"), @@ -948,7 +948,7 @@ public void testNestedSource() throws Exception { .get(); assertNoFailures(response); assertHitCount(response, 1); - assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().size(), equalTo(0)); // Check that inner hits contain _source even when it's disabled on the root request. @@ -958,7 +958,7 @@ public void testNestedSource() throws Exception { .get(); assertNoFailures(response); assertHitCount(response, 1); - assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getAt(0).getInnerHits().get("comments").getTotalHits().value(), equalTo(2L)); assertFalse(response.getHits().getAt(0).getInnerHits().get("comments").getAt(0).getSourceAsMap().isEmpty()); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/highlight/HighlighterSearchIT.java b/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/highlight/HighlighterSearchIT.java index f449a91a57279..6841c4b031f96 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/highlight/HighlighterSearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/fetch/subphase/highlight/HighlighterSearchIT.java @@ -3200,7 +3200,7 @@ public void testGeoFieldHighlightingWithDifferentHighlighters() throws IOExcepti ) .get(); assertNoFailures(search); - assertThat(search.getHits().getTotalHits().value, equalTo(1L)); + assertThat(search.getHits().getTotalHits().value(), equalTo(1L)); assertThat(search.getHits().getAt(0).getHighlightFields().get("text").fragments().length, equalTo(1)); } @@ -3244,7 +3244,7 @@ public void testGeoFieldHighlightingWhenQueryGetsRewritten() throws IOException, .setSource(new SearchSourceBuilder().query(query).highlighter(new HighlightBuilder().highlighterType("plain").field("jd"))) .get(); assertNoFailures(search); - assertThat(search.getHits().getTotalHits().value, equalTo(1L)); + assertThat(search.getHits().getTotalHits().value(), equalTo(1L)); } public void testKeywordFieldHighlighting() throws IOException, InterruptedException { @@ -3268,7 +3268,7 @@ public void testKeywordFieldHighlighting() throws IOException, InterruptedExcept ) .get(); assertNoFailures(search); - assertThat(search.getHits().getTotalHits().value, equalTo(1L)); + assertThat(search.getHits().getTotalHits().value(), equalTo(1L)); assertThat( search.getHits().getAt(0).getHighlightFields().get("keyword_field").getFragments()[0].string(), equalTo("some text") @@ -3432,7 +3432,7 @@ public void testHighlightQueryRewriteDatesWithNow() throws Exception { .get(); assertSearchResponse(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(1L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(1L)); assertHighlight(r1, 0, "field", 0, 1, equalTo("hello world")); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchFieldsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchFieldsIT.java index 60a6e59014e11..4fa50b4136bfe 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchFieldsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchFieldsIT.java @@ -264,26 +264,26 @@ public void testStoredFields() throws Exception { indexRandomForConcurrentSearch("test"); SearchResponse searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("field1").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field1").getValue().toString(), equalTo("value1")); // field2 is not stored, check that it is not extracted from source. searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("field2").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(0)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field2"), nullValue()); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("field3").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("*3").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); @@ -294,27 +294,27 @@ public void testStoredFields() throws Exception { .addStoredField("field1") .addStoredField("field2") .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(2)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); assertThat(searchResponse.getHits().getAt(0).getFields().get("field1").getValue().toString(), equalTo("value1")); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("field*").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(2)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); assertThat(searchResponse.getHits().getAt(0).getFields().get("field1").getValue().toString(), equalTo("value1")); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("f*3").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("*").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getSourceAsMap(), nullValue()); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(2)); @@ -322,7 +322,7 @@ public void testStoredFields() throws Exception { assertThat(searchResponse.getHits().getAt(0).getFields().get("field3").getValue().toString(), equalTo("value3")); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).addStoredField("*").addStoredField("_source").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); assertThat(searchResponse.getHits().getAt(0).getSourceAsMap(), notNullValue()); assertThat(searchResponse.getHits().getAt(0).getFields().size(), equalTo(2)); @@ -388,7 +388,7 @@ public void testScriptDocAndFields() throws Exception { assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertFalse(response.getHits().getAt(0).hasSource()); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); Set fields = new HashSet<>(response.getHits().getAt(0).getFields().keySet()); @@ -417,7 +417,7 @@ public void testScriptDocAndFields() throws Exception { .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value * factor", params)) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); fields = new HashSet<>(response.getHits().getAt(0).getFields().keySet()); assertThat(fields, equalTo(singleton("sNum1"))); @@ -488,7 +488,7 @@ public void testScriptWithUnsignedLong() throws Exception { assertNoFailures(response); logger.info("running doc['unsigned_num1'].value"); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertFalse(response.getHits().getAt(0).hasSource()); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); Set fields = new HashSet<>(response.getHits().getAt(0).getFields().keySet()); @@ -517,7 +517,7 @@ public void testScriptWithUnsignedLong() throws Exception { .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['unsigned_num1'].value * factor", params)) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); fields = new HashSet<>(response.getHits().getAt(0).getFields().keySet()); assertThat(fields, equalTo(singleton("sNum1"))); @@ -612,7 +612,7 @@ public void testIdBasedScriptFields() throws Exception { assertNoFailures(response); - assertThat(response.getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(response.getHits().getTotalHits().value(), equalTo((long) numDocs)); for (int i = 0; i < numDocs; i++) { assertThat(response.getHits().getAt(i).getId(), equalTo(Integer.toString(i))); Set fields = new HashSet<>(response.getHits().getAt(i).getFields().keySet()); @@ -827,7 +827,7 @@ public void testStoredFieldsWithoutSource() throws Exception { .addStoredField("unsigned_long_field") .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); Set fields = new HashSet<>(searchResponse.getHits().getAt(0).getFields().keySet()); assertThat( @@ -873,7 +873,7 @@ public void testSearchFieldsMetadata() throws Exception { SearchResponse searchResponse = client().prepareSearch("my-index").addStoredField("field1").addStoredField("_routing").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).field("field1"), nullValue()); assertThat(searchResponse.getHits().getAt(0).field("_routing").getValue().toString(), equalTo("1")); } @@ -956,7 +956,7 @@ public void testGetFieldsComplexField() throws Exception { String field = "field1.field2.field3.field4"; SearchResponse searchResponse = client().prepareSearch("my-index").addStoredField(field).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).field(field).getValues().size(), equalTo(2)); assertThat(searchResponse.getHits().getAt(0).field(field).getValues().get(0).toString(), equalTo("value1")); assertThat(searchResponse.getHits().getAt(0).field(field).getValues().get(1).toString(), equalTo("value2")); @@ -1080,7 +1080,7 @@ public void testDocValueFields() throws Exception { .addDocValueField("flat_object_field1.foo"); SearchResponse searchResponse = builder.get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); Set fields = new HashSet<>(searchResponse.getHits().getAt(0).getFields().keySet()); assertThat( @@ -1123,7 +1123,7 @@ public void testDocValueFields() throws Exception { builder = client().prepareSearch().setQuery(matchAllQuery()).addDocValueField("*field"); searchResponse = builder.get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); fields = new HashSet<>(searchResponse.getHits().getAt(0).getFields().keySet()); assertThat( @@ -1180,7 +1180,7 @@ public void testDocValueFields() throws Exception { ; searchResponse = builder.get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); fields = new HashSet<>(searchResponse.getHits().getAt(0).getFields().keySet()); assertThat( @@ -1232,7 +1232,7 @@ public void testDocValueFields() throws Exception { .addDocValueField("date_field", "epoch_millis"); searchResponse = builder.get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); fields = new HashSet<>(searchResponse.getHits().getAt(0).getFields().keySet()); assertThat( diff --git a/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchIpFieldTermsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchIpFieldTermsIT.java index d28d07a07e785..301af04f7204f 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchIpFieldTermsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/fields/SearchIpFieldTermsIT.java @@ -167,7 +167,7 @@ private void assertTermsHitCount( SearchResponse result = client().prepareSearch(indexName).setQuery(boolQueryBuilder.must(ipTerms) // .filter(QueryBuilders.termsQuery("dummy_filter", "a", "b")) ).get(); - long hitsFound = Objects.requireNonNull(result.getHits().getTotalHits()).value; + long hitsFound = Objects.requireNonNull(result.getHits().getTotalHits()).value(); MatcherAssert.assertThat(field, hitsFound, equalTo(expectedMatches)); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/DecayFunctionScoreIT.java b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/DecayFunctionScoreIT.java index 0380b3c7ddb89..fd9caf6ee0081 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/DecayFunctionScoreIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/DecayFunctionScoreIT.java @@ -187,7 +187,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); response = client().search( searchRequest().searchType(SearchType.QUERY_THEN_FETCH) @@ -195,7 +195,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); @@ -204,7 +204,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { response = client().search(searchRequest().searchType(SearchType.QUERY_THEN_FETCH).source(searchSource().query(baseQuery))); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); response = client().search( searchRequest().searchType(SearchType.QUERY_THEN_FETCH) @@ -212,7 +212,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); @@ -221,7 +221,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { response = client().search(searchRequest().searchType(SearchType.QUERY_THEN_FETCH).source(searchSource().query(baseQuery))); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); response = client().search( searchRequest().searchType(SearchType.QUERY_THEN_FETCH) @@ -229,7 +229,7 @@ public void testDistanceScoreGeoLinGaussExp() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); @@ -294,7 +294,7 @@ public void testDistanceScoreGeoLinGaussExpWithOffset() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getScore(), equalTo(sh.getAt(0).getScore())); @@ -317,7 +317,7 @@ public void testDistanceScoreGeoLinGaussExpWithOffset() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getScore(), equalTo(sh.getAt(0).getScore())); @@ -338,7 +338,7 @@ public void testDistanceScoreGeoLinGaussExpWithOffset() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (numDummyDocs + 2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (numDummyDocs + 2))); assertThat(sh.getAt(0).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getId(), anyOf(equalTo("1"), equalTo("2"))); assertThat(sh.getAt(1).getScore(), equalTo(sh.getAt(0).getScore())); @@ -411,7 +411,7 @@ public void testBoostModeSettingWorks() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (2))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); @@ -421,7 +421,7 @@ public void testBoostModeSettingWorks() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (2))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); @@ -437,7 +437,7 @@ public void testBoostModeSettingWorks() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (2))); assertThat(sh.getAt(0).getId(), equalTo("2")); assertThat(sh.getAt(1).getId(), equalTo("1")); @@ -489,7 +489,7 @@ public void testParseGeoPoint() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(1.0, 1.e-5)); // this is equivalent to new GeoPoint(20, 11); just flipped so scores must be same @@ -504,7 +504,7 @@ public void testParseGeoPoint() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(1.0f, 1.e-5)); } @@ -547,7 +547,7 @@ public void testCombineModes() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(1.0, 1.e-5)); @@ -561,7 +561,7 @@ public void testCombineModes() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(0.5, 1.e-5)); @@ -575,7 +575,7 @@ public void testCombineModes() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(2.0 + 0.5, 1.e-5)); logger.info("--> Hit[0] {} Explanation:\n {}", sr.getHits().getAt(0).getId(), sr.getHits().getAt(0).getExplanation()); @@ -590,7 +590,7 @@ public void testCombineModes() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo((2.0 + 0.5) / 2, 1.e-5)); @@ -604,7 +604,7 @@ public void testCombineModes() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(0.5, 1.e-5)); @@ -618,7 +618,7 @@ public void testCombineModes() throws Exception { ); sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat((double) sh.getAt(0).getScore(), closeTo(2.0, 1.e-5)); @@ -666,7 +666,7 @@ public void testCombineModesExplain() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (1))); + assertThat(sh.getTotalHits().value(), equalTo((long) (1))); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(0).getExplanation().getDetails(), arrayWithSize(2)); assertThat(sh.getAt(0).getExplanation().getDetails()[0].getDetails(), arrayWithSize(2)); @@ -1180,7 +1180,7 @@ public void testMultiFieldOptions() throws Exception { SearchResponse sr = response.actionGet(); assertSearchHits(sr, "1", "2"); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) (2))); + assertThat(sh.getTotalHits().value(), equalTo((long) (2))); List lonlat = new ArrayList<>(); lonlat.add(20f); @@ -1319,7 +1319,7 @@ public void testDistanceScoreGeoLinGaussExplain() throws Exception { ); SearchResponse sr = response.actionGet(); SearchHits sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo(2L)); + assertThat(sh.getTotalHits().value(), equalTo(2L)); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); assertExplain(queryName, functionName, sr); @@ -1336,7 +1336,7 @@ public void testDistanceScoreGeoLinGaussExplain() throws Exception { sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo(2L)); + assertThat(sh.getTotalHits().value(), equalTo(2L)); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); assertExplain(queryName, functionName, sr); @@ -1356,7 +1356,7 @@ public void testDistanceScoreGeoLinGaussExplain() throws Exception { sr = response.actionGet(); sh = sr.getHits(); - assertThat(sh.getTotalHits().value, equalTo(2L)); + assertThat(sh.getTotalHits().value(), equalTo(2L)); assertThat(sh.getAt(0).getId(), equalTo("1")); assertThat(sh.getAt(1).getId(), equalTo("2")); assertExplain(queryName, functionName, sr); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/ExplainableScriptIT.java b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/ExplainableScriptIT.java index 0573dcfc4863d..50e1affabdbef 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/ExplainableScriptIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/ExplainableScriptIT.java @@ -190,7 +190,7 @@ public void testExplainScript() throws InterruptedException, IOException, Execut OpenSearchAssertions.assertNoFailures(response); SearchHits hits = response.getHits(); - assertThat(hits.getTotalHits().value, equalTo(20L)); + assertThat(hits.getTotalHits().value(), equalTo(20L)); int idCounter = 19; for (SearchHit hit : hits.getHits()) { assertThat(hit.getId(), equalTo(Integer.toString(idCounter))); @@ -237,7 +237,7 @@ public void testExplainScriptWithName() throws InterruptedException, IOException OpenSearchAssertions.assertNoFailures(response); SearchHits hits = response.getHits(); - assertThat(hits.getTotalHits().value, equalTo(1L)); + assertThat(hits.getTotalHits().value(), equalTo(1L)); assertThat(hits.getHits()[0].getId(), equalTo("1")); assertThat(hits.getHits()[0].getExplanation().getDetails(), arrayWithSize(2)); assertThat(hits.getHits()[0].getExplanation().getDetails()[0].getDescription(), containsString("_name: func1")); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/FunctionScoreIT.java b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/FunctionScoreIT.java index 4f267f0059291..f3fb3fdf34887 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/FunctionScoreIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/FunctionScoreIT.java @@ -212,9 +212,9 @@ public void testMinScoreFunctionScoreBasic() throws IOException, InterruptedExce searchRequest().source(searchSource().query(functionScoreQuery(scriptFunction(script)).setMinScore(minScore))) ).actionGet(); if (score < minScore) { - assertThat(searchResponse.getHits().getTotalHits().value, is(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), is(0L)); } else { - assertThat(searchResponse.getHits().getTotalHits().value, is(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), is(1L)); } searchResponse = client().search( @@ -230,9 +230,9 @@ public void testMinScoreFunctionScoreBasic() throws IOException, InterruptedExce ) ).actionGet(); if (score < minScore) { - assertThat(searchResponse.getHits().getTotalHits().value, is(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), is(0L)); } else { - assertThat(searchResponse.getHits().getTotalHits().value, is(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), is(1L)); } } @@ -276,9 +276,9 @@ public void testMinScoreFunctionScoreManyDocsAndRandomMinScore() throws IOExcept protected void assertMinScoreSearchResponses(int numDocs, SearchResponse searchResponse, int numMatchingDocs) { assertSearchResponse(searchResponse); - assertThat((int) searchResponse.getHits().getTotalHits().value, is(numMatchingDocs)); + assertThat((int) searchResponse.getHits().getTotalHits().value(), is(numMatchingDocs)); int pos = 0; - for (int hitId = numDocs - 1; (numDocs - hitId) < searchResponse.getHits().getTotalHits().value; hitId--) { + for (int hitId = numDocs - 1; (numDocs - hitId) < searchResponse.getHits().getTotalHits().value(); hitId--) { assertThat(searchResponse.getHits().getAt(pos).getId(), equalTo(Integer.toString(hitId))); pos++; } @@ -294,7 +294,7 @@ public void testWithEmptyFunctions() throws IOException, ExecutionException, Int SearchResponse termQuery = client().search(searchRequest().source(searchSource().explain(true).query(termQuery("text", "text")))) .get(); assertSearchResponse(termQuery); - assertThat(termQuery.getHits().getTotalHits().value, equalTo(1L)); + assertThat(termQuery.getHits().getTotalHits().value(), equalTo(1L)); float termQueryScore = termQuery.getHits().getAt(0).getScore(); for (CombineFunction combineFunction : CombineFunction.values()) { @@ -309,7 +309,7 @@ protected void testMinScoreApplied(CombineFunction boostMode, float expectedScor ) ).get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getScore(), equalTo(expectedScore)); response = client().search( @@ -319,6 +319,6 @@ protected void testMinScoreApplied(CombineFunction boostMode, float expectedScor ).get(); assertSearchResponse(response); - assertThat(response.getHits().getTotalHits().value, equalTo(0L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(0L)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/QueryRescorerIT.java b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/QueryRescorerIT.java index 5121d5023fd95..f1f56a41cb539 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/QueryRescorerIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/QueryRescorerIT.java @@ -174,7 +174,7 @@ public void testRescorePhrase() throws Exception { ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getMaxScore(), equalTo(searchResponse.getHits().getHits()[0].getScore())); assertThat(searchResponse.getHits().getHits()[0].getId(), equalTo("1")); assertThat(searchResponse.getHits().getHits()[1].getId(), equalTo("3")); @@ -450,7 +450,7 @@ private static void assertEquivalent(String query, SearchResponse plain, SearchR assertNoFailures(rescored); SearchHits leftHits = plain.getHits(); SearchHits rightHits = rescored.getHits(); - assertThat(leftHits.getTotalHits().value, equalTo(rightHits.getTotalHits().value)); + assertThat(leftHits.getTotalHits().value(), equalTo(rightHits.getTotalHits().value())); assertThat(leftHits.getHits().length, equalTo(rightHits.getHits().length)); SearchHit[] hits = leftHits.getHits(); SearchHit[] rHits = rightHits.getHits(); @@ -865,7 +865,7 @@ public void testRescorePhaseWithInvalidSort() throws Exception { .setTrackScores(true) .addRescorer(new QueryRescorerBuilder(matchAllQuery()).setRescoreQueryWeight(100.0f), 50) .get(); - assertThat(resp.getHits().getTotalHits().value, equalTo(5L)); + assertThat(resp.getHits().getTotalHits().value(), equalTo(5L)); assertThat(resp.getHits().getHits().length, equalTo(5)); for (SearchHit hit : resp.getHits().getHits()) { assertThat(hit.getScore(), equalTo(101f)); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/RandomScoreFunctionIT.java b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/RandomScoreFunctionIT.java index f1205ba0f1e93..6c6d13e673064 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/functionscore/RandomScoreFunctionIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/functionscore/RandomScoreFunctionIT.java @@ -303,7 +303,7 @@ public void testSeedReportedInExplain() throws Exception { .setExplain(true) .get(); assertNoFailures(resp); - assertEquals(1, resp.getHits().getTotalHits().value); + assertEquals(1, resp.getHits().getTotalHits().value()); SearchHit firstHit = resp.getHits().getAt(0); assertThat(firstHit.getExplanation().toString(), containsString("" + seed)); } @@ -330,7 +330,7 @@ public void testSeedAndNameReportedInExplain() throws Exception { .setExplain(true) .get(); assertNoFailures(resp); - assertEquals(1, resp.getHits().getTotalHits().value); + assertEquals(1, resp.getHits().getTotalHits().value()); SearchHit firstHit = resp.getHits().getAt(0); assertThat(firstHit.getExplanation().getDetails(), arrayWithSize(2)); // "description": "*:* (_name: query1)" @@ -348,11 +348,11 @@ public void testNoDocs() throws Exception { .setQuery(functionScoreQuery(matchAllQuery(), randomFunction().seed(1234).setField(SeqNoFieldMapper.NAME))) .get(); assertNoFailures(resp); - assertEquals(0, resp.getHits().getTotalHits().value); + assertEquals(0, resp.getHits().getTotalHits().value()); resp = client().prepareSearch("test").setQuery(functionScoreQuery(matchAllQuery(), randomFunction())).get(); assertNoFailures(resp); - assertEquals(0, resp.getHits().getTotalHits().value); + assertEquals(0, resp.getHits().getTotalHits().value()); } public void testScoreRange() throws Exception { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoBoundingBoxQueryIT.java b/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoBoundingBoxQueryIT.java index 2f48ea0f64e35..d6e99349cb934 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoBoundingBoxQueryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoBoundingBoxQueryIT.java @@ -128,7 +128,7 @@ public void testSimpleBoundingBoxTest() throws Exception { SearchResponse searchResponse = client().prepareSearch() // from NY .setQuery(geoBoundingBoxQuery("location").setCorners(40.73, -74.1, 40.717, -73.99)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getHits().length, equalTo(2)); for (SearchHit hit : searchResponse.getHits()) { assertThat(hit.getId(), anyOf(equalTo("1"), equalTo("3"), equalTo("5"))); @@ -137,7 +137,7 @@ public void testSimpleBoundingBoxTest() throws Exception { searchResponse = client().prepareSearch() // from NY .setQuery(geoBoundingBoxQuery("location").setCorners(40.73, -74.1, 40.717, -73.99).type("indexed")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getHits().length, equalTo(2)); for (SearchHit hit : searchResponse.getHits()) { assertThat(hit.getId(), anyOf(equalTo("1"), equalTo("3"), equalTo("5"))); @@ -181,7 +181,7 @@ public void testLimit2BoundingBox() throws Exception { .filter(geoBoundingBoxQuery("location").setCorners(74.579421999999994, 143.5, -66.668903999999998, 113.96875)) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery( boolQuery().must(termQuery("userid", 880)) @@ -191,7 +191,7 @@ public void testLimit2BoundingBox() throws Exception { ) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery( @@ -199,7 +199,7 @@ public void testLimit2BoundingBox() throws Exception { .filter(geoBoundingBoxQuery("location").setCorners(74.579421999999994, 143.5, -66.668903999999998, 113.96875)) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery( boolQuery().must(termQuery("userid", 534)) @@ -209,7 +209,7 @@ public void testLimit2BoundingBox() throws Exception { ) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testCompleteLonRange() throws Exception { @@ -246,7 +246,7 @@ public void testCompleteLonRange() throws Exception { SearchResponse searchResponse = client().prepareSearch() .setQuery(geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(50, -180, -50, 180)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery( geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE) @@ -254,11 +254,11 @@ public void testCompleteLonRange() throws Exception { .type("indexed") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery(geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(90, -180, -90, 180)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); searchResponse = client().prepareSearch() .setQuery( geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE) @@ -266,27 +266,27 @@ public void testCompleteLonRange() throws Exception { .type("indexed") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); searchResponse = client().prepareSearch() .setQuery(geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(50, 0, -50, 360)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery( geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(50, 0, -50, 360).type("indexed") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch() .setQuery(geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(90, 0, -90, 360)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); searchResponse = client().prepareSearch() .setQuery( geoBoundingBoxQuery("location").setValidationMethod(GeoValidationMethod.COERCE).setCorners(90, 0, -90, 360).type("indexed") ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); } } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoDistanceIT.java b/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoDistanceIT.java index 272f07e874fdf..e4b7cc00495d5 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoDistanceIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/geo/AbstractGeoDistanceIT.java @@ -191,7 +191,7 @@ public void testSimpleDistanceQuery() { SearchResponse searchResponse = client().prepareSearch() // from NY .setQuery(QueryBuilders.geoDistanceQuery("location").point(40.5, -73.9).distance(25, DistanceUnit.KILOMETERS)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getHits().length, equalTo(2)); for (SearchHit hit : searchResponse.getHits()) { assertThat(hit.getId(), anyOf(equalTo("7"), equalTo("4"))); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/geo/GeoShapeIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/search/geo/GeoShapeIntegrationIT.java index 6dbffa019382d..098d39523e3fd 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/geo/GeoShapeIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/geo/GeoShapeIntegrationIT.java @@ -194,7 +194,7 @@ public void testIgnoreMalformed() throws Exception { indexRandom(true, client().prepareIndex("test").setId("0").setSource("shape", polygonGeoJson)); SearchResponse searchResponse = client().prepareSearch("test").setQuery(matchAllQuery()).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testMappingUpdate() throws Exception { @@ -250,7 +250,7 @@ public void testIndexShapeRouting() throws Exception { .setQuery(geoShapeQuery("shape", "0").indexedShapeIndex("test").indexedShapeRouting("ABC")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testIndexPolygonDateLine() throws Exception { @@ -292,19 +292,19 @@ public void testIndexPolygonDateLine() throws Exception { .setQuery(geoShapeQuery("shape", new PointBuilder(-179.75, 1))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("quad").setQuery(geoShapeQuery("shape", new PointBuilder(90, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); searchResponse = client().prepareSearch("quad").setQuery(geoShapeQuery("shape", new PointBuilder(-180, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("quad").setQuery(geoShapeQuery("shape", new PointBuilder(180, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } finally { ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); @@ -313,19 +313,19 @@ public void testIndexPolygonDateLine() throws Exception { SearchResponse searchResponse = client().prepareSearch("vector").setQuery(geoShapeQuery("shape", new PointBuilder(90, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); searchResponse = client().prepareSearch("vector").setQuery(geoShapeQuery("shape", new PointBuilder(-179.75, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("vector").setQuery(geoShapeQuery("shape", new PointBuilder(-180, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("vector").setQuery(geoShapeQuery("shape", new PointBuilder(180, 1))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } private String findNodeName(String index) { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/geo/LegacyGeoShapeIntegrationIT.java b/server/src/internalClusterTest/java/org/opensearch/search/geo/LegacyGeoShapeIntegrationIT.java index e9115cf7dfbce..fdff223788065 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/geo/LegacyGeoShapeIntegrationIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/geo/LegacyGeoShapeIntegrationIT.java @@ -190,7 +190,7 @@ public void testIgnoreMalformed() throws Exception { indexRandom(true, client().prepareIndex("test").setId("0").setSource("shape", polygonGeoJson)); SearchResponse searchResponse = client().prepareSearch("test").setQuery(matchAllQuery()).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } /** @@ -226,7 +226,7 @@ public void testIndexShapeRouting() throws Exception { .setQuery(geoShapeQuery("shape", "0").indexedShapeIndex("test").indexedShapeRouting("ABC")) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } /** @@ -253,7 +253,7 @@ public void testLegacyCircle() throws Exception { // test self crossing of circles SearchResponse searchResponse = client().prepareSearch("test").setQuery(geoShapeQuery("shape", new Circle(30, 50, 77000))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testDisallowExpensiveQueries() throws InterruptedException, IOException { @@ -281,7 +281,7 @@ public void testDisallowExpensiveQueries() throws InterruptedException, IOExcept SearchResponse searchResponse = client().prepareSearch("test") .setQuery(geoShapeQuery("shape", new Circle(30, 50, 77000))) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", false)); @@ -303,7 +303,7 @@ public void testDisallowExpensiveQueries() throws InterruptedException, IOExcept updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); searchResponse = client().prepareSearch("test").setQuery(geoShapeQuery("shape", new Circle(30, 50, 77000))).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } finally { ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/morelikethis/MoreLikeThisIT.java b/server/src/internalClusterTest/java/org/opensearch/search/morelikethis/MoreLikeThisIT.java index 36fc5de0a5cf7..a9cc85af2d17f 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/morelikethis/MoreLikeThisIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/morelikethis/MoreLikeThisIT.java @@ -853,7 +853,7 @@ public void testWithRouting() throws IOException, InterruptedException { moreLikeThisQueryBuilder.minTermFreq(1); moreLikeThisQueryBuilder.minDocFreq(1); SearchResponse searchResponse = client().prepareSearch("index").setQuery(moreLikeThisQueryBuilder).get(); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); } // Issue #29678 diff --git a/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedExplainIT.java b/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedExplainIT.java index 2efec6a63e6c1..dd1ab81043081 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedExplainIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedExplainIT.java @@ -105,7 +105,7 @@ public void testExplainMultipleDocs() throws Exception { .setExplain(true) .get(); assertNoFailures(nonConSearchResp); - assertThat(nonConSearchResp.getHits().getTotalHits().value, equalTo(1L)); + assertThat(nonConSearchResp.getHits().getTotalHits().value(), equalTo(1L)); Explanation nonConSearchExplain = nonConSearchResp.getHits().getHits()[0].getExplanation(); assertThat(nonConSearchExplain.getValue(), equalTo(nonConSearchResp.getHits().getHits()[0].getScore())); @@ -121,7 +121,7 @@ public void testExplainMultipleDocs() throws Exception { .setExplain(true) .get(); assertNoFailures(conSearchResp); - assertThat(conSearchResp.getHits().getTotalHits().value, equalTo(1L)); + assertThat(conSearchResp.getHits().getTotalHits().value(), equalTo(1L)); Explanation conSearchExplain = conSearchResp.getHits().getHits()[0].getExplanation(); assertThat(conSearchExplain.getValue(), equalTo(conSearchResp.getHits().getHits()[0].getScore())); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedIT.java b/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedIT.java index 19e38da1aed05..9aad23cdb9544 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/nested/SimpleNestedIT.java @@ -96,9 +96,9 @@ public void testSimpleNested() throws Exception { // check on no data, see it works SearchResponse searchResponse = client().prepareSearch("test").get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); searchResponse = client().prepareSearch("test").setQuery(termQuery("n_field1", "n_value1_1")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); client().prepareIndex("test") .setId("1") @@ -129,27 +129,27 @@ public void testSimpleNested() throws Exception { assertDocumentCount("test", 3); searchResponse = client().prepareSearch("test").setQuery(termQuery("n_field1", "n_value1_1")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); // search for something that matches the nested doc, and see that we don't find the nested doc searchResponse = client().prepareSearch("test").setQuery(matchAllQuery()).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test").setQuery(termQuery("n_field1", "n_value1_1")).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); // now, do a nested query searchResponse = client().prepareSearch("test") .setQuery(nestedQuery("nested1", termQuery("nested1.n_field1", "n_value1_1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery(nestedQuery("nested1", termQuery("nested1.n_field1", "n_value1_1"), ScoreMode.Avg)) .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); // add another doc, one that would match if it was not nested... @@ -185,7 +185,7 @@ public void testSimpleNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); // filter searchResponse = client().prepareSearch("test") @@ -201,7 +201,7 @@ public void testSimpleNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); // check with type prefix searchResponse = client().prepareSearch("test") @@ -214,7 +214,7 @@ public void testSimpleNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); // check delete, so all is gone... DeleteResponse deleteResponse = client().prepareDelete("test", "2").get(); @@ -227,7 +227,7 @@ public void testSimpleNested() throws Exception { .setQuery(nestedQuery("nested1", termQuery("nested1.n_field1", "n_value1_1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } public void testMultiNested() throws Exception { @@ -295,13 +295,13 @@ public void testMultiNested() throws Exception { .setQuery(nestedQuery("nested1", termQuery("nested1.field1", "1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery(nestedQuery("nested1.nested2", termQuery("nested1.nested2.field2", "2"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -314,7 +314,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -327,7 +327,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -340,7 +340,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -353,7 +353,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -366,7 +366,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -379,7 +379,7 @@ public void testMultiNested() throws Exception { ) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(0L)); } // When IncludeNestedDocsQuery is wrapped in a FilteredQuery then a in-finite loop occurs b/c of a bug in @@ -493,7 +493,7 @@ public void testExplainWithSingleDoc() throws Exception { .setExplain(true) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); Explanation explanation = searchResponse.getHits().getHits()[0].getExplanation(); assertThat(explanation.getValue(), equalTo(searchResponse.getHits().getHits()[0].getScore())); assertThat(explanation.toString(), startsWith("0.36464313 = Score based on 2 child docs in range from 0 to 1")); @@ -1669,7 +1669,7 @@ public void testCheckFixedBitSetCache() throws Exception { .setQuery(nestedQuery("array1", termQuery("array1.field1", "value1"), ScoreMode.Avg)) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(5L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(5L)); } clusterStatsResponse = client().admin().cluster().prepareClusterStats().get(); assertThat(clusterStatsResponse.getIndicesStats().getSegments().getBitsetMemoryInBytes(), greaterThan(0L)); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/preference/SearchPreferenceIT.java b/server/src/internalClusterTest/java/org/opensearch/search/preference/SearchPreferenceIT.java index bc9eeb528b031..3ed24f86d2127 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/preference/SearchPreferenceIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/preference/SearchPreferenceIT.java @@ -163,25 +163,25 @@ public void testSimplePreference() throws InterruptedException { indexRandomForConcurrentSearch("test"); SearchResponse searchResponse = client().prepareSearch().setQuery(matchAllQuery()).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("_local").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("_primary").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("_primary_first").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("_replica").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("_replica_first").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setPreference("1234").execute().actionGet(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/profile/query/QueryProfilerIT.java b/server/src/internalClusterTest/java/org/opensearch/search/profile/query/QueryProfilerIT.java index 412a94aaf1b3e..ba47e0f49ed2d 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/profile/query/QueryProfilerIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/profile/query/QueryProfilerIT.java @@ -202,10 +202,10 @@ public void testProfileMatchesRegular() throws Exception { ); } - if (vanillaResponse.getHits().getTotalHits().value != profileResponse.getHits().getTotalHits().value) { + if (vanillaResponse.getHits().getTotalHits().value() != profileResponse.getHits().getTotalHits().value()) { Set vanillaSet = new HashSet<>(Arrays.asList(vanillaResponse.getHits().getHits())); Set profileSet = new HashSet<>(Arrays.asList(profileResponse.getHits().getHits())); - if (vanillaResponse.getHits().getTotalHits().value > profileResponse.getHits().getTotalHits().value) { + if (vanillaResponse.getHits().getTotalHits().value() > profileResponse.getHits().getTotalHits().value()) { vanillaSet.removeAll(profileSet); fail("Vanilla hits were larger than profile hits. Non-overlapping elements were: " + vanillaSet.toString()); } else { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/query/ExistsIT.java b/server/src/internalClusterTest/java/org/opensearch/search/query/ExistsIT.java index b95542382e5fc..6168e6b22e93b 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/query/ExistsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/query/ExistsIT.java @@ -159,7 +159,7 @@ public void testExists() throws Exception { assertEquals( String.format(Locale.ROOT, "exists(%s, %d) mapping: %s response: %s", fieldName, count, mapping.toString(), resp), count, - resp.getHits().getTotalHits().value + resp.getHits().getTotalHits().value() ); } catch (AssertionError e) { for (SearchHit searchHit : allDocs.getHits()) { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/query/MultiMatchQueryIT.java b/server/src/internalClusterTest/java/org/opensearch/search/query/MultiMatchQueryIT.java index 392f8b036b7a2..8f541a9dfe3ae 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/query/MultiMatchQueryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/query/MultiMatchQueryIT.java @@ -385,7 +385,7 @@ public void testPhraseType() { ) ) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, greaterThan(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), greaterThan(1L)); searchResponse = client().prepareSearch("test") .setQuery( @@ -464,8 +464,8 @@ public void testSingleField() throws NoSuchFieldException, IllegalAccessExceptio .get(); assertThat( "field: " + field + " query: " + builder.toString(), - multiMatchResp.getHits().getTotalHits().value, - equalTo(matchResp.getHits().getTotalHits().value) + multiMatchResp.getHits().getTotalHits().value(), + equalTo(matchResp.getHits().getTotalHits().value()) ); SearchHits hits = multiMatchResp.getHits(); if (field.startsWith("missing")) { @@ -480,7 +480,7 @@ public void testSingleField() throws NoSuchFieldException, IllegalAccessExceptio } public void testCutoffFreq() throws ExecutionException, InterruptedException { - final long numDocs = client().prepareSearch("test").setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value; + final long numDocs = client().prepareSearch("test").setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value(); MatchQuery.Type type = MatchQuery.Type.BOOLEAN; Float cutoffFrequency = randomBoolean() ? Math.min(1, numDocs * 1.f / between(10, 20)) : 1.f / between(10, 20); SearchResponse searchResponse = client().prepareSearch("test") @@ -515,7 +515,7 @@ public void testCutoffFreq() throws ExecutionException, InterruptedException { .get(); assertFirstHit(searchResponse, anyOf(hasId("theone"), hasId("theother"))); assertThat(searchResponse.getHits().getHits()[0].getScore(), greaterThan(searchResponse.getHits().getHits()[1].getScore())); - long size = searchResponse.getHits().getTotalHits().value; + long size = searchResponse.getHits().getTotalHits().value(); searchResponse = client().prepareSearch("test") .setQuery( @@ -526,7 +526,7 @@ public void testCutoffFreq() throws ExecutionException, InterruptedException { ) .get(); assertFirstHit(searchResponse, anyOf(hasId("theone"), hasId("theother"))); - assertThat("common terms expected to be a way smaller result set", size, lessThan(searchResponse.getHits().getTotalHits().value)); + assertThat("common terms expected to be a way smaller result set", size, lessThan(searchResponse.getHits().getTotalHits().value())); cutoffFrequency = randomBoolean() ? Math.min(1, numDocs * 1.f / between(10, 20)) : 1.f / between(10, 20); searchResponse = client().prepareSearch("test") @@ -580,7 +580,13 @@ public void testCutoffFreq() throws ExecutionException, InterruptedException { public void testEquivalence() { - final int numDocs = (int) client().prepareSearch("test").setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value; + final int numDocs = (int) client().prepareSearch("test") + .setSize(0) + .setQuery(matchAllQuery()) + .get() + .getHits() + .getTotalHits() + .value(); int numIters = scaledRandomIntBetween(5, 10); for (int i = 0; i < numIters; i++) { { @@ -874,7 +880,7 @@ public void testCrossFieldMode() throws ExecutionException, InterruptedException ) .get(); assertFirstHit(searchResponse, anyOf(hasId("theother"), hasId("theone"))); - long numResults = searchResponse.getHits().getTotalHits().value; + long numResults = searchResponse.getHits().getTotalHits().value(); searchResponse = client().prepareSearch("test") .setQuery( @@ -885,7 +891,7 @@ public void testCrossFieldMode() throws ExecutionException, InterruptedException ) ) .get(); - assertThat(numResults, lessThan(searchResponse.getHits().getTotalHits().value)); + assertThat(numResults, lessThan(searchResponse.getHits().getTotalHits().value())); assertFirstHit(searchResponse, hasId("theone")); // test group based on analyzer -- all fields are grouped into a cross field search @@ -1053,7 +1059,7 @@ private static void assertEquivalent(String query, SearchResponse left, SearchRe assertNoFailures(right); SearchHits leftHits = left.getHits(); SearchHits rightHits = right.getHits(); - assertThat(leftHits.getTotalHits().value, equalTo(rightHits.getTotalHits().value)); + assertThat(leftHits.getTotalHits().value(), equalTo(rightHits.getTotalHits().value())); assertThat(leftHits.getHits().length, equalTo(rightHits.getHits().length)); SearchHit[] hits = leftHits.getHits(); SearchHit[] rHits = rightHits.getHits(); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/query/QueryStringIT.java b/server/src/internalClusterTest/java/org/opensearch/search/query/QueryStringIT.java index 8841638328ea4..7c17cd02a4250 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/query/QueryStringIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/query/QueryStringIT.java @@ -476,7 +476,7 @@ public void testFieldAliasOnDisallowedFieldType() throws Exception { } private void assertHits(SearchHits hits, String... ids) { - assertThat(hits.getTotalHits().value, equalTo((long) ids.length)); + assertThat(hits.getTotalHits().value(), equalTo((long) ids.length)); Set hitIds = new HashSet<>(); for (SearchHit hit : hits.getHits()) { hitIds.add(hit.getId()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/query/SearchQueryIT.java b/server/src/internalClusterTest/java/org/opensearch/search/query/SearchQueryIT.java index 3cf63e2f19a16..cf4b0106f8090 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/query/SearchQueryIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/query/SearchQueryIT.java @@ -35,7 +35,7 @@ import com.carrotsearch.randomizedtesting.annotations.ParametersFactory; import org.apache.lucene.analysis.pattern.PatternReplaceCharFilter; -import org.apache.lucene.index.IndexReader; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.join.ScoreMode; import org.apache.lucene.tests.analysis.MockTokenizer; @@ -309,7 +309,7 @@ public void testConstantScoreQuery() throws Exception { for (int i = 0; i < queryRounds; i++) { MatchQueryBuilder matchQuery = matchQuery("f", English.intToEnglish(between(0, num))); searchResponse = client().prepareSearch("test_1").setQuery(constantScoreQuery(matchQuery)).setSize(num).get(); - long totalHits = searchResponse.getHits().getTotalHits().value; + long totalHits = searchResponse.getHits().getTotalHits().value(); SearchHits hits = searchResponse.getHits(); for (SearchHit searchHit : hits) { assertThat(searchHit, hasScore(1.0f)); @@ -322,7 +322,7 @@ public void testConstantScoreQuery() throws Exception { .setSize(num) .get(); hits = searchResponse.getHits(); - assertThat(hits.getTotalHits().value, equalTo(totalHits)); + assertThat(hits.getTotalHits().value(), equalTo(totalHits)); if (totalHits > 1) { float expected = hits.getAt(0).getScore(); for (SearchHit searchHit : hits) { @@ -383,7 +383,7 @@ public void testCommonTermsQuery() throws Exception { searchResponse = client().prepareSearch() .setQuery(commonTermsQuery("field1", "the quick brown").cutoffFrequency(3).lowFreqOperator(Operator.AND)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertFirstHit(searchResponse, hasId("1")); assertSecondHit(searchResponse, hasId("2")); @@ -2030,7 +2030,7 @@ public void testQueryStringParserCache() throws Exception { .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .setQuery(QueryBuilders.queryStringQuery("xyz").boost(100)) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); float first = response.getHits().getAt(0).getScore(); @@ -2040,7 +2040,7 @@ public void testQueryStringParserCache() throws Exception { .setQuery(QueryBuilders.queryStringQuery("xyz").boost(100)) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); float actual = response.getHits().getAt(0).getScore(); assertThat(i + " expected: " + first + " actual: " + actual, Float.compare(first, actual), equalTo(0)); @@ -2293,7 +2293,7 @@ public Map> getTokenizers() { } /** - * Test correct handling {@link SpanBooleanQueryRewriteWithMaxClause#rewrite(IndexReader, MultiTermQuery)}. That rewrite method is e.g. + * Test correct handling {@link SpanBooleanQueryRewriteWithMaxClause#rewrite(IndexSearcher, MultiTermQuery)}. That rewrite method is e.g. * set for fuzzy queries with "constant_score" rewrite nested inside a `span_multi` query and would cause NPEs due to an unset * {@link AttributeSource}. */ @@ -2313,7 +2313,7 @@ public void testIssueFuzzyInsideSpanMulti() throws InterruptedException { * asserts the search response hits include the expected ids */ private void assertHits(SearchHits hits, String... ids) { - assertThat(hits.getTotalHits().value, equalTo((long) ids.length)); + assertThat(hits.getTotalHits().value(), equalTo((long) ids.length)); Set hitIds = new HashSet<>(); for (SearchHit hit : hits.getHits()) { hitIds.add(hit.getId()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/query/SimpleQueryStringIT.java b/server/src/internalClusterTest/java/org/opensearch/search/query/SimpleQueryStringIT.java index f9ccdbd62de1c..ca32c854897c0 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/query/SimpleQueryStringIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/query/SimpleQueryStringIT.java @@ -768,7 +768,7 @@ public void testDynamicClauseCountUpdate() throws Exception { } private void assertHits(SearchHits hits, String... ids) { - assertThat(hits.getTotalHits().value, equalTo((long) ids.length)); + assertThat(hits.getTotalHits().value(), equalTo((long) ids.length)); Set hitIds = new HashSet<>(); for (SearchHit hit : hits.getHits()) { hitIds.add(hit.getId()); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/scriptfilter/ScriptQuerySearchIT.java b/server/src/internalClusterTest/java/org/opensearch/search/scriptfilter/ScriptQuerySearchIT.java index 7dbc61a3ced39..28119c8a4770d 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/scriptfilter/ScriptQuerySearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/scriptfilter/ScriptQuerySearchIT.java @@ -94,18 +94,18 @@ public static class CustomScriptPlugin extends MockScriptPlugin { protected Map, Object>> pluginScripts() { Map, Object>> scripts = new HashMap<>(); - scripts.put("doc['num1'].value", vars -> { + scripts.put("doc['num1'].value()", vars -> { Map doc = (Map) vars.get("doc"); return doc.get("num1"); }); - scripts.put("doc['num1'].value > 1", vars -> { + scripts.put("doc['num1'].value() > 1", vars -> { Map doc = (Map) vars.get("doc"); ScriptDocValues.Doubles num1 = (ScriptDocValues.Doubles) doc.get("num1"); return num1.getValue() > 1; }); - scripts.put("doc['num1'].value > param1", vars -> { + scripts.put("doc['num1'].value() > param1", vars -> { Integer param1 = (Integer) vars.get("param1"); Map doc = (Map) vars.get("doc"); @@ -167,7 +167,7 @@ public void testCustomScriptBinaryField() throws Exception { ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getId(), equalTo("2")); assertThat(response.getHits().getAt(0).getFields().get("sbinaryData").getValues().get(0), equalTo(16)); @@ -210,14 +210,16 @@ public void testCustomScriptBoost() throws Exception { refresh(); indexRandomForConcurrentSearch("test"); - logger.info("running doc['num1'].value > 1"); + logger.info("running doc['num1'].value() > 1"); SearchResponse response = client().prepareSearch() - .setQuery(scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value > 1", Collections.emptyMap()))) + .setQuery( + scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value() > 1", Collections.emptyMap())) + ) .addSort("num1", SortOrder.ASC) - .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value", Collections.emptyMap())) + .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value()", Collections.emptyMap())) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(2L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(2L)); assertThat(response.getHits().getAt(0).getId(), equalTo("2")); assertThat(response.getHits().getAt(0).getFields().get("sNum1").getValues().get(0), equalTo(2.0)); assertThat(response.getHits().getAt(1).getId(), equalTo("3")); @@ -226,14 +228,14 @@ public void testCustomScriptBoost() throws Exception { Map params = new HashMap<>(); params.put("param1", 2); - logger.info("running doc['num1'].value > param1"); + logger.info("running doc['num1'].value() > param1"); response = client().prepareSearch() - .setQuery(scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value > param1", params))) + .setQuery(scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value() > param1", params))) .addSort("num1", SortOrder.ASC) - .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value", Collections.emptyMap())) + .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value()", Collections.emptyMap())) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getId(), equalTo("3")); assertThat(response.getHits().getAt(0).getFields().get("sNum1").getValues().get(0), equalTo(3.0)); @@ -241,12 +243,12 @@ public void testCustomScriptBoost() throws Exception { params.put("param1", -1); logger.info("running doc['num1'].value > param1"); response = client().prepareSearch() - .setQuery(scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value > param1", params))) + .setQuery(scriptQuery(new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value() > param1", params))) .addSort("num1", SortOrder.ASC) - .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value", Collections.emptyMap())) + .addScriptField("sNum1", new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value()", Collections.emptyMap())) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(3L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(3L)); assertThat(response.getHits().getAt(0).getId(), equalTo("1")); assertThat(response.getHits().getAt(0).getFields().get("sNum1").getValues().get(0), equalTo(1.0)); assertThat(response.getHits().getAt(1).getId(), equalTo("2")); @@ -266,7 +268,7 @@ public void testDisallowExpensiveQueries() throws InterruptedException { indexRandomForConcurrentSearch("test-index"); // Execute with search.allow_expensive_queries = null => default value = false => success - Script script = new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value > 1", Collections.emptyMap()); + Script script = new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value() > 1", Collections.emptyMap()); SearchResponse resp = client().prepareSearch("test-index").setQuery(scriptQuery(script)).get(); assertNoFailures(resp); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/scroll/DuelScrollIT.java b/server/src/internalClusterTest/java/org/opensearch/search/scroll/DuelScrollIT.java index 55b3cfeef7419..5d44fe98873e9 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/scroll/DuelScrollIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/scroll/DuelScrollIT.java @@ -82,7 +82,7 @@ public void testDuelQueryThenFetch() throws Exception { .get(); assertNoFailures(control); SearchHits sh = control.getHits(); - assertThat(sh.getTotalHits().value, equalTo((long) context.numDocs)); + assertThat(sh.getTotalHits().value(), equalTo((long) context.numDocs)); assertThat(sh.getHits().length, equalTo(context.numDocs)); SearchResponse searchScrollResponse = client().prepareSearch("index") @@ -93,7 +93,7 @@ public void testDuelQueryThenFetch() throws Exception { .get(); assertNoFailures(searchScrollResponse); - assertThat(searchScrollResponse.getHits().getTotalHits().value, equalTo((long) context.numDocs)); + assertThat(searchScrollResponse.getHits().getTotalHits().value(), equalTo((long) context.numDocs)); assertThat(searchScrollResponse.getHits().getHits().length, equalTo(context.scrollRequestSize)); int counter = 0; @@ -106,7 +106,7 @@ public void testDuelQueryThenFetch() throws Exception { while (true) { searchScrollResponse = client().prepareSearchScroll(scrollId).setScroll("10m").get(); assertNoFailures(searchScrollResponse); - assertThat(searchScrollResponse.getHits().getTotalHits().value, equalTo((long) context.numDocs)); + assertThat(searchScrollResponse.getHits().getTotalHits().value(), equalTo((long) context.numDocs)); if (searchScrollResponse.getHits().getHits().length == 0) { break; } @@ -273,7 +273,7 @@ private void testDuelIndexOrder(SearchType searchType, boolean trackScores, int try { while (true) { assertNoFailures(scroll); - assertEquals(control.getHits().getTotalHits().value, scroll.getHits().getTotalHits().value); + assertEquals(control.getHits().getTotalHits().value(), scroll.getHits().getTotalHits().value()); assertEquals(control.getHits().getMaxScore(), scroll.getHits().getMaxScore(), 0.01f); if (scroll.getHits().getHits().length == 0) { break; @@ -286,7 +286,7 @@ private void testDuelIndexOrder(SearchType searchType, boolean trackScores, int scrollDocs += scroll.getHits().getHits().length; scroll = client().prepareSearchScroll(scroll.getScrollId()).setScroll("10m").get(); } - assertEquals(control.getHits().getTotalHits().value, scrollDocs); + assertEquals(control.getHits().getTotalHits().value(), scrollDocs); } catch (AssertionError e) { logger.info("Control:\n{}", control); logger.info("Scroll size={}, from={}:\n{}", size, scrollDocs, scroll); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/scroll/SearchScrollIT.java b/server/src/internalClusterTest/java/org/opensearch/search/scroll/SearchScrollIT.java index 35b5a7949b20b..a99ccae1ffdd1 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/scroll/SearchScrollIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/scroll/SearchScrollIT.java @@ -140,7 +140,7 @@ public void testSimpleScrollQueryThenFetch() throws Exception { try { long counter = 0; - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -148,7 +148,7 @@ public void testSimpleScrollQueryThenFetch() throws Exception { searchResponse = client().prepareSearchScroll(searchResponse.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -156,7 +156,7 @@ public void testSimpleScrollQueryThenFetch() throws Exception { searchResponse = client().prepareSearchScroll(searchResponse.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(30)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -195,7 +195,7 @@ public void testSimpleScrollQueryThenFetchSmallSizeUnevenDistribution() throws E try { long counter = 0; - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -204,7 +204,7 @@ public void testSimpleScrollQueryThenFetchSmallSizeUnevenDistribution() throws E for (int i = 0; i < 32; i++) { searchResponse = client().prepareSearchScroll(searchResponse.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -214,7 +214,7 @@ public void testSimpleScrollQueryThenFetchSmallSizeUnevenDistribution() throws E // and now, the last one is one searchResponse = client().prepareSearchScroll(searchResponse.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -223,7 +223,7 @@ public void testSimpleScrollQueryThenFetchSmallSizeUnevenDistribution() throws E // a the last is zero searchResponse = client().prepareSearchScroll(searchResponse.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(0)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -254,21 +254,21 @@ public void testScrollAndUpdateIndex() throws Exception { client().admin().indices().prepareRefresh().get(); indexRandomForConcurrentSearch("test"); - assertThat(client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value, equalTo(500L)); + assertThat(client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value(), equalTo(500L)); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), equalTo(500L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), equalTo(500L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), equalTo(0L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), equalTo(0L) ); @@ -289,21 +289,21 @@ public void testScrollAndUpdateIndex() throws Exception { } while (searchResponse.getHits().getHits().length > 0); client().admin().indices().prepareRefresh().get(); - assertThat(client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value, equalTo(500L)); + assertThat(client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value(), equalTo(500L)); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), equalTo(0L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), equalTo(0L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), equalTo(500L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), equalTo(500L) ); } finally { @@ -346,13 +346,13 @@ public void testSimpleScrollQueryThenFetch_clearScrollIds() throws Exception { long counter1 = 0; long counter2 = 0; - assertThat(searchResponse1.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse1.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse1.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse1.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter1++)); } - assertThat(searchResponse2.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse2.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse2.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse2.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter2++)); @@ -362,13 +362,13 @@ public void testSimpleScrollQueryThenFetch_clearScrollIds() throws Exception { searchResponse2 = client().prepareSearchScroll(searchResponse2.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse1.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse1.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse1.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse1.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter1++)); } - assertThat(searchResponse2.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse2.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse2.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse2.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter2++)); @@ -467,13 +467,13 @@ public void testSimpleScrollQueryThenFetchClearAllScrollIds() throws Exception { long counter1 = 0; long counter2 = 0; - assertThat(searchResponse1.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse1.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse1.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse1.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter1++)); } - assertThat(searchResponse2.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse2.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse2.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse2.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter2++)); @@ -483,13 +483,13 @@ public void testSimpleScrollQueryThenFetchClearAllScrollIds() throws Exception { searchResponse2 = client().prepareSearchScroll(searchResponse2.getScrollId()).setScroll(TimeValue.timeValueMinutes(2)).get(); - assertThat(searchResponse1.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse1.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse1.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse1.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter1++)); } - assertThat(searchResponse2.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse2.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse2.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse2.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter2++)); @@ -607,7 +607,7 @@ public void testCloseAndReopenOrDeleteWithActiveScroll() throws InterruptedExcep .addSort("field", SortOrder.ASC) .get(); long counter = 0; - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(100L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(100L)); assertThat(searchResponse.getHits().getHits().length, equalTo(35)); for (SearchHit hit : searchResponse.getHits()) { assertThat(((Number) hit.getSortValues()[0]).longValue(), equalTo(counter++)); @@ -720,7 +720,7 @@ public void testInvalidScrollKeepAlive() throws IOException, InterruptedExceptio .setScroll(TimeValue.timeValueMinutes(5)) .get(); assertNotNull(searchResponse.getScrollId()); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(2L)); assertThat(searchResponse.getHits().getHits().length, equalTo(1)); exc = expectThrows( diff --git a/server/src/internalClusterTest/java/org/opensearch/search/searchafter/SearchAfterIT.java b/server/src/internalClusterTest/java/org/opensearch/search/searchafter/SearchAfterIT.java index 13c510ff21338..89c73d2e59485 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/searchafter/SearchAfterIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/searchafter/SearchAfterIT.java @@ -244,7 +244,7 @@ public void testWithNullStrings() throws InterruptedException { .setQuery(matchAllQuery()) .searchAfter(new Object[] { 0, null }) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, Matchers.equalTo(2L)); + assertThat(searchResponse.getHits().getTotalHits().value(), Matchers.equalTo(2L)); assertThat(searchResponse.getHits().getHits().length, Matchers.equalTo(1)); assertThat(searchResponse.getHits().getHits()[0].getSourceAsMap().get("field1"), Matchers.equalTo(100)); assertThat(searchResponse.getHits().getHits()[0].getSourceAsMap().get("field2"), Matchers.equalTo("toto")); diff --git a/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java b/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java index 98e749aa48cac..2865fb7e991f8 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/simple/SimpleSearchIT.java @@ -375,8 +375,8 @@ public void doTestSimpleTerminateAfterTrackTotalHitsUpTo(int size) throws Except .setTrackTotalHitsUpTo(5) .get(); assertTrue(searchResponse.isTerminatedEarly()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); - assertEquals(GREATER_THAN_OR_EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); + assertEquals(GREATER_THAN_OR_EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); // For size = 0, the following queries terminate early, but hits and relation can vary. if (size > 0) { @@ -387,8 +387,8 @@ public void doTestSimpleTerminateAfterTrackTotalHitsUpTo(int size) throws Except .setTrackTotalHitsUpTo(10) .get(); assertTrue(searchResponse.isTerminatedEarly()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); - assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); + assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); searchResponse = client().prepareSearch("test") .setQuery(QueryBuilders.rangeQuery("field").gte(1).lte(numDocs)) @@ -397,8 +397,8 @@ public void doTestSimpleTerminateAfterTrackTotalHitsUpTo(int size) throws Except .setTrackTotalHitsUpTo(5) .get(); assertTrue(searchResponse.isTerminatedEarly()); - assertEquals(5, searchResponse.getHits().getTotalHits().value); - assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); + assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); } searchResponse = client().prepareSearch("test") @@ -412,9 +412,9 @@ public void doTestSimpleTerminateAfterTrackTotalHitsUpTo(int size) throws Except // Since terminate_after < track_total_hits, we need to do a range check. assertHitCount(searchResponse, 5, numDocs); } else { - assertEquals(5, searchResponse.getHits().getTotalHits().value); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); } - assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); searchResponse = client().prepareSearch("test") .setQuery(QueryBuilders.rangeQuery("field").gte(1).lte(numDocs)) @@ -423,16 +423,16 @@ public void doTestSimpleTerminateAfterTrackTotalHitsUpTo(int size) throws Except .setTrackTotalHits(true) .get(); assertFalse(searchResponse.isTerminatedEarly()); - assertEquals(numDocs, searchResponse.getHits().getTotalHits().value); - assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(numDocs, searchResponse.getHits().getTotalHits().value()); + assertEquals(EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); searchResponse = client().prepareSearch("test") .setQuery(QueryBuilders.rangeQuery("field").gte(1).lte(numDocs)) .setSize(size) .setTrackTotalHitsUpTo(5) .get(); - assertEquals(5, searchResponse.getHits().getTotalHits().value); - assertEquals(GREATER_THAN_OR_EQUAL_TO, searchResponse.getHits().getTotalHits().relation); + assertEquals(5, searchResponse.getHits().getTotalHits().value()); + assertEquals(GREATER_THAN_OR_EQUAL_TO, searchResponse.getHits().getTotalHits().relation()); } public void testSimpleTerminateAfterTrackTotalHitsUpToRandomSize0() throws Exception { @@ -675,7 +675,7 @@ public void testTermQueryBigInt() throws Exception { parser.nextToken(); TermQueryBuilder query = TermQueryBuilder.fromXContent(parser); SearchResponse searchResponse = client().prepareSearch("idx").setQuery(query).get(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); } public void testIndexOnlyFloatField() throws IOException { @@ -692,7 +692,7 @@ public void testIndexOnlyFloatField() throws IOException { parser.nextToken(); ConstantScoreQueryBuilder query = ConstantScoreQueryBuilder.fromXContent(parser); SearchResponse searchResponse = client().prepareSearch("idx").setQuery(query).get(); - assertEquals(1, searchResponse.getHits().getTotalHits().value); + assertEquals(1, searchResponse.getHits().getTotalHits().value()); } public void testTooLongRegexInRegexpQuery() throws Exception { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/slice/SearchSliceIT.java b/server/src/internalClusterTest/java/org/opensearch/search/slice/SearchSliceIT.java index ea73f9ee1a2be..b381d7cbc18f8 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/slice/SearchSliceIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/slice/SearchSliceIT.java @@ -205,7 +205,7 @@ private void assertSearchSlicesWithPIT(SearchRequestBuilder request, String fiel SliceBuilder sliceBuilder = new SliceBuilder(field, id, numSlice); SearchResponse searchResponse = request.slice(sliceBuilder).setFrom(0).get(); totalResults += searchResponse.getHits().getHits().length; - int expectedSliceResults = (int) searchResponse.getHits().getTotalHits().value; + int expectedSliceResults = (int) searchResponse.getHits().getTotalHits().value(); int numSliceResults = searchResponse.getHits().getHits().length; for (SearchHit hit : searchResponse.getHits().getHits()) { assertTrue(keys.add(hit.getId())); @@ -231,7 +231,7 @@ public void testWithPreferenceAndRoutings() throws Exception { setupIndex(totalDocs, numShards); { SearchResponse sr = client().prepareSearch("test").setQuery(matchAllQuery()).setPreference("_shards:1,4").setSize(0).get(); - int numDocs = (int) sr.getHits().getTotalHits().value; + int numDocs = (int) sr.getHits().getTotalHits().value(); int max = randomIntBetween(2, numShards * 3); int fetchSize = randomIntBetween(10, 100); SearchRequestBuilder request = client().prepareSearch("test") @@ -244,7 +244,7 @@ public void testWithPreferenceAndRoutings() throws Exception { } { SearchResponse sr = client().prepareSearch("test").setQuery(matchAllQuery()).setRouting("foo", "bar").setSize(0).get(); - int numDocs = (int) sr.getHits().getTotalHits().value; + int numDocs = (int) sr.getHits().getTotalHits().value(); int max = randomIntBetween(2, numShards * 3); int fetchSize = randomIntBetween(10, 100); SearchRequestBuilder request = client().prepareSearch("test") @@ -266,7 +266,7 @@ public void testWithPreferenceAndRoutings() throws Exception { .get() ); SearchResponse sr = client().prepareSearch("alias1", "alias3").setQuery(matchAllQuery()).setSize(0).get(); - int numDocs = (int) sr.getHits().getTotalHits().value; + int numDocs = (int) sr.getHits().getTotalHits().value(); int max = randomIntBetween(2, numShards * 3); int fetchSize = randomIntBetween(10, 100); SearchRequestBuilder request = client().prepareSearch("alias1", "alias3") @@ -323,7 +323,7 @@ private void assertSearchSlicesWithScroll(SearchRequestBuilder request, String f SliceBuilder sliceBuilder = new SliceBuilder(field, id, numSlice); SearchResponse searchResponse = request.slice(sliceBuilder).get(); totalResults += searchResponse.getHits().getHits().length; - int expectedSliceResults = (int) searchResponse.getHits().getTotalHits().value; + int expectedSliceResults = (int) searchResponse.getHits().getTotalHits().value(); int numSliceResults = searchResponse.getHits().getHits().length; String scrollId = searchResponse.getScrollId(); for (SearchHit hit : searchResponse.getHits().getHits()) { diff --git a/server/src/internalClusterTest/java/org/opensearch/search/sort/FieldSortIT.java b/server/src/internalClusterTest/java/org/opensearch/search/sort/FieldSortIT.java index cc837019d0b42..2017557852865 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/sort/FieldSortIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/sort/FieldSortIT.java @@ -347,7 +347,7 @@ public void testRandomSorting() throws IOException, InterruptedException, Execut .addSort("dense_bytes", SortOrder.ASC) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) numDocs)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) numDocs)); assertThat(searchResponse.getHits().getHits().length, equalTo(size)); Set> entrySet = denseBytes.entrySet(); Iterator> iterator = entrySet.iterator(); @@ -367,7 +367,7 @@ public void testRandomSorting() throws IOException, InterruptedException, Execut .addSort("sparse_bytes", SortOrder.ASC) .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo((long) sparseBytes.size())); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo((long) sparseBytes.size())); assertThat(searchResponse.getHits().getHits().length, equalTo(size)); Set> entrySet = sparseBytes.entrySet(); Iterator> iterator = entrySet.iterator(); @@ -941,7 +941,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -953,7 +953,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -965,7 +965,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -977,7 +977,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -990,7 +990,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1002,7 +1002,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1014,7 +1014,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1026,7 +1026,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1039,7 +1039,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1051,7 +1051,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1063,7 +1063,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1075,7 +1075,7 @@ public void testSortMissingNumbers() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1145,7 +1145,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("3", "2"))); @@ -1158,7 +1158,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("3", "2"))); @@ -1171,7 +1171,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(0).getId(), is(oneOf("2", "1"))); assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("1", "2"))); @@ -1185,7 +1185,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1197,7 +1197,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1209,7 +1209,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1222,7 +1222,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("3", "2"))); @@ -1235,7 +1235,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("3", "2"))); @@ -1248,7 +1248,7 @@ public void testSortMissingNumbersMinMax() throws Exception { .get(); assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); // The order here could be unstable (depends on document order) since missing == field value assertThat(searchResponse.getHits().getAt(0).getId(), is(oneOf("2", "1"))); assertThat(searchResponse.getHits().getAt(1).getId(), is(oneOf("1", "2"))); @@ -1299,7 +1299,7 @@ public void testSortMissingStrings() throws IOException, InterruptedException { .get(); assertThat(Arrays.toString(searchResponse.getShardFailures()), searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1311,7 +1311,7 @@ public void testSortMissingStrings() throws IOException, InterruptedException { .get(); assertThat(Arrays.toString(searchResponse.getShardFailures()), searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("2")); @@ -1323,7 +1323,7 @@ public void testSortMissingStrings() throws IOException, InterruptedException { .get(); assertThat(Arrays.toString(searchResponse.getShardFailures()), searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1335,7 +1335,7 @@ public void testSortMissingStrings() throws IOException, InterruptedException { .get(); assertThat(Arrays.toString(searchResponse.getShardFailures()), searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).getId(), equalTo("2")); assertThat(searchResponse.getHits().getAt(2).getId(), equalTo("3")); @@ -1480,7 +1480,7 @@ public void testSortMVField() throws Exception { .addSort("long_values", SortOrder.ASC) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1494,7 +1494,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("long_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1512,7 +1512,7 @@ public void testSortMVField() throws Exception { .addSort(SortBuilders.fieldSort("long_values").order(SortOrder.DESC).sortMode(SortMode.SUM)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1530,7 +1530,7 @@ public void testSortMVField() throws Exception { .addSort(SortBuilders.fieldSort("long_values").order(SortOrder.DESC).sortMode(SortMode.AVG)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1548,7 +1548,7 @@ public void testSortMVField() throws Exception { .addSort(SortBuilders.fieldSort("long_values").order(SortOrder.DESC).sortMode(SortMode.MEDIAN)) .get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1562,7 +1562,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("int_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1576,7 +1576,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("int_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1590,7 +1590,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("short_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1604,7 +1604,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("short_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1618,7 +1618,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("byte_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1632,7 +1632,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("byte_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1646,7 +1646,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("float_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1660,7 +1660,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("float_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1674,7 +1674,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("half_float_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1688,7 +1688,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("half_float_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1702,7 +1702,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("double_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1716,7 +1716,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("double_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -1730,7 +1730,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("string_values", SortOrder.ASC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(3))); @@ -1744,7 +1744,7 @@ public void testSortMVField() throws Exception { searchResponse = client().prepareSearch().setQuery(matchAllQuery()).setSize(10).addSort("string_values", SortOrder.DESC).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getHits().length, equalTo(3)); assertThat(searchResponse.getHits().getAt(0).getId(), equalTo(Integer.toString(2))); @@ -2067,7 +2067,7 @@ public void testSortDuelBetweenSingleShardAndMultiShardIndex() throws Exception SearchResponse singleShardResponse = client().prepareSearch("test2").setFrom(from).setSize(size).addSort(sortField, order).get(); assertNoFailures(singleShardResponse); - assertThat(multiShardResponse.getHits().getTotalHits().value, equalTo(singleShardResponse.getHits().getTotalHits().value)); + assertThat(multiShardResponse.getHits().getTotalHits().value(), equalTo(singleShardResponse.getHits().getTotalHits().value())); assertThat(multiShardResponse.getHits().getHits().length, equalTo(singleShardResponse.getHits().getHits().length)); for (int i = 0; i < multiShardResponse.getHits().getHits().length; i++) { assertThat( @@ -2091,13 +2091,13 @@ public void testCustomFormat() throws Exception { SearchResponse response = client().prepareSearch("test").addSort(SortBuilders.fieldSort("ip")).get(); assertSearchResponse(response); - assertEquals(2, response.getHits().getTotalHits().value); + assertEquals(2, response.getHits().getTotalHits().value()); assertArrayEquals(new String[] { "192.168.1.7" }, response.getHits().getAt(0).getSortValues()); assertArrayEquals(new String[] { "2001:db8::ff00:42:8329" }, response.getHits().getAt(1).getSortValues()); response = client().prepareSearch("test").addSort(SortBuilders.fieldSort("ip")).searchAfter(new Object[] { "192.168.1.7" }).get(); assertSearchResponse(response); - assertEquals(2, response.getHits().getTotalHits().value); + assertEquals(2, response.getHits().getTotalHits().value()); assertEquals(1, response.getHits().getHits().length); assertArrayEquals(new String[] { "2001:db8::ff00:42:8329" }, response.getHits().getAt(0).getSortValues()); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/sort/SimpleSortIT.java b/server/src/internalClusterTest/java/org/opensearch/search/sort/SimpleSortIT.java index cb8b508c4496b..dc434e6fa3ff9 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/sort/SimpleSortIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/sort/SimpleSortIT.java @@ -398,7 +398,7 @@ public void testDocumentsWithNullValue() throws Exception { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).field("id").getValue(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).field("id").getValue(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).field("id").getValue(), equalTo("2")); @@ -411,7 +411,7 @@ public void testDocumentsWithNullValue() throws Exception { assertNoFailures(searchResponse); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).field("id").getValue(), equalTo("1")); assertThat(searchResponse.getHits().getAt(1).field("id").getValue(), equalTo("3")); assertThat(searchResponse.getHits().getAt(2).field("id").getValue(), equalTo("2")); @@ -430,7 +430,7 @@ public void testDocumentsWithNullValue() throws Exception { } assertThat(searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(3L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(3L)); assertThat(searchResponse.getHits().getAt(0).field("id").getValue(), equalTo("3")); assertThat(searchResponse.getHits().getAt(1).field("id").getValue(), equalTo("1")); assertThat(searchResponse.getHits().getAt(2).field("id").getValue(), equalTo("2")); @@ -450,7 +450,7 @@ public void testDocumentsWithNullValue() throws Exception { } assertThat(searchResponse.getFailedShards(), equalTo(0)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value(), equalTo(1L)); assertThat(searchResponse.getHits().getAt(0).field("id").getValue(), equalTo("2")); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/source/MetadataFetchingIT.java b/server/src/internalClusterTest/java/org/opensearch/search/source/MetadataFetchingIT.java index 4c1e47ef8da99..f7dbac270e3aa 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/source/MetadataFetchingIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/source/MetadataFetchingIT.java @@ -104,12 +104,12 @@ public void testInnerHits() throws InterruptedException { ) ) .get(); - assertThat(response.getHits().getTotalHits().value, equalTo(1L)); + assertThat(response.getHits().getTotalHits().value(), equalTo(1L)); assertThat(response.getHits().getAt(0).getId(), nullValue()); assertThat(response.getHits().getAt(0).getSourceAsString(), nullValue()); assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1)); SearchHits hits = response.getHits().getAt(0).getInnerHits().get("nested"); - assertThat(hits.getTotalHits().value, equalTo(1L)); + assertThat(hits.getTotalHits().value(), equalTo(1L)); assertThat(hits.getAt(0).getId(), nullValue()); assertThat(hits.getAt(0).getSourceAsString(), nullValue()); } diff --git a/server/src/internalClusterTest/java/org/opensearch/search/suggest/CompletionSuggestSearchIT.java b/server/src/internalClusterTest/java/org/opensearch/search/suggest/CompletionSuggestSearchIT.java index c72e128a88045..a54bc8b6e5fff 100644 --- a/server/src/internalClusterTest/java/org/opensearch/search/suggest/CompletionSuggestSearchIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/search/suggest/CompletionSuggestSearchIT.java @@ -1381,7 +1381,7 @@ public void testPrunedSegments() throws IOException { refresh(); assertSuggestions("b"); - assertThat(2L, equalTo(client().prepareSearch(INDEX).setSize(0).get().getHits().getTotalHits().value)); + assertThat(2L, equalTo(client().prepareSearch(INDEX).setSize(0).get().getHits().getTotalHits().value())); for (IndexShardSegments seg : client().admin().indices().prepareSegments().get().getIndices().get(INDEX)) { ShardSegments[] shards = seg.getShards(); for (ShardSegments shardSegments : shards) { diff --git a/server/src/internalClusterTest/java/org/opensearch/similarity/SimilarityIT.java b/server/src/internalClusterTest/java/org/opensearch/similarity/SimilarityIT.java index b89541c647580..7b1d2fa1daec7 100644 --- a/server/src/internalClusterTest/java/org/opensearch/similarity/SimilarityIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/similarity/SimilarityIT.java @@ -107,14 +107,14 @@ public void testCustomBM25Similarity() throws Exception { .setQuery(matchQuery("field1", "quick brown fox")) .execute() .actionGet(); - assertThat(bm25SearchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(bm25SearchResponse.getHits().getTotalHits().value(), equalTo(1L)); float bm25Score = bm25SearchResponse.getHits().getHits()[0].getScore(); SearchResponse booleanSearchResponse = client().prepareSearch() .setQuery(matchQuery("field2", "quick brown fox")) .execute() .actionGet(); - assertThat(booleanSearchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(booleanSearchResponse.getHits().getTotalHits().value(), equalTo(1L)); float defaultScore = booleanSearchResponse.getHits().getHits()[0].getScore(); assertThat(bm25Score, not(equalTo(defaultScore))); diff --git a/server/src/internalClusterTest/java/org/opensearch/snapshots/ConcurrentSnapshotsIT.java b/server/src/internalClusterTest/java/org/opensearch/snapshots/ConcurrentSnapshotsIT.java index 26f8ab5dd8d09..cab750a421f0d 100644 --- a/server/src/internalClusterTest/java/org/opensearch/snapshots/ConcurrentSnapshotsIT.java +++ b/server/src/internalClusterTest/java/org/opensearch/snapshots/ConcurrentSnapshotsIT.java @@ -1000,7 +1000,7 @@ public void testQueuedSnapshotsWaitingForShardReady() throws Exception { index(testIndex, "_doc", Integer.toString(i), "foo", "bar" + i); } refresh(); - assertThat(client().prepareSearch(testIndex).setSize(0).get().getHits().getTotalHits().value, equalTo(100L)); + assertThat(client().prepareSearch(testIndex).setSize(0).get().getHits().getTotalHits().value(), equalTo(100L)); logger.info("--> start relocations"); allowNodes(testIndex, 1); diff --git a/server/src/main/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumerWrapper.java b/server/src/main/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumerWrapper.java index 67ee45f4c9306..580f7a1cc576b 100644 --- a/server/src/main/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumerWrapper.java +++ b/server/src/main/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesConsumerWrapper.java @@ -27,12 +27,20 @@ public class Lucene90DocValuesConsumerWrapper implements Closeable { public Lucene90DocValuesConsumerWrapper( SegmentWriteState state, + int skipIndexIntervalSize, String dataCodec, String dataExtension, String metaCodec, String metaExtension ) throws IOException { - lucene90DocValuesConsumer = new Lucene90DocValuesConsumer(state, dataCodec, dataExtension, metaCodec, metaExtension); + lucene90DocValuesConsumer = new Lucene90DocValuesConsumer( + state, + skipIndexIntervalSize, + dataCodec, + dataExtension, + metaCodec, + metaExtension + ); } public Lucene90DocValuesConsumer getLucene90DocValuesConsumer() { diff --git a/server/src/main/java/org/apache/lucene/queries/BinaryDocValuesRangeQuery.java b/server/src/main/java/org/apache/lucene/queries/BinaryDocValuesRangeQuery.java index 963044a3f58d4..ca4e375bd4a02 100644 --- a/server/src/main/java/org/apache/lucene/queries/BinaryDocValuesRangeQuery.java +++ b/server/src/main/java/org/apache/lucene/queries/BinaryDocValuesRangeQuery.java @@ -42,6 +42,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.util.BytesRef; @@ -84,7 +85,7 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { final BinaryDocValues values = context.reader().getBinaryDocValues(fieldName); if (values == null) { return null; @@ -129,7 +130,9 @@ public float matchCost() { return 4; // at most 4 comparisons } }; - return new ConstantScoreScorer(this, score(), scoreMode, iterator); + + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, iterator); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/apache/lucene/queries/BlendedTermQuery.java b/server/src/main/java/org/apache/lucene/queries/BlendedTermQuery.java index 34e1e210d7137..2edc684677ea5 100644 --- a/server/src/main/java/org/apache/lucene/queries/BlendedTermQuery.java +++ b/server/src/main/java/org/apache/lucene/queries/BlendedTermQuery.java @@ -47,6 +47,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.TermQuery; import org.apache.lucene.util.ArrayUtil; +import org.apache.lucene.util.IOSupplier; import org.apache.lucene.util.InPlaceMergeSorter; import java.io.IOException; @@ -208,7 +209,12 @@ private TermStates adjustTTF(IndexReaderContext readerContext, TermStates termCo int df = termContext.docFreq(); long ttf = sumTTF; for (int i = 0; i < len; i++) { - TermState termState = termContext.get(leaves.get(i)); + final IOSupplier termStateSupplier = termContext.get(leaves.get(i)); + if (termStateSupplier == null) { + continue; + } + + final TermState termState = termStateSupplier.get(); if (termState == null) { continue; } @@ -232,10 +238,16 @@ private static TermStates adjustDF(IndexReaderContext readerContext, TermStates } TermStates newCtx = new TermStates(readerContext); for (int i = 0; i < len; ++i) { - TermState termState = ctx.get(leaves.get(i)); + final IOSupplier termStateSupplier = ctx.get(leaves.get(i)); + if (termStateSupplier == null) { + continue; + } + + final TermState termState = termStateSupplier.get(); if (termState == null) { continue; } + newCtx.register(termState, i, newDocFreq, newTTF); newDocFreq = 0; newTTF = 0; @@ -385,7 +397,7 @@ protected Query topLevelQuery(Term[] terms, TermStates[] ctx, int[] docFreqs, in if (low.clauses().isEmpty()) { BooleanQuery.Builder queryBuilder = new BooleanQuery.Builder(); for (BooleanClause booleanClause : high) { - queryBuilder.add(booleanClause.getQuery(), Occur.MUST); + queryBuilder.add(booleanClause.query(), Occur.MUST); } return queryBuilder.build(); } else if (high.clauses().isEmpty()) { diff --git a/server/src/main/java/org/apache/lucene/search/grouping/CollapseTopFieldDocs.java b/server/src/main/java/org/apache/lucene/search/grouping/CollapseTopFieldDocs.java index 961587113173d..4ab1eee4e089f 100644 --- a/server/src/main/java/org/apache/lucene/search/grouping/CollapseTopFieldDocs.java +++ b/server/src/main/java/org/apache/lucene/search/grouping/CollapseTopFieldDocs.java @@ -192,10 +192,10 @@ public static CollapseTopFieldDocs merge(Sort sort, int start, int size, Collaps final CollapseTopFieldDocs shard = shardHits[shardIDX]; // totalHits can be non-zero even if no hits were // collected, when searchAfter was used: - totalHitCount += shard.totalHits.value; + totalHitCount += shard.totalHits.value(); // If any hit count is a lower bound then the merged // total hit count is a lower bound as well - if (shard.totalHits.relation == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { + if (shard.totalHits.relation() == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { totalHitsRelation = TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO; } if (CollectionUtils.isEmpty(shard.scoreDocs) == false) { diff --git a/server/src/main/java/org/apache/lucene/search/grouping/CollapsingDocValuesSource.java b/server/src/main/java/org/apache/lucene/search/grouping/CollapsingDocValuesSource.java index d747bb47a30ad..dd7f79f8339f5 100644 --- a/server/src/main/java/org/apache/lucene/search/grouping/CollapsingDocValuesSource.java +++ b/server/src/main/java/org/apache/lucene/search/grouping/CollapsingDocValuesSource.java @@ -229,7 +229,7 @@ public void setNextReader(LeafReaderContext readerContext) throws IOException { public boolean advanceExact(int target) throws IOException { if (sorted.advanceExact(target)) { ord = (int) sorted.nextOrd(); - if (sorted.nextOrd() != SortedSetDocValues.NO_MORE_ORDS) { + if (sorted.docValueCount() != 1) { throw new IllegalStateException( "failed to collapse " + target + ", the collapse field must be single valued" ); diff --git a/server/src/main/java/org/apache/lucene/util/automaton/MinimizationOperations.java b/server/src/main/java/org/apache/lucene/util/automaton/MinimizationOperations.java new file mode 100644 index 0000000000000..8c9964cfb5286 --- /dev/null +++ b/server/src/main/java/org/apache/lucene/util/automaton/MinimizationOperations.java @@ -0,0 +1,331 @@ +/* + * dk.brics.automaton + * + * Copyright (c) 2001-2009 Anders Moeller + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +/* + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.apache.lucene.util.automaton; + +import org.apache.lucene.internal.hppc.IntArrayList; +import org.apache.lucene.internal.hppc.IntCursor; +import org.apache.lucene.internal.hppc.IntHashSet; + +import java.util.BitSet; +import java.util.LinkedList; + +/** + * Operations for minimizing automata. + * Copied from https://github.com/apache/lucene/blob/main/lucene/core/src/test/org/apache/lucene/util/automaton/MinimizationOperations.java + * + * @opensearch.experimental + */ +public final class MinimizationOperations { + + private MinimizationOperations() {} + + /** + * Minimizes (and determinizes if not already deterministic) the given automaton using Hopcroft's + * algorithm. + * + * @param determinizeWorkLimit maximum effort to spend determinizing the automaton. Set higher to + * allow more complex queries and lower to prevent memory exhaustion. Use {@link + * Operations#DEFAULT_DETERMINIZE_WORK_LIMIT} as a decent default if you don't otherwise know + * what to specify. + */ + public static Automaton minimize(Automaton a, int determinizeWorkLimit) { + + if (a.getNumStates() == 0 || (a.isAccept(0) == false && a.getNumTransitions(0) == 0)) { + // Fastmatch for common case + return new Automaton(); + } + a = Operations.determinize(a, determinizeWorkLimit); + // a.writeDot("adet"); + if (a.getNumTransitions(0) == 1) { + Transition t = new Transition(); + a.getTransition(0, 0, t); + if (t.dest == 0 && t.min == Character.MIN_CODE_POINT && t.max == Character.MAX_CODE_POINT) { + // Accepts all strings + return a; + } + } + a = Operations.totalize(a); + // a.writeDot("atot"); + + // initialize data structures + final int[] sigma = a.getStartPoints(); + final int sigmaLen = sigma.length, statesLen = a.getNumStates(); + + final IntArrayList[][] reverse = new IntArrayList[statesLen][sigmaLen]; + final IntHashSet[] partition = new IntHashSet[statesLen]; + final IntArrayList[] splitblock = new IntArrayList[statesLen]; + final int[] block = new int[statesLen]; + final StateList[][] active = new StateList[statesLen][sigmaLen]; + final StateListNode[][] active2 = new StateListNode[statesLen][sigmaLen]; + final LinkedList pending = new LinkedList<>(); + final BitSet pending2 = new BitSet(sigmaLen * statesLen); + final BitSet split = new BitSet(statesLen), refine = new BitSet(statesLen), refine2 = new BitSet(statesLen); + for (int q = 0; q < statesLen; q++) { + splitblock[q] = new IntArrayList(); + partition[q] = new IntHashSet(); + for (int x = 0; x < sigmaLen; x++) { + active[q][x] = StateList.EMPTY; + } + } + // find initial partition and reverse edges + Transition transition = new Transition(); + for (int q = 0; q < statesLen; q++) { + final int j = a.isAccept(q) ? 0 : 1; + partition[j].add(q); + block[q] = j; + transition.source = q; + transition.transitionUpto = -1; + for (int x = 0; x < sigmaLen; x++) { + final IntArrayList[] r = reverse[a.next(transition, sigma[x])]; + if (r[x] == null) { + r[x] = new IntArrayList(); + } + r[x].add(q); + } + } + // initialize active sets + for (int j = 0; j <= 1; j++) { + for (int x = 0; x < sigmaLen; x++) { + for (IntCursor qCursor : partition[j]) { + int q = qCursor.value; + if (reverse[q][x] != null) { + StateList stateList = active[j][x]; + if (stateList == StateList.EMPTY) { + stateList = new StateList(); + active[j][x] = stateList; + } + active2[q][x] = stateList.add(q); + } + } + } + } + + // initialize pending + for (int x = 0; x < sigmaLen; x++) { + final int j = (active[0][x].size <= active[1][x].size) ? 0 : 1; + pending.add(new IntPair(j, x)); + pending2.set(x * statesLen + j); + } + + // process pending until fixed point + int k = 2; + // System.out.println("start min"); + while (!pending.isEmpty()) { + // System.out.println(" cycle pending"); + final IntPair ip = pending.removeFirst(); + final int p = ip.n1; + final int x = ip.n2; + // System.out.println(" pop n1=" + ip.n1 + " n2=" + ip.n2); + pending2.clear(x * statesLen + p); + // find states that need to be split off their blocks + for (StateListNode m = active[p][x].first; m != null; m = m.next) { + final IntArrayList r = reverse[m.q][x]; + if (r != null) { + for (IntCursor iCursor : r) { + final int i = iCursor.value; + if (!split.get(i)) { + split.set(i); + final int j = block[i]; + splitblock[j].add(i); + if (!refine2.get(j)) { + refine2.set(j); + refine.set(j); + } + } + } + } + } + + // refine blocks + for (int j = refine.nextSetBit(0); j >= 0; j = refine.nextSetBit(j + 1)) { + final IntArrayList sb = splitblock[j]; + if (sb.size() < partition[j].size()) { + final IntHashSet b1 = partition[j]; + final IntHashSet b2 = partition[k]; + for (IntCursor iCursor : sb) { + final int s = iCursor.value; + b1.remove(s); + b2.add(s); + block[s] = k; + for (int c = 0; c < sigmaLen; c++) { + final StateListNode sn = active2[s][c]; + if (sn != null && sn.sl == active[j][c]) { + sn.remove(); + StateList stateList = active[k][c]; + if (stateList == StateList.EMPTY) { + stateList = new StateList(); + active[k][c] = stateList; + } + active2[s][c] = stateList.add(s); + } + } + } + // update pending + for (int c = 0; c < sigmaLen; c++) { + final int aj = active[j][c].size, ak = active[k][c].size, ofs = c * statesLen; + if (!pending2.get(ofs + j) && 0 < aj && aj <= ak) { + pending2.set(ofs + j); + pending.add(new IntPair(j, c)); + } else { + pending2.set(ofs + k); + pending.add(new IntPair(k, c)); + } + } + k++; + } + refine2.clear(j); + for (IntCursor iCursor : sb) { + final int s = iCursor.value; + split.clear(s); + } + sb.clear(); + } + refine.clear(); + } + + Automaton result = new Automaton(); + + Transition t = new Transition(); + + // System.out.println(" k=" + k); + + // make a new state for each equivalence class, set initial state + int[] stateMap = new int[statesLen]; + int[] stateRep = new int[k]; + + result.createState(); + + // System.out.println("min: k=" + k); + for (int n = 0; n < k; n++) { + // System.out.println(" n=" + n); + + boolean isInitial = partition[n].contains(0); + + int newState; + if (isInitial) { + // System.out.println(" isInitial!"); + newState = 0; + } else { + newState = result.createState(); + } + + // System.out.println(" newState=" + newState); + + for (IntCursor qCursor : partition[n]) { + int q = qCursor.value; + stateMap[q] = newState; + // System.out.println(" q=" + q + " isAccept?=" + a.isAccept(q)); + result.setAccept(newState, a.isAccept(q)); + stateRep[newState] = q; // select representative + } + } + + // build transitions and set acceptance + for (int n = 0; n < k; n++) { + int numTransitions = a.initTransition(stateRep[n], t); + for (int i = 0; i < numTransitions; i++) { + a.getNextTransition(t); + // System.out.println(" add trans"); + result.addTransition(n, stateMap[t.dest], t.min, t.max); + } + } + result.finishState(); + // System.out.println(result.getNumStates() + " states"); + + return Operations.removeDeadStates(result); + } + + static final class IntPair { + + final int n1, n2; + + IntPair(int n1, int n2) { + this.n1 = n1; + this.n2 = n2; + } + } + + static final class StateList { + + // Empty list that should never be mutated, used as a memory saving optimization instead of null + // so we don't need to branch the read path in #minimize + static final StateList EMPTY = new StateList(); + + int size; + + StateListNode first, last; + + StateListNode add(int q) { + assert this != EMPTY; + return new StateListNode(q, this); + } + } + + static final class StateListNode { + + final int q; + + StateListNode next, prev; + + final StateList sl; + + StateListNode(int q, StateList sl) { + this.q = q; + this.sl = sl; + if (sl.size++ == 0) sl.first = sl.last = this; + else { + sl.last.next = this; + prev = sl.last; + sl.last = this; + } + } + + void remove() { + sl.size--; + if (sl.first == this) sl.first = next; + else prev.next = next; + if (sl.last == this) sl.last = prev; + else next.prev = prev; + } + } +} diff --git a/server/src/main/java/org/apache/lucene/util/packed/XPackedInts.java b/server/src/main/java/org/apache/lucene/util/packed/XPackedInts.java index 4260d34ead7c9..4d45485ce2f7c 100644 --- a/server/src/main/java/org/apache/lucene/util/packed/XPackedInts.java +++ b/server/src/main/java/org/apache/lucene/util/packed/XPackedInts.java @@ -534,7 +534,7 @@ public static Reader getDirectReader(IndexInput in) throws IOException { */ public static Mutable getMutable(int valueCount, int bitsPerValue, float acceptableOverheadRatio) { final FormatAndBits formatAndBits = fastestFormatAndBits(valueCount, bitsPerValue, acceptableOverheadRatio); - return getMutable(valueCount, formatAndBits.bitsPerValue, formatAndBits.format); + return getMutable(valueCount, formatAndBits.bitsPerValue(), formatAndBits.format()); } /** @@ -629,7 +629,13 @@ public static Writer getWriter(DataOutput out, int valueCount, int bitsPerValue, assert valueCount >= 0; final FormatAndBits formatAndBits = fastestFormatAndBits(valueCount, bitsPerValue, acceptableOverheadRatio); - final XWriter writer = getWriterNoHeader(out, formatAndBits.format, valueCount, formatAndBits.bitsPerValue, DEFAULT_BUFFER_SIZE); + final XWriter writer = getWriterNoHeader( + out, + formatAndBits.format(), + valueCount, + formatAndBits.bitsPerValue(), + DEFAULT_BUFFER_SIZE + ); writer.writeHeader(); return writer; } diff --git a/server/src/main/java/org/opensearch/action/search/BottomSortValuesCollector.java b/server/src/main/java/org/opensearch/action/search/BottomSortValuesCollector.java index bce8d9fb2b1ca..73d821ff472a2 100644 --- a/server/src/main/java/org/opensearch/action/search/BottomSortValuesCollector.java +++ b/server/src/main/java/org/opensearch/action/search/BottomSortValuesCollector.java @@ -77,7 +77,7 @@ SearchSortValuesAndFormats getBottomSortValues() { } synchronized void consumeTopDocs(TopFieldDocs topDocs, DocValueFormat[] sortValuesFormat) { - totalHits += topDocs.totalHits.value; + totalHits += topDocs.totalHits.value(); if (validateShardSortFields(topDocs.fields) == false) { return; } diff --git a/server/src/main/java/org/opensearch/action/search/SearchPhaseController.java b/server/src/main/java/org/opensearch/action/search/SearchPhaseController.java index d63695447e365..43132b5cf58ab 100644 --- a/server/src/main/java/org/opensearch/action/search/SearchPhaseController.java +++ b/server/src/main/java/org/opensearch/action/search/SearchPhaseController.java @@ -818,8 +818,8 @@ TotalHits getTotalHits() { void add(TopDocsAndMaxScore topDocs, boolean timedOut, Boolean terminatedEarly) { if (trackTotalHitsUpTo != SearchContext.TRACK_TOTAL_HITS_DISABLED) { - totalHits += topDocs.topDocs.totalHits.value; - if (topDocs.topDocs.totalHits.relation == Relation.GREATER_THAN_OR_EQUAL_TO) { + totalHits += topDocs.topDocs.totalHits.value(); + if (topDocs.topDocs.totalHits.relation() == Relation.GREATER_THAN_OR_EQUAL_TO) { totalHitsRelation = TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO; } } diff --git a/server/src/main/java/org/opensearch/bootstrap/BootstrapChecks.java b/server/src/main/java/org/opensearch/bootstrap/BootstrapChecks.java index 485dd43a5999c..0e0b4e9be261a 100644 --- a/server/src/main/java/org/opensearch/bootstrap/BootstrapChecks.java +++ b/server/src/main/java/org/opensearch/bootstrap/BootstrapChecks.java @@ -703,7 +703,7 @@ String jvmVendor() { } String javaVersion() { - return Constants.JAVA_VERSION; + return Runtime.version().toString(); } } diff --git a/server/src/main/java/org/opensearch/common/lucene/Lucene.java b/server/src/main/java/org/opensearch/common/lucene/Lucene.java index 361ee859e3e6a..4845ee9ae09c9 100644 --- a/server/src/main/java/org/opensearch/common/lucene/Lucene.java +++ b/server/src/main/java/org/opensearch/common/lucene/Lucene.java @@ -110,7 +110,7 @@ * @opensearch.internal */ public class Lucene { - public static final String LATEST_CODEC = "Lucene912"; + public static final String LATEST_CODEC = "Lucene101"; public static final String SOFT_DELETES_FIELD = "__soft_deletes"; @@ -423,8 +423,8 @@ public static ScoreDoc readScoreDoc(StreamInput in) throws IOException { private static final Class GEO_DISTANCE_SORT_TYPE_CLASS = LatLonDocValuesField.newDistanceSort("some_geo_field", 0, 0).getClass(); public static void writeTotalHits(StreamOutput out, TotalHits totalHits) throws IOException { - out.writeVLong(totalHits.value); - out.writeEnum(totalHits.relation); + out.writeVLong(totalHits.value()); + out.writeEnum(totalHits.relation()); } public static void writeTopDocs(StreamOutput out, TopDocsAndMaxScore topDocs) throws IOException { diff --git a/server/src/main/java/org/opensearch/common/lucene/MinimumScoreCollector.java b/server/src/main/java/org/opensearch/common/lucene/MinimumScoreCollector.java index 32581f2c843d5..901f9e820ad3f 100644 --- a/server/src/main/java/org/opensearch/common/lucene/MinimumScoreCollector.java +++ b/server/src/main/java/org/opensearch/common/lucene/MinimumScoreCollector.java @@ -34,11 +34,13 @@ import org.apache.lucene.index.LeafReaderContext; import org.apache.lucene.search.Collector; +import org.apache.lucene.search.FilterCollector; +import org.apache.lucene.search.FilterLeafCollector; import org.apache.lucene.search.LeafCollector; import org.apache.lucene.search.Scorable; import org.apache.lucene.search.ScoreCachingWrappingScorer; import org.apache.lucene.search.ScoreMode; -import org.apache.lucene.search.SimpleCollector; +import org.apache.lucene.search.Weight; import java.io.IOException; @@ -47,46 +49,46 @@ * * @opensearch.internal */ -public class MinimumScoreCollector extends SimpleCollector { +public class MinimumScoreCollector extends FilterCollector { - private final Collector collector; private final float minimumScore; - private Scorable scorer; - private LeafCollector leafCollector; - public MinimumScoreCollector(Collector collector, float minimumScore) { - this.collector = collector; + super(collector); this.minimumScore = minimumScore; } public Collector getCollector() { - return collector; + return in; } @Override - public void setScorer(Scorable scorer) throws IOException { - if (!(scorer instanceof ScoreCachingWrappingScorer)) { - scorer = ScoreCachingWrappingScorer.wrap(scorer); - } - this.scorer = scorer; - leafCollector.setScorer(scorer); - } + public LeafCollector getLeafCollector(LeafReaderContext context) throws IOException { + return ScoreCachingWrappingScorer.wrap(new FilterLeafCollector(super.getLeafCollector(context)) { + Scorable scorer; - @Override - public void collect(int doc) throws IOException { - if (scorer.score() >= minimumScore) { - leafCollector.collect(doc); - } + @Override + public void setScorer(Scorable scorer) throws IOException { + this.scorer = scorer; + in.setScorer(scorer); + } + + @Override + public void collect(int doc) throws IOException { + if (scorer.score() >= minimumScore) { + in.collect(doc); + } + } + }); } @Override - public void doSetNextReader(LeafReaderContext context) throws IOException { - leafCollector = collector.getLeafCollector(context); + public void setWeight(Weight weight) { + // Not redirecting to delegate collector to maintain same behaviour when this extended SimpleCollector. } @Override public ScoreMode scoreMode() { - return collector.scoreMode() == ScoreMode.TOP_SCORES ? ScoreMode.TOP_SCORES : ScoreMode.COMPLETE; + return in.scoreMode() == ScoreMode.TOP_SCORES ? ScoreMode.TOP_SCORES : ScoreMode.COMPLETE; } } diff --git a/server/src/main/java/org/opensearch/common/lucene/index/FilterableTermsEnum.java b/server/src/main/java/org/opensearch/common/lucene/index/FilterableTermsEnum.java index 224c5b600c930..4e1ce331571b6 100644 --- a/server/src/main/java/org/opensearch/common/lucene/index/FilterableTermsEnum.java +++ b/server/src/main/java/org/opensearch/common/lucene/index/FilterableTermsEnum.java @@ -50,6 +50,7 @@ import org.apache.lucene.util.BitSet; import org.apache.lucene.util.Bits; import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.IOBooleanSupplier; import org.opensearch.common.Nullable; import java.io.IOException; @@ -256,4 +257,9 @@ public ImpactsEnum impacts(int flags) throws IOException { public BytesRef next() throws IOException { throw new UnsupportedOperationException(UNSUPPORTED_MESSAGE); } + + @Override + public IOBooleanSupplier prepareSeekExact(BytesRef text) throws IOException { + throw new UnsupportedOperationException(UNSUPPORTED_MESSAGE); + } } diff --git a/server/src/main/java/org/opensearch/common/lucene/search/AutomatonQueries.java b/server/src/main/java/org/opensearch/common/lucene/search/AutomatonQueries.java index ada5bc0598478..e98d6634513e8 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/AutomatonQueries.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/AutomatonQueries.java @@ -89,20 +89,20 @@ public static AutomatonQuery caseInsensitiveTermQuery(Term term) { } /** - * Build an automaton matching a wildcard pattern, ASCII case insensitive, if the method is null, then will use {@link MultiTermQuery#CONSTANT_SCORE_REWRITE}. + * Build an automaton matching a wildcard pattern, ASCII case insensitive, if the method is null, then will use {@link MultiTermQuery#CONSTANT_SCORE_BLENDED_REWRITE}. */ public static AutomatonQuery caseInsensitiveWildcardQuery(Term wildcardquery, MultiTermQuery.RewriteMethod method) { return createAutomatonQuery(wildcardquery, toCaseInsensitiveWildcardAutomaton(wildcardquery, Integer.MAX_VALUE), method); } /** - * Build an automaton matching a given pattern with rewrite method, if the rewrite method is null, then will use {@link MultiTermQuery#CONSTANT_SCORE_REWRITE}. + * Build an automaton matching a given pattern with rewrite method, if the rewrite method is null, then will use {@link MultiTermQuery#CONSTANT_SCORE_BLENDED_REWRITE}. */ public static AutomatonQuery createAutomatonQuery(Term term, Automaton automaton, MultiTermQuery.RewriteMethod method) { if (method == null) { - method = MultiTermQuery.CONSTANT_SCORE_REWRITE; + method = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; } - return new AutomatonQuery(term, automaton, Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, false, method); + return new AutomatonQuery(term, automaton, false, method); } /** diff --git a/server/src/main/java/org/opensearch/common/lucene/search/Queries.java b/server/src/main/java/org/opensearch/common/lucene/search/Queries.java index 125eab9512be8..d042938280717 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/Queries.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/Queries.java @@ -37,15 +37,15 @@ import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanClause.Occur; import org.apache.lucene.search.BooleanQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; import org.apache.lucene.search.Explanation; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; -import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Weight; import org.opensearch.OpenSearchException; import org.opensearch.common.Nullable; @@ -91,7 +91,7 @@ public static Query newLenientFieldQuery(String field, RuntimeException e) { * Creates a new non-nested docs query */ public static Query newNonNestedFilter() { - return new DocValuesFieldExistsQuery(SeqNoFieldMapper.PRIMARY_TERM_NAME); + return new FieldExistsQuery(SeqNoFieldMapper.PRIMARY_TERM_NAME); } public static BooleanQuery filtered(@Nullable Query query, @Nullable Query filter) { @@ -137,7 +137,7 @@ public static Query applyMinimumShouldMatch(BooleanQuery query, @Nullable String } int optionalClauses = 0; for (BooleanClause c : query.clauses()) { - if (c.getOccur() == BooleanClause.Occur.SHOULD) { + if (c.occur() == BooleanClause.Occur.SHOULD) { optionalClauses++; } } @@ -232,7 +232,7 @@ public Explanation explain(LeafReaderContext context, int doc) { } @Override - public Scorer scorer(LeafReaderContext context) { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { return null; } diff --git a/server/src/main/java/org/opensearch/common/lucene/search/SpanBooleanQueryRewriteWithMaxClause.java b/server/src/main/java/org/opensearch/common/lucene/search/SpanBooleanQueryRewriteWithMaxClause.java index 65cffa208a47f..399f71de05371 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/SpanBooleanQueryRewriteWithMaxClause.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/SpanBooleanQueryRewriteWithMaxClause.java @@ -43,7 +43,7 @@ import org.apache.lucene.queries.spans.SpanOrQuery; import org.apache.lucene.queries.spans.SpanQuery; import org.apache.lucene.queries.spans.SpanTermQuery; -import org.apache.lucene.search.BooleanQuery; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.Query; import org.apache.lucene.util.AttributeSource; @@ -67,7 +67,7 @@ public class SpanBooleanQueryRewriteWithMaxClause extends SpanMultiTermQueryWrap private final boolean hardLimit; public SpanBooleanQueryRewriteWithMaxClause() { - this(BooleanQuery.getMaxClauseCount(), true); + this(IndexSearcher.getMaxClauseCount(), true); } public SpanBooleanQueryRewriteWithMaxClause(int maxExpansions, boolean hardLimit) { @@ -84,11 +84,11 @@ public boolean isHardLimit() { } @Override - public SpanQuery rewrite(IndexReader reader, MultiTermQuery query) throws IOException { + public SpanQuery rewrite(IndexSearcher searcher, MultiTermQuery query) throws IOException { final MultiTermQuery.RewriteMethod delegate = new MultiTermQuery.RewriteMethod() { @Override - public Query rewrite(IndexReader reader, MultiTermQuery query) throws IOException { - Collection queries = collectTerms(reader, query); + public Query rewrite(IndexSearcher searcher, MultiTermQuery query) throws IOException { + Collection queries = collectTerms(searcher.getIndexReader(), query); if (queries.size() == 0) { return new SpanMatchNoDocsQuery(query.getField(), "no expansion found for " + query.toString()); } else if (queries.size() == 1) { @@ -124,7 +124,7 @@ private Collection collectTerms(IndexReader reader, MultiTermQuery qu + query.toString() + " ] " + "exceeds maxClauseCount [ Boolean maxClauseCount is set to " - + BooleanQuery.getMaxClauseCount() + + IndexSearcher.getMaxClauseCount() + "]" ); } else { @@ -137,6 +137,6 @@ private Collection collectTerms(IndexReader reader, MultiTermQuery qu return queries; } }; - return (SpanQuery) delegate.rewrite(reader, query); + return (SpanQuery) delegate.rewrite(searcher, query); } } diff --git a/server/src/main/java/org/opensearch/common/lucene/search/XMoreLikeThis.java b/server/src/main/java/org/opensearch/common/lucene/search/XMoreLikeThis.java index 49148890abd55..539ebb58d8b5d 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/XMoreLikeThis.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/XMoreLikeThis.java @@ -65,6 +65,7 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.DocIdSetIterator; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.similarities.ClassicSimilarity; @@ -256,7 +257,7 @@ public final class XMoreLikeThis { /** * Return a Query with no more than this many terms. * - * @see BooleanQuery#getMaxClauseCount + * @see IndexSearcher#getMaxClauseCount * @see #getMaxQueryTerms * @see #setMaxQueryTerms */ @@ -711,7 +712,7 @@ private void addToQuery(PriorityQueue q, BooleanQuery.Builder query) try { query.add(tq, BooleanClause.Occur.SHOULD); - } catch (BooleanQuery.TooManyClauses ignore) { + } catch (IndexSearcher.TooManyClauses ignore) { break; } } diff --git a/server/src/main/java/org/opensearch/common/lucene/search/function/FunctionScoreQuery.java b/server/src/main/java/org/opensearch/common/lucene/search/function/FunctionScoreQuery.java index cb93e80288a98..4e6ddc0375130 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/function/FunctionScoreQuery.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/function/FunctionScoreQuery.java @@ -412,12 +412,17 @@ private FunctionFactorScorer functionScorer(LeafReaderContext context) throws IO } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { Scorer scorer = functionScorer(context); if (scorer != null && minScore != null) { scorer = new MinScoreScorer(this, scorer, minScore); } - return scorer; + + if (scorer != null) { + return new DefaultScorerSupplier(scorer); + } else { + return null; + } } @Override @@ -518,7 +523,7 @@ private FunctionFactorScorer( CombineFunction scoreCombiner, boolean needsScores ) throws IOException { - super(scorer, w); + super(scorer); this.scoreMode = scoreMode; this.functions = functions; this.leafFunctions = leafFunctions; diff --git a/server/src/main/java/org/opensearch/common/lucene/search/function/MinScoreScorer.java b/server/src/main/java/org/opensearch/common/lucene/search/function/MinScoreScorer.java index 30be06489dea5..e2b6b9a8f4feb 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/function/MinScoreScorer.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/function/MinScoreScorer.java @@ -53,7 +53,7 @@ final class MinScoreScorer extends Scorer { private float curScore; MinScoreScorer(Weight weight, Scorer scorer, float minScore) { - super(weight); + super(); this.in = scorer; this.minScore = minScore; } diff --git a/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreFunction.java b/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreFunction.java index 38c356a8be4b0..54299e15de92a 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreFunction.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreFunction.java @@ -52,14 +52,8 @@ public class ScriptScoreFunction extends ScoreFunction { static final class CannedScorer extends Scorable { - protected int docid; protected float score; - @Override - public int docID() { - return docid; - } - @Override public float score() { return score; @@ -104,7 +98,6 @@ public LeafScoreFunction getLeafScoreFunction(LeafReaderContext ctx) throws IOEx @Override public double score(int docId, float subQueryScore) throws IOException { leafScript.setDocument(docId); - scorer.docid = docId; scorer.score = subQueryScore; double result = leafScript.execute(null); if (result < 0f) { @@ -118,7 +111,6 @@ public Explanation explainScore(int docId, Explanation subQueryScore) throws IOE Explanation exp; if (leafScript instanceof ExplainableScoreScript) { leafScript.setDocument(docId); - scorer.docid = docId; scorer.score = subQueryScore.getValue().floatValue(); exp = ((ExplainableScoreScript) leafScript).explain(subQueryScore, functionName); } else { diff --git a/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreQuery.java b/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreQuery.java index 5aff09d715622..6a2b7ded93b4f 100644 --- a/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreQuery.java +++ b/server/src/main/java/org/opensearch/common/lucene/search/function/ScriptScoreQuery.java @@ -45,6 +45,7 @@ import org.apache.lucene.search.Scorable; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.util.Bits; @@ -124,29 +125,73 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new Weight(this) { @Override - public BulkScorer bulkScorer(LeafReaderContext context) throws IOException { - if (minScore == null) { - final BulkScorer subQueryBulkScorer = subQueryWeight.bulkScorer(context); - if (subQueryBulkScorer == null) { - return null; + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { + final Weight weight = this; + return new ScorerSupplier() { + private Scorer scorer; + private BulkScorer bulkScorer; + + @Override + public BulkScorer bulkScorer() throws IOException { + if (minScore == null) { + final BulkScorer subQueryBulkScorer = subQueryWeight.bulkScorer(context); + if (subQueryBulkScorer == null) { + bulkScorer = null; + } else { + bulkScorer = new ScriptScoreBulkScorer( + subQueryBulkScorer, + subQueryScoreMode, + makeScoreScript(context), + boost + ); + } + } else { + final Scorer scorer = get(Long.MAX_VALUE); + if (scorer != null) { + bulkScorer = new Weight.DefaultBulkScorer(scorer); + } + } + + return bulkScorer; } - return new ScriptScoreBulkScorer(subQueryBulkScorer, subQueryScoreMode, makeScoreScript(context), boost); - } else { - return super.bulkScorer(context); - } - } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - Scorer subQueryScorer = subQueryWeight.scorer(context); - if (subQueryScorer == null) { - return null; - } - Scorer scriptScorer = new ScriptScorer(this, makeScoreScript(context), subQueryScorer, subQueryScoreMode, boost, null); - if (minScore != null) { - scriptScorer = new MinScoreScorer(this, scriptScorer, minScore); - } - return scriptScorer; + @Override + public Scorer get(long leadCost) throws IOException { + final Scorer subQueryScorer = subQueryWeight.scorer(context); + + if (subQueryScorer == null) { + scorer = null; + } else { + Scorer scriptScorer = new ScriptScorer( + weight, + makeScoreScript(context), + subQueryScorer, + subQueryScoreMode, + boost, + null + ); + if (minScore != null) { + scriptScorer = new MinScoreScorer(weight, scriptScorer, minScore); + } + scorer = scriptScorer; + } + + return scorer; + } + + @Override + public long cost() { + if (scorer != null) { + return scorer.iterator().cost(); + } else if (bulkScorer != null) { + return bulkScorer.cost(); + } else { + // We have no prior knowledge of how many docs might match for any given query term, + // so we assume that all docs could be a match. + return Integer.MAX_VALUE; + } + } + }; } @Override @@ -265,7 +310,7 @@ private static class ScriptScorer extends Scorer { float boost, ExplanationHolder explanation ) { - super(weight); + super(); this.scoreScript = scoreScript; if (subQueryScoreMode == ScoreMode.COMPLETE) { scoreScript.setScorer(subQueryScorer); @@ -325,6 +370,7 @@ private static class ScriptScorable extends Scorable { private final Scorable subQueryScorer; private final float boost; private final ExplanationHolder explanation; + private int docId; ScriptScorable( ScoreScript scoreScript, @@ -342,9 +388,12 @@ private static class ScriptScorable extends Scorable { this.explanation = explanation; } + void setDocument(int docId) { + this.docId = docId; + } + @Override public float score() throws IOException { - int docId = docID(); scoreScript.setDocument(docId); float score = (float) scoreScript.execute(explanation); if (score < 0f || Float.isNaN(score)) { @@ -359,11 +408,6 @@ public float score() throws IOException { } return score * boost; } - - @Override - public int docID() { - return subQueryScorer.docID(); - } } /** @@ -392,9 +436,18 @@ public int score(LeafCollector collector, Bits acceptDocs, int min, int max) thr private LeafCollector wrapCollector(LeafCollector collector) { return new FilterLeafCollector(collector) { + private ScriptScorable scriptScorable; + @Override public void setScorer(Scorable scorer) throws IOException { - in.setScorer(new ScriptScorable(scoreScript, scorer, subQueryScoreMode, boost, null)); + scriptScorable = new ScriptScorable(scoreScript, scorer, subQueryScoreMode, boost, null); + in.setScorer(scriptScorable); + } + + @Override + public void collect(int doc) throws IOException { + scriptScorable.setDocument(doc); + super.collect(doc); } }; } diff --git a/server/src/main/java/org/opensearch/common/util/concurrent/ThreadContext.java b/server/src/main/java/org/opensearch/common/util/concurrent/ThreadContext.java index 75a7ef94978d4..8b8183d405323 100644 --- a/server/src/main/java/org/opensearch/common/util/concurrent/ThreadContext.java +++ b/server/src/main/java/org/opensearch/common/util/concurrent/ThreadContext.java @@ -221,6 +221,7 @@ public Writeable captureAsWriteable() { * * permission org.opensearch.secure_sm.ThreadContextPermission "stashWithOrigin"; */ + @SuppressWarnings("removal") public StoredContext stashWithOrigin(String origin) { SecurityManager sm = System.getSecurityManager(); if (sm != null) { @@ -243,6 +244,7 @@ public StoredContext stashWithOrigin(String origin) { * * permission org.opensearch.secure_sm.ThreadContextPermission "stashAndMergeHeaders"; */ + @SuppressWarnings("removal") public StoredContext stashAndMergeHeaders(Map headers) { SecurityManager sm = System.getSecurityManager(); if (sm != null) { @@ -601,6 +603,7 @@ boolean isDefaultContext() { * * permission org.opensearch.secure_sm.ThreadContextPermission "markAsSystemContext"; */ + @SuppressWarnings("removal") public void markAsSystemContext() { SecurityManager sm = System.getSecurityManager(); if (sm != null) { diff --git a/server/src/main/java/org/opensearch/common/xcontent/support/XContentMapValues.java b/server/src/main/java/org/opensearch/common/xcontent/support/XContentMapValues.java index a87edbb949d39..d3fa44c5afb66 100644 --- a/server/src/main/java/org/opensearch/common/xcontent/support/XContentMapValues.java +++ b/server/src/main/java/org/opensearch/common/xcontent/support/XContentMapValues.java @@ -246,9 +246,9 @@ public static Map filter(Map map, String[] includes, * For instance, if the original simple regex is `foo`, this will translate * it into `foo` OR `foo.*`. */ private static Automaton makeMatchDotsInFieldNames(Automaton automaton) { - return Operations.union( - automaton, - Operations.concatenate(Arrays.asList(automaton, Automata.makeChar('.'), Automata.makeAnyString())) + return Operations.determinize( + Operations.union(automaton, Operations.concatenate(Arrays.asList(automaton, Automata.makeChar('.'), Automata.makeAnyString()))), + Operations.DEFAULT_DETERMINIZE_WORK_LIMIT ); } diff --git a/server/src/main/java/org/opensearch/index/IndexModule.java b/server/src/main/java/org/opensearch/index/IndexModule.java index 79de97dc96fba..691efe74eea96 100644 --- a/server/src/main/java/org/opensearch/index/IndexModule.java +++ b/server/src/main/java/org/opensearch/index/IndexModule.java @@ -38,7 +38,6 @@ import org.apache.lucene.index.LeafReader; import org.apache.lucene.search.similarities.BM25Similarity; import org.apache.lucene.search.similarities.Similarity; -import org.apache.lucene.store.MMapDirectory; import org.apache.lucene.util.Constants; import org.opensearch.Version; import org.opensearch.client.Client; @@ -590,7 +589,7 @@ public static DataLocalityType getValueOf(final String localityType) { } public static Type defaultStoreType(final boolean allowMmap) { - if (allowMmap && Constants.JRE_IS_64BIT && MMapDirectory.UNMAP_SUPPORTED) { + if (allowMmap && Constants.JRE_IS_64BIT) { return Type.HYBRIDFS; } else { return Type.NIOFS; diff --git a/server/src/main/java/org/opensearch/index/codec/CodecService.java b/server/src/main/java/org/opensearch/index/codec/CodecService.java index 3a93795ef61ec..1568e262fb6b9 100644 --- a/server/src/main/java/org/opensearch/index/codec/CodecService.java +++ b/server/src/main/java/org/opensearch/index/codec/CodecService.java @@ -34,8 +34,8 @@ import org.apache.logging.log4j.Logger; import org.apache.lucene.codecs.Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec.Mode; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec.Mode; import org.opensearch.common.Nullable; import org.opensearch.common.collect.MapBuilder; import org.opensearch.index.IndexSettings; @@ -70,10 +70,10 @@ public CodecService(@Nullable MapperService mapperService, IndexSettings indexSe final MapBuilder codecs = MapBuilder.newMapBuilder(); assert null != indexSettings; if (mapperService == null) { - codecs.put(DEFAULT_CODEC, new Lucene912Codec()); - codecs.put(LZ4, new Lucene912Codec()); - codecs.put(BEST_COMPRESSION_CODEC, new Lucene912Codec(Mode.BEST_COMPRESSION)); - codecs.put(ZLIB, new Lucene912Codec(Mode.BEST_COMPRESSION)); + codecs.put(DEFAULT_CODEC, new Lucene101Codec()); + codecs.put(LZ4, new Lucene101Codec()); + codecs.put(BEST_COMPRESSION_CODEC, new Lucene101Codec(Mode.BEST_COMPRESSION)); + codecs.put(ZLIB, new Lucene101Codec(Mode.BEST_COMPRESSION)); } else { // CompositeCodec still delegates to PerFieldMappingPostingFormatCodec // We can still support all the compression codecs when composite index is present diff --git a/server/src/main/java/org/opensearch/index/codec/PerFieldMappingPostingFormatCodec.java b/server/src/main/java/org/opensearch/index/codec/PerFieldMappingPostingFormatCodec.java index 0ed8c3880f2ea..b6cc0a79ca62a 100644 --- a/server/src/main/java/org/opensearch/index/codec/PerFieldMappingPostingFormatCodec.java +++ b/server/src/main/java/org/opensearch/index/codec/PerFieldMappingPostingFormatCodec.java @@ -36,8 +36,8 @@ import org.apache.lucene.codecs.Codec; import org.apache.lucene.codecs.DocValuesFormat; import org.apache.lucene.codecs.PostingsFormat; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.codecs.lucene90.Lucene90DocValuesFormat; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; import org.opensearch.common.lucene.Lucene; import org.opensearch.index.codec.fuzzy.FuzzyFilterPostingsFormat; import org.opensearch.index.codec.fuzzy.FuzzySetFactory; @@ -59,7 +59,7 @@ * * @opensearch.internal */ -public class PerFieldMappingPostingFormatCodec extends Lucene912Codec { +public class PerFieldMappingPostingFormatCodec extends Lucene101Codec { private final Logger logger; private final MapperService mapperService; private final DocValuesFormat dvFormat = new Lucene90DocValuesFormat(); diff --git a/server/src/main/java/org/opensearch/index/codec/composite/CompositeCodecFactory.java b/server/src/main/java/org/opensearch/index/codec/composite/CompositeCodecFactory.java index 674773bb86354..ff300339bb125 100644 --- a/server/src/main/java/org/opensearch/index/codec/composite/CompositeCodecFactory.java +++ b/server/src/main/java/org/opensearch/index/codec/composite/CompositeCodecFactory.java @@ -10,9 +10,9 @@ import org.apache.logging.log4j.Logger; import org.apache.lucene.codecs.Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.opensearch.common.annotation.ExperimentalApi; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.index.mapper.MapperService; import java.util.HashMap; @@ -32,16 +32,16 @@ public class CompositeCodecFactory { // we can use this to track the latest composite codec - public static final String COMPOSITE_CODEC = Composite912Codec.COMPOSITE_INDEX_CODEC_NAME; + public static final String COMPOSITE_CODEC = Composite101Codec.COMPOSITE_INDEX_CODEC_NAME; public CompositeCodecFactory() {} public Map getCompositeIndexCodecs(MapperService mapperService, Logger logger) { Map codecs = new HashMap<>(); - codecs.put(DEFAULT_CODEC, new Composite912Codec(Lucene912Codec.Mode.BEST_SPEED, mapperService, logger)); - codecs.put(LZ4, new Composite912Codec(Lucene912Codec.Mode.BEST_SPEED, mapperService, logger)); - codecs.put(BEST_COMPRESSION_CODEC, new Composite912Codec(Lucene912Codec.Mode.BEST_COMPRESSION, mapperService, logger)); - codecs.put(ZLIB, new Composite912Codec(Lucene912Codec.Mode.BEST_COMPRESSION, mapperService, logger)); + codecs.put(DEFAULT_CODEC, new Composite101Codec(Lucene101Codec.Mode.BEST_SPEED, mapperService, logger)); + codecs.put(LZ4, new Composite101Codec(Lucene101Codec.Mode.BEST_SPEED, mapperService, logger)); + codecs.put(BEST_COMPRESSION_CODEC, new Composite101Codec(Lucene101Codec.Mode.BEST_COMPRESSION, mapperService, logger)); + codecs.put(ZLIB, new Composite101Codec(Lucene101Codec.Mode.BEST_COMPRESSION, mapperService, logger)); return codecs; } } diff --git a/server/src/main/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactory.java b/server/src/main/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactory.java index 4b3f62b6171da..6d79275606d24 100644 --- a/server/src/main/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactory.java +++ b/server/src/main/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactory.java @@ -33,9 +33,28 @@ public static DocValuesConsumer getDocValuesConsumerForCompositeCodec( String dataExtension, String metaCodec, String metaExtension + ) throws IOException { + return getDocValuesConsumerForCompositeCodec( + state, + 4096, + /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ dataCodec, + dataExtension, + metaCodec, + metaExtension + ); + } + + public static DocValuesConsumer getDocValuesConsumerForCompositeCodec( + SegmentWriteState state, + int skipIndexIntervalSize, + String dataCodec, + String dataExtension, + String metaCodec, + String metaExtension ) throws IOException { Lucene90DocValuesConsumerWrapper lucene90DocValuesConsumerWrapper = new Lucene90DocValuesConsumerWrapper( state, + skipIndexIntervalSize, dataCodec, dataExtension, metaCodec, diff --git a/server/src/main/java/org/opensearch/index/codec/composite/composite101/Composite101Codec.java b/server/src/main/java/org/opensearch/index/codec/composite/composite101/Composite101Codec.java new file mode 100644 index 0000000000000..5925cdadf0f17 --- /dev/null +++ b/server/src/main/java/org/opensearch/index/codec/composite/composite101/Composite101Codec.java @@ -0,0 +1,58 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +package org.opensearch.index.codec.composite.composite101; + +import org.apache.logging.log4j.Logger; +import org.apache.lucene.codecs.Codec; +import org.apache.lucene.codecs.DocValuesFormat; +import org.apache.lucene.codecs.FilterCodec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.opensearch.common.annotation.ExperimentalApi; +import org.opensearch.index.codec.PerFieldMappingPostingFormatCodec; +import org.opensearch.index.codec.composite.composite912.Composite912DocValuesFormat; +import org.opensearch.index.mapper.MapperService; + +/** + * Extends the Codec to support new file formats for composite indices eg: star tree index + * based on the mappings. + * + * @opensearch.experimental + */ +@ExperimentalApi +public class Composite101Codec extends FilterCodec { + public static final String COMPOSITE_INDEX_CODEC_NAME = "Composite101Codec"; + private final MapperService mapperService; + + // needed for SPI - this is used in reader path + public Composite101Codec() { + this(COMPOSITE_INDEX_CODEC_NAME, new Lucene101Codec(), null); + } + + public Composite101Codec(Lucene101Codec.Mode compressionMode, MapperService mapperService, Logger logger) { + this(COMPOSITE_INDEX_CODEC_NAME, new PerFieldMappingPostingFormatCodec(compressionMode, mapperService, logger), mapperService); + } + + /** + * Sole constructor. When subclassing this codec, create a no-arg ctor and pass the delegate codec and a unique name to + * this ctor. + * + * @param name name of the codec + * @param delegate codec delegate + * @param mapperService mapper service instance + */ + protected Composite101Codec(String name, Codec delegate, MapperService mapperService) { + super(name, delegate); + this.mapperService = mapperService; + } + + @Override + public DocValuesFormat docValuesFormat() { + return new Composite912DocValuesFormat(mapperService); + } +} diff --git a/server/src/main/java/org/opensearch/index/codec/composite/composite101/package-info.java b/server/src/main/java/org/opensearch/index/codec/composite/composite101/package-info.java new file mode 100644 index 0000000000000..3c9c366f2e74d --- /dev/null +++ b/server/src/main/java/org/opensearch/index/codec/composite/composite101/package-info.java @@ -0,0 +1,12 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +/** + * Responsible for handling all composite index codecs and operations associated with Composite100 codec + */ +package org.opensearch.index.codec.composite.composite101; diff --git a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912Codec.java b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912Codec.java index 6f7a3c6852cf6..35b28db54733a 100644 --- a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912Codec.java +++ b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912Codec.java @@ -8,13 +8,11 @@ package org.opensearch.index.codec.composite.composite912; -import org.apache.logging.log4j.Logger; +import org.apache.lucene.backward_codecs.lucene912.Lucene912Codec; import org.apache.lucene.codecs.Codec; import org.apache.lucene.codecs.DocValuesFormat; import org.apache.lucene.codecs.FilterCodec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; import org.opensearch.common.annotation.ExperimentalApi; -import org.opensearch.index.codec.PerFieldMappingPostingFormatCodec; import org.opensearch.index.mapper.MapperService; /** @@ -33,10 +31,6 @@ public Composite912Codec() { this(COMPOSITE_INDEX_CODEC_NAME, new Lucene912Codec(), null); } - public Composite912Codec(Lucene912Codec.Mode compressionMode, MapperService mapperService, Logger logger) { - this(COMPOSITE_INDEX_CODEC_NAME, new PerFieldMappingPostingFormatCodec(compressionMode, mapperService, logger), mapperService); - } - /** * Sole constructor. When subclassing this codec, create a no-arg ctor and pass the delegate codec and a unique name to * this ctor. diff --git a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesReader.java b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesReader.java index 38d3f4867e89b..7178ffbadf9f1 100644 --- a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesReader.java +++ b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesReader.java @@ -14,6 +14,8 @@ import org.apache.lucene.codecs.DocValuesProducer; import org.apache.lucene.index.BinaryDocValues; import org.apache.lucene.index.CorruptIndexException; +import org.apache.lucene.index.DocValues; +import org.apache.lucene.index.DocValuesSkipper; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; @@ -25,6 +27,7 @@ import org.apache.lucene.index.SortedSetDocValues; import org.apache.lucene.store.ChecksumIndexInput; import org.apache.lucene.store.IndexInput; +import org.apache.lucene.store.ReadAdvice; import org.opensearch.common.annotation.ExperimentalApi; import org.opensearch.common.util.io.IOUtils; import org.opensearch.index.codec.composite.CompositeIndexFieldInfo; @@ -62,7 +65,6 @@ public class Composite912DocValuesReader extends DocValuesProducer implements Co private final DocValuesProducer delegate; private IndexInput dataIn; - private ChecksumIndexInput metaIn; private final Map compositeIndexInputMap = new LinkedHashMap<>(); private final Map compositeIndexMetadataMap = new LinkedHashMap<>(); private final List fields; @@ -87,10 +89,10 @@ public Composite912DocValuesReader(DocValuesProducer producer, SegmentReadState ); boolean success = false; - try { + try (ChecksumIndexInput metaIn = readState.directory.openChecksumInput(metaFileName)) { - // initialize meta input - dataIn = readState.directory.openInput(dataFileName, readState.context); + // initialize data input + dataIn = readState.directory.openInput(dataFileName, readState.context.withReadAdvice(ReadAdvice.NORMAL)); CodecUtil.checkIndexHeader( dataIn, Composite912DocValuesFormat.DATA_CODEC_NAME, @@ -100,8 +102,7 @@ public Composite912DocValuesReader(DocValuesProducer producer, SegmentReadState readState.segmentSuffix ); - // initialize data input - metaIn = readState.directory.openChecksumInput(metaFileName, readState.context); + // initialize meta input Throwable priorE = null; try { CodecUtil.checkIndexHeader( @@ -257,17 +258,16 @@ public void close() throws IOException { delegate.close(); boolean success = false; try { - IOUtils.close(metaIn, dataIn); + IOUtils.close(dataIn); IOUtils.close(compositeDocValuesProducer); success = true; } finally { if (!success) { - IOUtils.closeWhileHandlingException(metaIn, dataIn); + IOUtils.closeWhileHandlingException(dataIn); } compositeIndexInputMap.clear(); compositeIndexMetadataMap.clear(); fields.clear(); - metaIn = null; dataIn = null; } } @@ -295,4 +295,21 @@ public CompositeIndexValues getCompositeIndexValues(CompositeIndexFieldInfo comp } + /** + * Returns the sorted numeric doc values for the given sorted numeric field. + * If the sorted numeric field is null, it returns an empty doc id set iterator. + *

+ * Sorted numeric field can be null for cases where the segment doesn't hold a particular value. + * + * @param sortedNumeric the sorted numeric doc values for a field + * @return empty sorted numeric values if the field is not present, else sortedNumeric + */ + public static SortedNumericDocValues getSortedNumericDocValues(SortedNumericDocValues sortedNumeric) { + return sortedNumeric == null ? DocValues.emptySortedNumeric() : sortedNumeric; + } + + @Override + public DocValuesSkipper getSkipper(FieldInfo field) throws IOException { + return delegate.getSkipper(field); + } } diff --git a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesWriter.java b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesWriter.java index ca52d8bf4bca0..55fbef3754cf9 100644 --- a/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesWriter.java +++ b/server/src/main/java/org/opensearch/index/codec/composite/composite912/Composite912DocValuesWriter.java @@ -101,6 +101,7 @@ public Composite912DocValuesWriter(DocValuesConsumer delegate, SegmentWriteState this.compositeDocValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( consumerWriteState, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, diff --git a/server/src/main/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtils.java b/server/src/main/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtils.java index 240a727678d6f..e018aefe99b19 100644 --- a/server/src/main/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtils.java +++ b/server/src/main/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtils.java @@ -7,6 +7,7 @@ */ package org.opensearch.index.compositeindex.datacube.startree.utils; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -100,6 +101,7 @@ public static FieldInfo getFieldInfo(String fieldName, DocValuesType docValuesTy true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, docValuesType, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, @@ -133,6 +135,7 @@ public static FieldInfo getFieldInfo(String fieldName, DocValuesType docValuesTy true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, docValuesType, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, diff --git a/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesRangeQuery.java b/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesRangeQuery.java index 234c67cc637f1..ca6c1f74bb535 100644 --- a/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesRangeQuery.java +++ b/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesRangeQuery.java @@ -22,6 +22,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.opensearch.common.Numbers; @@ -106,7 +107,7 @@ public boolean isCacheable(LeafReaderContext ctx) { } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { SortedNumericDocValues values = getValues(context.reader(), field); if (values == null) { return null; @@ -148,7 +149,8 @@ public float matchCost() { } }; } - return new ConstantScoreScorer(this, score(), scoreMode, iterator); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, iterator); + return new DefaultScorerSupplier(scorer); } }; } diff --git a/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesSetQuery.java b/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesSetQuery.java index 669dbb1e1bfc7..7f4f47054207e 100644 --- a/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesSetQuery.java +++ b/server/src/main/java/org/opensearch/index/document/SortedUnsignedLongDocValuesSetQuery.java @@ -22,6 +22,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.util.LongHashSet; @@ -93,7 +94,7 @@ public boolean isCacheable(LeafReaderContext ctx) { } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { SortedNumericDocValues values = getValues(context.reader(), field); if (values == null) { return null; @@ -139,7 +140,8 @@ public float matchCost() { } }; } - return new ConstantScoreScorer(this, score(), scoreMode, iterator); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, iterator); + return new DefaultScorerSupplier(scorer); } }; } diff --git a/server/src/main/java/org/opensearch/index/engine/Engine.java b/server/src/main/java/org/opensearch/index/engine/Engine.java index c945d082c9a35..db08ea1164f68 100644 --- a/server/src/main/java/org/opensearch/index/engine/Engine.java +++ b/server/src/main/java/org/opensearch/index/engine/Engine.java @@ -57,7 +57,6 @@ import org.apache.lucene.search.similarities.Similarity; import org.apache.lucene.store.AlreadyClosedException; import org.apache.lucene.store.Directory; -import org.apache.lucene.store.IOContext; import org.opensearch.ExceptionsHelper; import org.opensearch.action.index.IndexRequest; import org.opensearch.common.Nullable; @@ -981,9 +980,7 @@ private Map getSegmentFileSizes(SegmentReader segmentReader) { boolean useCompoundFile = segmentCommitInfo.info.getUseCompoundFile(); if (useCompoundFile) { try { - directory = engineConfig.getCodec() - .compoundFormat() - .getCompoundReader(segmentReader.directory(), segmentCommitInfo.info, IOContext.READ); + directory = engineConfig.getCodec().compoundFormat().getCompoundReader(segmentReader.directory(), segmentCommitInfo.info); } catch (IOException e) { logger.warn( () -> new ParameterizedMessage( diff --git a/server/src/main/java/org/opensearch/index/engine/LuceneChangesSnapshot.java b/server/src/main/java/org/opensearch/index/engine/LuceneChangesSnapshot.java index 00d15478f9866..72ccc097e20d6 100644 --- a/server/src/main/java/org/opensearch/index/engine/LuceneChangesSnapshot.java +++ b/server/src/main/java/org/opensearch/index/engine/LuceneChangesSnapshot.java @@ -46,6 +46,7 @@ import org.apache.lucene.search.SortField; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopFieldCollector; +import org.apache.lucene.search.TopFieldCollectorManager; import org.apache.lucene.util.ArrayUtil; import org.opensearch.common.lucene.Lucene; import org.opensearch.common.lucene.search.Queries; @@ -122,7 +123,7 @@ final class LuceneChangesSnapshot implements Translog.Snapshot { this.indexSearcher.setQueryCache(null); this.parallelArray = new ParallelArray(this.searchBatchSize); final TopDocs topDocs = searchOperations(null, accurateCount); - this.totalHits = Math.toIntExact(topDocs.totalHits.value); + this.totalHits = Math.toIntExact(topDocs.totalHits.value()); this.scoreDocs = topDocs.scoreDocs; fillParallelArray(scoreDocs, parallelArray); } @@ -263,12 +264,13 @@ static int countNumberOfHistoryOperations(Engine.Searcher searcher, long fromSeq private TopDocs searchOperations(FieldDoc after, boolean accurate) throws IOException { final Query rangeQuery = operationsRangeQuery(Math.max(fromSeqNo, lastSeenSeqNo), toSeqNo); final Sort sortedBySeqNo = new Sort(new SortField(SeqNoFieldMapper.NAME, SortField.Type.LONG)); - final TopFieldCollector topFieldCollector = TopFieldCollector.create( + final TopFieldCollector topFieldCollector = new TopFieldCollectorManager( sortedBySeqNo, searchBatchSize, after, - accurate ? Integer.MAX_VALUE : 0 - ); + accurate ? Integer.MAX_VALUE : 0, + false + ).newCollector(); indexSearcher.search(rangeQuery, topFieldCollector); return topFieldCollector.topDocs(); } diff --git a/server/src/main/java/org/opensearch/index/engine/RecoverySourcePruneMergePolicy.java b/server/src/main/java/org/opensearch/index/engine/RecoverySourcePruneMergePolicy.java index 493ccbb69a244..112346299a427 100644 --- a/server/src/main/java/org/opensearch/index/engine/RecoverySourcePruneMergePolicy.java +++ b/server/src/main/java/org/opensearch/index/engine/RecoverySourcePruneMergePolicy.java @@ -36,6 +36,7 @@ import org.apache.lucene.codecs.StoredFieldsReader; import org.apache.lucene.index.BinaryDocValues; import org.apache.lucene.index.CodecReader; +import org.apache.lucene.index.DocValuesSkipper; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FilterCodecReader; import org.apache.lucene.index.FilterNumericDocValues; @@ -217,6 +218,11 @@ public void checkIntegrity() throws IOException { public void close() throws IOException { in.close(); } + + @Override + public DocValuesSkipper getSkipper(FieldInfo field) throws IOException { + return in.getSkipper(field); + } } /** diff --git a/server/src/main/java/org/opensearch/index/engine/TranslogLeafReader.java b/server/src/main/java/org/opensearch/index/engine/TranslogLeafReader.java index 94b8c6181de4e..e459358c16934 100644 --- a/server/src/main/java/org/opensearch/index/engine/TranslogLeafReader.java +++ b/server/src/main/java/org/opensearch/index/engine/TranslogLeafReader.java @@ -33,10 +33,11 @@ import org.apache.lucene.index.BinaryDocValues; import org.apache.lucene.index.ByteVectorValues; +import org.apache.lucene.index.DocValuesSkipIndexType; +import org.apache.lucene.index.DocValuesSkipper; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; -import org.apache.lucene.index.Fields; import org.apache.lucene.index.FloatVectorValues; import org.apache.lucene.index.IndexOptions; import org.apache.lucene.index.LeafMetaData; @@ -82,6 +83,7 @@ public final class TranslogLeafReader extends LeafReader { false, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, @@ -101,6 +103,7 @@ public final class TranslogLeafReader extends LeafReader { false, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, @@ -120,6 +123,7 @@ public final class TranslogLeafReader extends LeafReader { false, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, @@ -202,11 +206,6 @@ public LeafMetaData getMetaData() { throw new UnsupportedOperationException(); } - @Override - public Fields getTermVectors(int docID) { - throw new UnsupportedOperationException(); - } - @Override public TermVectors termVectors() throws IOException { throw new UnsupportedOperationException(); @@ -222,11 +221,6 @@ public int maxDoc() { return 1; } - @Override - public void document(int docID, StoredFieldVisitor visitor) throws IOException { - storedFields().document(docID, visitor); - } - @Override public StoredFields storedFields() throws IOException { return new StoredFields() { @@ -282,4 +276,9 @@ public void searchNearestVectors(String field, byte[] target, KnnCollector k, Bi public void searchNearestVectors(String field, float[] target, KnnCollector k, Bits acceptDocs) throws IOException { throw new UnsupportedOperationException(); } + + @Override + public DocValuesSkipper getDocValuesSkipper(String field) throws IOException { + throw new UnsupportedOperationException(); + } } diff --git a/server/src/main/java/org/opensearch/index/fielddata/FieldData.java b/server/src/main/java/org/opensearch/index/fielddata/FieldData.java index 6db6bbccacae5..fd988831d43c5 100644 --- a/server/src/main/java/org/opensearch/index/fielddata/FieldData.java +++ b/server/src/main/java/org/opensearch/index/fielddata/FieldData.java @@ -427,19 +427,12 @@ public void get(List list) throws IOException { */ public static SortedBinaryDocValues toString(final SortedSetDocValues values) { return new SortedBinaryDocValues() { - private int count = 0; - @Override public boolean advanceExact(int doc) throws IOException { if (values.advanceExact(doc) == false) { return false; } - for (int i = 0;; ++i) { - if (values.nextOrd() == SortedSetDocValues.NO_MORE_ORDS) { - count = i; - break; - } - } + // reset the iterator on the current doc boolean advanced = values.advanceExact(doc); assert advanced; @@ -448,7 +441,7 @@ public boolean advanceExact(int doc) throws IOException { @Override public int docValueCount() { - return count; + return values.docValueCount(); } @Override diff --git a/server/src/main/java/org/opensearch/index/fielddata/ordinals/GlobalOrdinalMapping.java b/server/src/main/java/org/opensearch/index/fielddata/ordinals/GlobalOrdinalMapping.java index cc5415aacd276..8f2bd0f5f5cab 100644 --- a/server/src/main/java/org/opensearch/index/fielddata/ordinals/GlobalOrdinalMapping.java +++ b/server/src/main/java/org/opensearch/index/fielddata/ordinals/GlobalOrdinalMapping.java @@ -51,6 +51,8 @@ final class GlobalOrdinalMapping extends SortedSetDocValues { private final OrdinalMap ordinalMap; private final LongValues mapping; private final TermsEnum[] lookups; + private int docValueCount = 0; + private int nextOrd = 0; GlobalOrdinalMapping(OrdinalMap ordinalMap, SortedSetDocValues values, TermsEnum[] lookups, int segmentIndex) { super(); @@ -71,14 +73,27 @@ public long getGlobalOrd(long segmentOrd) { @Override public boolean advanceExact(int target) throws IOException { - return values.advanceExact(target); + nextOrd = 0; /* reset next ordinal */ + docValueCount = 0; /* reset docValueCount */ + if (values.advanceExact(target)) { + // Some SortedSetDocValues implementations like MultiOrdinals#MultiDocs do change + // docValueCount() return value after each nextOrd() call, so we prefetch the value + // here. + docValueCount = values.docValueCount(); + return true; + } else { + return false; + } } @Override public long nextOrd() throws IOException { + if (++nextOrd > docValueCount) { + return SortedSetDocValues.NO_MORE_DOCS; + } long segmentOrd = values.nextOrd(); - if (segmentOrd == SortedSetDocValues.NO_MORE_ORDS) { - return SortedSetDocValues.NO_MORE_ORDS; + if (segmentOrd == SortedSetDocValues.NO_MORE_DOCS) { + return SortedSetDocValues.NO_MORE_DOCS; } else { return getGlobalOrd(segmentOrd); } diff --git a/server/src/main/java/org/opensearch/index/fielddata/ordinals/MultiOrdinals.java b/server/src/main/java/org/opensearch/index/fielddata/ordinals/MultiOrdinals.java index daccb5dfe9fca..ea91e6ddd1820 100644 --- a/server/src/main/java/org/opensearch/index/fielddata/ordinals/MultiOrdinals.java +++ b/server/src/main/java/org/opensearch/index/fielddata/ordinals/MultiOrdinals.java @@ -68,13 +68,13 @@ public static boolean significantlySmallerThanSinglePackedOrdinals( float acceptableOverheadRatio ) { int bitsPerOrd = PackedInts.bitsRequired(numOrds); - bitsPerOrd = PackedInts.fastestFormatAndBits(numDocsWithValue, bitsPerOrd, acceptableOverheadRatio).bitsPerValue; + bitsPerOrd = PackedInts.fastestFormatAndBits(numDocsWithValue, bitsPerOrd, acceptableOverheadRatio).bitsPerValue(); // Compute the worst-case number of bits per value for offsets in the worst case, eg. if no docs have a value at the // beginning of the block and all docs have one at the end of the block final float avgValuesPerDoc = (float) numDocsWithValue / maxDoc; final int maxDelta = (int) Math.ceil(OFFSETS_PAGE_SIZE * (1 - avgValuesPerDoc) * avgValuesPerDoc); int bitsPerOffset = PackedInts.bitsRequired(maxDelta) + 1; // +1 because of the sign - bitsPerOffset = PackedInts.fastestFormatAndBits(maxDoc, bitsPerOffset, acceptableOverheadRatio).bitsPerValue; + bitsPerOffset = PackedInts.fastestFormatAndBits(maxDoc, bitsPerOffset, acceptableOverheadRatio).bitsPerValue(); final long expectedMultiSizeInBytes = (long) numDocsWithValue * bitsPerOrd + (long) maxDoc * bitsPerOffset; final long expectedSingleSizeInBytes = (long) maxDoc * bitsPerOrd; @@ -219,7 +219,7 @@ public boolean advanceExact(int docId) throws IOException { @Override public long nextOrd() throws IOException { if (currentOffset == currentEndOffset) { - return SortedSetDocValues.NO_MORE_ORDS; + return SortedSetDocValues.NO_MORE_DOCS; } else { return ords.get(currentOffset++); } diff --git a/server/src/main/java/org/opensearch/index/get/ShardGetService.java b/server/src/main/java/org/opensearch/index/get/ShardGetService.java index d4eeb8aae8e24..082a06c8f7773 100644 --- a/server/src/main/java/org/opensearch/index/get/ShardGetService.java +++ b/server/src/main/java/org/opensearch/index/get/ShardGetService.java @@ -32,6 +32,7 @@ package org.opensearch.index.get; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -323,6 +324,7 @@ private GetResult innerGetLoadFromStoredFields( false, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, diff --git a/server/src/main/java/org/opensearch/index/mapper/CompletionFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/CompletionFieldMapper.java index fe48adf3249a3..a78d181e362b2 100644 --- a/server/src/main/java/org/opensearch/index/mapper/CompletionFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/CompletionFieldMapper.java @@ -35,7 +35,7 @@ import org.apache.lucene.document.FieldType; import org.apache.lucene.index.IndexOptions; import org.apache.lucene.index.Term; -import org.apache.lucene.search.suggest.document.Completion912PostingsFormat; +import org.apache.lucene.search.suggest.document.Completion101PostingsFormat; import org.apache.lucene.search.suggest.document.CompletionAnalyzer; import org.apache.lucene.search.suggest.document.CompletionQuery; import org.apache.lucene.search.suggest.document.FuzzyCompletionQuery; @@ -330,7 +330,7 @@ public ContextMappings getContextMappings() { */ public static synchronized PostingsFormat postingsFormat() { if (postingsFormat == null) { - postingsFormat = new Completion912PostingsFormat(); + postingsFormat = new Completion101PostingsFormat(); } return postingsFormat; } diff --git a/server/src/main/java/org/opensearch/index/mapper/ConstantKeywordFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/ConstantKeywordFieldMapper.java index 02c2214c18e72..14b54b803b62a 100644 --- a/server/src/main/java/org/opensearch/index/mapper/ConstantKeywordFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/ConstantKeywordFieldMapper.java @@ -16,6 +16,7 @@ import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.automaton.Automaton; import org.apache.lucene.util.automaton.ByteRunAutomaton; +import org.apache.lucene.util.automaton.Operations; import org.apache.lucene.util.automaton.RegExp; import org.opensearch.OpenSearchParseException; import org.opensearch.common.Nullable; @@ -175,9 +176,10 @@ public Query regexpQuery( @Nullable MultiTermQuery.RewriteMethod method, QueryShardContext context ) { - Automaton automaton = new RegExp(value, syntaxFlags, matchFlags).toAutomaton( - RegexpQuery.DEFAULT_PROVIDER, - maxDeterminizedStates + final RegExp regExp = new RegExp(value, syntaxFlags, matchFlags); + final Automaton automaton = Operations.determinize( + regExp.toAutomaton(RegexpQuery.DEFAULT_PROVIDER), + Operations.DEFAULT_DETERMINIZE_WORK_LIMIT ); ByteRunAutomaton byteRunAutomaton = new ByteRunAutomaton(automaton); BytesRef valueBytes = BytesRefs.toBytesRef(this.value); diff --git a/server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java index effee53d7cf63..ec5b128bf3173 100644 --- a/server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java @@ -32,6 +32,7 @@ package org.opensearch.index.mapper; +import org.apache.lucene.document.LongField; import org.apache.lucene.document.LongPoint; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.document.StoredField; @@ -142,7 +143,7 @@ public long parsePointAsMillis(byte[] value) { @Override protected Query distanceFeatureQuery(String field, float boost, long origin, TimeValue pivot) { - return LongPoint.newDistanceFeatureQuery(field, boost, origin, pivot.getMillis()); + return LongField.newDistanceFeatureQuery(field, boost, origin, pivot.getMillis()); } }, NANOSECONDS(DATE_NANOS_CONTENT_TYPE, NumericType.DATE_NANOSECONDS) { @@ -168,7 +169,7 @@ public long parsePointAsMillis(byte[] value) { @Override protected Query distanceFeatureQuery(String field, float boost, long origin, TimeValue pivot) { - return LongPoint.newDistanceFeatureQuery(field, boost, origin, pivot.getNanos()); + return LongField.newDistanceFeatureQuery(field, boost, origin, pivot.getNanos()); } }; diff --git a/server/src/main/java/org/opensearch/index/mapper/DerivedFieldSupportedTypes.java b/server/src/main/java/org/opensearch/index/mapper/DerivedFieldSupportedTypes.java index c2bfaffa206a5..a468a2cb6471f 100644 --- a/server/src/main/java/org/opensearch/index/mapper/DerivedFieldSupportedTypes.java +++ b/server/src/main/java/org/opensearch/index/mapper/DerivedFieldSupportedTypes.java @@ -12,11 +12,9 @@ import org.apache.lucene.document.Field; import org.apache.lucene.document.FieldType; import org.apache.lucene.document.FloatField; -import org.apache.lucene.document.InetAddressPoint; import org.apache.lucene.document.KeywordField; import org.apache.lucene.document.LatLonPoint; import org.apache.lucene.document.LongField; -import org.apache.lucene.document.LongPoint; import org.apache.lucene.document.TextField; import org.apache.lucene.index.IndexOptions; import org.apache.lucene.index.IndexableField; @@ -70,7 +68,7 @@ public enum DerivedFieldSupportedTypes { ); return builder.build(context); }, - name -> o -> new LongPoint(name, (long) o), + name -> o -> new LongField(name, (long) o, Field.Store.NO), formatter -> o -> formatter == null ? DateFieldMapper.getDefaultDateTimeFormatter().formatMillis((long) o) : formatter.formatMillis((long) o) @@ -95,7 +93,7 @@ public enum DerivedFieldSupportedTypes { } else { address = InetAddresses.forString(o.toString()); } - return new InetAddressPoint(name, address); + return IpFieldMapper.buildInetAddressField(name, address); }, formatter -> o -> o), KEYWORD("keyword", (name, context, indexAnalyzers) -> { FieldType dummyFieldType = new FieldType(); diff --git a/server/src/main/java/org/opensearch/index/mapper/DerivedFieldType.java b/server/src/main/java/org/opensearch/index/mapper/DerivedFieldType.java index fe81f19d74b21..cea03d29b8ae7 100644 --- a/server/src/main/java/org/opensearch/index/mapper/DerivedFieldType.java +++ b/server/src/main/java/org/opensearch/index/mapper/DerivedFieldType.java @@ -237,34 +237,6 @@ public Query rangeQuery( ); } - @Override - public Query fuzzyQuery( - Object value, - Fuzziness fuzziness, - int prefixLength, - int maxExpansions, - boolean transpositions, - QueryShardContext context - ) { - Query query = typeFieldMapper.mappedFieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, context); - DerivedFieldQuery derivedFieldQuery = new DerivedFieldQuery( - query, - () -> valueFetcher(context, context.lookup(), null), - context.lookup(), - getIndexAnalyzer(), - indexableFieldGenerator, - derivedField.getIgnoreMalformed() - ); - return Optional.ofNullable(getPrefilterFieldType(context)) - .map( - prefilterFieldType -> createConjuctionQuery( - prefilterFieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, context), - derivedFieldQuery - ) - ) - .orElse(derivedFieldQuery); - } - @Override public Query fuzzyQuery( Object value, diff --git a/server/src/main/java/org/opensearch/index/mapper/DocCountFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/DocCountFieldMapper.java index db4770adf6666..240d7fed16b60 100644 --- a/server/src/main/java/org/opensearch/index/mapper/DocCountFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/DocCountFieldMapper.java @@ -33,7 +33,7 @@ import org.apache.lucene.document.Field; import org.apache.lucene.document.NumericDocValuesField; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.Query; import org.opensearch.core.xcontent.XContentParser; import org.opensearch.core.xcontent.XContentParserUtils; @@ -109,7 +109,7 @@ public String familyTypeName() { @Override public Query existsQuery(QueryShardContext context) { - return new DocValuesFieldExistsQuery(NAME); + return new FieldExistsQuery(NAME); } @Override diff --git a/server/src/main/java/org/opensearch/index/mapper/FlatObjectFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/FlatObjectFieldMapper.java index 424172e844989..b921afb3157a8 100644 --- a/server/src/main/java/org/opensearch/index/mapper/FlatObjectFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/FlatObjectFieldMapper.java @@ -446,12 +446,7 @@ public Query rangeQuery(Object lowerTerm, Object upperTerm, boolean includeLower BytesRef upperTermBytes = upperTerm == null ? null : indexedValueForSearch(rewriteSearchValue(upperTerm)); Automaton a2 = TermRangeQuery.toAutomaton(lowerTermBytes, upperTermBytes, includeLower, includeUpper); Automaton termAutomaton = Operations.intersection(a1, a2); - indexQuery = new AutomatonQuery( - new Term(getSearchField()), - termAutomaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - true - ); + indexQuery = new AutomatonQuery(new Term(getSearchField()), termAutomaton, true); } } if (hasDocValues()) { @@ -462,13 +457,7 @@ public Query rangeQuery(Object lowerTerm, Object upperTerm, boolean includeLower BytesRef upperDvBytes = upperTerm == null ? null : indexedValueForSearch(dvPrefix + rewriteSearchValue(upperTerm)); Automaton a2 = TermRangeQuery.toAutomaton(lowerDvBytes, upperDvBytes, includeLower, includeUpper); Automaton dvAutomaton = Operations.intersection(a1, a2); - dvQuery = new AutomatonQuery( - new Term(getSearchField()), - dvAutomaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - true, - DOC_VALUES_REWRITE - ); + dvQuery = new AutomatonQuery(new Term(getSearchField()), dvAutomaton, true, DOC_VALUES_REWRITE); } assert indexQuery != null || dvQuery != null; diff --git a/server/src/main/java/org/opensearch/index/mapper/IdFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/IdFieldMapper.java index 658f4228cb0c6..a386e8b11eb38 100644 --- a/server/src/main/java/org/opensearch/index/mapper/IdFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/IdFieldMapper.java @@ -37,6 +37,7 @@ import org.apache.lucene.index.IndexOptions; import org.apache.lucene.index.LeafReaderContext; import org.apache.lucene.search.MatchAllDocsQuery; +import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.SortField; import org.apache.lucene.search.TermInSetQuery; @@ -65,7 +66,9 @@ import org.opensearch.search.sort.SortOrder; import java.io.IOException; +import java.util.ArrayList; import java.util.Arrays; +import java.util.Collection; import java.util.Collections; import java.util.List; import java.util.function.Supplier; @@ -163,15 +166,15 @@ public Query existsQuery(QueryShardContext context) { @Override public Query termsQuery(List values, QueryShardContext context) { failIfNotIndexed(); - BytesRef[] bytesRefs = new BytesRef[values.size()]; - for (int i = 0; i < bytesRefs.length; i++) { + Collection bytesRefs = new ArrayList<>(values.size()); + for (int i = 0; i < values.size(); i++) { Object idObject = values.get(i); if (idObject instanceof BytesRef) { idObject = ((BytesRef) idObject).utf8ToString(); } - bytesRefs[i] = Uid.encodeId(idObject.toString()); + bytesRefs.add(Uid.encodeId(idObject.toString())); } - return new TermInSetQuery(name(), bytesRefs); + return new TermInSetQuery(MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, name(), bytesRefs); } @Override diff --git a/server/src/main/java/org/opensearch/index/mapper/IpFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/IpFieldMapper.java index 1283aa302c111..9fe5d3f240e2c 100644 --- a/server/src/main/java/org/opensearch/index/mapper/IpFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/IpFieldMapper.java @@ -32,9 +32,12 @@ package org.opensearch.index.mapper; +import org.apache.lucene.document.Field; +import org.apache.lucene.document.FieldType; import org.apache.lucene.document.InetAddressPoint; import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.document.StoredField; +import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.SortedSetDocValues; import org.apache.lucene.sandbox.search.MultiRangeQuery; import org.apache.lucene.search.BooleanClause; @@ -451,7 +454,8 @@ public IpScriptDocValues(SortedSetDocValues in) { public void setNextDocId(int docId) throws IOException { count = 0; if (in.advanceExact(docId)) { - for (long ord = in.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = in.nextOrd()) { + for (long ord = in.nextOrd(); ord != SortedSetDocValues.NO_MORE_DOCS && count < in.docValueCount(); ord = in + .nextOrd()) { ords = ArrayUtil.grow(ords, count + 1); ords[count++] = ord; } @@ -613,12 +617,14 @@ protected void parseCreateField(ParseContext context) throws IOException { } } - if (indexed) { + if (indexed && hasDocValues) { + context.doc().add(new InetAddressField(fieldType().name(), address)); + } else if (indexed) { context.doc().add(new InetAddressPoint(fieldType().name(), address)); - } - if (hasDocValues) { + } else if (hasDocValues) { context.doc().add(new SortedSetDocValuesField(fieldType().name(), new BytesRef(InetAddressPoint.encode(address)))); - } else if (stored || indexed) { + } + if ((stored || indexed) && hasDocValues == false) { createFieldNamesField(context); } if (stored) { @@ -626,6 +632,30 @@ protected void parseCreateField(ParseContext context) throws IOException { } } + public static InetAddressField buildInetAddressField(String name, InetAddress value) { + return new InetAddressField(name, value); + } + + /** + * Field type that combines dimensional points and doc values for IP fields + */ + public static class InetAddressField extends Field { + private static final FieldType FIELD_TYPE = new FieldType(); + static { + FIELD_TYPE.setDimensions(1, InetAddressPoint.BYTES); + FIELD_TYPE.setDocValuesType(DocValuesType.SORTED_SET); + FIELD_TYPE.freeze(); + } + + /** + * Create new instance + */ + public InetAddressField(String name, InetAddress value) { + super(name, FIELD_TYPE); + fieldsData = new BytesRef(InetAddressPoint.encode(value)); + } + } + @Override public ParametrizedFieldMapper.Builder getMergeBuilder() { return new Builder(simpleName(), ignoreMalformedByDefault, indexCreatedVersion).init(this); diff --git a/server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java index 4436e74c821c3..77235bbac969e 100644 --- a/server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java @@ -70,7 +70,9 @@ import java.io.IOException; import java.io.UncheckedIOException; +import java.util.ArrayList; import java.util.Arrays; +import java.util.Collection; import java.util.Collections; import java.util.List; import java.util.Map; @@ -447,21 +449,21 @@ public Query termsQuery(List values, QueryShardContext context) { if (!context.keywordFieldIndexOrDocValuesEnabled()) { return super.termsQuery(values, context); } - BytesRef[] iBytesRefs = new BytesRef[values.size()]; - BytesRef[] dVByteRefs = new BytesRef[values.size()]; - for (int i = 0; i < iBytesRefs.length; i++) { - iBytesRefs[i] = indexedValueForSearch(values.get(i)); - dVByteRefs[i] = indexedValueForSearch(rewriteForDocValue(values.get(i))); + Collection iBytesRefs = new ArrayList<>(values.size()); + Collection dVByteRefs = new ArrayList<>(values.size()); + for (int i = 0; i < values.size(); i++) { + iBytesRefs.add(indexedValueForSearch(values.get(i))); + dVByteRefs.add(indexedValueForSearch(rewriteForDocValue(values.get(i)))); } - Query indexQuery = new TermInSetQuery(name(), iBytesRefs); + Query indexQuery = new TermInSetQuery(MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, name(), iBytesRefs); Query dvQuery = new TermInSetQuery(MultiTermQuery.DOC_VALUES_REWRITE, name(), dVByteRefs); return new IndexOrDocValuesQuery(indexQuery, dvQuery); } // if we only have doc_values enabled, we construct a new query with doc_values re-written if (hasDocValues()) { - BytesRef[] bytesRefs = new BytesRef[values.size()]; - for (int i = 0; i < bytesRefs.length; i++) { - bytesRefs[i] = indexedValueForSearch(rewriteForDocValue(values.get(i))); + Collection bytesRefs = new ArrayList<>(values.size()); + for (int i = 0; i < values.size(); i++) { + bytesRefs.add(indexedValueForSearch(rewriteForDocValue(values.get(i)))); } return new TermInSetQuery(MultiTermQuery.DOC_VALUES_REWRITE, name(), bytesRefs); } @@ -645,7 +647,7 @@ public Query fuzzyQuery( MultiTermQuery.DOC_VALUES_REWRITE ); } - return super.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, context); + return super.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, method, context); } @Override diff --git a/server/src/main/java/org/opensearch/index/mapper/MappedFieldType.java b/server/src/main/java/org/opensearch/index/mapper/MappedFieldType.java index 66d4654e543a2..4a0740dbf1b3c 100644 --- a/server/src/main/java/org/opensearch/index/mapper/MappedFieldType.java +++ b/server/src/main/java/org/opensearch/index/mapper/MappedFieldType.java @@ -34,8 +34,6 @@ import org.apache.lucene.analysis.TokenStream; import org.apache.lucene.index.IndexReader; -import org.apache.lucene.index.PrefixCodedTerms; -import org.apache.lucene.index.PrefixCodedTerms.TermIterator; import org.apache.lucene.index.Term; import org.apache.lucene.queries.intervals.IntervalsSource; import org.apache.lucene.queries.spans.SpanMultiTermQueryWrapper; @@ -44,13 +42,13 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MultiTermQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.BytesRefIterator; import org.opensearch.ExceptionsHelper; import org.opensearch.OpenSearchParseException; import org.opensearch.common.Nullable; @@ -258,19 +256,6 @@ public Query rangeQuery( throw new IllegalArgumentException("Field [" + name + "] of type [" + typeName() + "] does not support range queries"); } - public Query fuzzyQuery( - Object value, - Fuzziness fuzziness, - int prefixLength, - int maxExpansions, - boolean transpositions, - QueryShardContext context - ) { - throw new IllegalArgumentException( - "Can only use fuzzy queries on keyword and text fields - not on [" + name + "] which is of type [" + typeName() + "]" - ); - } - // Fuzzy Query with re-write method public Query fuzzyQuery( Object value, @@ -345,9 +330,9 @@ public Query regexpQuery( public Query existsQuery(QueryShardContext context) { if (hasDocValues()) { - return new DocValuesFieldExistsQuery(name()); + return new FieldExistsQuery(name()); } else if (getTextSearchInfo().hasNorms()) { - return new NormsFieldExistsQuery(name()); + return new FieldExistsQuery(name()); } else { return new TermQuery(new Term(FieldNamesFieldMapper.NAME, name())); } @@ -497,19 +482,19 @@ public DocValueFormat docValueFormat(@Nullable String format, ZoneId timeZone) { /** * Extract a {@link Term} from a query created with {@link #termQuery} by * recursively removing {@link BoostQuery} wrappers. + * @throws IOException * @throws IllegalArgumentException if the wrapped query is not a {@link TermQuery} */ - public static Term extractTerm(Query termQuery) { + public static Term extractTerm(Query termQuery) throws IOException { while (termQuery instanceof BoostQuery) { termQuery = ((BoostQuery) termQuery).getQuery(); } if (termQuery instanceof TermInSetQuery) { TermInSetQuery tisQuery = (TermInSetQuery) termQuery; - PrefixCodedTerms terms = tisQuery.getTermData(); - if (terms.size() == 1) { - TermIterator it = terms.iterator(); + if (tisQuery.getTermsCount() == 1) { + BytesRefIterator it = tisQuery.getBytesRefIterator(); BytesRef term = it.next(); - return new Term(it.field(), term); + return new Term(tisQuery.getField(), term); } } if (termQuery instanceof TermQuery == false) { diff --git a/server/src/main/java/org/opensearch/index/mapper/StringFieldType.java b/server/src/main/java/org/opensearch/index/mapper/StringFieldType.java index 682ccc13f769d..ed2d3d6c25db5 100644 --- a/server/src/main/java/org/opensearch/index/mapper/StringFieldType.java +++ b/server/src/main/java/org/opensearch/index/mapper/StringFieldType.java @@ -79,30 +79,6 @@ public StringFieldType( super(name, isSearchable, isStored, hasDocValues, textSearchInfo, meta); } - @Override - public Query fuzzyQuery( - Object value, - Fuzziness fuzziness, - int prefixLength, - int maxExpansions, - boolean transpositions, - QueryShardContext context - ) { - if (context.allowExpensiveQueries() == false) { - throw new OpenSearchException( - "[fuzzy] queries cannot be executed when '" + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false." - ); - } - failIfNotIndexed(); - return new FuzzyQuery( - new Term(name(), indexedValueForSearch(value)), - fuzziness.asDistance(BytesRefs.toString(value)), - prefixLength, - maxExpansions, - transpositions - ); - } - @Override public Query fuzzyQuery( Object value, @@ -144,7 +120,7 @@ public Query prefixQuery(String value, MultiTermQuery.RewriteMethod method, bool } failIfNotIndexed(); if (method == null) { - method = MultiTermQuery.CONSTANT_SCORE_REWRITE; + method = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; } if (caseInsensitive) { return AutomatonQueries.caseInsensitivePrefixQuery((new Term(name(), indexedValueForSearch(value))), method); @@ -228,7 +204,7 @@ public Query wildcardQuery( return AutomatonQueries.caseInsensitiveWildcardQuery(term, method); } if (method == null) { - method = MultiTermQuery.CONSTANT_SCORE_REWRITE; + method = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; } return new WildcardQuery(term, Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, method); } @@ -249,7 +225,7 @@ public Query regexpQuery( } failIfNotIndexed(); if (method == null) { - method = MultiTermQuery.CONSTANT_SCORE_REWRITE; + method = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; } return new RegexpQuery( new Term(name(), indexedValueForSearch(value)), diff --git a/server/src/main/java/org/opensearch/index/mapper/TermBasedFieldType.java b/server/src/main/java/org/opensearch/index/mapper/TermBasedFieldType.java index 78dae2d2c27fc..da01c6490ea37 100644 --- a/server/src/main/java/org/opensearch/index/mapper/TermBasedFieldType.java +++ b/server/src/main/java/org/opensearch/index/mapper/TermBasedFieldType.java @@ -34,6 +34,7 @@ import org.apache.lucene.index.Term; import org.apache.lucene.search.BoostQuery; +import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; @@ -42,6 +43,8 @@ import org.opensearch.common.lucene.search.AutomatonQueries; import org.opensearch.index.query.QueryShardContext; +import java.util.ArrayList; +import java.util.Collection; import java.util.List; import java.util.Map; @@ -93,11 +96,11 @@ public Query termQuery(Object value, QueryShardContext context) { @Override public Query termsQuery(List values, QueryShardContext context) { failIfNotIndexed(); - BytesRef[] bytesRefs = new BytesRef[values.size()]; - for (int i = 0; i < bytesRefs.length; i++) { - bytesRefs[i] = indexedValueForSearch(values.get(i)); + Collection bytesRefs = new ArrayList<>(values.size()); + for (int i = 0; i < values.size(); i++) { + bytesRefs.add(indexedValueForSearch(values.get(i))); } - return new TermInSetQuery(name(), bytesRefs); + return new TermInSetQuery(MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, name(), bytesRefs); } } diff --git a/server/src/main/java/org/opensearch/index/mapper/TextFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/TextFieldMapper.java index ba053a3aeee1d..cebd0cfa43b2b 100644 --- a/server/src/main/java/org/opensearch/index/mapper/TextFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/TextFieldMapper.java @@ -846,6 +846,7 @@ public Query prefixQuery(String value, MultiTermQuery.RewriteMethod method, bool Query tq = prefixFieldType.prefixQuery(value, method, caseInsensitive, context); if (method == null || method == MultiTermQuery.CONSTANT_SCORE_REWRITE + || method == MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE || method == MultiTermQuery.CONSTANT_SCORE_BOOLEAN_REWRITE) { return new ConstantScoreQuery(tq); } diff --git a/server/src/main/java/org/opensearch/index/mapper/WildcardFieldMapper.java b/server/src/main/java/org/opensearch/index/mapper/WildcardFieldMapper.java index 7342c6f9f23bd..e45eaa7766c3c 100644 --- a/server/src/main/java/org/opensearch/index/mapper/WildcardFieldMapper.java +++ b/server/src/main/java/org/opensearch/index/mapper/WildcardFieldMapper.java @@ -37,6 +37,7 @@ import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.automaton.Automaton; import org.apache.lucene.util.automaton.CompiledAutomaton; +import org.apache.lucene.util.automaton.Operations; import org.apache.lucene.util.automaton.RegExp; import org.opensearch.common.lucene.BytesRefs; import org.opensearch.common.lucene.Lucene; @@ -422,6 +423,7 @@ public Query fuzzyQuery( int prefixLength, int maxExpansions, boolean transpositions, + MultiTermQuery.RewriteMethod method, QueryShardContext context ) { // TODO: Not sure if we can reasonably describe a fuzzy query in terms of n-grams without exploding the cardinality @@ -449,7 +451,7 @@ public Query wildcardQuery(String value, MultiTermQuery.RewriteMethod method, bo finalValue = value; } Predicate matchPredicate; - Automaton automaton = WildcardQuery.toAutomaton(new Term(name(), finalValue)); + Automaton automaton = WildcardQuery.toAutomaton(new Term(name(), finalValue), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); CompiledAutomaton compiledAutomaton = new CompiledAutomaton(automaton); if (compiledAutomaton.type == CompiledAutomaton.AUTOMATON_TYPE.SINGLE) { // when type equals SINGLE, #compiledAutomaton.runAutomaton is null @@ -614,7 +616,7 @@ public Query regexpQuery( final boolean caseInsensitive = matchFlags == RegExp.ASCII_CASE_INSENSITIVE; RegExp regExp = new RegExp(finalValue, syntaxFlags, matchFlags); - Automaton automaton = regExp.toAutomaton(maxDeterminizedStates); + Automaton automaton = Operations.determinize(regExp.toAutomaton(), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); CompiledAutomaton compiledAutomaton = new CompiledAutomaton(automaton); Predicate regexpPredicate; @@ -705,7 +707,7 @@ private static Query regexpToQuery(String fieldName, RegExp regExp, boolean case return new MatchAllDocsQuery(); } if (query.clauses().size() == 1) { - return query.iterator().next().getQuery(); + return query.iterator().next().query(); } else if (query.clauses().size() == 0) { return new MatchAllDocsQuery(); } @@ -862,18 +864,8 @@ public Query rewrite(IndexSearcher indexSearcher) throws IOException { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { Weight firstPhaseWeight = firstPhaseQuery.createWeight(searcher, scoreMode, boost); return new ConstantScoreWeight(this, boost) { - @Override - public Scorer scorer(LeafReaderContext leafReaderContext) throws IOException { - ScorerSupplier supplier = scorerSupplier(leafReaderContext); - if (supplier == null) { - return null; - } - return supplier.get(Long.MAX_VALUE); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { - Weight weight = this; ScorerSupplier firstPhaseSupplier = firstPhaseWeight.scorerSupplier(context); if (firstPhaseSupplier == null) { return null; @@ -904,7 +896,7 @@ public float matchCost() { return MATCH_COST_ESTIMATE; } }; - return new ConstantScoreScorer(weight, score(), scoreMode, twoPhaseIterator); + return new ConstantScoreScorer(score(), scoreMode, twoPhaseIterator); } @Override diff --git a/server/src/main/java/org/opensearch/index/query/DerivedFieldQuery.java b/server/src/main/java/org/opensearch/index/query/DerivedFieldQuery.java index dcc02726cb0ef..04469db9c46ec 100644 --- a/server/src/main/java/org/opensearch/index/query/DerivedFieldQuery.java +++ b/server/src/main/java/org/opensearch/index/query/DerivedFieldQuery.java @@ -20,6 +20,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.opensearch.index.mapper.DerivedFieldValueFetcher; @@ -93,7 +94,7 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { DocIdSetIterator approximation; approximation = DocIdSetIterator.all(context.reader().maxDoc()); @@ -130,7 +131,8 @@ public float matchCost() { return 1000f; } }; - return new ConstantScoreScorer(this, score(), scoreMode, twoPhase); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, twoPhase); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/index/query/FuzzyQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/FuzzyQueryBuilder.java index 93c32bbedcef4..297c25476663e 100644 --- a/server/src/main/java/org/opensearch/index/query/FuzzyQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/FuzzyQueryBuilder.java @@ -357,12 +357,12 @@ protected Query doToQuery(QueryShardContext context) throws IOException { throw new IllegalStateException("Rewrite first"); } String rewrite = this.rewrite; - Query query = fieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, null, context); - if (query instanceof MultiTermQuery) { - MultiTermQuery.RewriteMethod rewriteMethod = QueryParsers.parseRewriteMethod(rewrite, null, LoggingDeprecationHandler.INSTANCE); - QueryParsers.setRewriteMethod((MultiTermQuery) query, rewriteMethod); - } - return query; + MultiTermQuery.RewriteMethod rewriteMethod = QueryParsers.parseRewriteMethod( + rewrite, + FuzzyQuery.defaultRewriteMethod(maxExpansions), + LoggingDeprecationHandler.INSTANCE + ); + return fieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, rewriteMethod, context); } @Override diff --git a/server/src/main/java/org/opensearch/index/query/IntervalBuilder.java b/server/src/main/java/org/opensearch/index/query/IntervalBuilder.java index 0e42e79f67d0c..d37519120b325 100644 --- a/server/src/main/java/org/opensearch/index/query/IntervalBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/IntervalBuilder.java @@ -43,7 +43,7 @@ import org.apache.lucene.queries.intervals.IntervalMatchesIterator; import org.apache.lucene.queries.intervals.Intervals; import org.apache.lucene.queries.intervals.IntervalsSource; -import org.apache.lucene.search.BooleanQuery; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.graph.GraphTokenStreamFiniteStrings; @@ -229,7 +229,7 @@ protected List analyzeGraph(TokenStream source) throws IOExcept List clauses = new ArrayList<>(); int[] articulationPoints = graph.articulationPoints(); int lastState = 0; - int maxClauseCount = BooleanQuery.getMaxClauseCount(); + int maxClauseCount = IndexSearcher.getMaxClauseCount(); for (int i = 0; i <= articulationPoints.length; i++) { int start = lastState; int end = -1; @@ -244,7 +244,7 @@ protected List analyzeGraph(TokenStream source) throws IOExcept TokenStream ts = it.next(); IntervalsSource phrase = combineSources(analyzeTerms(ts), 0, IntervalMode.ORDERED); if (paths.size() >= maxClauseCount) { - throw new BooleanQuery.TooManyClauses(); + throw new IndexSearcher.TooManyClauses(); } paths.add(phrase); } diff --git a/server/src/main/java/org/opensearch/index/query/IntervalsSourceProvider.java b/server/src/main/java/org/opensearch/index/query/IntervalsSourceProvider.java index 5a57dfed14f69..1fc596011d84a 100644 --- a/server/src/main/java/org/opensearch/index/query/IntervalsSourceProvider.java +++ b/server/src/main/java/org/opensearch/index/query/IntervalsSourceProvider.java @@ -39,7 +39,9 @@ import org.apache.lucene.queries.intervals.IntervalsSource; import org.apache.lucene.search.FuzzyQuery; import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.automaton.Automaton; import org.apache.lucene.util.automaton.CompiledAutomaton; +import org.apache.lucene.util.automaton.Operations; import org.apache.lucene.util.automaton.RegExp; import org.opensearch.common.unit.Fuzziness; import org.opensearch.core.ParseField; @@ -718,7 +720,8 @@ public IntervalsSource getSource(QueryShardContext context, MappedFieldType fiel flags, caseInsensitive ? RegExp.ASCII_CASE_INSENSITIVE : 0 ); - final CompiledAutomaton automaton = new CompiledAutomaton(regexp.toAutomaton()); + final Automaton automaton = Operations.determinize(regexp.toAutomaton(), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); + final CompiledAutomaton compiledAutomaton = new CompiledAutomaton(automaton); if (useField != null) { fieldType = context.fieldMapper(useField); @@ -726,14 +729,14 @@ public IntervalsSource getSource(QueryShardContext context, MappedFieldType fiel checkPositions(fieldType); IntervalsSource regexpSource = maxExpansions == null - ? Intervals.multiterm(automaton, regexp.toString()) - : Intervals.multiterm(automaton, maxExpansions, regexp.toString()); + ? Intervals.multiterm(compiledAutomaton, regexp.toString()) + : Intervals.multiterm(compiledAutomaton, maxExpansions, regexp.toString()); return Intervals.fixField(useField, regexpSource); } else { checkPositions(fieldType); return maxExpansions == null - ? Intervals.multiterm(automaton, regexp.toString()) - : Intervals.multiterm(automaton, maxExpansions, regexp.toString()); + ? Intervals.multiterm(compiledAutomaton, regexp.toString()) + : Intervals.multiterm(compiledAutomaton, maxExpansions, regexp.toString()); } } diff --git a/server/src/main/java/org/opensearch/index/query/MultiMatchQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/MultiMatchQueryBuilder.java index 6227e5d2fa806..b336df33860e2 100644 --- a/server/src/main/java/org/opensearch/index/query/MultiMatchQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/MultiMatchQueryBuilder.java @@ -822,7 +822,13 @@ protected Query doToQuery(QueryShardContext context) throws IOException { multiMatchQuery.setMaxExpansions(maxExpansions); multiMatchQuery.setOccur(operator.toBooleanClauseOccur()); if (fuzzyRewrite != null) { - multiMatchQuery.setFuzzyRewriteMethod(QueryParsers.parseRewriteMethod(fuzzyRewrite, null, LoggingDeprecationHandler.INSTANCE)); + multiMatchQuery.setFuzzyRewriteMethod( + QueryParsers.parseRewriteMethod( + fuzzyRewrite, + FuzzyQuery.defaultRewriteMethod(maxExpansions), + LoggingDeprecationHandler.INSTANCE + ) + ); } if (tieBreaker != null) { multiMatchQuery.setTieBreaker(tieBreaker); diff --git a/server/src/main/java/org/opensearch/index/query/NestedQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/NestedQueryBuilder.java index 6cf34c5b7db30..a15353cc13f52 100644 --- a/server/src/main/java/org/opensearch/index/query/NestedQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/NestedQueryBuilder.java @@ -40,8 +40,8 @@ import org.apache.lucene.search.Query; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopDocsCollector; -import org.apache.lucene.search.TopFieldCollector; -import org.apache.lucene.search.TopScoreDocCollector; +import org.apache.lucene.search.TopFieldCollectorManager; +import org.apache.lucene.search.TopScoreDocCollectorManager; import org.apache.lucene.search.TotalHitCountCollector; import org.apache.lucene.search.TotalHits; import org.apache.lucene.search.Weight; @@ -490,12 +490,12 @@ public TopDocsAndMaxScore topDocs(SearchHit hit) throws IOException { TopDocsCollector topDocsCollector; MaxScoreCollector maxScoreCollector = null; if (sort() != null) { - topDocsCollector = TopFieldCollector.create(sort().sort, topN, Integer.MAX_VALUE); + topDocsCollector = new TopFieldCollectorManager(sort().sort, topN, null, Integer.MAX_VALUE, false).newCollector(); if (trackScores()) { maxScoreCollector = new MaxScoreCollector(); } } else { - topDocsCollector = TopScoreDocCollector.create(topN, Integer.MAX_VALUE); + topDocsCollector = new TopScoreDocCollectorManager(topN, null, Integer.MAX_VALUE, false).newCollector(); maxScoreCollector = new MaxScoreCollector(); } intersect(weight, innerHitQueryWeight, MultiCollector.wrap(topDocsCollector, maxScoreCollector), ctx); diff --git a/server/src/main/java/org/opensearch/index/query/QueryStringQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/QueryStringQueryBuilder.java index 3d8fbd5fc436d..5e943137951a2 100644 --- a/server/src/main/java/org/opensearch/index/query/QueryStringQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/QueryStringQueryBuilder.java @@ -940,7 +940,13 @@ protected Query doToQuery(QueryShardContext context) throws IOException { queryParser.setFuzziness(fuzziness); queryParser.setFuzzyPrefixLength(fuzzyPrefixLength); queryParser.setFuzzyMaxExpansions(fuzzyMaxExpansions); - queryParser.setFuzzyRewriteMethod(QueryParsers.parseRewriteMethod(this.fuzzyRewrite, LoggingDeprecationHandler.INSTANCE)); + queryParser.setFuzzyRewriteMethod( + QueryParsers.parseRewriteMethod( + this.fuzzyRewrite, + FuzzyQuery.defaultRewriteMethod(fuzzyMaxExpansions), + LoggingDeprecationHandler.INSTANCE + ) + ); queryParser.setMultiTermRewriteMethod(QueryParsers.parseRewriteMethod(this.rewrite, LoggingDeprecationHandler.INSTANCE)); queryParser.setTimeZone(timeZone); queryParser.setDeterminizeWorkLimit(maxDeterminizedStates); diff --git a/server/src/main/java/org/opensearch/index/query/RegexpFlag.java b/server/src/main/java/org/opensearch/index/query/RegexpFlag.java index b7e7569f67127..f5eb1ebc98d04 100644 --- a/server/src/main/java/org/opensearch/index/query/RegexpFlag.java +++ b/server/src/main/java/org/opensearch/index/query/RegexpFlag.java @@ -63,7 +63,7 @@ public enum RegexpFlag { /** * Enables complement expression of the form: {@code ~<expression>} */ - COMPLEMENT(RegExp.COMPLEMENT), + COMPLEMENT(RegExp.DEPRECATED_COMPLEMENT), /** * Enables empty language expression: {@code #} diff --git a/server/src/main/java/org/opensearch/index/query/ScriptQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/ScriptQueryBuilder.java index ded6fd0528c33..75b8e14a5ca1f 100644 --- a/server/src/main/java/org/opensearch/index/query/ScriptQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/ScriptQueryBuilder.java @@ -41,6 +41,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.opensearch.OpenSearchException; @@ -215,7 +216,7 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { DocIdSetIterator approximation = DocIdSetIterator.all(context.reader().maxDoc()); final FilterScript leafScript = filterScript.newInstance(context); TwoPhaseIterator twoPhase = new TwoPhaseIterator(approximation) { @@ -232,7 +233,8 @@ public float matchCost() { return 1000f; } }; - return new ConstantScoreScorer(this, score(), scoreMode, twoPhase); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, twoPhase); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/index/query/SourceFieldMatchQuery.java b/server/src/main/java/org/opensearch/index/query/SourceFieldMatchQuery.java index e9abcb698f68f..d189c90233da8 100644 --- a/server/src/main/java/org/opensearch/index/query/SourceFieldMatchQuery.java +++ b/server/src/main/java/org/opensearch/index/query/SourceFieldMatchQuery.java @@ -18,6 +18,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.opensearch.index.mapper.MappedFieldType; @@ -88,7 +89,7 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { Scorer scorer = weight.scorer(context); if (scorer == null) { @@ -121,7 +122,7 @@ public float matchCost() { return 1000f; } }; - return new ConstantScoreScorer(this, score(), ScoreMode.TOP_DOCS, twoPhase); + return new DefaultScorerSupplier(new ConstantScoreScorer(score(), ScoreMode.TOP_DOCS, twoPhase)); } @Override diff --git a/server/src/main/java/org/opensearch/index/query/TermsSetQueryBuilder.java b/server/src/main/java/org/opensearch/index/query/TermsSetQueryBuilder.java index e2cf7384ecac7..5582feb86bf5e 100644 --- a/server/src/main/java/org/opensearch/index/query/TermsSetQueryBuilder.java +++ b/server/src/main/java/org/opensearch/index/query/TermsSetQueryBuilder.java @@ -35,7 +35,6 @@ import org.apache.lucene.index.SortedNumericDocValues; import org.apache.lucene.index.Term; import org.apache.lucene.sandbox.search.CoveringQuery; -import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.DoubleValues; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.LongValues; @@ -251,8 +250,8 @@ protected Query doToQuery(QueryShardContext context) { return Queries.newMatchNoDocsQuery("No terms supplied for \"" + getName() + "\" query."); } // Fail before we attempt to create the term queries: - if (values.size() > BooleanQuery.getMaxClauseCount()) { - throw new BooleanQuery.TooManyClauses(); + if (values.size() > IndexSearcher.getMaxClauseCount()) { + throw new IndexSearcher.TooManyClauses(); } List queries = createTermQueries(context); diff --git a/server/src/main/java/org/opensearch/index/query/support/QueryParsers.java b/server/src/main/java/org/opensearch/index/query/support/QueryParsers.java index af8f7c7b4b8a7..d55c8330a5d9c 100644 --- a/server/src/main/java/org/opensearch/index/query/support/QueryParsers.java +++ b/server/src/main/java/org/opensearch/index/query/support/QueryParsers.java @@ -45,6 +45,7 @@ public final class QueryParsers { public static final ParseField CONSTANT_SCORE = new ParseField("constant_score"); + public static final ParseField CONSTANT_SCORE_BLENDED = new ParseField("constant_score_blended"); public static final ParseField SCORING_BOOLEAN = new ParseField("scoring_boolean"); public static final ParseField CONSTANT_SCORE_BOOLEAN = new ParseField("constant_score_boolean"); public static final ParseField TOP_TERMS = new ParseField("top_terms_"); @@ -55,15 +56,8 @@ private QueryParsers() { } - public static void setRewriteMethod(MultiTermQuery query, @Nullable MultiTermQuery.RewriteMethod rewriteMethod) { - if (rewriteMethod == null) { - return; - } - query.setRewriteMethod(rewriteMethod); - } - public static MultiTermQuery.RewriteMethod parseRewriteMethod(@Nullable String rewriteMethod, DeprecationHandler deprecationHandler) { - return parseRewriteMethod(rewriteMethod, MultiTermQuery.CONSTANT_SCORE_REWRITE, deprecationHandler); + return parseRewriteMethod(rewriteMethod, MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, deprecationHandler); } public static MultiTermQuery.RewriteMethod parseRewriteMethod( @@ -83,6 +77,9 @@ public static MultiTermQuery.RewriteMethod parseRewriteMethod( if (CONSTANT_SCORE_BOOLEAN.match(rewriteMethod, deprecationHandler)) { return MultiTermQuery.CONSTANT_SCORE_BOOLEAN_REWRITE; } + if (CONSTANT_SCORE_BLENDED.match(rewriteMethod, deprecationHandler)) { + return MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; + } int firstDigit = -1; for (int i = 0; i < rewriteMethod.length(); ++i) { diff --git a/server/src/main/java/org/opensearch/index/reindex/ClientScrollableHitSource.java b/server/src/main/java/org/opensearch/index/reindex/ClientScrollableHitSource.java index 55d018af46970..8c76b0a62f6bb 100644 --- a/server/src/main/java/org/opensearch/index/reindex/ClientScrollableHitSource.java +++ b/server/src/main/java/org/opensearch/index/reindex/ClientScrollableHitSource.java @@ -175,7 +175,7 @@ private Response wrapSearchResponse(SearchResponse response) { } hits = unmodifiableList(hits); } - long total = response.getHits().getTotalHits().value; + long total = response.getHits().getTotalHits().value(); return new Response(response.isTimedOut(), failures, total, hits, response.getScrollId()); } diff --git a/server/src/main/java/org/opensearch/index/search/MatchQuery.java b/server/src/main/java/org/opensearch/index/search/MatchQuery.java index 86ea799ab311d..11c95c4250805 100644 --- a/server/src/main/java/org/opensearch/index/search/MatchQuery.java +++ b/server/src/main/java/org/opensearch/index/search/MatchQuery.java @@ -52,6 +52,7 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostAttribute; import org.apache.lucene.search.FuzzyQuery; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; @@ -70,7 +71,6 @@ import org.opensearch.index.mapper.MatchOnlyTextFieldMapper; import org.opensearch.index.mapper.TextFieldMapper; import org.opensearch.index.query.QueryShardContext; -import org.opensearch.index.query.support.QueryParsers; import java.io.IOException; import java.util.ArrayList; @@ -359,10 +359,10 @@ private Query createCommonTermsQuery( private Query boolToExtendedCommonTermsQuery(BooleanQuery bq, Occur highFreqOccur, Occur lowFreqOccur, float maxTermFrequency) { ExtendedCommonTermsQuery query = new ExtendedCommonTermsQuery(highFreqOccur, lowFreqOccur, maxTermFrequency); for (BooleanClause clause : bq.clauses()) { - if ((clause.getQuery() instanceof TermQuery) == false) { + if ((clause.query() instanceof TermQuery) == false) { return bq; } - query.add(((TermQuery) clause.getQuery()).getTerm()); + query.add(((TermQuery) clause.query()).getTerm()); } return query; } @@ -600,13 +600,15 @@ private SpanQuery createSpanQuery(TokenStream in, String field, boolean isPrefix protected Query newTermQuery(Term term, float boost) { Supplier querySupplier; if (fuzziness != null) { - querySupplier = () -> { - Query query = fieldType.fuzzyQuery(term.text(), fuzziness, fuzzyPrefixLength, maxExpansions, transpositions, context); - if (query instanceof FuzzyQuery) { - QueryParsers.setRewriteMethod((FuzzyQuery) query, fuzzyRewriteMethod); - } - return query; - }; + querySupplier = () -> fieldType.fuzzyQuery( + term.text(), + fuzziness, + fuzzyPrefixLength, + maxExpansions, + transpositions, + fuzzyRewriteMethod, + context + ); } else { querySupplier = () -> fieldType.termQuery(term.bytes(), context); } @@ -829,7 +831,7 @@ private Query analyzeGraphPhrase(TokenStream source, String field, Type type, in List clauses = new ArrayList<>(); int[] articulationPoints = graph.articulationPoints(); int lastState = 0; - int maxClauseCount = BooleanQuery.getMaxClauseCount(); + int maxClauseCount = IndexSearcher.getMaxClauseCount(); for (int i = 0; i <= articulationPoints.length; i++) { int start = lastState; int end = -1; @@ -847,7 +849,7 @@ private Query analyzeGraphPhrase(TokenStream source, String field, Type type, in SpanQuery q = createSpanQuery(ts, field, usePrefix); if (q != null) { if (queries.size() >= maxClauseCount) { - throw new BooleanQuery.TooManyClauses(); + throw new IndexSearcher.TooManyClauses(); } queries.add(q); } @@ -861,14 +863,14 @@ private Query analyzeGraphPhrase(TokenStream source, String field, Type type, in Term[] terms = graph.getTerms(field, start); assert terms.length > 0; if (terms.length >= maxClauseCount) { - throw new BooleanQuery.TooManyClauses(); + throw new IndexSearcher.TooManyClauses(); } queryPos = newSpanQuery(terms, usePrefix); } if (queryPos != null) { if (clauses.size() >= maxClauseCount) { - throw new BooleanQuery.TooManyClauses(); + throw new IndexSearcher.TooManyClauses(); } clauses.add(queryPos); } diff --git a/server/src/main/java/org/opensearch/index/search/MultiMatchQuery.java b/server/src/main/java/org/opensearch/index/search/MultiMatchQuery.java index 8c0c87e8c9d0c..a2a8dd9795117 100644 --- a/server/src/main/java/org/opensearch/index/search/MultiMatchQuery.java +++ b/server/src/main/java/org/opensearch/index/search/MultiMatchQuery.java @@ -221,7 +221,7 @@ private class BlendedQueryBuilder extends MatchQueryBuilder { protected Query newSynonymQuery(String field, TermAndBoost[] terms) { BytesRef[] values = new BytesRef[terms.length]; for (int i = 0; i < terms.length; i++) { - values[i] = terms[i].term; + values[i] = terms[i].term(); } return blendTerms(context, values, commonTermsCutoff, tieBreaker, lenient, blendedFields); } diff --git a/server/src/main/java/org/opensearch/index/search/NestedHelper.java b/server/src/main/java/org/opensearch/index/search/NestedHelper.java index 50e7e41b95be7..33094455b0382 100644 --- a/server/src/main/java/org/opensearch/index/search/NestedHelper.java +++ b/server/src/main/java/org/opensearch/index/search/NestedHelper.java @@ -32,7 +32,6 @@ package org.opensearch.index.search; -import org.apache.lucene.index.PrefixCodedTerms; import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanClause.Occur; import org.apache.lucene.search.BooleanQuery; @@ -76,11 +75,9 @@ public boolean mightMatchNestedDocs(Query query) { // cover a high majority of use-cases return mightMatchNestedDocs(((TermQuery) query).getTerm().field()); } else if (query instanceof TermInSetQuery) { - PrefixCodedTerms terms = ((TermInSetQuery) query).getTermData(); - if (terms.size() > 0) { - PrefixCodedTerms.TermIterator it = terms.iterator(); - it.next(); - return mightMatchNestedDocs(it.field()); + final TermInSetQuery termInSetQuery = (TermInSetQuery) query; + if (termInSetQuery.getTermsCount() > 0) { + return mightMatchNestedDocs(termInSetQuery.getField()); } else { return false; } @@ -95,13 +92,13 @@ public boolean mightMatchNestedDocs(Query query) { return bq.clauses() .stream() .filter(BooleanClause::isRequired) - .map(BooleanClause::getQuery) + .map(BooleanClause::query) .allMatch(this::mightMatchNestedDocs); } else { return bq.clauses() .stream() - .filter(c -> c.getOccur() == Occur.SHOULD) - .map(BooleanClause::getQuery) + .filter(c -> c.occur() == Occur.SHOULD) + .map(BooleanClause::query) .anyMatch(this::mightMatchNestedDocs); } } else if (query instanceof OpenSearchToParentBlockJoinQuery) { @@ -148,11 +145,9 @@ public boolean mightMatchNonNestedDocs(Query query, String nestedPath) { } else if (query instanceof TermQuery) { return mightMatchNonNestedDocs(((TermQuery) query).getTerm().field(), nestedPath); } else if (query instanceof TermInSetQuery) { - PrefixCodedTerms terms = ((TermInSetQuery) query).getTermData(); - if (terms.size() > 0) { - PrefixCodedTerms.TermIterator it = terms.iterator(); - it.next(); - return mightMatchNonNestedDocs(it.field(), nestedPath); + final TermInSetQuery termInSetQuery = (TermInSetQuery) query; + if (termInSetQuery.getTermsCount() > 0) { + return mightMatchNonNestedDocs(termInSetQuery.getField(), nestedPath); } else { return false; } @@ -167,13 +162,13 @@ public boolean mightMatchNonNestedDocs(Query query, String nestedPath) { return bq.clauses() .stream() .filter(BooleanClause::isRequired) - .map(BooleanClause::getQuery) + .map(BooleanClause::query) .allMatch(q -> mightMatchNonNestedDocs(q, nestedPath)); } else { return bq.clauses() .stream() - .filter(c -> c.getOccur() == Occur.SHOULD) - .map(BooleanClause::getQuery) + .filter(c -> c.occur() == Occur.SHOULD) + .map(BooleanClause::query) .anyMatch(q -> mightMatchNonNestedDocs(q, nestedPath)); } } else { diff --git a/server/src/main/java/org/opensearch/index/search/QueryStringQueryParser.java b/server/src/main/java/org/opensearch/index/search/QueryStringQueryParser.java index 7d6cbc0064535..9be20edeb7b9f 100644 --- a/server/src/main/java/org/opensearch/index/search/QueryStringQueryParser.java +++ b/server/src/main/java/org/opensearch/index/search/QueryStringQueryParser.java @@ -110,7 +110,7 @@ public class QueryStringQueryParser extends XQueryParser { private ZoneId timeZone; private Fuzziness fuzziness = Fuzziness.AUTO; private int fuzzyMaxExpansions = FuzzyQuery.defaultMaxExpansions; - private MultiTermQuery.RewriteMethod fuzzyRewriteMethod = MultiTermQuery.CONSTANT_SCORE_REWRITE; + private MultiTermQuery.RewriteMethod fuzzyRewriteMethod = MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; private boolean fuzzyTranspositions = FuzzyQuery.defaultTranspositions; /** @@ -514,6 +514,7 @@ private Query getFuzzyQuerySingle(String field, String termStr, float minSimilar getFuzzyPrefixLength(), fuzzyMaxExpansions, fuzzyTranspositions, + fuzzyRewriteMethod, context ); } catch (RuntimeException e) { @@ -527,11 +528,7 @@ private Query getFuzzyQuerySingle(String field, String termStr, float minSimilar @Override protected Query newFuzzyQuery(Term term, float minimumSimilarity, int prefixLength) { int numEdits = Fuzziness.build(minimumSimilarity).asDistance(term.text()); - if (fuzzyRewriteMethod != null) { - return new FuzzyQuery(term, numEdits, prefixLength, fuzzyMaxExpansions, fuzzyTranspositions, fuzzyRewriteMethod); - } else { - return new FuzzyQuery(term, numEdits, prefixLength, fuzzyMaxExpansions, fuzzyTranspositions); - } + return new FuzzyQuery(term, numEdits, prefixLength, fuzzyMaxExpansions, fuzzyTranspositions, fuzzyRewriteMethod); } @Override diff --git a/server/src/main/java/org/opensearch/index/search/SimpleQueryStringQueryParser.java b/server/src/main/java/org/opensearch/index/search/SimpleQueryStringQueryParser.java index ce43aaf14c982..f345b35960f9f 100644 --- a/server/src/main/java/org/opensearch/index/search/SimpleQueryStringQueryParser.java +++ b/server/src/main/java/org/opensearch/index/search/SimpleQueryStringQueryParser.java @@ -159,6 +159,7 @@ public Query newFuzzyQuery(String text, int fuzziness) { settings.fuzzyPrefixLength, settings.fuzzyMaxExpansions, settings.fuzzyTranspositions, + null, context ); disjuncts.add(wrapWithBoost(query, entry.getValue())); diff --git a/server/src/main/java/org/opensearch/index/search/comparators/HalfFloatComparator.java b/server/src/main/java/org/opensearch/index/search/comparators/HalfFloatComparator.java index b2e2ba8001b88..fa9de2845e48a 100644 --- a/server/src/main/java/org/opensearch/index/search/comparators/HalfFloatComparator.java +++ b/server/src/main/java/org/opensearch/index/search/comparators/HalfFloatComparator.java @@ -13,6 +13,7 @@ import org.apache.lucene.search.LeafFieldComparator; import org.apache.lucene.search.Pruning; import org.apache.lucene.search.comparators.NumericComparator; +import org.apache.lucene.util.BitUtil; import java.io.IOException; @@ -52,6 +53,16 @@ public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws I return new HalfFloatLeafComparator(context); } + @Override + protected long missingValueAsComparableLong() { + return HalfFloatPoint.halfFloatToSortableShort(missingValue); + } + + @Override + protected long sortableBytesToLong(byte[] bytes) { + return sortableBytesToShort(bytes, 0); + } + /** Leaf comparator for {@link HalfFloatComparator} that provides skipping functionality */ public class HalfFloatLeafComparator extends NumericLeafComparator { @@ -90,23 +101,22 @@ public void copy(int slot, int doc) throws IOException { } @Override - protected int compareMissingValueWithBottomValue() { - return Float.compare(missingValue, bottom); + protected long bottomAsComparableLong() { + return HalfFloatPoint.halfFloatToSortableShort(bottom); } @Override - protected int compareMissingValueWithTopValue() { - return Float.compare(missingValue, topValue); - } - - @Override - protected void encodeBottom(byte[] packedValue) { - HalfFloatPoint.encodeDimension(bottom, packedValue, 0); + protected long topAsComparableLong() { + return HalfFloatPoint.halfFloatToSortableShort(topValue); } + } - @Override - protected void encodeTop(byte[] packedValue) { - HalfFloatPoint.encodeDimension(topValue, packedValue, 0); - } + /** + * Copy of HalfFloatPoint::sortableBytesToShort since it is not exposed + */ + private static short sortableBytesToShort(byte[] encoded, int offset) { + short x = (short) BitUtil.VH_BE_SHORT.get(encoded, offset); + // Re-flip the sign bit to restore the original value: + return (short) (x ^ 0x8000); } } diff --git a/server/src/main/java/org/opensearch/index/search/comparators/UnsignedLongComparator.java b/server/src/main/java/org/opensearch/index/search/comparators/UnsignedLongComparator.java index 2b6bd9933e553..0e9588d845acb 100644 --- a/server/src/main/java/org/opensearch/index/search/comparators/UnsignedLongComparator.java +++ b/server/src/main/java/org/opensearch/index/search/comparators/UnsignedLongComparator.java @@ -13,6 +13,7 @@ import org.apache.lucene.search.LeafFieldComparator; import org.apache.lucene.search.Pruning; import org.apache.lucene.search.comparators.NumericComparator; +import org.apache.lucene.util.NumericUtils; import org.opensearch.common.Numbers; import java.io.IOException; @@ -50,6 +51,20 @@ public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws I return new UnsignedLongLeafComparator(context); } + @Override + protected long missingValueAsComparableLong() { + // Apache Lucene will not use unsigned comparison for long values, so we have to convert to double + // first (with possible lost of precision) and than to sortable long. + return NumericUtils.doubleToSortableLong(missingValue.doubleValue()); + } + + @Override + protected long sortableBytesToLong(byte[] bytes) { + // Apache Lucene will not use unsigned comparison for long values, so we have to convert to double + // first (with possible lost of precision) and than to sortable long. + return NumericUtils.doubleToSortableLong(NumericUtils.sortableBytesToBigInt(bytes, 0, BigIntegerPoint.BYTES).doubleValue()); + } + /** Leaf comparator for {@link UnsignedLongComparator} that provides skipping functionality */ public class UnsignedLongLeafComparator extends NumericLeafComparator { @@ -88,23 +103,17 @@ public void copy(int slot, int doc) throws IOException { } @Override - protected void encodeBottom(byte[] packedValue) { - BigIntegerPoint.encodeDimension(bottom, packedValue, 0); - } - - @Override - protected void encodeTop(byte[] packedValue) { - BigIntegerPoint.encodeDimension(topValue, packedValue, 0); - } - - @Override - protected int compareMissingValueWithBottomValue() { - return missingValue.compareTo(bottom); + protected long bottomAsComparableLong() { + // Apache Lucene will not use unsigned comparison for long values, so we have to convert to double + // first (with possible lost of precision) and than to sortable long. + return NumericUtils.doubleToSortableLong(bottom.doubleValue()); } @Override - protected int compareMissingValueWithTopValue() { - return missingValue.compareTo(topValue); + protected long topAsComparableLong() { + // Apache Lucene will not use unsigned comparison for long values, so we have to convert to double + // first (with possible lost of precision) and than to sortable long. + return NumericUtils.doubleToSortableLong(topValue.doubleValue()); } } } diff --git a/server/src/main/java/org/opensearch/index/shard/RemoveCorruptedLuceneSegmentsAction.java b/server/src/main/java/org/opensearch/index/shard/RemoveCorruptedLuceneSegmentsAction.java index 93ef2c4abf09d..8e78515d3d218 100644 --- a/server/src/main/java/org/opensearch/index/shard/RemoveCorruptedLuceneSegmentsAction.java +++ b/server/src/main/java/org/opensearch/index/shard/RemoveCorruptedLuceneSegmentsAction.java @@ -32,6 +32,7 @@ package org.opensearch.index.shard; import org.apache.lucene.index.CheckIndex; +import org.apache.lucene.index.CheckIndex.Level; import org.apache.lucene.store.Directory; import org.apache.lucene.store.Lock; import org.opensearch.OpenSearchException; @@ -58,7 +59,7 @@ public Tuple getCleanStatus final CheckIndex.Status status; try (CheckIndex checker = new CheckIndex(indexDirectory, writeLock)) { - checker.setChecksumsOnly(true); + checker.setLevel(Level.MIN_LEVEL_FOR_CHECKSUM_CHECKS); checker.setInfoStream(printStream, verbose); status = checker.checkIndex(null); @@ -89,7 +90,7 @@ public void execute(Terminal terminal, Directory indexDirectory, Lock writeLock, final CheckIndex.Status status; try (CheckIndex checker = new CheckIndex(indexDirectory, writeLock)) { - checker.setChecksumsOnly(true); + checker.setLevel(Level.MIN_LEVEL_FOR_CHECKSUM_CHECKS); checker.setInfoStream(printStream, verbose); status = checker.checkIndex(null); diff --git a/server/src/main/java/org/opensearch/index/shard/SearchOperationListener.java b/server/src/main/java/org/opensearch/index/shard/SearchOperationListener.java index 5e63262df0d70..6fb8ccd8e5e4e 100644 --- a/server/src/main/java/org/opensearch/index/shard/SearchOperationListener.java +++ b/server/src/main/java/org/opensearch/index/shard/SearchOperationListener.java @@ -33,6 +33,7 @@ import org.apache.logging.log4j.Logger; import org.apache.logging.log4j.message.ParameterizedMessage; +import org.apache.lucene.search.IndexSearcher; import org.opensearch.ExceptionsHelper; import org.opensearch.common.annotation.PublicApi; import org.opensearch.search.internal.ReaderContext; @@ -73,7 +74,7 @@ default void onQueryPhase(SearchContext searchContext, long tookInNanos) {} /** * Executed before the slice execution in - * {@link org.opensearch.search.internal.ContextIndexSearcher#search(List, org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)}. + * {@link org.opensearch.search.internal.ContextIndexSearcher#search(IndexSearcher.LeafReaderContextPartition[], org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)}. * This will be called once per slice in concurrent search and only once in non-concurrent search. * @param searchContext the current search context */ @@ -81,7 +82,7 @@ default void onPreSliceExecution(SearchContext searchContext) {} /** * Executed if the slice execution in - * {@link org.opensearch.search.internal.ContextIndexSearcher#search(List, org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)} failed. + * {@link org.opensearch.search.internal.ContextIndexSearcher#search(IndexSearcher.LeafReaderContextPartition[], org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)} failed. * This will be called once per slice in concurrent search and only once in non-concurrent search. * @param searchContext the current search context */ @@ -89,7 +90,7 @@ default void onFailedSliceExecution(SearchContext searchContext) {} /** * Executed after the slice execution in - * {@link org.opensearch.search.internal.ContextIndexSearcher#search(List, org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)} successfully finished. + * {@link org.opensearch.search.internal.ContextIndexSearcher#search(IndexSearcher.LeafReaderContextPartition[], org.apache.lucene.search.Weight, org.apache.lucene.search.Collector)} successfully finished. * This will be called once per slice in concurrent search and only once in non-concurrent search. * Note: this is not invoked if the slice execution failed.* * @param searchContext the current search context diff --git a/server/src/main/java/org/opensearch/index/shard/ShardSplittingQuery.java b/server/src/main/java/org/opensearch/index/shard/ShardSplittingQuery.java index 219ead931797a..4c1c4796ef6b2 100644 --- a/server/src/main/java/org/opensearch/index/shard/ShardSplittingQuery.java +++ b/server/src/main/java/org/opensearch/index/shard/ShardSplittingQuery.java @@ -48,6 +48,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.search.join.BitSetProducer; @@ -95,7 +96,7 @@ public String toString() { } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { LeafReader leafReader = context.reader(); FixedBitSet bitSet = new FixedBitSet(leafReader.maxDoc()); Terms terms = leafReader.terms(RoutingFieldMapper.NAME); @@ -130,7 +131,8 @@ public Scorer scorer(LeafReaderContext context) throws IOException { TwoPhaseIterator twoPhaseIterator = parentBitSet == null ? new RoutingPartitionedDocIdSetIterator(visitor) : new NestedRoutingPartitionedDocIdSetIterator(visitor, parentBitSet); - return new ConstantScoreScorer(this, score(), scoreMode, twoPhaseIterator); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, twoPhaseIterator); + return new DefaultScorerSupplier(scorer); } else { // here we potentially guard the docID consumers with our parent bitset if we have one. // this ensures that we are only marking root documents in the nested case and if necessary @@ -171,7 +173,8 @@ public Scorer scorer(LeafReaderContext context) throws IOException { } } - return new ConstantScoreScorer(this, score(), scoreMode, new BitSetIterator(bitSet, bitSet.length())); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, new BitSetIterator(bitSet, bitSet.length())); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/index/store/CompositeDirectory.java b/server/src/main/java/org/opensearch/index/store/CompositeDirectory.java index eb89c86ae687f..1c9a8b98fb86e 100644 --- a/server/src/main/java/org/opensearch/index/store/CompositeDirectory.java +++ b/server/src/main/java/org/opensearch/index/store/CompositeDirectory.java @@ -24,7 +24,6 @@ import org.opensearch.index.store.remote.filecache.CachedFullFileIndexInput; import org.opensearch.index.store.remote.filecache.CachedIndexInput; import org.opensearch.index.store.remote.filecache.FileCache; -import org.opensearch.index.store.remote.utils.BlockIOContext; import org.opensearch.index.store.remote.utils.FileTypeUtils; import org.opensearch.index.store.remote.utils.TransferManager; @@ -68,7 +67,7 @@ public CompositeDirectory(Directory localDirectory, Directory remoteDirectory, F this.fileCache = fileCache; transferManager = new TransferManager( (name, position, length) -> new InputStreamIndexInput( - remoteDirectory.openInput(name, new BlockIOContext(IOContext.DEFAULT, position, length)), + CompositeDirectory.this.remoteDirectory.openBlockInput(name, position, length, IOContext.DEFAULT), length ), fileCache diff --git a/server/src/main/java/org/opensearch/index/store/FsDirectoryFactory.java b/server/src/main/java/org/opensearch/index/store/FsDirectoryFactory.java index c963f8aa95b8d..db1cc9e843e73 100644 --- a/server/src/main/java/org/opensearch/index/store/FsDirectoryFactory.java +++ b/server/src/main/java/org/opensearch/index/store/FsDirectoryFactory.java @@ -56,6 +56,7 @@ import java.nio.file.Path; import java.util.HashSet; import java.util.Set; +import java.util.function.BiPredicate; /** * Factory for a filesystem directory @@ -100,12 +101,12 @@ protected Directory newFSDirectory(Path location, LockFactory lockFactory, Index final Set nioExtensions = new HashSet<>(indexSettings.getValue(IndexModule.INDEX_STORE_HYBRID_NIO_EXTENSIONS)); if (primaryDirectory instanceof MMapDirectory) { MMapDirectory mMapDirectory = (MMapDirectory) primaryDirectory; - return new HybridDirectory(lockFactory, setPreload(mMapDirectory, lockFactory, preLoadExtensions), nioExtensions); + return new HybridDirectory(lockFactory, setPreload(mMapDirectory, preLoadExtensions), nioExtensions); } else { return primaryDirectory; } case MMAPFS: - return setPreload(new MMapDirectory(location, lockFactory), lockFactory, preLoadExtensions); + return setPreload(new MMapDirectory(location, lockFactory), preLoadExtensions); // simplefs was removed in Lucene 9; support for enum is maintained for bwc case SIMPLEFS: case NIOFS: @@ -115,15 +116,9 @@ protected Directory newFSDirectory(Path location, LockFactory lockFactory, Index } } - public static MMapDirectory setPreload(MMapDirectory mMapDirectory, LockFactory lockFactory, Set preLoadExtensions) - throws IOException { - assert mMapDirectory.getPreload() == false; + public static MMapDirectory setPreload(MMapDirectory mMapDirectory, Set preLoadExtensions) throws IOException { if (preLoadExtensions.isEmpty() == false) { - if (preLoadExtensions.contains("*")) { - mMapDirectory.setPreload(true); - } else { - return new PreLoadMMapDirectory(mMapDirectory, lockFactory, preLoadExtensions); - } + mMapDirectory.setPreload(createPreloadPredicate(preLoadExtensions)); } return mMapDirectory; } @@ -136,6 +131,20 @@ public static boolean isHybridFs(Directory directory) { return unwrap instanceof HybridDirectory; } + static BiPredicate createPreloadPredicate(Set preLoadExtensions) { + if (preLoadExtensions.contains("*")) { + return MMapDirectory.ALL_FILES; + } else { + return (s, f) -> { + int dotIndex = s.lastIndexOf('.'); + if (dotIndex > 0) { + return preLoadExtensions.contains(s.substring(dotIndex + 1)); + } + return false; + }; + } + } + /** * A hybrid directory implementation * @@ -181,55 +190,4 @@ MMapDirectory getDelegate() { return delegate; } } - - /** - * Pre loaded mmap directory - * - * @opensearch.internal - */ - // TODO it would be nice to share code between PreLoadMMapDirectory and HybridDirectory but due to the nesting aspect of - // directories here makes it tricky. It would be nice to allow MMAPDirectory to pre-load on a per IndexInput basis. - static final class PreLoadMMapDirectory extends MMapDirectory { - private final MMapDirectory delegate; - private final Set preloadExtensions; - - PreLoadMMapDirectory(MMapDirectory delegate, LockFactory lockFactory, Set preload) throws IOException { - super(delegate.getDirectory(), lockFactory); - super.setPreload(false); - this.delegate = delegate; - this.delegate.setPreload(true); - this.preloadExtensions = preload; - assert getPreload() == false; - } - - @Override - public void setPreload(boolean preload) { - throw new IllegalArgumentException("can't set preload on a preload-wrapper"); - } - - @Override - public IndexInput openInput(String name, IOContext context) throws IOException { - if (useDelegate(name)) { - // we need to do these checks on the outer directory since the inner doesn't know about pending deletes - ensureOpen(); - ensureCanRead(name); - return delegate.openInput(name, context); - } - return super.openInput(name, context); - } - - @Override - public synchronized void close() throws IOException { - IOUtils.close(super::close, delegate); - } - - boolean useDelegate(String name) { - final String extension = FileSwitchDirectory.getExtension(name); - return preloadExtensions.contains(extension); - } - - MMapDirectory getDelegate() { - return delegate; - } - } } diff --git a/server/src/main/java/org/opensearch/index/store/RemoteDirectory.java b/server/src/main/java/org/opensearch/index/store/RemoteDirectory.java index aa856aa22313d..949113cce87d6 100644 --- a/server/src/main/java/org/opensearch/index/store/RemoteDirectory.java +++ b/server/src/main/java/org/opensearch/index/store/RemoteDirectory.java @@ -32,7 +32,6 @@ import org.opensearch.core.action.ActionListener; import org.opensearch.core.common.unit.ByteSizeUnit; import org.opensearch.index.store.exception.ChecksumCombinationException; -import org.opensearch.index.store.remote.utils.BlockIOContext; import java.io.FileNotFoundException; import java.io.IOException; @@ -202,15 +201,42 @@ public IndexInput openInput(String name, IOContext context) throws IOException { return openInput(name, fileLength(name), context); } + /** + * Opens a stream for reading one block from the existing file and returns {@link RemoteIndexInput} enclosing + * the block stream. + * @param name the name of an existing file. + * @param position block start position + * @param length block length + * @param fileLength file length + * @param context desired {@link IOContext} context + * @return the {@link RemoteIndexInput} enclosing the block stream + * @throws IOException in case of I/O error + * @throws NoSuchFileException if the file does not exist + */ + public IndexInput openBlockInput(String name, long position, long length, long fileLength, IOContext context) throws IOException { + try { + return getBlockInput(name, position, length, fileLength); + } catch (Exception e) { + logger.error("Exception while reading blob for file: " + name + " for path " + blobContainer.path()); + throw e; + } + } + + /** + * Opens a stream for reading the existing file and returns {@link RemoteIndexInput} enclosing + * the stream. + * @param name the name of an existing file. + * @param fileLength file length + * @param context desired {@link IOContext} context + * @return the {@link RemoteIndexInput} enclosing the block stream + * @throws IOException in case of I/O error + * @throws NoSuchFileException if the file does not exist + */ public IndexInput openInput(String name, long fileLength, IOContext context) throws IOException { InputStream inputStream = null; try { - if (context instanceof BlockIOContext) { - return getBlockInput(name, fileLength, (BlockIOContext) context); - } else { - inputStream = blobContainer.readBlob(name); - return new RemoteIndexInput(name, downloadRateLimiter.apply(inputStream), fileLength); - } + inputStream = blobContainer.readBlob(name); + return new RemoteIndexInput(name, downloadRateLimiter.apply(inputStream), fileLength); } catch (Exception e) { // In case the RemoteIndexInput creation fails, close the input stream to avoid file handler leak. if (inputStream != null) { @@ -441,10 +467,8 @@ private long calculateChecksumOfChecksum(Directory directory, String file) throw } } - private IndexInput getBlockInput(String name, long fileLength, BlockIOContext blockIOContext) throws IOException { - long position = blockIOContext.getBlockStart(); - long length = blockIOContext.getBlockSize(); - if (position < 0 || length < 0 || (position + length > fileLength)) { + private IndexInput getBlockInput(String name, long position, long length, long fileLength) throws IOException { + if (position < 0 || length <= 0 || (position + length > fileLength)) { throw new IllegalArgumentException("Invalid values of block start and size"); } byte[] bytes; diff --git a/server/src/main/java/org/opensearch/index/store/RemoteSegmentStoreDirectory.java b/server/src/main/java/org/opensearch/index/store/RemoteSegmentStoreDirectory.java index d51fe0643575e..941cf047347f7 100644 --- a/server/src/main/java/org/opensearch/index/store/RemoteSegmentStoreDirectory.java +++ b/server/src/main/java/org/opensearch/index/store/RemoteSegmentStoreDirectory.java @@ -494,6 +494,7 @@ public IndexOutput createOutput(String name, IOContext context) throws IOExcepti * Opens a stream for reading an existing file and returns {@link RemoteIndexInput} enclosing the stream. * * @param name the name of an existing file. + * @param context desired {@link IOContext} context * @throws IOException in case of I/O error * @throws NoSuchFileException if the file does not exist either in cache or remote segment store */ @@ -508,6 +509,28 @@ public IndexInput openInput(String name, IOContext context) throws IOException { } } + /** + * Opens a stream for reading one block from the existing file and returns {@link RemoteIndexInput} enclosing + * the block stream. + * @param name the name of an existing file. + * @param position block start position + * @param length block length + * @param context desired {@link IOContext} context + * @return the {@link RemoteIndexInput} enclosing the block stream + * @throws IOException in case of I/O error + * @throws NoSuchFileException if the file does not exist + */ + + public IndexInput openBlockInput(String name, long position, long length, IOContext context) throws IOException { + String remoteFilename = getExistingRemoteFilename(name); + long fileLength = fileLength(name); + if (remoteFilename != null) { + return remoteDataDirectory.openBlockInput(remoteFilename, position, length, fileLength, context); + } else { + throw new NoSuchFileException(name); + } + } + /** * Copies a file from the source directory to a remote based on multi-stream upload support. * If vendor plugin supports uploading multiple parts in parallel, BlobContainer#writeBlobByStreams diff --git a/server/src/main/java/org/opensearch/index/store/Store.java b/server/src/main/java/org/opensearch/index/store/Store.java index e7de61ff764f6..48ed4d9e75dcb 100644 --- a/server/src/main/java/org/opensearch/index/store/Store.java +++ b/server/src/main/java/org/opensearch/index/store/Store.java @@ -737,7 +737,7 @@ private static void failIfCorrupted(Directory directory) throws IOException { List ex = new ArrayList<>(); for (String file : files) { if (file.startsWith(CORRUPTED_MARKER_NAME_PREFIX)) { - try (ChecksumIndexInput input = directory.openChecksumInput(file, IOContext.READONCE)) { + try (ChecksumIndexInput input = directory.openChecksumInput(file)) { CodecUtil.checkHeader(input, CODEC, CORRUPTED_MARKER_CODEC_VERSION, CORRUPTED_MARKER_CODEC_VERSION); final int size = input.readVInt(); final byte[] buffer = new byte[size]; diff --git a/server/src/main/java/org/opensearch/index/store/remote/directory/RemoteSnapshotDirectory.java b/server/src/main/java/org/opensearch/index/store/remote/directory/RemoteSnapshotDirectory.java index 19ecee67bdb96..b9d772c7e7410 100644 --- a/server/src/main/java/org/opensearch/index/store/remote/directory/RemoteSnapshotDirectory.java +++ b/server/src/main/java/org/opensearch/index/store/remote/directory/RemoteSnapshotDirectory.java @@ -38,7 +38,7 @@ */ public final class RemoteSnapshotDirectory extends Directory { - public static final Version SEARCHABLE_SNAPSHOT_EXTENDED_COMPATIBILITY_MINIMUM_VERSION = LegacyESVersion.V_6_0_0; + public static final Version SEARCHABLE_SNAPSHOT_EXTENDED_COMPATIBILITY_MINIMUM_VERSION = LegacyESVersion.V_7_2_0; private static final String VIRTUAL_FILE_PREFIX = BlobStoreRepository.VIRTUAL_DATA_BLOB_PREFIX; diff --git a/server/src/main/java/org/opensearch/index/store/remote/utils/BlockIOContext.java b/server/src/main/java/org/opensearch/index/store/remote/utils/BlockIOContext.java deleted file mode 100644 index a78dd85d6f194..0000000000000 --- a/server/src/main/java/org/opensearch/index/store/remote/utils/BlockIOContext.java +++ /dev/null @@ -1,53 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * - * The OpenSearch Contributors require contributions made to - * this file be licensed under the Apache-2.0 license or a - * compatible open source license. - */ - -package org.opensearch.index.store.remote.utils; - -import org.apache.lucene.store.IOContext; -import org.opensearch.common.annotation.ExperimentalApi; - -/** - * BlockIOContext is an extension of IOContext which can be used to pass block related information to the openInput() method of any directory - * - * @opensearch.experimental - */ -@ExperimentalApi -public class BlockIOContext extends IOContext { - - private long blockStart; - private long blockSize; - - /** - * Constructor to initialise BlockIOContext with block related information - */ - public BlockIOContext(IOContext ctx, long blockStart, long blockSize) { - super(ctx.context); - verifyBlockStartAndSize(blockStart, blockSize); - this.blockStart = blockStart; - this.blockSize = blockSize; - } - - /** - * Getter for blockStart - */ - public long getBlockStart() { - return blockStart; - } - - /** - * Getter for blockSize - */ - public long getBlockSize() { - return blockSize; - } - - private void verifyBlockStartAndSize(long blockStart, long blockSize) { - if (blockStart < 0) throw new IllegalArgumentException("blockStart must be greater than or equal to 0"); - if (blockSize <= 0) throw new IllegalArgumentException(("blockSize must be greater than 0")); - } -} diff --git a/server/src/main/java/org/opensearch/index/store/remote/utils/TransferManager.java b/server/src/main/java/org/opensearch/index/store/remote/utils/TransferManager.java index 77a8ccfafbac2..2b619c26f49c0 100644 --- a/server/src/main/java/org/opensearch/index/store/remote/utils/TransferManager.java +++ b/server/src/main/java/org/opensearch/index/store/remote/utils/TransferManager.java @@ -67,6 +67,7 @@ public TransferManager(final StreamReader streamReader, final FileCache fileCach * @param blobFetchRequest to fetch * @return future of IndexInput augmented with internal caching maintenance tasks */ + @SuppressWarnings("removal") public IndexInput fetchBlob(BlobFetchRequest blobFetchRequest) throws IOException { final Path key = blobFetchRequest.getFilePath(); logger.trace("fetchBlob called for {}", key.toString()); @@ -106,7 +107,6 @@ public IndexInput fetchBlob(BlobFetchRequest blobFetchRequest) throws IOExceptio } } - @SuppressWarnings("removal") private static FileCachedIndexInput createIndexInput(FileCache fileCache, StreamReader streamReader, BlobFetchRequest request) { try { // This local file cache is ref counted and may not strictly enforce configured capacity. @@ -141,7 +141,7 @@ private static FileCachedIndexInput createIndexInput(FileCache fileCache, Stream } } } - final IndexInput luceneIndexInput = request.getDirectory().openInput(request.getFileName(), IOContext.READ); + final IndexInput luceneIndexInput = request.getDirectory().openInput(request.getFileName(), IOContext.DEFAULT); return new FileCachedIndexInput(fileCache, request.getFilePath(), luceneIndexInput); } catch (IOException e) { throw new UncheckedIOException(e); diff --git a/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java b/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java index 52ed311a1eb92..f16b16ce395b7 100644 --- a/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java +++ b/server/src/main/java/org/opensearch/indices/IndicesQueryCache.java @@ -35,13 +35,11 @@ import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.lucene.index.LeafReaderContext; -import org.apache.lucene.search.BulkScorer; import org.apache.lucene.search.Explanation; import org.apache.lucene.search.LRUQueryCache; import org.apache.lucene.search.Query; import org.apache.lucene.search.QueryCache; import org.apache.lucene.search.QueryCachingPolicy; -import org.apache.lucene.search.Scorer; import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Weight; import org.opensearch.common.annotation.PublicApi; @@ -170,24 +168,12 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio return in.explain(context, doc); } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - shardKeyMap.add(context.reader()); - return in.scorer(context); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { shardKeyMap.add(context.reader()); return in.scorerSupplier(context); } - @Override - public BulkScorer bulkScorer(LeafReaderContext context) throws IOException { - shardKeyMap.add(context.reader()); - return in.bulkScorer(context); - } - @Override public int count(LeafReaderContext context) throws IOException { shardKeyMap.add(context.reader()); diff --git a/server/src/main/java/org/opensearch/indices/SystemIndexDescriptor.java b/server/src/main/java/org/opensearch/indices/SystemIndexDescriptor.java index f3212b1e2fae1..c886b6c0c0607 100644 --- a/server/src/main/java/org/opensearch/indices/SystemIndexDescriptor.java +++ b/server/src/main/java/org/opensearch/indices/SystemIndexDescriptor.java @@ -32,7 +32,9 @@ package org.opensearch.indices; +import org.apache.lucene.util.automaton.Automaton; import org.apache.lucene.util.automaton.CharacterRunAutomaton; +import org.apache.lucene.util.automaton.Operations; import org.opensearch.common.annotation.PublicApi; import org.opensearch.common.regex.Regex; @@ -74,7 +76,8 @@ public SystemIndexDescriptor(String indexPattern, String description) { ); } this.indexPattern = indexPattern; - this.indexPatternAutomaton = new CharacterRunAutomaton(Regex.simpleMatchToAutomaton(indexPattern)); + Automaton a = Operations.determinize(Regex.simpleMatchToAutomaton(indexPattern), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); + this.indexPatternAutomaton = new CharacterRunAutomaton(a); this.description = description; } diff --git a/server/src/main/java/org/opensearch/indices/replication/SegmentFileTransferHandler.java b/server/src/main/java/org/opensearch/indices/replication/SegmentFileTransferHandler.java index 80ba5146535aa..756751cf0b87d 100644 --- a/server/src/main/java/org/opensearch/indices/replication/SegmentFileTransferHandler.java +++ b/server/src/main/java/org/opensearch/indices/replication/SegmentFileTransferHandler.java @@ -110,7 +110,7 @@ protected void onNewResource(StoreFileMetadata md) throws IOException { // Segments* files require IOContext.READONCE // https://github.com/apache/lucene/blob/b2d3a2b37e00f19a74949097736be8fd64745f61/lucene/test-framework/src/java/org/apache/lucene/tests/store/MockDirectoryWrapper.java#L817 if (md.name().startsWith(IndexFileNames.SEGMENTS) == false) { - final IndexInput indexInput = store.directory().openInput(md.name(), IOContext.READ); + final IndexInput indexInput = store.directory().openInput(md.name(), IOContext.DEFAULT); currentInput = new InputStreamIndexInput(indexInput, md.length()) { @Override public void close() throws IOException { diff --git a/server/src/main/java/org/opensearch/lucene/queries/MinDocQuery.java b/server/src/main/java/org/opensearch/lucene/queries/MinDocQuery.java index 5c904d8a7770d..197e20ef30799 100644 --- a/server/src/main/java/org/opensearch/lucene/queries/MinDocQuery.java +++ b/server/src/main/java/org/opensearch/lucene/queries/MinDocQuery.java @@ -42,6 +42,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Weight; import java.io.IOException; @@ -103,14 +104,15 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo } return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { final int maxDoc = context.reader().maxDoc(); if (context.docBase + maxDoc <= minDoc) { return null; } final int segmentMinDoc = Math.max(0, minDoc - context.docBase); final DocIdSetIterator disi = new MinDocIterator(segmentMinDoc, maxDoc); - return new ConstantScoreScorer(this, score(), scoreMode, disi); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, disi); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/lucene/queries/SearchAfterSortedDocQuery.java b/server/src/main/java/org/opensearch/lucene/queries/SearchAfterSortedDocQuery.java index 600ba5b5a92d8..81778c3387863 100644 --- a/server/src/main/java/org/opensearch/lucene/queries/SearchAfterSortedDocQuery.java +++ b/server/src/main/java/org/opensearch/lucene/queries/SearchAfterSortedDocQuery.java @@ -45,6 +45,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Sort; import org.apache.lucene.search.SortField; import org.apache.lucene.search.Weight; @@ -91,8 +92,8 @@ public SearchAfterSortedDocQuery(Sort sort, FieldDoc after) { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { return new ConstantScoreWeight(this, 1.0f) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - Sort segmentSort = context.reader().getMetaData().getSort(); + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { + Sort segmentSort = context.reader().getMetaData().sort(); if (segmentSort == null || Lucene.canEarlyTerminate(sort, segmentSort) == false) { throw new IOException("search sort :[" + sort.getSort() + "] does not match the index sort:[" + segmentSort + "]"); } @@ -104,7 +105,8 @@ public Scorer scorer(LeafReaderContext context) throws IOException { return null; } final DocIdSetIterator disi = new MinDocQuery.MinDocIterator(firstDoc, maxDoc); - return new ConstantScoreScorer(this, score(), scoreMode, disi); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, disi); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/node/Node.java b/server/src/main/java/org/opensearch/node/Node.java index 704a23890b07a..c2c4e68dae257 100644 --- a/server/src/main/java/org/opensearch/node/Node.java +++ b/server/src/main/java/org/opensearch/node/Node.java @@ -488,8 +488,8 @@ protected Node( Constants.OS_ARCH, Constants.JVM_VENDOR, Constants.JVM_NAME, - Constants.JAVA_VERSION, - Constants.JVM_VERSION + System.getProperty("java.version"), + Runtime.version().toString() ); if (jvmInfo.getBundledJdk()) { logger.info("JVM home [{}], using bundled JDK/JRE [{}]", System.getProperty("java.home"), jvmInfo.getUsingBundledJdk()); diff --git a/server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java b/server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java index 93a7dc0cb06af..54c4a3332e005 100644 --- a/server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java +++ b/server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java @@ -4650,7 +4650,7 @@ private void snapshotFile( ) throws IOException { final BlobContainer shardContainer = shardContainer(indexId, shardId); final String file = fileInfo.physicalName(); - try (IndexInput indexInput = store.openVerifyingInput(file, IOContext.READ, fileInfo.metadata())) { + try (IndexInput indexInput = store.openVerifyingInput(file, IOContext.DEFAULT, fileInfo.metadata())) { for (int i = 0; i < fileInfo.numberOfParts(); i++) { final long partBytes = fileInfo.partBytes(i); diff --git a/server/src/main/java/org/opensearch/rest/action/cat/RestCountAction.java b/server/src/main/java/org/opensearch/rest/action/cat/RestCountAction.java index 9c054ffe1bcc7..cc85f4141023c 100644 --- a/server/src/main/java/org/opensearch/rest/action/cat/RestCountAction.java +++ b/server/src/main/java/org/opensearch/rest/action/cat/RestCountAction.java @@ -99,7 +99,7 @@ public RestChannelConsumer doCatRequest(final RestRequest request, final NodeCli return channel -> client.search(countRequest, new RestResponseListener(channel) { @Override public RestResponse buildResponse(SearchResponse countResponse) throws Exception { - assert countResponse.getHits().getTotalHits().relation == TotalHits.Relation.EQUAL_TO; + assert countResponse.getHits().getTotalHits().relation() == TotalHits.Relation.EQUAL_TO; return RestTable.buildResponse(buildTable(request, countResponse), channel); } }); @@ -117,7 +117,7 @@ protected Table getTableWithHeader(final RestRequest request) { private Table buildTable(RestRequest request, SearchResponse response) { Table table = getTableWithHeader(request); table.startRow(); - table.addCell(response.getHits().getTotalHits().value); + table.addCell(response.getHits().getTotalHits().value()); table.endRow(); return table; diff --git a/server/src/main/java/org/opensearch/rest/action/search/RestCountAction.java b/server/src/main/java/org/opensearch/rest/action/search/RestCountAction.java index acedd5a884596..56c20ed956037 100644 --- a/server/src/main/java/org/opensearch/rest/action/search/RestCountAction.java +++ b/server/src/main/java/org/opensearch/rest/action/search/RestCountAction.java @@ -118,7 +118,7 @@ public RestResponse buildResponse(SearchResponse response, XContentBuilder build if (terminateAfter != DEFAULT_TERMINATE_AFTER) { builder.field("terminated_early", response.isTerminatedEarly()); } - builder.field("count", response.getHits().getTotalHits().value); + builder.field("count", response.getHits().getTotalHits().value()); buildBroadcastShardsHeader( builder, request, diff --git a/server/src/main/java/org/opensearch/search/MultiValueMode.java b/server/src/main/java/org/opensearch/search/MultiValueMode.java index fa2e776eca67a..930308abadca0 100644 --- a/server/src/main/java/org/opensearch/search/MultiValueMode.java +++ b/server/src/main/java/org/opensearch/search/MultiValueMode.java @@ -666,7 +666,9 @@ protected BytesRef pick( @Override protected int pick(SortedSetDocValues values) throws IOException { long maxOrd = -1; - for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = values.nextOrd()) { + int count = values.docValueCount(); + long ord; + while ((count-- > 0) && (ord = values.nextOrd()) != SortedSetDocValues.NO_MORE_DOCS) { maxOrd = ord; } return Math.toIntExact(maxOrd); diff --git a/server/src/main/java/org/opensearch/search/SearchHits.java b/server/src/main/java/org/opensearch/search/SearchHits.java index 8232643b353f5..963ce82e636cf 100644 --- a/server/src/main/java/org/opensearch/search/SearchHits.java +++ b/server/src/main/java/org/opensearch/search/SearchHits.java @@ -219,12 +219,12 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws builder.startObject(Fields.HITS); boolean totalHitAsInt = params.paramAsBoolean(RestSearchAction.TOTAL_HITS_AS_INT_PARAM, false); if (totalHitAsInt) { - long total = totalHits == null ? -1 : totalHits.value; + long total = totalHits == null ? -1 : totalHits.value(); builder.field(Fields.TOTAL, total); } else if (totalHits != null) { builder.startObject(Fields.TOTAL); - builder.field("value", totalHits.value); - builder.field("relation", totalHits.relation == Relation.EQUAL_TO ? "eq" : "gte"); + builder.field("value", totalHits.value()); + builder.field("relation", totalHits.relation() == Relation.EQUAL_TO ? "eq" : "gte"); builder.endObject(); } if (Float.isNaN(maxScore)) { diff --git a/server/src/main/java/org/opensearch/search/SearchModule.java b/server/src/main/java/org/opensearch/search/SearchModule.java index 3a746259af3b5..40e0293f88f07 100644 --- a/server/src/main/java/org/opensearch/search/SearchModule.java +++ b/server/src/main/java/org/opensearch/search/SearchModule.java @@ -32,7 +32,7 @@ package org.opensearch.search; -import org.apache.lucene.search.BooleanQuery; +import org.apache.lucene.search.IndexSearcher; import org.opensearch.common.NamedRegistry; import org.opensearch.common.Nullable; import org.opensearch.common.geo.GeoShapeType; @@ -324,7 +324,7 @@ public class SearchModule { * Constructs a new SearchModule object *

* NOTE: This constructor should not be called in production unless an accurate {@link Settings} object is provided. - * When constructed, a static flag is set in Lucene {@link BooleanQuery#setMaxClauseCount} according to the settings. + * When constructed, a static flag is set in Lucene {@link IndexSearcher#setMaxClauseCount} according to the settings. * @param settings Current settings * @param plugins List of included {@link SearchPlugin} objects. */ diff --git a/server/src/main/java/org/opensearch/search/aggregations/MultiBucketCollector.java b/server/src/main/java/org/opensearch/search/aggregations/MultiBucketCollector.java index 9915848ee2e1d..edbad58017785 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/MultiBucketCollector.java +++ b/server/src/main/java/org/opensearch/search/aggregations/MultiBucketCollector.java @@ -38,7 +38,6 @@ import org.apache.lucene.search.LeafCollector; import org.apache.lucene.search.MultiCollector; import org.apache.lucene.search.Scorable; -import org.apache.lucene.search.ScoreCachingWrappingScorer; import org.apache.lucene.search.ScoreMode; import java.io.IOException; @@ -176,7 +175,54 @@ public LeafBucketCollector getLeafCollector(LeafReaderContext context) throws IO case 1: return leafCollectors.get(0); default: - return new MultiLeafBucketCollector(leafCollectors, cacheScores); + LeafBucketCollector collector = new MultiLeafBucketCollector(leafCollectors, cacheScores); + if (cacheScores) { + collector = new ScoreCachingLeafBucketCollector(collector); + } + return collector; + } + } + + private static class ScoreCachingLeafBucketCollector extends LeafBucketCollector { + private final LeafBucketCollector in; + private ScoreCachingWrappingScorer scoreCachingWrappingScorer; + + public ScoreCachingLeafBucketCollector(LeafBucketCollector in) { + this.in = in; + } + + @Override + public void setScorer(Scorable scorer) throws IOException { + scoreCachingWrappingScorer = new ScoreCachingWrappingScorer(scorer); + in.setScorer(scoreCachingWrappingScorer); + } + + @Override + public void collect(int doc, long owningBucketOrd) throws IOException { + if (scoreCachingWrappingScorer != null) { + scoreCachingWrappingScorer.curDoc = doc; + } + in.collect(doc, owningBucketOrd); + } + } + + private static class ScoreCachingWrappingScorer extends Scorable { + private int lastDoc = -1; + private int curDoc = -1; + private float curScore; + private final Scorable in; + + ScoreCachingWrappingScorer(Scorable in) { + this.in = in; + } + + @Override + public float score() throws IOException { + if (lastDoc != curDoc) { + curScore = in.score(); + lastDoc = curDoc; + } + return curScore; } } @@ -199,9 +245,6 @@ private MultiLeafBucketCollector(List collectors, boolean c @Override public void setScorer(Scorable scorer) throws IOException { - if (cacheScores) { - scorer = ScoreCachingWrappingScorer.wrap(scorer); - } for (int i = 0; i < numCollectors; ++i) { final LeafCollector c = collectors[i]; c.setScorer(scorer); diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java index cfe716eb57ca8..7a0bffa0cf74a 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregator.java @@ -384,7 +384,7 @@ private boolean isMaybeMultivalued(LeafReaderContext context, SortField sortFiel * optimization and null if index sort is not applicable. */ private Sort buildIndexSortPrefix(LeafReaderContext context) throws IOException { - Sort indexSort = context.reader().getMetaData().getSort(); + Sort indexSort = context.reader().getMetaData().sort(); if (indexSort == null) { return null; } diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/GlobalOrdinalValuesSource.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/GlobalOrdinalValuesSource.java index 3e5c53d470f79..227dce543cfe9 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/GlobalOrdinalValuesSource.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/GlobalOrdinalValuesSource.java @@ -50,7 +50,7 @@ import java.io.IOException; -import static org.apache.lucene.index.SortedSetDocValues.NO_MORE_ORDS; +import static org.apache.lucene.index.SortedSetDocValues.NO_MORE_DOCS; /** * A {@link SingleDimensionValuesSource} for global ordinals. @@ -176,7 +176,8 @@ LeafBucketCollector getLeafCollector(LeafReaderContext context, LeafBucketCollec public void collect(int doc, long bucket) throws IOException { if (dvs.advanceExact(doc)) { long ord; - while ((ord = dvs.nextOrd()) != NO_MORE_ORDS) { + int count = dvs.docValueCount(); + while ((count-- > 0) && (ord = dvs.nextOrd()) != NO_MORE_DOCS) { currentValue = ord; next.collect(doc, bucket); } @@ -206,7 +207,8 @@ public void collect(int doc, long bucket) throws IOException { if (currentValueIsSet == false) { if (dvs.advanceExact(doc)) { long ord; - while ((ord = dvs.nextOrd()) != NO_MORE_ORDS) { + int count = dvs.docValueCount(); + while ((count-- > 0) && (ord = dvs.nextOrd()) != NO_MORE_DOCS) { if (term.equals(lookup.lookupOrd(ord))) { currentValueIsSet = true; currentValue = ord; diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/LongValuesSource.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/LongValuesSource.java index 48e080c1576dd..da855e3d14bce 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/LongValuesSource.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/composite/LongValuesSource.java @@ -39,7 +39,7 @@ import org.apache.lucene.index.SortedNumericDocValues; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexOrDocValuesQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.PointRangeQuery; @@ -242,8 +242,8 @@ static boolean checkMatchAllOrRangeQuery(Query query, String fieldName) { } else if (query instanceof PointRangeQuery) { PointRangeQuery pointQuery = (PointRangeQuery) query; return fieldName.equals(pointQuery.getField()); - } else if (query instanceof DocValuesFieldExistsQuery) { - DocValuesFieldExistsQuery existsQuery = (DocValuesFieldExistsQuery) query; + } else if (query instanceof FieldExistsQuery) { + FieldExistsQuery existsQuery = (FieldExistsQuery) query; return fieldName.equals(existsQuery.getField()); } else { return false; diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/nested/NestedAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/nested/NestedAggregator.java index db8979d611b4f..d807e625c8d7b 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/nested/NestedAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/nested/NestedAggregator.java @@ -257,7 +257,6 @@ void processBufferedChildBuckets() throws IOException { int childDocId = res.v2(); for (; childDocId < currentParentDoc; childDocId = childDocs.nextDoc()) { - cachedScorer.doc = childDocId; for (var bucket : bucketBuffer) { collectBucket(sub, childDocId, bucket); } @@ -272,19 +271,12 @@ void processBufferedChildBuckets() throws IOException { * @opensearch.internal */ private static class CachedScorable extends Scorable { - int doc; float score; @Override public final float score() { return score; } - - @Override - public int docID() { - return doc; - } - } } diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregator.java index a48649af99be3..cea8cb093b95e 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregator.java @@ -202,7 +202,11 @@ abstract static class SortedSetRangeLeafCollector extends LeafBucketCollectorBas public void collect(int doc, long bucket) throws IOException { if (values.advanceExact(doc)) { int lo = 0; - for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = values.nextOrd()) { + for (int i = 0; i < values.docValueCount(); i++) { + long ord = values.nextOrd(); + if (ord == SortedSetDocValues.NO_MORE_DOCS) { + break; + } lo = collect(doc, ord, bucket, lo); } } diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/BestDocsDeferringCollector.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/BestDocsDeferringCollector.java index 080142185f82e..89106f7ec1e7a 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/BestDocsDeferringCollector.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/BestDocsDeferringCollector.java @@ -39,7 +39,7 @@ import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopDocsCollector; -import org.apache.lucene.search.TopScoreDocCollector; +import org.apache.lucene.search.TopScoreDocCollectorManager; import org.apache.lucene.util.RamUsageEstimator; import org.opensearch.OpenSearchException; import org.opensearch.common.lease.Releasable; @@ -124,7 +124,7 @@ public void collect(int doc, long bucket) throws IOException { // Designed to be overridden by subclasses that may score docs by criteria // other than Lucene score protected TopDocsCollector createTopDocsCollector(int size) throws IOException { - return TopScoreDocCollector.create(size, Integer.MAX_VALUE); + return new TopScoreDocCollectorManager(size, null, Integer.MAX_VALUE, false).newCollector(); } // Can be overridden by subclasses that have a different priority queue implementation @@ -239,7 +239,6 @@ class PerSegmentCollects extends Scorable { private LeafReaderContext readerContext; int maxDocId = Integer.MIN_VALUE; private float currentScore; - private int currentDocId = -1; private Scorable currentScorer; PerSegmentCollects(LeafReaderContext readerContext) throws IOException { @@ -274,7 +273,6 @@ public void replayRelatedMatches(List sd) throws IOException { leafCollector.setScorer(this); currentScore = 0; - currentDocId = -1; if (maxDocId < 0) { return; } @@ -284,7 +282,6 @@ public void replayRelatedMatches(List sd) throws IOException { int rebased = scoreDoc.doc - readerContext.docBase; if ((rebased >= 0) && (rebased <= maxDocId)) { currentScore = scoreDoc.score; - currentDocId = rebased; // We stored the bucket ID in Lucene's shardIndex property // for convenience. leafCollector.collect(rebased, scoreDoc.shardIndex); @@ -301,11 +298,6 @@ public float score() throws IOException { return currentScore; } - @Override - public int docID() { - return currentDocId; - } - public void collect(int docId, long parentBucket) throws IOException { perBucketSamples = bigArrays.grow(perBucketSamples, parentBucket + 1); PerParentBucketSamples sampler = perBucketSamples.get((int) parentBucket); diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/DiversifiedOrdinalsSamplerAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/DiversifiedOrdinalsSamplerAggregator.java index 953a45971919c..a26e057b82ccb 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/DiversifiedOrdinalsSamplerAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/sampler/DiversifiedOrdinalsSamplerAggregator.java @@ -148,7 +148,7 @@ public boolean advanceExact(int target) throws IOException { value = globalOrds.nextOrd(); // Check there isn't a second value for this // document - if (globalOrds.nextOrd() != SortedSetDocValues.NO_MORE_ORDS) { + if (globalOrds.docValueCount() != 1) { throw new IllegalArgumentException("Sample diversifying key must be a single valued-field"); } return true; diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java index 9e40f7b4c9b3e..9dbc97f7d2cb6 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/GlobalOrdinalsStringTermsAggregator.java @@ -79,8 +79,7 @@ import java.util.function.LongUnaryOperator; import static org.opensearch.search.aggregations.InternalOrder.isKeyOrder; -import static org.apache.lucene.index.SortedSetDocValues.NO_MORE_ORDS; -import static org.apache.lucene.search.DocIdSetIterator.NO_MORE_DOCS; +import static org.apache.lucene.index.SortedSetDocValues.NO_MORE_DOCS; /** * An aggregator of string values that relies on global ordinals in order to build buckets. @@ -289,7 +288,9 @@ public void collect(int doc, long owningBucketOrd) throws IOException { if (false == globalOrds.advanceExact(doc)) { return; } - for (long globalOrd = globalOrds.nextOrd(); globalOrd != NO_MORE_ORDS; globalOrd = globalOrds.nextOrd()) { + int count = globalOrds.docValueCount(); + long globalOrd; + while ((count-- > 0) && (globalOrd = globalOrds.nextOrd()) != SortedSetDocValues.NO_MORE_DOCS) { collectionStrategy.collectGlobalOrd(owningBucketOrd, doc, globalOrd, sub); } } @@ -301,7 +302,9 @@ public void collect(int doc, long owningBucketOrd) throws IOException { if (false == globalOrds.advanceExact(doc)) { return; } - for (long globalOrd = globalOrds.nextOrd(); globalOrd != NO_MORE_ORDS; globalOrd = globalOrds.nextOrd()) { + int count = globalOrds.docValueCount(); + long globalOrd; + while ((count-- > 0) && (globalOrd = globalOrds.nextOrd()) != SortedSetDocValues.NO_MORE_DOCS) { if (false == acceptedGlobalOrdinals.test(globalOrd)) { continue; } @@ -478,7 +481,9 @@ public void collect(int doc, long owningBucketOrd) throws IOException { if (false == segmentOrds.advanceExact(doc)) { return; } - for (long segmentOrd = segmentOrds.nextOrd(); segmentOrd != NO_MORE_ORDS; segmentOrd = segmentOrds.nextOrd()) { + int count = segmentOrds.docValueCount(); + long segmentOrd; + while ((count-- > 0) && (segmentOrd = segmentOrds.nextOrd()) != SortedSetDocValues.NO_MORE_DOCS) { long docCount = docCountProvider.getDocCount(doc); segmentDocCounts.increment(segmentOrd + 1, docCount); } diff --git a/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/IncludeExclude.java b/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/IncludeExclude.java index 20f294bc7199b..867c76e39487e 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/IncludeExclude.java +++ b/server/src/main/java/org/opensearch/search/aggregations/bucket/terms/IncludeExclude.java @@ -744,7 +744,12 @@ private Automaton toAutomaton(int maxRegExLength) { } else if (excludeValues != null) { a = Operations.minus(a, Automata.makeStringUnion(excludeValues), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); } - return a; + + if (a.isDeterministic()) { + return a; + } else { + return Operations.determinize(a, Operations.DEFAULT_DETERMINIZE_WORK_LIMIT); + } } private static void validateRegExpStringLength(String source, int maxRegexLength) { diff --git a/server/src/main/java/org/opensearch/search/aggregations/metrics/CardinalityAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/metrics/CardinalityAggregator.java index 0f3d975960364..d578c37af8818 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/metrics/CardinalityAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/metrics/CardinalityAggregator.java @@ -226,7 +226,7 @@ private boolean tryScoreWithPruningCollector(LeafReaderContext ctx, Collector pr return false; } Bits liveDocs = ctx.reader().getLiveDocs(); - scorer.score(pruningCollector, liveDocs); + scorer.score(pruningCollector, liveDocs, 0, DocIdSetIterator.NO_MORE_DOCS); pruningCollector.postCollect(); Releasables.close(pruningCollector); } catch (Exception e) { @@ -354,7 +354,7 @@ private static class PruningCollector extends Collector { this.queue = new DisiPriorityQueue(postingMap.size()); for (Scorer scorer : postingMap.values()) { - queue.add(new DisiWrapper(scorer)); + queue.add(new DisiWrapper(scorer, false)); } competitiveIterator = new DisjunctionDISI(queue); @@ -554,7 +554,9 @@ public void collect(int doc, long bucketOrd) throws IOException { visitedOrds.set(bucketOrd, bits); } if (values.advanceExact(doc)) { - for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = values.nextOrd()) { + int count = values.docValueCount(); + long ord; + while ((count-- > 0) && (ord = values.nextOrd()) != SortedSetDocValues.NO_MORE_DOCS) { bits.set((int) ord); } } diff --git a/server/src/main/java/org/opensearch/search/aggregations/metrics/InternalTopHits.java b/server/src/main/java/org/opensearch/search/aggregations/metrics/InternalTopHits.java index e82f49aa13600..fab6113b397da 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/metrics/InternalTopHits.java +++ b/server/src/main/java/org/opensearch/search/aggregations/metrics/InternalTopHits.java @@ -183,7 +183,7 @@ public InternalAggregation reduce(List aggregations, Reduce } while (shardDocs[scoreDoc.shardIndex].scoreDocs[position] != scoreDoc); hits[i] = shardHits[scoreDoc.shardIndex].getAt(position); } - assert reducedTopDocs.totalHits.relation == Relation.EQUAL_TO; + assert reducedTopDocs.totalHits.relation() == Relation.EQUAL_TO; return new InternalTopHits( name, this.from, @@ -224,8 +224,8 @@ public boolean equals(Object obj) { InternalTopHits other = (InternalTopHits) obj; if (from != other.from) return false; if (size != other.size) return false; - if (topDocs.topDocs.totalHits.value != other.topDocs.topDocs.totalHits.value) return false; - if (topDocs.topDocs.totalHits.relation != other.topDocs.topDocs.totalHits.relation) return false; + if (topDocs.topDocs.totalHits.value() != other.topDocs.topDocs.totalHits.value()) return false; + if (topDocs.topDocs.totalHits.relation() != other.topDocs.topDocs.totalHits.relation()) return false; if (topDocs.topDocs.scoreDocs.length != other.topDocs.topDocs.scoreDocs.length) return false; for (int d = 0; d < topDocs.topDocs.scoreDocs.length; d++) { ScoreDoc thisDoc = topDocs.topDocs.scoreDocs[d]; @@ -251,8 +251,8 @@ public int hashCode() { int hashCode = super.hashCode(); hashCode = 31 * hashCode + Integer.hashCode(from); hashCode = 31 * hashCode + Integer.hashCode(size); - hashCode = 31 * hashCode + Long.hashCode(topDocs.topDocs.totalHits.value); - hashCode = 31 * hashCode + topDocs.topDocs.totalHits.relation.hashCode(); + hashCode = 31 * hashCode + Long.hashCode(topDocs.topDocs.totalHits.value()); + hashCode = 31 * hashCode + topDocs.topDocs.totalHits.relation().hashCode(); for (int d = 0; d < topDocs.topDocs.scoreDocs.length; d++) { ScoreDoc doc = topDocs.topDocs.scoreDocs[d]; hashCode = 31 * hashCode + doc.doc; diff --git a/server/src/main/java/org/opensearch/search/aggregations/metrics/MetricInspectionHelper.java b/server/src/main/java/org/opensearch/search/aggregations/metrics/MetricInspectionHelper.java index 47e27da5394b3..8408036772673 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/metrics/MetricInspectionHelper.java +++ b/server/src/main/java/org/opensearch/search/aggregations/metrics/MetricInspectionHelper.java @@ -78,7 +78,7 @@ public static boolean hasValue(InternalTDigestPercentiles agg) { } public static boolean hasValue(InternalTopHits agg) { - return (agg.getHits().getTotalHits().value == 0 + return (agg.getHits().getTotalHits().value() == 0 && Double.isNaN(agg.getHits().getMaxScore()) && Double.isNaN(agg.getTopDocs().maxScore)) == false; } diff --git a/server/src/main/java/org/opensearch/search/aggregations/metrics/TopHitsAggregator.java b/server/src/main/java/org/opensearch/search/aggregations/metrics/TopHitsAggregator.java index 2d78360f20bc6..a6caf7377c27d 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/metrics/TopHitsAggregator.java +++ b/server/src/main/java/org/opensearch/search/aggregations/metrics/TopHitsAggregator.java @@ -43,8 +43,9 @@ import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopDocsCollector; import org.apache.lucene.search.TopFieldCollector; +import org.apache.lucene.search.TopFieldCollectorManager; import org.apache.lucene.search.TopFieldDocs; -import org.apache.lucene.search.TopScoreDocCollector; +import org.apache.lucene.search.TopScoreDocCollectorManager; import org.apache.lucene.search.TotalHits; import org.opensearch.OpenSearchException; import org.opensearch.action.search.MaxScoreCollector; @@ -157,12 +158,15 @@ public void collect(int docId, long bucket) throws IOException { // but here we create collectors ourselves and we need prevent OOM because of crazy an offset and size. topN = Math.min(topN, subSearchContext.searcher().getIndexReader().maxDoc()); if (sort == null) { - collectors = new Collectors(TopScoreDocCollector.create(topN, Integer.MAX_VALUE), null); + collectors = new Collectors( + new TopScoreDocCollectorManager(topN, null, Integer.MAX_VALUE, false).newCollector(), + null + ); } else { // TODO: can we pass trackTotalHits=subSearchContext.trackTotalHits(){ // Note that this would require to catch CollectionTerminatedException collectors = new Collectors( - TopFieldCollector.create(sort.sort, topN, Integer.MAX_VALUE), + new TopFieldCollectorManager(sort.sort, topN, null, Integer.MAX_VALUE, false).newCollector(), subSearchContext.trackScores() ? new MaxScoreCollector() : null ); } diff --git a/server/src/main/java/org/opensearch/search/aggregations/support/MissingValues.java b/server/src/main/java/org/opensearch/search/aggregations/support/MissingValues.java index d21737a8366b2..a5c685a0930e2 100644 --- a/server/src/main/java/org/opensearch/search/aggregations/support/MissingValues.java +++ b/server/src/main/java/org/opensearch/search/aggregations/support/MissingValues.java @@ -305,10 +305,10 @@ public long nextOrd() throws IOException { return values.nextOrd(); } else { // we want to return the next missing ord but set this to - // NO_MORE_ORDS so on the next call we indicate there are no + // NO_MORE_DOCS so on the next call we indicate there are no // more values long ordToReturn = nextMissingOrd; - nextMissingOrd = SortedSetDocValues.NO_MORE_ORDS; + nextMissingOrd = SortedSetDocValues.NO_MORE_DOCS; return ordToReturn; } } @@ -368,15 +368,17 @@ public long nextOrd() throws IOException { final long ord = values.nextOrd(); if (ord < insertedOrd) { return ord; + } else if (ord == SortedSetDocValues.NO_MORE_DOCS /* no more docs */) { + return SortedSetDocValues.NO_MORE_DOCS; } else { return ord + 1; } } else { // we want to return the next missing ord but set this to - // NO_MORE_ORDS so on the next call we indicate there are no + // NO_MORE_DOCS so on the next call we indicate there are no // more values long ordToReturn = nextMissingOrd; - nextMissingOrd = SortedSetDocValues.NO_MORE_ORDS; + nextMissingOrd = SortedSetDocValues.NO_MORE_DOCS; return ordToReturn; } } diff --git a/server/src/main/java/org/opensearch/search/approximate/ApproximatePointRangeQuery.java b/server/src/main/java/org/opensearch/search/approximate/ApproximatePointRangeQuery.java index 6ff01f5f39d36..fe3d66c674a82 100644 --- a/server/src/main/java/org/opensearch/search/approximate/ApproximatePointRangeQuery.java +++ b/server/src/main/java/org/opensearch/search/approximate/ApproximatePointRangeQuery.java @@ -359,7 +359,7 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti public Scorer get(long leadCost) throws IOException { intersectLeft(values.getPointTree(), visitor, docCount); DocIdSetIterator iterator = result.build().iterator(); - return new ConstantScoreScorer(weight, score(), scoreMode, iterator); + return new ConstantScoreScorer(score(), scoreMode, iterator); } @Override @@ -387,7 +387,7 @@ public long cost() { public Scorer get(long leadCost) throws IOException { intersectRight(values.getPointTree(), visitor, docCount); DocIdSetIterator iterator = result.build().iterator(); - return new ConstantScoreScorer(weight, score(), scoreMode, iterator); + return new ConstantScoreScorer(score(), scoreMode, iterator); } @Override @@ -404,15 +404,6 @@ public long cost() { } } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - ScorerSupplier scorerSupplier = scorerSupplier(context); - if (scorerSupplier == null) { - return null; - } - return scorerSupplier.get(Long.MAX_VALUE); - } - @Override public int count(LeafReaderContext context) throws IOException { return pointRangeQueryWeight.count(context); diff --git a/server/src/main/java/org/opensearch/search/fetch/subphase/highlight/FragmentBuilderHelper.java b/server/src/main/java/org/opensearch/search/fetch/subphase/highlight/FragmentBuilderHelper.java index e037ae5544a07..2e94a7ba0b3f6 100644 --- a/server/src/main/java/org/opensearch/search/fetch/subphase/highlight/FragmentBuilderHelper.java +++ b/server/src/main/java/org/opensearch/search/fetch/subphase/highlight/FragmentBuilderHelper.java @@ -75,13 +75,13 @@ public static WeightedFragInfo fixWeightedFragInfo(MappedFieldType fieldType, Fi CollectionUtil.introSort(subInfos, new Comparator() { @Override public int compare(SubInfo o1, SubInfo o2) { - int startOffset = o1.getTermsOffsets().get(0).getStartOffset(); - int startOffset2 = o2.getTermsOffsets().get(0).getStartOffset(); + int startOffset = o1.termsOffsets().get(0).getStartOffset(); + int startOffset2 = o2.termsOffsets().get(0).getStartOffset(); return FragmentBuilderHelper.compare(startOffset, startOffset2); } }); return new WeightedFragInfo( - Math.min(fragInfo.getSubInfos().get(0).getTermsOffsets().get(0).getStartOffset(), fragInfo.getStartOffset()), + Math.min(fragInfo.getSubInfos().get(0).termsOffsets().get(0).getStartOffset(), fragInfo.getStartOffset()), fragInfo.getEndOffset(), subInfos, fragInfo.getTotalBoost() diff --git a/server/src/main/java/org/opensearch/search/internal/ContextIndexSearcher.java b/server/src/main/java/org/opensearch/search/internal/ContextIndexSearcher.java index aa8212e8dad69..dacccbed79f72 100644 --- a/server/src/main/java/org/opensearch/search/internal/ContextIndexSearcher.java +++ b/server/src/main/java/org/opensearch/search/internal/ContextIndexSearcher.java @@ -54,6 +54,7 @@ import org.apache.lucene.search.ScoreDoc; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TermStatistics; import org.apache.lucene.search.TopFieldDocs; import org.apache.lucene.search.TotalHits; @@ -238,9 +239,10 @@ public void search( final List collectors = new ArrayList<>(leaves.size()); for (LeafReaderContext ctx : leaves) { final Collector collector = manager.newCollector(); - searchLeaf(ctx, weight, collector); + searchLeaf(ctx, 0, DocIdSetIterator.NO_MORE_DOCS, weight, collector); collectors.add(collector); } + TopFieldDocs mergedTopDocs = (TopFieldDocs) manager.reduce(collectors); // Lucene sets shards indexes during merging of topDocs from different collectors // We need to reset shard index; OpenSearch will set shard index later during reduce stage @@ -253,6 +255,12 @@ public void search( result.topDocs(new TopDocsAndMaxScore(mergedTopDocs, Float.NaN), formats); } + @Override + public void search(Query query, Collector collector) throws IOException { + super.search(query, collector); + searchContext.bucketCollectorProcessor().processPostCollection(collector); + } + public void search( Query query, CollectorManager manager, @@ -273,7 +281,7 @@ public void search( } @Override - protected void search(List leaves, Weight weight, Collector collector) throws IOException { + protected void search(LeafReaderContextPartition[] partitions, Weight weight, Collector collector) throws IOException { searchContext.indexShard().getSearchOperationListener().onPreSliceExecution(searchContext); try { // Time series based workload by default traverses segments in desc order i.e. latest to the oldest order. @@ -281,12 +289,12 @@ protected void search(List leaves, Weight weight, Collector c // That can slow down ASC order queries on timestamp workload. So to avoid that slowdown, we will reverse leaf // reader order here. if (searchContext.shouldUseTimeSeriesDescSortOptimization()) { - for (int i = leaves.size() - 1; i >= 0; i--) { - searchLeaf(leaves.get(i), weight, collector); + for (int i = partitions.length - 1; i >= 0; i--) { + searchLeaf(partitions[i].ctx, partitions[i].minDocId, partitions[i].maxDocId, weight, collector); } } else { - for (int i = 0; i < leaves.size(); i++) { - searchLeaf(leaves.get(i), weight, collector); + for (LeafReaderContextPartition partition : partitions) { + searchLeaf(partition.ctx, partition.minDocId, partition.maxDocId, weight, collector); } } searchContext.bucketCollectorProcessor().processPostCollection(collector); @@ -304,7 +312,7 @@ protected void search(List leaves, Weight weight, Collector c * the provided ctx. */ @Override - protected void searchLeaf(LeafReaderContext ctx, Weight weight, Collector collector) throws IOException { + protected void searchLeaf(LeafReaderContext ctx, int minDocId, int maxDocId, Weight weight, Collector collector) throws IOException { // Check if at all we need to call this leaf for collecting results. if (canMatch(ctx) == false) { @@ -336,7 +344,7 @@ protected void searchLeaf(LeafReaderContext ctx, Weight weight, Collector collec BulkScorer bulkScorer = weight.bulkScorer(ctx); if (bulkScorer != null) { try { - bulkScorer.score(leafCollector, liveDocs); + bulkScorer.score(leafCollector, liveDocs, minDocId, maxDocId); } catch (CollectionTerminatedException e) { // collection was terminated prematurely // continue with the following leaf @@ -354,6 +362,8 @@ protected void searchLeaf(LeafReaderContext ctx, Weight weight, Collector collec scorer, liveDocsBitSet, leafCollector, + minDocId, + maxDocId, this.cancellable.isEnabled() ? cancellable::checkCancelled : () -> {} ); } catch (CollectionTerminatedException e) { @@ -386,18 +396,43 @@ public boolean isCacheable(LeafReaderContext ctx) { } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - return weight.scorer(context); - } - - @Override - public BulkScorer bulkScorer(LeafReaderContext context) throws IOException { - BulkScorer in = weight.bulkScorer(context); - if (in != null) { - return new CancellableBulkScorer(in, cancellable::checkCancelled); - } else { - return null; - } + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { + return new ScorerSupplier() { + private Scorer scorer; + private BulkScorer bulkScorer; + + @Override + public Scorer get(long leadCost) throws IOException { + scorer = weight.scorer(context); + return scorer; + } + + @Override + public BulkScorer bulkScorer() throws IOException { + final BulkScorer in = weight.bulkScorer(context); + + if (in != null) { + bulkScorer = new CancellableBulkScorer(in, cancellable::checkCancelled); + } else { + bulkScorer = null; + } + + return bulkScorer; + } + + @Override + public long cost() { + if (scorer != null) { + return scorer.iterator().cost(); + } else if (bulkScorer != null) { + return bulkScorer.cost(); + } else { + // We have no prior knowledge of how many docs might match for any given query term, + // so we assume that all docs could be a match. + return Integer.MAX_VALUE; + } + } + }; } @Override @@ -425,6 +460,17 @@ private static BitSet getSparseBitSetOrNull(Bits liveDocs) { static void intersectScorerAndBitSet(Scorer scorer, BitSet acceptDocs, LeafCollector collector, Runnable checkCancelled) throws IOException { + intersectScorerAndBitSet(scorer, acceptDocs, collector, 0, DocIdSetIterator.NO_MORE_DOCS, checkCancelled); + } + + private static void intersectScorerAndBitSet( + Scorer scorer, + BitSet acceptDocs, + LeafCollector collector, + int minDocId, + int maxDocId, + Runnable checkCancelled + ) throws IOException { collector.setScorer(scorer); // ConjunctionDISI uses the DocIdSetIterator#cost() to order the iterators, so if roleBits has the lowest cardinality it should // be used first: @@ -433,7 +479,8 @@ static void intersectScorerAndBitSet(Scorer scorer, BitSet acceptDocs, LeafColle ); int seen = 0; checkCancelled.run(); - for (int docId = iterator.nextDoc(); docId < DocIdSetIterator.NO_MORE_DOCS; docId = iterator.nextDoc()) { + iterator.advance(minDocId); + for (int docId = iterator.docID(); docId < maxDocId; docId = iterator.nextDoc()) { if (++seen % CHECK_CANCELLED_SCORER_INTERVAL == 0) { checkCancelled.run(); } diff --git a/server/src/main/java/org/opensearch/search/internal/MaxTargetSliceSupplier.java b/server/src/main/java/org/opensearch/search/internal/MaxTargetSliceSupplier.java index 64984585f3ab6..e2c61615601a8 100644 --- a/server/src/main/java/org/opensearch/search/internal/MaxTargetSliceSupplier.java +++ b/server/src/main/java/org/opensearch/search/internal/MaxTargetSliceSupplier.java @@ -40,14 +40,14 @@ static IndexSearcher.LeafSlice[] getSlices(List leaves, int t // Sort by maxDoc, descending: sortedLeaves.sort(Collections.reverseOrder(Comparator.comparingInt(l -> l.reader().maxDoc()))); - final List> groupedLeaves = new ArrayList<>(targetSliceCount); + final List> groupedLeaves = new ArrayList<>(targetSliceCount); for (int i = 0; i < targetSliceCount; ++i) { groupedLeaves.add(new ArrayList<>()); } // distribute the slices in round-robin fashion for (int idx = 0; idx < sortedLeaves.size(); ++idx) { int currentGroup = idx % targetSliceCount; - groupedLeaves.get(currentGroup).add(sortedLeaves.get(idx)); + groupedLeaves.get(currentGroup).add(IndexSearcher.LeafReaderContextPartition.createForEntireSegment(sortedLeaves.get(idx))); } return groupedLeaves.stream().map(IndexSearcher.LeafSlice::new).toArray(IndexSearcher.LeafSlice[]::new); diff --git a/server/src/main/java/org/opensearch/search/profile/query/ProfileScorer.java b/server/src/main/java/org/opensearch/search/profile/query/ProfileScorer.java index 319281449195b..28b693ee03ad5 100644 --- a/server/src/main/java/org/opensearch/search/profile/query/ProfileScorer.java +++ b/server/src/main/java/org/opensearch/search/profile/query/ProfileScorer.java @@ -35,7 +35,6 @@ import org.apache.lucene.search.DocIdSetIterator; import org.apache.lucene.search.Scorer; import org.apache.lucene.search.TwoPhaseIterator; -import org.apache.lucene.search.Weight; import org.opensearch.search.profile.AbstractProfileBreakdown; import org.opensearch.search.profile.Timer; @@ -51,15 +50,12 @@ final class ProfileScorer extends Scorer { private final Scorer scorer; - private ProfileWeight profileWeight; private final Timer scoreTimer, nextDocTimer, advanceTimer, matchTimer, shallowAdvanceTimer, computeMaxScoreTimer, setMinCompetitiveScoreTimer; - ProfileScorer(ProfileWeight w, Scorer scorer, AbstractProfileBreakdown profile) throws IOException { - super(w); + ProfileScorer(Scorer scorer, AbstractProfileBreakdown profile) throws IOException { this.scorer = scorer; - this.profileWeight = w; scoreTimer = profile.getTimer(QueryTimingType.SCORE); nextDocTimer = profile.getTimer(QueryTimingType.NEXT_DOC); advanceTimer = profile.getTimer(QueryTimingType.ADVANCE); @@ -84,11 +80,6 @@ public float score() throws IOException { } } - @Override - public Weight getWeight() { - return profileWeight; - } - @Override public Collection getChildren() throws IOException { return scorer.getChildren(); diff --git a/server/src/main/java/org/opensearch/search/profile/query/ProfileWeight.java b/server/src/main/java/org/opensearch/search/profile/query/ProfileWeight.java index c7e70d8d88007..f190a9734c1a5 100644 --- a/server/src/main/java/org/opensearch/search/profile/query/ProfileWeight.java +++ b/server/src/main/java/org/opensearch/search/profile/query/ProfileWeight.java @@ -33,7 +33,6 @@ package org.opensearch.search.profile.query; import org.apache.lucene.index.LeafReaderContext; -import org.apache.lucene.search.BulkScorer; import org.apache.lucene.search.Collector; import org.apache.lucene.search.Explanation; import org.apache.lucene.search.Query; @@ -63,15 +62,6 @@ public ProfileWeight(Query query, Weight subQueryWeight, ContextualProfileBreakd this.profile = profile; } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - ScorerSupplier supplier = scorerSupplier(context); - if (supplier == null) { - return null; - } - return supplier.get(Long.MAX_VALUE); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { Timer timer = profile.context(context).getTimer(QueryTimingType.BUILD_SCORER); @@ -86,14 +76,13 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti return null; } - final ProfileWeight weight = this; return new ScorerSupplier() { @Override public Scorer get(long loadCost) throws IOException { timer.start(); try { - return new ProfileScorer(weight, subQueryScorerSupplier.get(loadCost), profile.context(context)); + return new ProfileScorer(subQueryScorerSupplier.get(loadCost), profile.context(context)); } finally { timer.stop(); } @@ -111,18 +100,6 @@ public long cost() { }; } - @Override - public BulkScorer bulkScorer(LeafReaderContext context) throws IOException { - // We use the default bulk scorer instead of the specialized one. The reason - // is that Lucene's BulkScorers do everything at once: finding matches, - // scoring them and calling the collector, so they make it impossible to - // see where time is spent, which is the purpose of query profiling. - // The default bulk scorer will pull a scorer and iterate over matches, - // this might be a significantly different execution path for some queries - // like disjunctions, but in general this is what is done anyway - return super.bulkScorer(context); - } - @Override public Explanation explain(LeafReaderContext context, int doc) throws IOException { return subQueryWeight.explain(context, doc); diff --git a/server/src/main/java/org/opensearch/search/query/BitmapDocValuesQuery.java b/server/src/main/java/org/opensearch/search/query/BitmapDocValuesQuery.java index 9fce29ae7fbfb..307dc63fd0ac8 100644 --- a/server/src/main/java/org/opensearch/search/query/BitmapDocValuesQuery.java +++ b/server/src/main/java/org/opensearch/search/query/BitmapDocValuesQuery.java @@ -20,6 +20,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.util.Accountable; @@ -61,7 +62,7 @@ public BitmapDocValuesQuery(String field, RoaringBitmap bitmap) { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { SortedNumericDocValues values = DocValues.getSortedNumeric(context.reader(), field); final NumericDocValues singleton = DocValues.unwrapSingleton(values); final TwoPhaseIterator iterator; @@ -102,7 +103,8 @@ public float matchCost() { } }; } - return new ConstantScoreScorer(this, score(), scoreMode, iterator); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, iterator); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/search/query/BitmapIndexQuery.java b/server/src/main/java/org/opensearch/search/query/BitmapIndexQuery.java index 87a7c132be848..2451c3d8d7d4a 100644 --- a/server/src/main/java/org/opensearch/search/query/BitmapIndexQuery.java +++ b/server/src/main/java/org/opensearch/search/query/BitmapIndexQuery.java @@ -97,18 +97,8 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo // get cardinality is not cheap enough to do when supplying scorers, so do it once per weight final long cardinality = bitmap.getLongCardinality(); - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - ScorerSupplier scorerSupplier = scorerSupplier(context); - if (scorerSupplier == null) { - return null; - } - return scorerSupplier.get(Long.MAX_VALUE); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { - final Weight weight = this; LeafReader reader = context.reader(); // get the point value which should be one dimension, since bitmap saves integers PointValues values = reader.getPointValues(field); @@ -128,7 +118,7 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti @Override public Scorer get(long leadCost) throws IOException { values.intersect(visitor); - return new ConstantScoreScorer(weight, score(), scoreMode, result.build().iterator()); + return new ConstantScoreScorer(score(), scoreMode, result.build().iterator()); } @Override diff --git a/server/src/main/java/org/opensearch/search/query/QueryPhase.java b/server/src/main/java/org/opensearch/search/query/QueryPhase.java index 55b7c0bc5178d..58be02cc413dd 100644 --- a/server/src/main/java/org/opensearch/search/query/QueryPhase.java +++ b/server/src/main/java/org/opensearch/search/query/QueryPhase.java @@ -388,7 +388,7 @@ private static boolean canEarlyTerminate(IndexReader reader, SortAndFormats sort } final Sort sort = sortAndFormats.sort; for (LeafReaderContext ctx : reader.leaves()) { - Sort indexSort = ctx.reader().getMetaData().getSort(); + Sort indexSort = ctx.reader().getMetaData().sort(); if (indexSort == null || Lucene.canEarlyTerminate(sort, indexSort) == false) { return false; } diff --git a/server/src/main/java/org/opensearch/search/query/TopDocsCollectorContext.java b/server/src/main/java/org/opensearch/search/query/TopDocsCollectorContext.java index f780f6fe32af2..dab7f30db6a7b 100644 --- a/server/src/main/java/org/opensearch/search/query/TopDocsCollectorContext.java +++ b/server/src/main/java/org/opensearch/search/query/TopDocsCollectorContext.java @@ -46,8 +46,8 @@ import org.apache.lucene.search.Collector; import org.apache.lucene.search.CollectorManager; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; import org.apache.lucene.search.FieldDoc; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MultiCollector; import org.apache.lucene.search.Query; @@ -58,9 +58,9 @@ import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.TopDocsCollector; -import org.apache.lucene.search.TopFieldCollector; +import org.apache.lucene.search.TopFieldCollectorManager; import org.apache.lucene.search.TopFieldDocs; -import org.apache.lucene.search.TopScoreDocCollector; +import org.apache.lucene.search.TopScoreDocCollectorManager; import org.apache.lucene.search.TotalHitCountCollector; import org.apache.lucene.search.TotalHits; import org.apache.lucene.search.grouping.CollapseTopFieldDocs; @@ -347,9 +347,10 @@ private static TopDocsCollector createCollector( int hitCountThreshold ) { if (sortAndFormats == null) { - return TopScoreDocCollector.create(numHits, searchAfter, hitCountThreshold); + return new TopScoreDocCollectorManager(numHits, searchAfter, hitCountThreshold, false).newCollector(); } else { - return TopFieldCollector.create(sortAndFormats.sort, numHits, (FieldDoc) searchAfter, hitCountThreshold); + return new TopFieldCollectorManager(sortAndFormats.sort, numHits, (FieldDoc) searchAfter, hitCountThreshold, false) + .newCollector(); } } @@ -362,16 +363,17 @@ private static TopDocsCollector createCollector( if (sortAndFormats == null) { // See please https://github.com/apache/lucene/pull/450, should be fixed in 9.x if (searchAfter != null) { - return TopScoreDocCollector.createSharedManager( + return new TopScoreDocCollectorManager( numHits, new FieldDoc(searchAfter.doc, searchAfter.score), - hitCountThreshold + hitCountThreshold, + true ); } else { - return TopScoreDocCollector.createSharedManager(numHits, null, hitCountThreshold); + return new TopScoreDocCollectorManager(numHits, null, hitCountThreshold, true); } } else { - return TopFieldCollector.createSharedManager(sortAndFormats.sort, numHits, (FieldDoc) searchAfter, hitCountThreshold); + return new TopFieldCollectorManager(sortAndFormats.sort, numHits, (FieldDoc) searchAfter, hitCountThreshold, true); } } @@ -581,7 +583,7 @@ TopDocsAndMaxScore newTopDocs(final TopDocs topDocs, final float maxScore, final // artificially reducing the number of total hits and doc scores. ScoreDoc[] scoreDocs = topDocs.scoreDocs; if (terminatedAfter != null) { - if (totalHits.value > terminatedAfter) { + if (totalHits.value() > terminatedAfter) { totalHits = new TotalHits(terminatedAfter, TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO); } @@ -735,8 +737,8 @@ static int shortcutTotalHitCount(IndexReader reader, Query query) throws IOExcep count += context.reader().docFreq(term); } return count; - } else if (query.getClass() == DocValuesFieldExistsQuery.class && reader.hasDeletions() == false) { - final String field = ((DocValuesFieldExistsQuery) query).getField(); + } else if (query.getClass() == FieldExistsQuery.class && reader.hasDeletions() == false) { + final String field = ((FieldExistsQuery) query).getField(); int count = 0; for (LeafReaderContext context : reader.leaves()) { FieldInfos fieldInfos = context.reader().getFieldInfos(); diff --git a/server/src/main/java/org/opensearch/search/slice/DocValuesSliceQuery.java b/server/src/main/java/org/opensearch/search/slice/DocValuesSliceQuery.java index 856e103193463..33c300c69abaf 100644 --- a/server/src/main/java/org/opensearch/search/slice/DocValuesSliceQuery.java +++ b/server/src/main/java/org/opensearch/search/slice/DocValuesSliceQuery.java @@ -41,6 +41,7 @@ import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.opensearch.common.util.BitMixer; @@ -65,7 +66,7 @@ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float bo return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { final SortedNumericDocValues values = DocValues.getSortedNumeric(context.reader(), getField()); final DocIdSetIterator approximation = DocIdSetIterator.all(context.reader().maxDoc()); final TwoPhaseIterator twoPhase = new TwoPhaseIterator(approximation) { @@ -90,7 +91,8 @@ public float matchCost() { return 10; } }; - return new ConstantScoreScorer(this, score(), scoreMode, twoPhase); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, twoPhase); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/search/slice/TermsSliceQuery.java b/server/src/main/java/org/opensearch/search/slice/TermsSliceQuery.java index 05f36b0d6f3cf..4515b470895bc 100644 --- a/server/src/main/java/org/opensearch/search/slice/TermsSliceQuery.java +++ b/server/src/main/java/org/opensearch/search/slice/TermsSliceQuery.java @@ -44,6 +44,7 @@ import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Weight; import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.DocIdSetBuilder; @@ -74,10 +75,11 @@ public TermsSliceQuery(String field, int id, int max) { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { final DocIdSet disi = build(context.reader()); final DocIdSetIterator leafIt = disi.iterator(); - return new ConstantScoreScorer(this, score(), scoreMode, leafIt); + final Scorer scorer = new ConstantScoreScorer(score(), scoreMode, leafIt); + return new DefaultScorerSupplier(scorer); } @Override diff --git a/server/src/main/java/org/opensearch/search/sort/ScoreSortBuilder.java b/server/src/main/java/org/opensearch/search/sort/ScoreSortBuilder.java index 1be49e5ca81ce..3d69fe97a3035 100644 --- a/server/src/main/java/org/opensearch/search/sort/ScoreSortBuilder.java +++ b/server/src/main/java/org/opensearch/search/sort/ScoreSortBuilder.java @@ -144,7 +144,6 @@ public void setScorer(Scorable scorer) { @Override protected boolean advanceExact(int doc) throws IOException { - assert doc == scorer.docID() : "expected scorer to be on [" + doc + "] but was on [" + scorer.docID() + "]"; /* We will never be called by documents that don't match the * query and they'll all have a score, thus `true`. */ score = scorer.score(); diff --git a/server/src/main/java/org/opensearch/search/sort/SortedWiderNumericSortField.java b/server/src/main/java/org/opensearch/search/sort/SortedWiderNumericSortField.java index 7f61b7cca3501..0b12e22224cd8 100644 --- a/server/src/main/java/org/opensearch/search/sort/SortedWiderNumericSortField.java +++ b/server/src/main/java/org/opensearch/search/sort/SortedWiderNumericSortField.java @@ -19,6 +19,7 @@ import org.apache.lucene.search.Pruning; import org.apache.lucene.search.SortedNumericSortField; import org.apache.lucene.search.comparators.NumericComparator; +import org.apache.lucene.util.NumericUtils; import java.io.IOException; import java.util.Comparator; @@ -94,6 +95,21 @@ public int compareValues(Number first, Number second) { return comparator.compare(first, second); } } + + @Override + protected long missingValueAsComparableLong() { + return switch (missingValue) { + case Double d -> NumericUtils.doubleToSortableLong(d); + case Float f -> NumericUtils.floatToSortableInt(f); + case Number n -> n.longValue(); + case null -> 0L; + }; + } + + @Override + protected long sortableBytesToLong(byte[] bytes) { + throw new UnsupportedOperationException(); + } }; } diff --git a/server/src/main/java/org/opensearch/search/suggest/completion/CompletionSuggester.java b/server/src/main/java/org/opensearch/search/suggest/completion/CompletionSuggester.java index fabb67d7fb841..bffd72a29465a 100644 --- a/server/src/main/java/org/opensearch/search/suggest/completion/CompletionSuggester.java +++ b/server/src/main/java/org/opensearch/search/suggest/completion/CompletionSuggester.java @@ -34,6 +34,7 @@ import org.apache.lucene.index.LeafReaderContext; import org.apache.lucene.search.BulkScorer; import org.apache.lucene.search.CollectionTerminatedException; +import org.apache.lucene.search.DocIdSetIterator; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.LeafCollector; import org.apache.lucene.search.Weight; @@ -112,7 +113,7 @@ private static void suggest(IndexSearcher searcher, CompletionQuery query, TopSu LeafCollector leafCollector = null; try { leafCollector = collector.getLeafCollector(context); - scorer.score(leafCollector, context.reader().getLiveDocs()); + scorer.score(leafCollector, context.reader().getLiveDocs(), 0, DocIdSetIterator.NO_MORE_DOCS); } catch (CollectionTerminatedException e) { // collection was terminated prematurely // continue with the following leaf diff --git a/server/src/main/java/org/opensearch/search/suggest/phrase/DirectCandidateGenerator.java b/server/src/main/java/org/opensearch/search/suggest/phrase/DirectCandidateGenerator.java index 1a00cb9465771..4fa6e262e89ad 100644 --- a/server/src/main/java/org/opensearch/search/suggest/phrase/DirectCandidateGenerator.java +++ b/server/src/main/java/org/opensearch/search/suggest/phrase/DirectCandidateGenerator.java @@ -131,7 +131,7 @@ public DirectCandidateGenerator( */ @Override public boolean isKnownWord(BytesRef term) throws IOException { - return termStats(term).docFreq > 0; + return termStats(term).docFreq() > 0; } /* (non-Javadoc) @@ -173,7 +173,7 @@ public CandidateSet drawCandidates(CandidateSet set) throws IOException { because that's what {@link DirectSpellChecker#suggestSimilar} expects when filtering terms. */ - int threshold = thresholdTermFrequency(original.termStats.docFreq); + int threshold = thresholdTermFrequency(original.termStats.docFreq()); if (threshold == Integer.MAX_VALUE) { // the threshold is the max possible frequency so we can skip the search return set; @@ -267,7 +267,7 @@ public void nextToken() throws IOException { } private double score(TermStats termStats, double errorScore, long dictionarySize) { - return errorScore * (((double) termStats.totalTermFreq + 1) / ((double) dictionarySize + 1)); + return errorScore * (((double) termStats.totalTermFreq() + 1) / ((double) dictionarySize + 1)); } // package protected for test diff --git a/server/src/main/java/org/opensearch/search/suggest/phrase/LaplaceScorer.java b/server/src/main/java/org/opensearch/search/suggest/phrase/LaplaceScorer.java index 7ac1d14576ea1..2c307e9464705 100644 --- a/server/src/main/java/org/opensearch/search/suggest/phrase/LaplaceScorer.java +++ b/server/src/main/java/org/opensearch/search/suggest/phrase/LaplaceScorer.java @@ -64,7 +64,7 @@ protected double scoreUnigram(Candidate word) throws IOException { @Override protected double scoreBigram(Candidate word, Candidate w_1) throws IOException { join(separator, spare, w_1.term, word.term); - return (alpha + frequency(spare.get())) / (w_1.termStats.totalTermFreq + alpha * numTerms); + return (alpha + frequency(spare.get())) / (w_1.termStats.totalTermFreq() + alpha * numTerms); } @Override diff --git a/server/src/main/java/org/opensearch/search/suggest/phrase/LinearInterpolatingScorer.java b/server/src/main/java/org/opensearch/search/suggest/phrase/LinearInterpolatingScorer.java index e012dde78c59e..6da733e97efdd 100644 --- a/server/src/main/java/org/opensearch/search/suggest/phrase/LinearInterpolatingScorer.java +++ b/server/src/main/java/org/opensearch/search/suggest/phrase/LinearInterpolatingScorer.java @@ -86,7 +86,7 @@ protected double scoreBigram(Candidate word, Candidate w_1) throws IOException { if (count < 1) { return unigramLambda * scoreUnigram(word); } - return bigramLambda * (count / (0.5d + w_1.termStats.totalTermFreq)) + unigramLambda * scoreUnigram(word); + return bigramLambda * (count / (0.5d + w_1.termStats.totalTermFreq())) + unigramLambda * scoreUnigram(word); } @Override diff --git a/server/src/main/java/org/opensearch/search/suggest/phrase/NoisyChannelSpellChecker.java b/server/src/main/java/org/opensearch/search/suggest/phrase/NoisyChannelSpellChecker.java index e8ba90f353f02..c4b531229ae7b 100644 --- a/server/src/main/java/org/opensearch/search/suggest/phrase/NoisyChannelSpellChecker.java +++ b/server/src/main/java/org/opensearch/search/suggest/phrase/NoisyChannelSpellChecker.java @@ -98,7 +98,7 @@ public void nextToken() throws IOException { if (posIncAttr.getPositionIncrement() == 0 && typeAttribute.type() == SynonymFilter.TYPE_SYNONYM) { assert currentSet != null; TermStats termStats = generator.termStats(term); - if (termStats.docFreq > 0) { + if (termStats.docFreq() > 0) { currentSet.addOneCandidate(generator.createCandidate(BytesRef.deepCopyOf(term), termStats, realWordLikelihood)); } } else { diff --git a/server/src/main/java/org/opensearch/search/suggest/phrase/StupidBackoffScorer.java b/server/src/main/java/org/opensearch/search/suggest/phrase/StupidBackoffScorer.java index 35de07015b853..4b66eb0f10301 100644 --- a/server/src/main/java/org/opensearch/search/suggest/phrase/StupidBackoffScorer.java +++ b/server/src/main/java/org/opensearch/search/suggest/phrase/StupidBackoffScorer.java @@ -63,7 +63,7 @@ protected double scoreBigram(Candidate word, Candidate w_1) throws IOException { if (count < 1) { return discount * scoreUnigram(word); } - return count / (w_1.termStats.totalTermFreq + 0.00000000001d); + return count / (w_1.termStats.totalTermFreq() + 0.00000000001d); } @Override @@ -78,7 +78,7 @@ protected double scoreTrigram(Candidate w, Candidate w_1, Candidate w_2) throws join(separator, spare, w_2.term, w_1.term, w.term); long trigramCount = frequency(spare.get()); if (trigramCount < 1) { - return discount * (bigramCount / (w_1.termStats.totalTermFreq + 0.00000000001d)); + return discount * (bigramCount / (w_1.termStats.totalTermFreq() + 0.00000000001d)); } return trigramCount / (bigramCount + 0.00000000001d); } diff --git a/server/src/main/java/org/opensearch/telemetry/tracing/listener/TraceableSearchRequestOperationsListener.java b/server/src/main/java/org/opensearch/telemetry/tracing/listener/TraceableSearchRequestOperationsListener.java index 71fb59194c447..e74f172c85189 100644 --- a/server/src/main/java/org/opensearch/telemetry/tracing/listener/TraceableSearchRequestOperationsListener.java +++ b/server/src/main/java/org/opensearch/telemetry/tracing/listener/TraceableSearchRequestOperationsListener.java @@ -69,7 +69,7 @@ public void onRequestEnd(SearchPhaseContext context, SearchRequestContext search // add response-related attributes on request end requestSpan.addAttribute( AttributeNames.TOTAL_HITS, - searchRequestContext.totalHits() == null ? 0 : searchRequestContext.totalHits().value + searchRequestContext.totalHits() == null ? 0 : searchRequestContext.totalHits().value() ); } } diff --git a/server/src/main/resources/META-INF/services/org.apache.lucene.codecs.Codec b/server/src/main/resources/META-INF/services/org.apache.lucene.codecs.Codec index 16742fd33ed43..b144b13d09753 100644 --- a/server/src/main/resources/META-INF/services/org.apache.lucene.codecs.Codec +++ b/server/src/main/resources/META-INF/services/org.apache.lucene.codecs.Codec @@ -1 +1,2 @@ org.opensearch.index.codec.composite.composite912.Composite912Codec +org.opensearch.index.codec.composite.composite101.Composite101Codec diff --git a/server/src/test/java/org/opensearch/VersionTests.java b/server/src/test/java/org/opensearch/VersionTests.java index 56fac9619e213..4577dcc6a66f0 100644 --- a/server/src/test/java/org/opensearch/VersionTests.java +++ b/server/src/test/java/org/opensearch/VersionTests.java @@ -290,9 +290,9 @@ public void testIsBeta() { } public void testIsAlpha() { - assertTrue(new Version(1000001, org.apache.lucene.util.Version.LUCENE_8_8_2).isAlpha()); - assertFalse(new Version(1000026, org.apache.lucene.util.Version.LUCENE_8_8_2).isAlpha()); - assertTrue(new Version(1000026, org.apache.lucene.util.Version.LUCENE_8_8_2).isBeta()); + assertTrue(new Version(1000001, org.apache.lucene.util.Version.LUCENE_9_12_0).isAlpha()); + assertFalse(new Version(1000026, org.apache.lucene.util.Version.LUCENE_9_12_0).isAlpha()); + assertTrue(new Version(1000026, org.apache.lucene.util.Version.LUCENE_9_12_0).isBeta()); assertTrue(Version.fromString("2.0.0-alpha14").isAlpha()); assertEquals(2000014 ^ MASK, Version.fromString("2.0.0-alpha14").id); assertTrue(Version.fromId(5000015 ^ MASK).isAlpha()); diff --git a/server/src/test/java/org/opensearch/action/search/DfsQueryPhaseTests.java b/server/src/test/java/org/opensearch/action/search/DfsQueryPhaseTests.java index 6952841c295e2..2dd2d9f9576a9 100644 --- a/server/src/test/java/org/opensearch/action/search/DfsQueryPhaseTests.java +++ b/server/src/test/java/org/opensearch/action/search/DfsQueryPhaseTests.java @@ -155,11 +155,11 @@ public void run() throws IOException { assertNotNull(responseRef.get()); assertNotNull(responseRef.get().get(0)); assertNull(responseRef.get().get(0).fetchResult()); - assertEquals(1, responseRef.get().get(0).queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, responseRef.get().get(0).queryResult().topDocs().topDocs.totalHits.value()); assertEquals(42, responseRef.get().get(0).queryResult().topDocs().topDocs.scoreDocs[0].doc); assertNotNull(responseRef.get().get(1)); assertNull(responseRef.get().get(1).fetchResult()); - assertEquals(1, responseRef.get().get(1).queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, responseRef.get().get(1).queryResult().topDocs().topDocs.totalHits.value()); assertEquals(84, responseRef.get().get(1).queryResult().topDocs().topDocs.scoreDocs[0].doc); assertTrue(mockSearchPhaseContext.releasedSearchContexts.isEmpty()); assertEquals(2, mockSearchPhaseContext.numSuccess.get()); @@ -240,7 +240,7 @@ public void run() throws IOException { assertNotNull(responseRef.get()); assertNotNull(responseRef.get().get(0)); assertNull(responseRef.get().get(0).fetchResult()); - assertEquals(1, responseRef.get().get(0).queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, responseRef.get().get(0).queryResult().topDocs().topDocs.totalHits.value()); assertEquals(42, responseRef.get().get(0).queryResult().topDocs().topDocs.scoreDocs[0].doc); assertNull(responseRef.get().get(1)); diff --git a/server/src/test/java/org/opensearch/action/search/FetchSearchPhaseTests.java b/server/src/test/java/org/opensearch/action/search/FetchSearchPhaseTests.java index 1eb3a44642806..c5db475e9db02 100644 --- a/server/src/test/java/org/opensearch/action/search/FetchSearchPhaseTests.java +++ b/server/src/test/java/org/opensearch/action/search/FetchSearchPhaseTests.java @@ -114,7 +114,7 @@ public void run() { mockSearchPhaseContext.assertNoFailure(); SearchResponse searchResponse = mockSearchPhaseContext.searchResponse.get(); assertNotNull(searchResponse); - assertEquals(numHits, searchResponse.getHits().getTotalHits().value); + assertEquals(numHits, searchResponse.getHits().getTotalHits().value()); if (numHits != 0) { assertEquals(42, searchResponse.getHits().getAt(0).docId()); } @@ -209,7 +209,7 @@ public void run() { mockSearchPhaseContext.assertNoFailure(); SearchResponse searchResponse = mockSearchPhaseContext.searchResponse.get(); assertNotNull(searchResponse); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); assertEquals(84, searchResponse.getHits().getAt(0).docId()); assertEquals(42, searchResponse.getHits().getAt(1).docId()); assertEquals(0, searchResponse.getFailedShards()); @@ -302,7 +302,7 @@ public void run() { mockSearchPhaseContext.assertNoFailure(); SearchResponse searchResponse = mockSearchPhaseContext.searchResponse.get(); assertNotNull(searchResponse); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); assertEquals(84, searchResponse.getHits().getAt(0).docId()); assertEquals(1, searchResponse.getFailedShards()); assertEquals(1, searchResponse.getSuccessfulShards()); @@ -387,7 +387,7 @@ public void run() { mockSearchPhaseContext.assertNoFailure(); SearchResponse searchResponse = mockSearchPhaseContext.searchResponse.get(); assertNotNull(searchResponse); - assertEquals(numHits, searchResponse.getHits().getTotalHits().value); + assertEquals(numHits, searchResponse.getHits().getTotalHits().value()); assertEquals(Math.min(numHits, resultSetSize), searchResponse.getHits().getHits().length); SearchHit[] hits = searchResponse.getHits().getHits(); for (int i = 0; i < hits.length; i++) { @@ -582,7 +582,7 @@ public void run() { mockSearchPhaseContext.assertNoFailure(); SearchResponse searchResponse = mockSearchPhaseContext.searchResponse.get(); assertNotNull(searchResponse); - assertEquals(2, searchResponse.getHits().getTotalHits().value); + assertEquals(2, searchResponse.getHits().getTotalHits().value()); assertEquals(1, searchResponse.getHits().getHits().length); assertEquals(84, searchResponse.getHits().getAt(0).docId()); assertEquals(0, searchResponse.getFailedShards()); diff --git a/server/src/test/java/org/opensearch/action/search/SearchPhaseControllerTests.java b/server/src/test/java/org/opensearch/action/search/SearchPhaseControllerTests.java index a927f733cc504..964f79d23447a 100644 --- a/server/src/test/java/org/opensearch/action/search/SearchPhaseControllerTests.java +++ b/server/src/test/java/org/opensearch/action/search/SearchPhaseControllerTests.java @@ -356,8 +356,8 @@ public void testMerge() { if (trackTotalHits == SearchContext.TRACK_TOTAL_HITS_DISABLED) { assertNull(mergedResponse.hits.getTotalHits()); } else { - assertThat(mergedResponse.hits.getTotalHits().value, equalTo(0L)); - assertEquals(mergedResponse.hits.getTotalHits().relation, Relation.EQUAL_TO); + assertThat(mergedResponse.hits.getTotalHits().value(), equalTo(0L)); + assertEquals(mergedResponse.hits.getTotalHits().relation(), Relation.EQUAL_TO); } for (SearchHit hit : mergedResponse.hits().getHits()) { SearchPhaseResult searchPhaseResult = fetchResults.get(hit.getShard().getShardId().id()); @@ -583,8 +583,8 @@ private static int getTotalQueryHits(AtomicArray results) { int resultCount = 0; for (SearchPhaseResult shardResult : results.asList()) { TopDocs topDocs = shardResult.queryResult().topDocs().topDocs; - assert topDocs.totalHits.relation == Relation.EQUAL_TO; - resultCount += (int) topDocs.totalHits.value; + assert topDocs.totalHits.relation() == Relation.EQUAL_TO; + resultCount += (int) topDocs.totalHits.value(); } return resultCount; } @@ -820,7 +820,7 @@ public void testConsumerConcurrently() throws Exception { assertEquals(max.get(), internalMax.getValue(), 0.0D); assertEquals(1, reduce.sortedTopDocs.scoreDocs.length); assertEquals(max.get(), reduce.maxScore, 0.0f); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertEquals(max.get(), reduce.sortedTopDocs.scoreDocs[0].score, 0.0f); assertFalse(reduce.sortedTopDocs.isSortedByField); assertNull(reduce.sortedTopDocs.sortFields); @@ -872,7 +872,7 @@ public void testConsumerOnlyAggs() throws Exception { assertEquals(max.get(), internalMax.getValue(), 0.0D); assertEquals(0, reduce.sortedTopDocs.scoreDocs.length); assertEquals(max.get(), reduce.maxScore, 0.0f); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertFalse(reduce.sortedTopDocs.isSortedByField); assertNull(reduce.sortedTopDocs.sortFields); assertNull(reduce.sortedTopDocs.collapseField); @@ -921,7 +921,7 @@ public void testConsumerOnlyHits() throws Exception { assertAggReduction(request); assertEquals(1, reduce.sortedTopDocs.scoreDocs.length); assertEquals(max.get(), reduce.maxScore, 0.0f); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertEquals(max.get(), reduce.sortedTopDocs.scoreDocs[0].score, 0.0f); assertFalse(reduce.sortedTopDocs.isSortedByField); assertNull(reduce.sortedTopDocs.sortFields); @@ -980,7 +980,7 @@ public void testReduceTopNWithFromOffset() throws Exception { ScoreDoc[] scoreDocs = reduce.sortedTopDocs.scoreDocs; assertEquals(5, scoreDocs.length); assertEquals(100.f, reduce.maxScore, 0.0f); - assertEquals(12, reduce.totalHits.value); + assertEquals(12, reduce.totalHits.value()); assertEquals(95.0f, scoreDocs[0].score, 0.0f); assertEquals(94.0f, scoreDocs[1].score, 0.0f); assertEquals(93.0f, scoreDocs[2].score, 0.0f); @@ -1025,7 +1025,7 @@ public void testConsumerSortByField() throws Exception { SearchPhaseController.ReducedQueryPhase reduce = consumer.reduce(); assertAggReduction(request); assertEquals(Math.min(expectedNumResults, size), reduce.sortedTopDocs.scoreDocs.length); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertEquals(max.get(), ((FieldDoc) reduce.sortedTopDocs.scoreDocs[0]).fields[0]); assertTrue(reduce.sortedTopDocs.isSortedByField); assertEquals(1, reduce.sortedTopDocs.sortFields.length); @@ -1074,7 +1074,7 @@ public void testConsumerFieldCollapsing() throws Exception { SearchPhaseController.ReducedQueryPhase reduce = consumer.reduce(); assertAggReduction(request); assertEquals(3, reduce.sortedTopDocs.scoreDocs.length); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertEquals(a, ((FieldDoc) reduce.sortedTopDocs.scoreDocs[0]).fields[0]); assertEquals(b, ((FieldDoc) reduce.sortedTopDocs.scoreDocs[1]).fields[0]); assertEquals(c, ((FieldDoc) reduce.sortedTopDocs.scoreDocs[2]).fields[0]); @@ -1186,7 +1186,7 @@ public void testConsumerSuggestions() throws Exception { assertEquals(maxScoreCompletion, reduce.sortedTopDocs.scoreDocs[0].score, 0f); assertEquals(0, reduce.sortedTopDocs.scoreDocs[0].doc); assertNotEquals(-1, reduce.sortedTopDocs.scoreDocs[0].shardIndex); - assertEquals(0, reduce.totalHits.value); + assertEquals(0, reduce.totalHits.value()); assertFalse(reduce.sortedTopDocs.isSortedByField); assertNull(reduce.sortedTopDocs.sortFields); assertNull(reduce.sortedTopDocs.collapseField); @@ -1277,7 +1277,7 @@ public void onFinalReduce(List shards, TotalHits totalHits, Interna assertEquals(max.get(), internalMax.getValue(), 0.0D); assertEquals(1, reduce.sortedTopDocs.scoreDocs.length); assertEquals(max.get(), reduce.maxScore, 0.0f); - assertEquals(expectedNumResults, reduce.totalHits.value); + assertEquals(expectedNumResults, reduce.totalHits.value()); assertEquals(max.get(), reduce.sortedTopDocs.scoreDocs[0].score, 0.0f); assertFalse(reduce.sortedTopDocs.isSortedByField); assertNull(reduce.sortedTopDocs.sortFields); diff --git a/server/src/test/java/org/opensearch/action/search/SearchQueryThenFetchAsyncActionTests.java b/server/src/test/java/org/opensearch/action/search/SearchQueryThenFetchAsyncActionTests.java index f6a06a51c7b43..1ccbbf4196505 100644 --- a/server/src/test/java/org/opensearch/action/search/SearchQueryThenFetchAsyncActionTests.java +++ b/server/src/test/java/org/opensearch/action/search/SearchQueryThenFetchAsyncActionTests.java @@ -274,11 +274,11 @@ public void run() { SearchPhaseController.ReducedQueryPhase phase = action.results.reduce(); assertThat(phase.numReducePhases, greaterThanOrEqualTo(1)); if (withScroll) { - assertThat(phase.totalHits.value, equalTo((long) numShards)); - assertThat(phase.totalHits.relation, equalTo(TotalHits.Relation.EQUAL_TO)); + assertThat(phase.totalHits.value(), equalTo((long) numShards)); + assertThat(phase.totalHits.relation(), equalTo(TotalHits.Relation.EQUAL_TO)); } else { - assertThat(phase.totalHits.value, equalTo(2L)); - assertThat(phase.totalHits.relation, equalTo(TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO)); + assertThat(phase.totalHits.value(), equalTo(2L)); + assertThat(phase.totalHits.relation(), equalTo(TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO)); } assertThat(phase.sortedTopDocs.scoreDocs.length, equalTo(1)); assertThat(phase.sortedTopDocs.scoreDocs[0], instanceOf(FieldDoc.class)); diff --git a/server/src/test/java/org/opensearch/action/search/SearchResponseMergerTests.java b/server/src/test/java/org/opensearch/action/search/SearchResponseMergerTests.java index 0eefa413c1864..e93f4553063ac 100644 --- a/server/src/test/java/org/opensearch/action/search/SearchResponseMergerTests.java +++ b/server/src/test/java/org/opensearch/action/search/SearchResponseMergerTests.java @@ -665,11 +665,11 @@ public void testMergeSearchHits() throws InterruptedException { TotalHits totalHits = null; if (trackTotalHitsUpTo != SearchContext.TRACK_TOTAL_HITS_DISABLED) { totalHits = new TotalHits(randomLongBetween(0, 1000), totalHitsRelation); - long previousValue = expectedTotalHits == null ? 0 : expectedTotalHits.value; - expectedTotalHits = new TotalHits(Math.min(previousValue + totalHits.value, trackTotalHitsUpTo), totalHitsRelation); + long previousValue = expectedTotalHits == null ? 0 : expectedTotalHits.value(); + expectedTotalHits = new TotalHits(Math.min(previousValue + totalHits.value(), trackTotalHitsUpTo), totalHitsRelation); } - final int numDocs = totalHits == null || totalHits.value >= requestedSize ? requestedSize : (int) totalHits.value; + final int numDocs = totalHits == null || totalHits.value() >= requestedSize ? requestedSize : (int) totalHits.value(); int scoreFactor = randomIntBetween(1, numResponses); float maxScore = scoreSort ? numDocs * scoreFactor : Float.NaN; SearchHit[] hits = randomSearchHitArray( @@ -771,8 +771,8 @@ public void testMergeSearchHits() throws InterruptedException { assertNull(searchHits.getTotalHits()); } else { assertNotNull(searchHits.getTotalHits()); - assertEquals(expectedTotalHits.value, searchHits.getTotalHits().value); - assertSame(expectedTotalHits.relation, searchHits.getTotalHits().relation); + assertEquals(expectedTotalHits.value(), searchHits.getTotalHits().value()); + assertSame(expectedTotalHits.relation(), searchHits.getTotalHits().relation()); } if (expectedMaxScore == Float.NEGATIVE_INFINITY) { assertTrue(Float.isNaN(searchHits.getMaxScore())); @@ -821,9 +821,9 @@ public void testMergeNoResponsesAdded() { assertEquals(0, response.getNumReducePhases()); assertFalse(response.isTimedOut()); assertNotNull(response.getHits().getTotalHits()); - assertEquals(0, response.getHits().getTotalHits().value); + assertEquals(0, response.getHits().getTotalHits().value()); assertEquals(0, response.getHits().getHits().length); - assertEquals(TotalHits.Relation.EQUAL_TO, response.getHits().getTotalHits().relation); + assertEquals(TotalHits.Relation.EQUAL_TO, response.getHits().getTotalHits().relation()); assertNull(response.getScrollId()); assertSame(InternalAggregations.EMPTY, response.getAggregations()); assertNull(response.getSuggest()); @@ -892,7 +892,7 @@ public void testMergeEmptySearchHitsWithNonEmpty() { () -> null ) ); - assertEquals(10, mergedResponse.getHits().getTotalHits().value); + assertEquals(10, mergedResponse.getHits().getTotalHits().value()); assertEquals(10, mergedResponse.getHits().getHits().length); assertEquals(2, mergedResponse.getTotalShards()); assertEquals(2, mergedResponse.getSuccessfulShards()); @@ -916,8 +916,8 @@ public void testMergeOnlyEmptyHits() { TotalHits totalHits = null; if (trackTotalHitsUpTo != SearchContext.TRACK_TOTAL_HITS_DISABLED) { totalHits = new TotalHits(randomLongBetween(0, 1000), totalHitsRelation); - long previousValue = expectedTotalHits == null ? 0 : expectedTotalHits.value; - expectedTotalHits = new TotalHits(Math.min(previousValue + totalHits.value, trackTotalHitsUpTo), totalHitsRelation); + long previousValue = expectedTotalHits == null ? 0 : expectedTotalHits.value(); + expectedTotalHits = new TotalHits(Math.min(previousValue + totalHits.value(), trackTotalHitsUpTo), totalHitsRelation); } SearchHits empty = new SearchHits(new SearchHit[0], totalHits, Float.NaN, null, null, null); InternalSearchResponse response = new InternalSearchResponse(empty, null, null, null, false, false, 1); diff --git a/server/src/test/java/org/opensearch/action/search/SearchResponseTests.java b/server/src/test/java/org/opensearch/action/search/SearchResponseTests.java index c9e59ab4ea04d..f5a53be43f267 100644 --- a/server/src/test/java/org/opensearch/action/search/SearchResponseTests.java +++ b/server/src/test/java/org/opensearch/action/search/SearchResponseTests.java @@ -434,8 +434,8 @@ public void testSerialization() throws IOException { if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); @@ -452,8 +452,8 @@ public void testSerializationWithSearchExtBuilders() throws IOException { if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); @@ -474,8 +474,8 @@ public void testSerializationWithSearchExtBuildersOnUnsupportedWriterVersion() t if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); diff --git a/server/src/test/java/org/opensearch/action/termvectors/AbstractTermVectorsTestCase.java b/server/src/test/java/org/opensearch/action/termvectors/AbstractTermVectorsTestCase.java index e84b5213be39e..92f8e132b691b 100644 --- a/server/src/test/java/org/opensearch/action/termvectors/AbstractTermVectorsTestCase.java +++ b/server/src/test/java/org/opensearch/action/termvectors/AbstractTermVectorsTestCase.java @@ -464,6 +464,6 @@ protected Fields getTermVectorsFromLucene(DirectoryReader directoryReader, TestD ScoreDoc[] scoreDocs = search.scoreDocs; assertEquals(1, scoreDocs.length); - return directoryReader.getTermVectors(scoreDocs[0].doc); + return directoryReader.termVectors().get(scoreDocs[0].doc); } } diff --git a/server/src/test/java/org/opensearch/action/termvectors/TermVectorsUnitTests.java b/server/src/test/java/org/opensearch/action/termvectors/TermVectorsUnitTests.java index 347670ffcdd00..999bbb8e10db7 100644 --- a/server/src/test/java/org/opensearch/action/termvectors/TermVectorsUnitTests.java +++ b/server/src/test/java/org/opensearch/action/termvectors/TermVectorsUnitTests.java @@ -132,7 +132,7 @@ private void writeEmptyTermVector(TermVectorsResponse outResponse) throws IOExce TopDocs search = s.search(new TermQuery(new Term("id", "abc")), 1); ScoreDoc[] scoreDocs = search.scoreDocs; int doc = scoreDocs[0].doc; - Fields fields = dr.getTermVectors(doc); + Fields fields = dr.termVectors().get(doc); EnumSet flags = EnumSet.of(Flag.Positions, Flag.Offsets); outResponse.setFields(fields, null, flags, fields); outResponse.setExists(true); @@ -167,7 +167,7 @@ private void writeStandardTermVector(TermVectorsResponse outResponse) throws IOE TopDocs search = s.search(new TermQuery(new Term("id", "abc")), 1); ScoreDoc[] scoreDocs = search.scoreDocs; int doc = scoreDocs[0].doc; - Fields termVectors = dr.getTermVectors(doc); + Fields termVectors = dr.termVectors().get(doc); EnumSet flags = EnumSet.of(Flag.Positions, Flag.Offsets); outResponse.setFields(termVectors, null, flags, termVectors); dr.close(); diff --git a/server/src/test/java/org/opensearch/bootstrap/SecurityTests.java b/server/src/test/java/org/opensearch/bootstrap/SecurityTests.java index 76353aea03257..ae57a1b0916cb 100644 --- a/server/src/test/java/org/opensearch/bootstrap/SecurityTests.java +++ b/server/src/test/java/org/opensearch/bootstrap/SecurityTests.java @@ -85,6 +85,7 @@ public void testProcessExecution() throws Exception { } catch (SecurityException expected) {} } + @SuppressWarnings("removal") public void testReadPolicyWithCodebases() throws IOException { final Map codebases = Map.of( "test-netty-tcnative-boringssl-static-2.0.61.Final-linux-x86_64.jar", diff --git a/server/src/test/java/org/opensearch/common/lucene/LuceneTests.java b/server/src/test/java/org/opensearch/common/lucene/LuceneTests.java index 3325495699e1a..df5e14b0c9e49 100644 --- a/server/src/test/java/org/opensearch/common/lucene/LuceneTests.java +++ b/server/src/test/java/org/opensearch/common/lucene/LuceneTests.java @@ -72,7 +72,6 @@ import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.Weight; import org.apache.lucene.store.Directory; -import org.apache.lucene.store.MMapDirectory; import org.apache.lucene.tests.analysis.MockAnalyzer; import org.apache.lucene.tests.index.RandomIndexWriter; import org.apache.lucene.tests.store.MockDirectoryWrapper; @@ -108,6 +107,8 @@ public class LuceneTests extends OpenSearchTestCase { private static final NamedWriteableRegistry EMPTY_REGISTRY = new NamedWriteableRegistry(Collections.emptyList()); + public static final String OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH = "/indices/bwc/os-1.3.0/testIndex-os-1.3.0.zip"; + public void testCleanIndex() throws IOException { MockDirectoryWrapper dir = newMockDirectory(); IndexWriterConfig iwc = newIndexWriterConfig(); @@ -204,10 +205,10 @@ public void testPruneUnreferencedFiles() throws IOException { assertEquals(3, open.maxDoc()); IndexSearcher s = new IndexSearcher(open); - assertEquals(s.search(new TermQuery(new Term("id", "1")), 1).totalHits.value, 1); - assertEquals(s.search(new TermQuery(new Term("id", "2")), 1).totalHits.value, 1); - assertEquals(s.search(new TermQuery(new Term("id", "3")), 1).totalHits.value, 1); - assertEquals(s.search(new TermQuery(new Term("id", "4")), 1).totalHits.value, 0); + assertEquals(s.search(new TermQuery(new Term("id", "1")), 1).totalHits.value(), 1); + assertEquals(s.search(new TermQuery(new Term("id", "2")), 1).totalHits.value(), 1); + assertEquals(s.search(new TermQuery(new Term("id", "3")), 1).totalHits.value(), 1); + assertEquals(s.search(new TermQuery(new Term("id", "4")), 1).totalHits.value(), 0); for (String file : dir.listAll()) { assertFalse("unexpected file: " + file, file.equals("segments_3") || file.startsWith("_2")); @@ -330,13 +331,12 @@ public void testNumDocs() throws IOException { /** * Tests whether old segments are readable and queryable based on the data documented - * in the README here. + * in the README here. */ public void testReadSegmentInfosExtendedCompatibility() throws IOException { - final String pathToTestIndex = "/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip"; - final Version minVersion = LegacyESVersion.V_6_0_0; + final Version minVersion = LegacyESVersion.V_7_2_0; Path tmp = createTempDir(); - TestUtil.unzip(getClass().getResourceAsStream(pathToTestIndex), tmp); + TestUtil.unzip(getClass().getResourceAsStream(OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH), tmp); try (MockDirectoryWrapper dir = newMockFSDirectory(tmp)) { // The standard API will throw an exception expectThrows(IndexFormatTooOldException.class, () -> Lucene.readSegmentInfos(dir)); @@ -466,11 +466,6 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio throw new UnsupportedOperationException(); } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - throw new UnsupportedOperationException(); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { return new ScorerSupplier() { @@ -526,18 +521,6 @@ public void testAsSequentialBitsUsesRandomAccess() throws IOException { } } - /** - * Test that the "unmap hack" is detected as supported by lucene. - * This works around the following bug: https://bugs.openjdk.java.net/browse/JDK-4724038 - *

- * While not guaranteed, current status is "Critical Internal API": http://openjdk.java.net/jeps/260 - * Additionally this checks we did not screw up the security logic around the hack. - */ - public void testMMapHackSupported() throws Exception { - // add assume's here if needed for certain platforms, but we should know if it does not work. - assertTrue("MMapDirectory does not support unmapping: " + MMapDirectory.UNMAP_NOT_SUPPORTED_REASON, MMapDirectory.UNMAP_SUPPORTED); - } - public void testWrapAllDocsLive() throws Exception { Directory dir = newDirectory(); IndexWriterConfig config = newIndexWriterConfig().setSoftDeletesField(Lucene.SOFT_DELETES_FIELD) diff --git a/server/src/test/java/org/opensearch/common/lucene/search/QueriesTests.java b/server/src/test/java/org/opensearch/common/lucene/search/QueriesTests.java index b9f128037a970..3cca29826378a 100644 --- a/server/src/test/java/org/opensearch/common/lucene/search/QueriesTests.java +++ b/server/src/test/java/org/opensearch/common/lucene/search/QueriesTests.java @@ -35,7 +35,7 @@ import org.apache.lucene.index.Term; import org.apache.lucene.search.BooleanClause.Occur; import org.apache.lucene.search.BooleanQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.TermQuery; import org.opensearch.Version; @@ -50,7 +50,7 @@ public void testNonNestedQuery() { // This is a custom query that extends AutomatonQuery and want to make sure the equals method works assertEquals(Queries.newNonNestedFilter(), Queries.newNonNestedFilter()); assertEquals(Queries.newNonNestedFilter().hashCode(), Queries.newNonNestedFilter().hashCode()); - assertEquals(Queries.newNonNestedFilter(), new DocValuesFieldExistsQuery(SeqNoFieldMapper.PRIMARY_TERM_NAME)); + assertEquals(Queries.newNonNestedFilter(), new FieldExistsQuery(SeqNoFieldMapper.PRIMARY_TERM_NAME)); } } diff --git a/server/src/test/java/org/opensearch/common/lucene/search/function/MinScoreScorerTests.java b/server/src/test/java/org/opensearch/common/lucene/search/function/MinScoreScorerTests.java index 8de2a54a3df37..b248f29ea5302 100644 --- a/server/src/test/java/org/opensearch/common/lucene/search/function/MinScoreScorerTests.java +++ b/server/src/test/java/org/opensearch/common/lucene/search/function/MinScoreScorerTests.java @@ -37,6 +37,7 @@ import org.apache.lucene.search.Explanation; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; import org.apache.lucene.tests.util.LuceneTestCase; @@ -89,7 +90,7 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { return null; } @@ -102,7 +103,7 @@ public boolean isCacheable(LeafReaderContext ctx) { private static Scorer scorer(int maxDoc, final int[] docs, final float[] scores, final boolean twoPhase) { final DocIdSetIterator iterator = twoPhase ? DocIdSetIterator.all(maxDoc) : iterator(docs); - return new Scorer(fakeWeight()) { + return new Scorer() { int lastScoredDoc = -1; diff --git a/server/src/test/java/org/opensearch/common/lucene/search/morelikethis/XMoreLikeThisTests.java b/server/src/test/java/org/opensearch/common/lucene/search/morelikethis/XMoreLikeThisTests.java index b907d739a435b..08efcf74624c9 100644 --- a/server/src/test/java/org/opensearch/common/lucene/search/morelikethis/XMoreLikeThisTests.java +++ b/server/src/test/java/org/opensearch/common/lucene/search/morelikethis/XMoreLikeThisTests.java @@ -99,7 +99,7 @@ public void testTopN() throws Exception { expectedTerms[idx++] = new Term("text", text); } for (BooleanClause clause : clauses) { - Term term = ((TermQuery) clause.getQuery()).getTerm(); + Term term = ((TermQuery) clause.query()).getTerm(); assertTrue(Arrays.asList(expectedTerms).contains(term)); } diff --git a/server/src/test/java/org/opensearch/deps/lucene/SimpleLuceneTests.java b/server/src/test/java/org/opensearch/deps/lucene/SimpleLuceneTests.java index 594a62679ccfc..e92500ba3e006 100644 --- a/server/src/test/java/org/opensearch/deps/lucene/SimpleLuceneTests.java +++ b/server/src/test/java/org/opensearch/deps/lucene/SimpleLuceneTests.java @@ -99,12 +99,12 @@ public void testSimpleNumericOps() throws Exception { IndexReader reader = DirectoryReader.open(indexWriter); IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); - Document doc = searcher.doc(topDocs.scoreDocs[0].doc); + Document doc = searcher.storedFields().document(topDocs.scoreDocs[0].doc); IndexableField f = doc.getField("test"); assertThat(f.numericValue(), equalTo(2)); topDocs = searcher.search(IntPoint.newExactQuery("test", 2), 1); - doc = searcher.doc(topDocs.scoreDocs[0].doc); + doc = searcher.storedFields().document(topDocs.scoreDocs[0].doc); f = doc.getField("test"); assertThat(f.stringValue(), equalTo("2")); @@ -129,7 +129,7 @@ public void testOrdering() throws Exception { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); final ArrayList fieldsOrder = new ArrayList<>(); - searcher.doc(topDocs.scoreDocs[0].doc, new StoredFieldVisitor() { + searcher.storedFields().document(topDocs.scoreDocs[0].doc, new StoredFieldVisitor() { @Override public Status needsField(FieldInfo fieldInfo) throws IOException { fieldsOrder.add(fieldInfo.name); diff --git a/server/src/test/java/org/opensearch/deps/lucene/VectorHighlighterTests.java b/server/src/test/java/org/opensearch/deps/lucene/VectorHighlighterTests.java index 880eaeca99bb0..ed5b23b148fa8 100644 --- a/server/src/test/java/org/opensearch/deps/lucene/VectorHighlighterTests.java +++ b/server/src/test/java/org/opensearch/deps/lucene/VectorHighlighterTests.java @@ -75,7 +75,7 @@ public void testVectorHighlighter() throws Exception { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); FastVectorHighlighter highlighter = new FastVectorHighlighter(); String fragment = highlighter.getBestFragment( @@ -106,7 +106,7 @@ public void testVectorHighlighterPrefixQuery() throws Exception { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); FastVectorHighlighter highlighter = new FastVectorHighlighter(); @@ -162,7 +162,7 @@ public void testVectorHighlighterNoStore() throws Exception { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); FastVectorHighlighter highlighter = new FastVectorHighlighter(); String fragment = highlighter.getBestFragment( @@ -188,7 +188,7 @@ public void testVectorHighlighterNoTermVector() throws Exception { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("_id", "1")), 1); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); FastVectorHighlighter highlighter = new FastVectorHighlighter(); String fragment = highlighter.getBestFragment( diff --git a/server/src/test/java/org/opensearch/gateway/MetadataStateFormatTests.java b/server/src/test/java/org/opensearch/gateway/MetadataStateFormatTests.java index 53f370471c91f..7978071c00a66 100644 --- a/server/src/test/java/org/opensearch/gateway/MetadataStateFormatTests.java +++ b/server/src/test/java/org/opensearch/gateway/MetadataStateFormatTests.java @@ -239,7 +239,7 @@ public static void corruptFile(Path fileToCorrupt, Logger logger) throws IOExcep } long checksumAfterCorruption; long actualChecksumAfterCorruption; - try (ChecksumIndexInput input = dir.openChecksumInput(fileToCorrupt.getFileName().toString(), IOContext.DEFAULT)) { + try (ChecksumIndexInput input = dir.openChecksumInput(fileToCorrupt.getFileName().toString())) { assertThat(input.getFilePointer(), is(0L)); input.seek(input.length() - 8); // one long is the checksum... 8 bytes checksumAfterCorruption = input.getChecksum(); diff --git a/server/src/test/java/org/opensearch/index/IndexServiceTests.java b/server/src/test/java/org/opensearch/index/IndexServiceTests.java index 5905e64cede1b..0e62953f52e1e 100644 --- a/server/src/test/java/org/opensearch/index/IndexServiceTests.java +++ b/server/src/test/java/org/opensearch/index/IndexServiceTests.java @@ -320,7 +320,7 @@ public void testRefreshActuallyWorks() throws Exception { // we are running on updateMetadata if the interval changes try (Engine.Searcher searcher = shard.acquireSearcher("test")) { TopDocs search = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, search.totalHits.value); + assertEquals(1, search.totalHits.value()); } }); assertFalse(refreshTask.isClosed()); @@ -336,7 +336,7 @@ public void testRefreshActuallyWorks() throws Exception { // this one becomes visible due to the force refresh we are running on updateMetadata if the interval changes try (Engine.Searcher searcher = shard.acquireSearcher("test")) { TopDocs search = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(2, search.totalHits.value); + assertEquals(2, search.totalHits.value()); } }); client().prepareIndex("test").setId("2").setSource("{\"foo\": \"bar\"}", MediaTypeRegistry.JSON).get(); @@ -344,7 +344,7 @@ public void testRefreshActuallyWorks() throws Exception { // this one becomes visible due to the scheduled refresh try (Engine.Searcher searcher = shard.acquireSearcher("test")) { TopDocs search = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(3, search.totalHits.value); + assertEquals(3, search.totalHits.value()); } }); } diff --git a/server/src/test/java/org/opensearch/index/codec/CodecTests.java b/server/src/test/java/org/opensearch/index/codec/CodecTests.java index cdd2c6d92f788..fa35996f14607 100644 --- a/server/src/test/java/org/opensearch/index/codec/CodecTests.java +++ b/server/src/test/java/org/opensearch/index/codec/CodecTests.java @@ -34,8 +34,8 @@ import org.apache.logging.log4j.LogManager; import org.apache.lucene.codecs.Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.codecs.lucene90.Lucene90StoredFieldsFormat; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; import org.apache.lucene.document.Document; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexWriter; @@ -48,7 +48,7 @@ import org.opensearch.env.Environment; import org.opensearch.index.IndexSettings; import org.opensearch.index.analysis.IndexAnalyzers; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.index.engine.EngineConfig; import org.opensearch.index.mapper.MapperService; import org.opensearch.index.similarity.SimilarityService; @@ -71,58 +71,58 @@ public class CodecTests extends OpenSearchTestCase { public void testResolveDefaultCodecs() throws Exception { CodecService codecService = createCodecService(false); assertThat(codecService.codec("default"), instanceOf(PerFieldMappingPostingFormatCodec.class)); - assertThat(codecService.codec("default"), instanceOf(Lucene912Codec.class)); + assertThat(codecService.codec("default"), instanceOf(Lucene101Codec.class)); } public void testDefault() throws Exception { Codec codec = createCodecService(false).codec("default"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_SPEED, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_SPEED, codec); } public void testDefaultWithCompositeIndex() throws Exception { Codec codec = createCodecService(false, true).codec("default"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_SPEED, codec); - assert codec instanceof Composite912Codec; + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_SPEED, codec); + assert codec instanceof Composite101Codec; } public void testBestCompression() throws Exception { Codec codec = createCodecService(false).codec("best_compression"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_COMPRESSION, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_COMPRESSION, codec); } public void testBestCompressionWithCompositeIndex() throws Exception { Codec codec = createCodecService(false, true).codec("best_compression"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_COMPRESSION, codec); - assert codec instanceof Composite912Codec; + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_COMPRESSION, codec); + assert codec instanceof Composite101Codec; } public void testLZ4() throws Exception { Codec codec = createCodecService(false).codec("lz4"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_SPEED, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_SPEED, codec); assert codec instanceof PerFieldMappingPostingFormatCodec; } public void testLZ4WithCompositeIndex() throws Exception { Codec codec = createCodecService(false, true).codec("lz4"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_SPEED, codec); - assert codec instanceof Composite912Codec; + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_SPEED, codec); + assert codec instanceof Composite101Codec; } public void testZlib() throws Exception { Codec codec = createCodecService(false).codec("zlib"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_COMPRESSION, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_COMPRESSION, codec); assert codec instanceof PerFieldMappingPostingFormatCodec; } public void testZlibWithCompositeIndex() throws Exception { Codec codec = createCodecService(false, true).codec("zlib"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_COMPRESSION, codec); - assert codec instanceof Composite912Codec; + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_COMPRESSION, codec); + assert codec instanceof Composite101Codec; } public void testResolveDefaultCodecsWithCompositeIndex() throws Exception { CodecService codecService = createCodecService(false, true); - assertThat(codecService.codec("default"), instanceOf(Composite912Codec.class)); + assertThat(codecService.codec("default"), instanceOf(Composite101Codec.class)); } public void testBestCompressionWithCompressionLevel() { @@ -157,12 +157,12 @@ public void testLuceneCodecsWithCompressionLevel() { public void testDefaultMapperServiceNull() throws Exception { Codec codec = createCodecService(true).codec("default"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_SPEED, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_SPEED, codec); } public void testBestCompressionMapperServiceNull() throws Exception { Codec codec = createCodecService(true).codec("best_compression"); - assertStoredFieldsCompressionEquals(Lucene912Codec.Mode.BEST_COMPRESSION, codec); + assertStoredFieldsCompressionEquals(Lucene101Codec.Mode.BEST_COMPRESSION, codec); } public void testExceptionCodecNull() { @@ -174,11 +174,11 @@ public void testExceptionIndexSettingsNull() { } // write some docs with it, inspect .si to see this was the used compression - private void assertStoredFieldsCompressionEquals(Lucene912Codec.Mode expected, Codec actual) throws Exception { + private void assertStoredFieldsCompressionEquals(Lucene101Codec.Mode expected, Codec actual) throws Exception { SegmentReader sr = getSegmentReader(actual); String v = sr.getSegmentInfo().info.getAttribute(Lucene90StoredFieldsFormat.MODE_KEY); assertNotNull(v); - assertEquals(expected, Lucene912Codec.Mode.valueOf(v)); + assertEquals(expected, Lucene101Codec.Mode.valueOf(v)); } private CodecService createCodecService(boolean isMapperServiceNull) throws IOException { diff --git a/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactoryTests.java b/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactoryTests.java index b39dd77ea225e..703eebecb95e1 100644 --- a/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactoryTests.java +++ b/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesConsumerFactoryTests.java @@ -9,7 +9,7 @@ package org.opensearch.index.codec.composite; import org.apache.lucene.codecs.DocValuesConsumer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; import org.apache.lucene.index.SegmentInfo; @@ -17,7 +17,7 @@ import org.apache.lucene.store.Directory; import org.apache.lucene.util.InfoStream; import org.apache.lucene.util.Version; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.test.OpenSearchTestCase; import org.junit.After; import org.junit.Before; @@ -44,12 +44,12 @@ public void testGetDocValuesConsumerForCompositeCodec() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_12_0, + Version.LUCENE_10_1_0, "test_segment", randomInt(), false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), @@ -73,7 +73,7 @@ public void testGetDocValuesConsumerForCompositeCodec() throws IOException { ); assertEquals("org.apache.lucene.codecs.lucene90.Lucene90DocValuesConsumer", consumer.getClass().getName()); - assertEquals(CompositeCodecFactory.COMPOSITE_CODEC, Composite912Codec.COMPOSITE_INDEX_CODEC_NAME); + assertEquals(CompositeCodecFactory.COMPOSITE_CODEC, Composite101Codec.COMPOSITE_INDEX_CODEC_NAME); consumer.close(); } diff --git a/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesProducerFactoryTests.java b/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesProducerFactoryTests.java index 7206828ff4723..c5ce4949a6440 100644 --- a/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesProducerFactoryTests.java +++ b/server/src/test/java/org/opensearch/index/codec/composite/LuceneDocValuesProducerFactoryTests.java @@ -10,7 +10,7 @@ import org.apache.lucene.codecs.DocValuesConsumer; import org.apache.lucene.codecs.DocValuesProducer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; import org.apache.lucene.index.SegmentInfo; @@ -48,12 +48,12 @@ public void testGetDocValuesProducerForCompositeCodec99() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_12_0, + Version.LUCENE_10_1_0, "test_segment", randomInt(), false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), diff --git a/server/src/test/java/org/opensearch/index/codec/composite/SortedNumericDocValuesWriterWrapperTests.java b/server/src/test/java/org/opensearch/index/codec/composite/SortedNumericDocValuesWriterWrapperTests.java index 54eead20ef354..018ae3b7208a8 100644 --- a/server/src/test/java/org/opensearch/index/codec/composite/SortedNumericDocValuesWriterWrapperTests.java +++ b/server/src/test/java/org/opensearch/index/codec/composite/SortedNumericDocValuesWriterWrapperTests.java @@ -8,6 +8,7 @@ package org.opensearch.index.codec.composite; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -38,6 +39,7 @@ public void setUp() throws Exception { true, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, @@ -62,10 +64,13 @@ public void testAddValue() throws IOException { assertNotNull(docValues); assertEquals(0, docValues.nextDoc()); + assertEquals(1, docValues.docValueCount()); assertEquals(10, docValues.nextValue()); assertEquals(1, docValues.nextDoc()); + assertEquals(1, docValues.docValueCount()); assertEquals(20, docValues.nextValue()); assertEquals(2, docValues.nextDoc()); + assertEquals(1, docValues.docValueCount()); assertEquals(30, docValues.nextValue()); } @@ -83,12 +88,12 @@ public void testMultipleValues() throws IOException { assertNotNull(docValues); assertEquals(0, docValues.nextDoc()); + assertEquals(2, docValues.docValueCount()); assertEquals(10, docValues.nextValue()); assertEquals(20, docValues.nextValue()); - assertThrows(IllegalStateException.class, docValues::nextValue); assertEquals(1, docValues.nextDoc()); + assertEquals(1, docValues.docValueCount()); assertEquals(30, docValues.nextValue()); - assertThrows(IllegalStateException.class, docValues::nextValue); } } diff --git a/server/src/test/java/org/opensearch/index/codec/composite/SortedSetDocValuesWriterWrapperTests.java b/server/src/test/java/org/opensearch/index/codec/composite/SortedSetDocValuesWriterWrapperTests.java index b0fdd712beafb..5adab4eee4265 100644 --- a/server/src/test/java/org/opensearch/index/codec/composite/SortedSetDocValuesWriterWrapperTests.java +++ b/server/src/test/java/org/opensearch/index/codec/composite/SortedSetDocValuesWriterWrapperTests.java @@ -8,6 +8,7 @@ package org.opensearch.index.codec.composite; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -40,6 +41,7 @@ public void setUp() throws Exception { true, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, @@ -89,10 +91,10 @@ public void testMultipleValues() throws IOException { assertEquals(0, docValues.nextDoc()); assertEquals(0, docValues.nextOrd()); assertEquals(1, docValues.nextOrd()); - assertEquals(-1, docValues.nextOrd()); + assertEquals(2, docValues.docValueCount()); assertEquals(1, docValues.nextDoc()); assertEquals(2, docValues.nextOrd()); - assertEquals(-1, docValues.nextOrd()); + assertEquals(1, docValues.docValueCount()); } } diff --git a/server/src/test/java/org/opensearch/index/codec/composite912/datacube/startree/AbstractStarTreeDVFormatTests.java b/server/src/test/java/org/opensearch/index/codec/composite912/datacube/startree/AbstractStarTreeDVFormatTests.java index 4dfd8c08575f2..2b9c174a5b6f5 100644 --- a/server/src/test/java/org/opensearch/index/codec/composite912/datacube/startree/AbstractStarTreeDVFormatTests.java +++ b/server/src/test/java/org/opensearch/index/codec/composite912/datacube/startree/AbstractStarTreeDVFormatTests.java @@ -13,7 +13,7 @@ import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.lucene.codecs.Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.tests.index.BaseDocValuesFormatTestCase; import org.apache.lucene.tests.util.LuceneTestCase; import org.opensearch.Version; @@ -29,7 +29,7 @@ import org.opensearch.core.xcontent.XContentBuilder; import org.opensearch.index.IndexSettings; import org.opensearch.index.MapperTestUtils; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.index.compositeindex.datacube.startree.StarTreeFieldConfiguration; import org.opensearch.index.compositeindex.datacube.startree.StarTreeIndexSettings; import org.opensearch.index.mapper.MapperService; @@ -90,7 +90,7 @@ protected Codec getCodec() { } catch (IOException e) { throw new RuntimeException(e); } - Codec codec = new Composite912Codec(Lucene912Codec.Mode.BEST_SPEED, mapperService, testLogger); + Codec codec = new Composite101Codec(Lucene101Codec.Mode.BEST_SPEED, mapperService, testLogger); return codec; } diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BaseStarTreeBuilderTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BaseStarTreeBuilderTests.java index ac729f6392f63..f2f7947888aaa 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BaseStarTreeBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BaseStarTreeBuilderTests.java @@ -10,7 +10,8 @@ import org.apache.lucene.codecs.DocValuesConsumer; import org.apache.lucene.codecs.DocValuesProducer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; @@ -108,12 +109,12 @@ public static void setup() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_12_0, + Version.LUCENE_10_1_0, "test_segment", 5, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), @@ -131,6 +132,7 @@ public static void setup() throws IOException { true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BuilderTestsUtils.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BuilderTestsUtils.java index 077bf0422ab50..e7c3d50c9572a 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BuilderTestsUtils.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/BuilderTestsUtils.java @@ -9,7 +9,8 @@ package org.opensearch.index.compositeindex.datacube.startree.builder; import org.apache.lucene.codecs.DocValuesProducer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; @@ -471,6 +472,7 @@ public static SegmentReadState getReadState( true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, dimensionFields.get(dimension), + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, @@ -499,6 +501,7 @@ public static SegmentReadState getReadState( true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, @@ -517,12 +520,12 @@ public static SegmentReadState getReadState( SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_11_0, + Version.LUCENE_10_1_0, "test_segment", numDocs, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), writeState.segmentInfo.getId(), new HashMap<>(), @@ -571,12 +574,12 @@ public static SegmentWriteState getWriteState(int numDocs, byte[] id, FieldInfo[ SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_11_0, + Version.LUCENE_10_1_0, "test_segment", numDocs, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), id, new HashMap<>(), diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuildMetricTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuildMetricTests.java index 95adae9335740..72ee197a93e18 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuildMetricTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuildMetricTests.java @@ -10,7 +10,8 @@ import org.apache.lucene.codecs.DocValuesConsumer; import org.apache.lucene.codecs.DocValuesProducer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; @@ -507,12 +508,12 @@ public void test_build_multipleStarTrees() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_11_0, + Version.LUCENE_10_1_0, "test_segment", 7, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), @@ -530,6 +531,7 @@ public void test_build_multipleStarTrees() throws IOException { true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, @@ -767,12 +769,12 @@ public void test_build_starTreeDataset() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_11_0, + Version.LUCENE_10_1_0, "test_segment", 7, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), @@ -790,6 +792,7 @@ public void test_build_starTreeDataset() throws IOException { true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderFlushFlowTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderFlushFlowTests.java index 70cc20fe4a9f6..7ecf175b5eb09 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderFlushFlowTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderFlushFlowTests.java @@ -102,6 +102,7 @@ public void testFlushFlow() throws IOException { */ this.docValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( writeState, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, @@ -191,6 +192,7 @@ public void testFlushFlowDimsReverse() throws IOException { writeState = getWriteState(6, writeState.segmentInfo.getId()); this.docValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( writeState, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, @@ -296,6 +298,7 @@ public void testFlushFlowBuild() throws IOException { SegmentWriteState consumerWriteState = getWriteState(DocIdSetIterator.NO_MORE_DOCS, writeState.segmentInfo.getId()); this.docValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( consumerWriteState, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, @@ -376,6 +379,7 @@ public void testFlushFlowWithTimestamps() throws IOException { ); this.docValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( writeState, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, @@ -457,6 +461,7 @@ public void testFlushFlowForKeywords() throws IOException { SegmentWriteState w = getWriteState(DocIdSetIterator.NO_MORE_DOCS, writeState.segmentInfo.getId()); this.docValuesConsumer = LuceneDocValuesConsumerFactory.getDocValuesConsumerForCompositeCodec( w, + 4096, /* Lucene90DocValuesFormat#DEFAULT_SKIP_INDEX_INTERVAL_SIZE */ Composite912DocValuesFormat.DATA_DOC_VALUES_CODEC, Composite912DocValuesFormat.DATA_DOC_VALUES_EXTENSION, Composite912DocValuesFormat.META_DOC_VALUES_CODEC, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderTestCase.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderTestCase.java index cca987b6f9b16..843a79d0877ce 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderTestCase.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreeBuilderTestCase.java @@ -12,6 +12,7 @@ import org.apache.lucene.codecs.DocValuesConsumer; import org.apache.lucene.codecs.DocValuesProducer; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexFileNames; @@ -145,6 +146,7 @@ public void setup() throws IOException { true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.RANGE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreesBuilderTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreesBuilderTests.java index 4ab21dbce059f..6c75e89959348 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreesBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/builder/StarTreesBuilderTests.java @@ -10,7 +10,7 @@ import org.apache.lucene.codecs.DocValuesConsumer; import org.apache.lucene.codecs.DocValuesProducer; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; import org.apache.lucene.index.SegmentInfo; @@ -60,12 +60,12 @@ public void setUp() throws Exception { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_12_0, + Version.LUCENE_10_1_0, "test_segment", 5, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/fileformats/meta/StarTreeMetadataTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/fileformats/meta/StarTreeMetadataTests.java index cc91d69be97c1..51dee33662290 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/fileformats/meta/StarTreeMetadataTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/fileformats/meta/StarTreeMetadataTests.java @@ -8,7 +8,8 @@ package org.opensearch.index.compositeindex.datacube.startree.fileformats.meta; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.FieldInfos; @@ -74,12 +75,12 @@ public void setup() throws IOException { SegmentInfo segmentInfo = new SegmentInfo( directory, Version.LATEST, - Version.LUCENE_9_12_0, + Version.LUCENE_10_1_0, "test_segment", 6, false, false, - new Lucene912Codec(), + new Lucene101Codec(), new HashMap<>(), UUID.randomUUID().toString().substring(0, 16).getBytes(StandardCharsets.UTF_8), new HashMap<>(), @@ -96,6 +97,7 @@ public void setup() throws IOException { true, IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, DocValuesType.SORTED_NUMERIC, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/SequentialDocValuesIteratorTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/SequentialDocValuesIteratorTests.java index 78d63800abd16..00233f48de6da 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/SequentialDocValuesIteratorTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/SequentialDocValuesIteratorTests.java @@ -9,6 +9,7 @@ package org.opensearch.index.compositeindex.datacube.startree.utils; import org.apache.lucene.codecs.DocValuesProducer; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -40,6 +41,7 @@ public static void setup() { true, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtilsTests.java b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtilsTests.java index 7e438c18d9ab9..7ff9e75de684f 100644 --- a/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtilsTests.java +++ b/server/src/test/java/org/opensearch/index/compositeindex/datacube/startree/utils/StarTreeUtilsTests.java @@ -62,7 +62,7 @@ private void assertFieldInfos(FieldInfo actualFieldInfo, String fieldName, Integ assertEquals(fieldNumber, actualFieldInfo.number, 0); assertFalse(actualFieldInfo.hasVectorValues()); assertTrue(actualFieldInfo.hasNorms()); - assertFalse(actualFieldInfo.hasVectors()); + assertFalse(actualFieldInfo.hasTermVectors()); assertEquals(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS, actualFieldInfo.getIndexOptions()); assertEquals(DocValuesType.SORTED_NUMERIC, actualFieldInfo.getDocValuesType()); assertEquals(-1, actualFieldInfo.getDocValuesGen()); diff --git a/server/src/test/java/org/opensearch/index/engine/CompletionStatsCacheTests.java b/server/src/test/java/org/opensearch/index/engine/CompletionStatsCacheTests.java index 6a2db930aff9b..a46aaf3a14043 100644 --- a/server/src/test/java/org/opensearch/index/engine/CompletionStatsCacheTests.java +++ b/server/src/test/java/org/opensearch/index/engine/CompletionStatsCacheTests.java @@ -32,14 +32,14 @@ package org.opensearch.index.engine; import org.apache.lucene.codecs.PostingsFormat; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.document.Document; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.search.Query; import org.apache.lucene.search.QueryCachingPolicy; -import org.apache.lucene.search.suggest.document.Completion912PostingsFormat; +import org.apache.lucene.search.suggest.document.Completion101PostingsFormat; import org.apache.lucene.search.suggest.document.SuggestField; import org.apache.lucene.store.Directory; import org.opensearch.OpenSearchException; @@ -69,8 +69,8 @@ public void testExceptionsAreNotCached() { public void testCompletionStatsCache() throws IOException, InterruptedException { final IndexWriterConfig indexWriterConfig = newIndexWriterConfig(); - final PostingsFormat postingsFormat = new Completion912PostingsFormat(); - indexWriterConfig.setCodec(new Lucene912Codec() { + final PostingsFormat postingsFormat = new Completion101PostingsFormat(); + indexWriterConfig.setCodec(new Lucene101Codec() { @Override public PostingsFormat getPostingsFormatForField(String field) { return postingsFormat; // all fields are suggest fields diff --git a/server/src/test/java/org/opensearch/index/engine/InternalEngineTests.java b/server/src/test/java/org/opensearch/index/engine/InternalEngineTests.java index 08a24a74fdc89..b49ea9af6d5c4 100644 --- a/server/src/test/java/org/opensearch/index/engine/InternalEngineTests.java +++ b/server/src/test/java/org/opensearch/index/engine/InternalEngineTests.java @@ -271,7 +271,7 @@ public void testVersionMapAfterAutoIDDocument() throws IOException { try (Engine.Searcher searcher = engine.acquireSearcher("test", Engine.SearcherScope.INTERNAL)) { assertEquals(1, searcher.getIndexReader().numDocs()); TopDocs search = searcher.search(new MatchAllDocsQuery(), 1); - org.apache.lucene.document.Document luceneDoc = searcher.doc(search.scoreDocs[0].doc); + org.apache.lucene.document.Document luceneDoc = searcher.storedFields().document(search.scoreDocs[0].doc); assertEquals("test", luceneDoc.get("value")); } @@ -284,7 +284,7 @@ public void testVersionMapAfterAutoIDDocument() throws IOException { try (Engine.Searcher searcher = engine.acquireSearcher("test")) { assertEquals(1, searcher.getIndexReader().numDocs()); TopDocs search = searcher.search(new MatchAllDocsQuery(), 1); - org.apache.lucene.document.Document luceneDoc = searcher.doc(search.scoreDocs[0].doc); + org.apache.lucene.document.Document luceneDoc = searcher.storedFields().document(search.scoreDocs[0].doc); assertEquals("updated", luceneDoc.get("value")); } @@ -732,7 +732,7 @@ public void testMergeSegmentsOnCommitDefault() throws Exception { engine.refresh("test"); segments = engine.segments(true); - assertThat(segments.size(), equalTo(2)); + assertThat(segments.size(), equalTo(1)); } } @@ -1004,7 +1004,7 @@ public void testTranslogRecoveryWithMultipleGenerations() throws IOException { recoveringEngine.refresh("test"); try (Engine.Searcher searcher = recoveringEngine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), docs); - assertEquals(docs, topDocs.totalHits.value); + assertEquals(docs, topDocs.totalHits.value()); } } finally { IOUtils.close(initialEngine, recoveringEngine, store); @@ -3909,7 +3909,7 @@ public void testSkipTranslogReplay() throws IOException { engine.translogManager().skipTranslogRecovery(); try (Engine.Searcher searcher = engine.acquireSearcher("test", Engine.SearcherScope.INTERNAL)) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), randomIntBetween(numDocs, numDocs + 10)); - assertThat(topDocs.totalHits.value, equalTo(0L)); + assertThat(topDocs.totalHits.value(), equalTo(0L)); } } } @@ -4000,7 +4000,7 @@ public void testTranslogReplay() throws IOException { assertThat(result.getVersion(), equalTo(2L)); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs + 1); - assertThat(topDocs.totalHits.value, equalTo(numDocs + 1L)); + assertThat(topDocs.totalHits.value(), equalTo(numDocs + 1L)); } engine.close(); @@ -4009,7 +4009,7 @@ public void testTranslogReplay() throws IOException { engine.refresh("warm_up"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs + 1); - assertThat(topDocs.totalHits.value, equalTo(numDocs + 1L)); + assertThat(topDocs.totalHits.value(), equalTo(numDocs + 1L)); } assertEquals(flush ? 1 : 2, translogHandler.appliedOperations()); engine.delete(new Engine.Delete(Integer.toString(randomId), newUid(doc), primaryTerm.get())); @@ -4020,7 +4020,7 @@ public void testTranslogReplay() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs); - assertThat(topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(topDocs.totalHits.value(), equalTo((long) numDocs)); } } @@ -4443,7 +4443,7 @@ public void testDoubleDeliveryPrimary() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } operation = appendOnlyPrimary(doc, false, 1, create); retry = appendOnlyPrimary(doc, true, 1, create); @@ -4478,7 +4478,7 @@ public void testDoubleDeliveryPrimary() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } } @@ -4536,7 +4536,7 @@ public void testDoubleDeliveryReplicaAppendingAndDeleteOnly() throws IOException engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); } } @@ -4560,7 +4560,7 @@ public void testDoubleDeliveryReplicaAppendingOnly() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } final boolean create = randomBoolean(); @@ -4600,7 +4600,7 @@ public void testDoubleDeliveryReplicaAppendingOnly() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } } @@ -4645,12 +4645,12 @@ public void testDoubleDeliveryReplica() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } if (engine.engineConfig.getIndexSettings().isSoftDeleteEnabled()) { List ops = readAllOperationsInLucene(engine); @@ -4725,7 +4725,7 @@ public void testRetryWithAutogeneratedIdWorksAndNoDuplicateDocs() throws IOExcep engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } index = new Engine.Index( @@ -4747,7 +4747,7 @@ public void testRetryWithAutogeneratedIdWorksAndNoDuplicateDocs() throws IOExcep replicaEngine.refresh("test"); try (Engine.Searcher searcher = replicaEngine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } } @@ -4817,7 +4817,7 @@ public void testRetryWithAutogeneratedIdsAndWrongOrderWorksAndNoDuplicateDocs() engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } Engine.Index secondIndexRequestReplica = new Engine.Index( @@ -4838,7 +4838,7 @@ public void testRetryWithAutogeneratedIdsAndWrongOrderWorksAndNoDuplicateDocs() replicaEngine.refresh("test"); try (Engine.Searcher searcher = replicaEngine.acquireSearcher("test")) { TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); } } @@ -6277,7 +6277,7 @@ public void testConcurrentAppendUpdateAndRefresh() throws InterruptedException, try (Engine.Searcher searcher = engine.acquireSearcher("test", Engine.SearcherScope.INTERNAL)) { TopDocs search = searcher.search(new MatchAllDocsQuery(), searcher.getIndexReader().numDocs()); for (int i = 0; i < search.scoreDocs.length; i++) { - org.apache.lucene.document.Document luceneDoc = searcher.doc(search.scoreDocs[i].doc); + org.apache.lucene.document.Document luceneDoc = searcher.storedFields().document(search.scoreDocs[i].doc); assertEquals("updated", luceneDoc.get("value")); } int totalNumDocs = numDocs - numDeletes.get(); @@ -7294,7 +7294,7 @@ public void testStoreHonorsLuceneVersion() throws IOException { engine.refresh("test"); try (Engine.Searcher searcher = engine.acquireSearcher("test")) { LeafReader leafReader = getOnlyLeafReader(searcher.getIndexReader()); - assertEquals(createdVersion.luceneVersion.major, leafReader.getMetaData().getCreatedVersionMajor()); + assertEquals(createdVersion.luceneVersion.major, leafReader.getMetaData().createdVersionMajor()); } } } diff --git a/server/src/test/java/org/opensearch/index/engine/LiveVersionMapTests.java b/server/src/test/java/org/opensearch/index/engine/LiveVersionMapTests.java index 8ac584b3fa923..1b584ce1cad5f 100644 --- a/server/src/test/java/org/opensearch/index/engine/LiveVersionMapTests.java +++ b/server/src/test/java/org/opensearch/index/engine/LiveVersionMapTests.java @@ -36,7 +36,6 @@ import org.apache.lucene.tests.util.TestUtil; import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.BytesRefBuilder; -import org.apache.lucene.util.Constants; import org.opensearch.common.lease.Releasable; import org.opensearch.index.translog.Translog; import org.opensearch.test.OpenSearchTestCase; @@ -86,22 +85,14 @@ public void testRamBytesUsed() throws Exception { } actualRamBytesUsed = RamUsageTester.ramUsed(map); estimatedRamBytesUsed = map.ramBytesUsed(); - long tolerance; - if (Constants.JRE_IS_MINIMUM_JAVA9) { - // With Java 9, RamUsageTester computes the memory usage of maps as - // the memory usage of an array that would contain exactly all keys - // and values. This is an under-estimation of the actual memory - // usage since it ignores the impact of the load factor and of the - // linked list/tree that is used to resolve collisions. So we use a - // bigger tolerance. - // less than 50% off - tolerance = actualRamBytesUsed / 2; - } else { - // Java 8 is more accurate by doing reflection into the actual JDK classes - // so we give it a lower error bound. - // less than 25% off - tolerance = actualRamBytesUsed / 4; - } + // With Java 9, RamUsageTester computes the memory usage of maps as + // the memory usage of an array that would contain exactly all keys + // and values. This is an under-estimation of the actual memory + // usage since it ignores the impact of the load factor and of the + // linked list/tree that is used to resolve collisions. So we use a + // bigger tolerance. + // less than 50% off + long tolerance = actualRamBytesUsed / 2; assertEquals(actualRamBytesUsed, estimatedRamBytesUsed, tolerance); } diff --git a/server/src/test/java/org/opensearch/index/engine/ReadOnlyEngineTests.java b/server/src/test/java/org/opensearch/index/engine/ReadOnlyEngineTests.java index 7c9a08d69d3c2..288822ab1589f 100644 --- a/server/src/test/java/org/opensearch/index/engine/ReadOnlyEngineTests.java +++ b/server/src/test/java/org/opensearch/index/engine/ReadOnlyEngineTests.java @@ -38,6 +38,7 @@ import org.apache.lucene.tests.util.TestUtil; import org.opensearch.Version; import org.opensearch.cluster.metadata.IndexMetadata; +import org.opensearch.common.lucene.LuceneTests; import org.opensearch.common.lucene.index.OpenSearchDirectoryReader; import org.opensearch.common.settings.Settings; import org.opensearch.common.util.FeatureFlags; @@ -243,9 +244,8 @@ public void testReadOldIndices() throws Exception { IOUtils.close(engine, store); // The index has one document in it, so the checkpoint cannot be NO_OPS_PERFORMED final AtomicLong globalCheckpoint = new AtomicLong(0); - final String pathToTestIndex = "/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip"; Path tmp = createTempDir(); - TestUtil.unzip(getClass().getResourceAsStream(pathToTestIndex), tmp); + TestUtil.unzip(getClass().getResourceAsStream(LuceneTests.OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH), tmp); FeatureFlagSetter.set(FeatureFlags.SEARCHABLE_SNAPSHOT_EXTENDED_COMPATIBILITY); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings( "index", @@ -266,9 +266,8 @@ public void testReadOldIndicesFailure() throws IOException { IOUtils.close(engine, store); // The index has one document in it, so the checkpoint cannot be NO_OPS_PERFORMED final AtomicLong globalCheckpoint = new AtomicLong(0); - final String pathToTestIndex = "/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip"; Path tmp = createTempDir(); - TestUtil.unzip(getClass().getResourceAsStream(pathToTestIndex), tmp); + TestUtil.unzip(getClass().getResourceAsStream(LuceneTests.OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH), tmp); try (Store store = createStore(newFSDirectory(tmp))) { EngineConfig config = config(defaultSettings, store, createTempDir(), newMergePolicy(), null, null, globalCheckpoint::get); try { diff --git a/server/src/test/java/org/opensearch/index/engine/SegmentTests.java b/server/src/test/java/org/opensearch/index/engine/SegmentTests.java index 840bfe37b979d..53453e94349cf 100644 --- a/server/src/test/java/org/opensearch/index/engine/SegmentTests.java +++ b/server/src/test/java/org/opensearch/index/engine/SegmentTests.java @@ -92,7 +92,7 @@ static Segment randomSegment() { segment.sizeInBytes = randomNonNegativeLong(); segment.docCount = randomIntBetween(1, Integer.MAX_VALUE); segment.delDocCount = randomIntBetween(0, segment.docCount); - segment.version = Version.LUCENE_8_0_0; + segment.version = Version.LUCENE_9_0_0; segment.compound = randomBoolean(); segment.mergeId = randomAlphaOfLengthBetween(1, 10); segment.segmentSort = randomIndexSort(); diff --git a/server/src/test/java/org/opensearch/index/fielddata/AbstractFieldDataImplTestCase.java b/server/src/test/java/org/opensearch/index/fielddata/AbstractFieldDataImplTestCase.java index 2b44e759f4ff9..52901efd54eca 100644 --- a/server/src/test/java/org/opensearch/index/fielddata/AbstractFieldDataImplTestCase.java +++ b/server/src/test/java/org/opensearch/index/fielddata/AbstractFieldDataImplTestCase.java @@ -128,7 +128,7 @@ public void testSingleValueAllSet() throws Exception { TopFieldDocs topDocs; SortField sortField = indexFieldData.sortField(null, MultiValueMode.MIN, null, false); topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs[0].doc, equalTo(1)); assertThat(toString(((FieldDoc) topDocs.scoreDocs[0]).fields[0]), equalTo(one())); assertThat(topDocs.scoreDocs[1].doc, equalTo(0)); @@ -138,7 +138,7 @@ public void testSingleValueAllSet() throws Exception { sortField = indexFieldData.sortField(null, MultiValueMode.MAX, null, true); topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs[0].doc, equalTo(2)); assertThat(topDocs.scoreDocs[1].doc, equalTo(0)); assertThat(topDocs.scoreDocs[2].doc, equalTo(1)); @@ -225,14 +225,14 @@ public void testMultiValueAllSet() throws Exception { IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(writer)); SortField sortField = indexFieldData.sortField(null, MultiValueMode.MIN, null, false); TopFieldDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs.length, equalTo(3)); assertThat(topDocs.scoreDocs[0].doc, equalTo(1)); assertThat(topDocs.scoreDocs[1].doc, equalTo(0)); assertThat(topDocs.scoreDocs[2].doc, equalTo(2)); sortField = indexFieldData.sortField(null, MultiValueMode.MAX, null, true); topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs.length, equalTo(3)); assertThat(topDocs.scoreDocs[0].doc, equalTo(0)); assertThat(topDocs.scoreDocs[1].doc, equalTo(2)); @@ -289,7 +289,7 @@ public void testSortMultiValuesFields() throws Exception { IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(writer)); SortField sortField = indexFieldData.sortField(null, MultiValueMode.MIN, null, false); TopFieldDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(8L)); + assertThat(topDocs.totalHits.value(), equalTo(8L)); assertThat(topDocs.scoreDocs.length, equalTo(8)); assertThat(topDocs.scoreDocs[0].doc, equalTo(7)); assertThat(((BytesRef) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).utf8ToString(), equalTo("!08")); @@ -310,7 +310,7 @@ public void testSortMultiValuesFields() throws Exception { sortField = indexFieldData.sortField(null, MultiValueMode.MAX, null, true); topDocs = searcher.search(new MatchAllDocsQuery(), 10, new Sort(sortField)); - assertThat(topDocs.totalHits.value, equalTo(8L)); + assertThat(topDocs.totalHits.value(), equalTo(8L)); assertThat(topDocs.scoreDocs.length, equalTo(8)); assertThat(topDocs.scoreDocs[0].doc, equalTo(6)); assertThat(((BytesRef) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).utf8ToString(), equalTo("10")); diff --git a/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java b/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java index 3aa698260686d..da314358475c4 100644 --- a/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java +++ b/server/src/test/java/org/opensearch/index/fielddata/AbstractStringFieldDataTestCase.java @@ -45,6 +45,7 @@ import org.apache.lucene.index.Term; import org.apache.lucene.index.TermsEnum; import org.apache.lucene.search.ConstantScoreQuery; +import org.apache.lucene.search.DocIdSetIterator; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -283,10 +284,10 @@ public void testActualMissingValue(boolean reverse) throws IOException { randomBoolean() ? numDocs : randomIntBetween(10, numDocs), new Sort(sortField) ); - assertEquals(numDocs, topDocs.totalHits.value); + assertEquals(numDocs, topDocs.totalHits.value()); BytesRef previousValue = reverse ? UnicodeUtil.BIG_TERM : new BytesRef(); for (int i = 0; i < topDocs.scoreDocs.length; ++i) { - final String docValue = searcher.doc(topDocs.scoreDocs[i].doc).get("value"); + final String docValue = searcher.storedFields().document(topDocs.scoreDocs[i].doc).get("value"); final BytesRef value = new BytesRef(docValue == null ? missingValue : docValue); if (reverse) { assertTrue(previousValue.compareTo(value) >= 0); @@ -342,15 +343,15 @@ public void testSortMissing(boolean first, boolean reverse) throws IOException { new Sort(sortField) ); // As of Lucene 9.0.0, totalHits may be a lower bound - if (topDocs.totalHits.relation == TotalHits.Relation.EQUAL_TO) { - assertEquals(numDocs, topDocs.totalHits.value); + if (topDocs.totalHits.relation() == TotalHits.Relation.EQUAL_TO) { + assertEquals(numDocs, topDocs.totalHits.value()); } else { - assertTrue(1000 <= topDocs.totalHits.value); - assertTrue(numDocs >= topDocs.totalHits.value); + assertTrue(1000 <= topDocs.totalHits.value()); + assertTrue(numDocs >= topDocs.totalHits.value()); } BytesRef previousValue = first ? null : reverse ? UnicodeUtil.BIG_TERM : new BytesRef(); for (int i = 0; i < topDocs.scoreDocs.length; ++i) { - final String docValue = searcher.doc(topDocs.scoreDocs[i].doc).get("value"); + final String docValue = searcher.storedFields().document(topDocs.scoreDocs[i].doc).get("value"); if (first && docValue == null) { assertNull(previousValue); } else if (!first && docValue != null) { @@ -449,7 +450,7 @@ public void testNestedSorting(MultiValueMode sortMode) throws IOException { assertTrue("expected " + docID + " to be a parent", parents.get(docID)); BytesRef cmpValue = null; for (int child = parents.prevSetBit(docID - 1) + 1; child < docID; ++child) { - String[] sVals = searcher.doc(child).getValues("text"); + String[] sVals = searcher.storedFields().document(child).getValues("text"); final BytesRef[] vals; if (sVals.length == 0) { vals = new BytesRef[0]; @@ -505,14 +506,14 @@ public void testGlobalOrdinals() throws Exception { assertThat(ord, equalTo(5L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("04")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); assertFalse(values.advanceExact(1)); assertTrue(values.advanceExact(2)); ord = values.nextOrd(); assertThat(ord, equalTo(4L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("03")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); // Second segment leaf = topLevelReader.leaves().get(1); @@ -529,7 +530,7 @@ public void testGlobalOrdinals() throws Exception { assertThat(ord, equalTo(7L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("06")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); assertTrue(values.advanceExact(1)); ord = values.nextOrd(); assertThat(ord, equalTo(7L)); @@ -541,7 +542,7 @@ public void testGlobalOrdinals() throws Exception { assertThat(ord, equalTo(9L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("08")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); assertFalse(values.advanceExact(2)); assertTrue(values.advanceExact(3)); ord = values.nextOrd(); @@ -554,7 +555,7 @@ public void testGlobalOrdinals() throws Exception { assertThat(ord, equalTo(11L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("10")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); // Third segment leaf = topLevelReader.leaves().get(2); @@ -571,7 +572,7 @@ public void testGlobalOrdinals() throws Exception { assertThat(ord, equalTo(2L)); assertThat(values.lookupOrd(ord).utf8ToString(), equalTo("!10")); ord = values.nextOrd(); - assertThat(ord, equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(ord, equalTo((long) DocIdSetIterator.NO_MORE_DOCS)); } public void testTermsEnum() throws Exception { diff --git a/server/src/test/java/org/opensearch/index/fielddata/ordinals/MultiOrdinalsTests.java b/server/src/test/java/org/opensearch/index/fielddata/ordinals/MultiOrdinalsTests.java index 2f9c2499cba04..64cb953a27cf3 100644 --- a/server/src/test/java/org/opensearch/index/fielddata/ordinals/MultiOrdinalsTests.java +++ b/server/src/test/java/org/opensearch/index/fielddata/ordinals/MultiOrdinalsTests.java @@ -131,7 +131,7 @@ public int compare(OrdAndId o1, OrdAndId o2) { for (Long ord : docOrds) { assertThat(docs.nextOrd(), equalTo(ord)); } - assertEquals(SortedSetDocValues.NO_MORE_ORDS, docs.nextOrd()); + assertEquals(SortedSetDocValues.NO_MORE_DOCS, docs.nextOrd()); } for (int i = docId + 1; i < ordAndId.id; i++) { assertFalse(singleOrds.advanceExact(i)); @@ -280,7 +280,7 @@ private void assertEquals(SortedSetDocValues docs, long[][] ordinalPlan) throws for (long ord : ords) { assertThat(docs.nextOrd(), equalTo(ord)); } - assertThat(docs.nextOrd(), equalTo(SortedSetDocValues.NO_MORE_ORDS)); + assertThat(docs.nextOrd(), equalTo((long) SortedSetDocValues.NO_MORE_DOCS)); } } } diff --git a/server/src/test/java/org/opensearch/index/fieldstats/FieldStatsProviderRefreshTests.java b/server/src/test/java/org/opensearch/index/fieldstats/FieldStatsProviderRefreshTests.java index 16ce40d588a1b..7529efe51f0c7 100644 --- a/server/src/test/java/org/opensearch/index/fieldstats/FieldStatsProviderRefreshTests.java +++ b/server/src/test/java/org/opensearch/index/fieldstats/FieldStatsProviderRefreshTests.java @@ -82,7 +82,7 @@ public void testQueryRewriteOnRefresh() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("a").lte("g")) .get(); assertSearchResponse(r1); - assertThat(r1.getHits().getTotalHits().value, equalTo(3L)); + assertThat(r1.getHits().getTotalHits().value(), equalTo(3L)); assertRequestCacheStats(0, 1); // Search again and check it hits the cache @@ -92,7 +92,7 @@ public void testQueryRewriteOnRefresh() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("a").lte("g")) .get(); assertSearchResponse(r2); - assertThat(r2.getHits().getTotalHits().value, equalTo(3L)); + assertThat(r2.getHits().getTotalHits().value(), equalTo(3L)); assertRequestCacheStats(1, 1); // Index some more documents in the query range and refresh @@ -107,7 +107,7 @@ public void testQueryRewriteOnRefresh() throws Exception { .setQuery(QueryBuilders.rangeQuery("s").gte("a").lte("g")) .get(); assertSearchResponse(r3); - assertThat(r3.getHits().getTotalHits().value, equalTo(5L)); + assertThat(r3.getHits().getTotalHits().value(), equalTo(5L)); assertRequestCacheStats(1, 2); } diff --git a/server/src/test/java/org/opensearch/index/mapper/BooleanFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/BooleanFieldTypeTests.java index aab63fc30efd7..2ddec0c628bac 100644 --- a/server/src/test/java/org/opensearch/index/mapper/BooleanFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/BooleanFieldTypeTests.java @@ -34,7 +34,7 @@ import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.Term; import org.apache.lucene.search.BoostQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.TermQuery; import org.apache.lucene.util.BytesRef; @@ -85,7 +85,7 @@ public void testTermsQuery() { List terms = new ArrayList<>(); terms.add(new BytesRef("true")); terms.add(new BytesRef("false")); - assertEquals(new DocValuesFieldExistsQuery("field"), ft.termsQuery(terms, null)); + assertEquals(new FieldExistsQuery("field"), ft.termsQuery(terms, null)); List newTerms = new ArrayList<>(); newTerms.add(new BytesRef("true")); @@ -108,7 +108,7 @@ public void testTermsQuery() { public void testRangeQuery() { BooleanFieldMapper.BooleanFieldType ft = new BooleanFieldMapper.BooleanFieldType("field"); - assertEquals(new DocValuesFieldExistsQuery("field"), ft.rangeQuery(false, true, true, true, null)); + assertEquals(new FieldExistsQuery("field"), ft.rangeQuery(false, true, true, true, null)); assertEquals(new TermQuery(new Term("field", "T")), ft.rangeQuery(false, true, false, true, null)); @@ -122,7 +122,7 @@ public void testRangeQuery() { assertEquals(new TermQuery(new Term("field", "F")), ft.rangeQuery(null, false, true, true, null)); - assertEquals(new DocValuesFieldExistsQuery("field"), ft.rangeQuery(false, null, true, true, null)); + assertEquals(new FieldExistsQuery("field"), ft.rangeQuery(false, null, true, true, null)); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> ft.rangeQuery("random", null, true, true, null)); diff --git a/server/src/test/java/org/opensearch/index/mapper/DateFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/DateFieldTypeTests.java index 52091d571ee72..febdb17201edd 100644 --- a/server/src/test/java/org/opensearch/index/mapper/DateFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/DateFieldTypeTests.java @@ -595,10 +595,10 @@ public void testDateResolutionForOverflow() throws IOException { ); TopDocs topDocs = searcher.search(rangeQuery, dates.size()); - assertEquals("Number of non-null date documents", dates.size() - numNullDates, topDocs.totalHits.value); + assertEquals("Number of non-null date documents", dates.size() - numNullDates, topDocs.totalHits.value()); for (ScoreDoc scoreDoc : topDocs.scoreDocs) { - org.apache.lucene.document.Document doc = reader.document(scoreDoc.doc); + org.apache.lucene.document.Document doc = reader.storedFields().document(scoreDoc.doc); IndexableField dateField = doc.getField(ft.name()); if (dateField != null) { long dateValue = dateField.numericValue().longValue(); @@ -622,7 +622,7 @@ public void testDateResolutionForOverflow() throws IOException { Query nullValueQuery = ftWithNullValue.termQuery("2020-01-01T00:00:00Z", context); topDocs = searcher.search(nullValueQuery, dates.size()); - assertEquals("Documents matching the 2020-01-01 date", 1, topDocs.totalHits.value); + assertEquals("Documents matching the 2020-01-01 date", 1, topDocs.totalHits.value()); IOUtils.close(reader, w, dir); } @@ -672,9 +672,9 @@ public void testDateFieldTypeWithNulls() throws IOException { for (int i = 0; i < 100; i++) { TopDocs topDocs = searcher.search(queryBuilder.build(), nullDocs + datedDocs, sort); - assertEquals("Total hits should match total documents", nullDocs + datedDocs, topDocs.totalHits.value); + assertEquals("Total hits should match total documents", nullDocs + datedDocs, topDocs.totalHits.value()); for (ScoreDoc scoreDoc : topDocs.scoreDocs) { - org.apache.lucene.document.Document doc = reader.document(scoreDoc.doc); + org.apache.lucene.document.Document doc = reader.storedFields().document(scoreDoc.doc); IndexableField dateField = doc.getField(ft.name()); if (dateField != null) { long dateValue = dateField.numericValue().longValue(); diff --git a/server/src/test/java/org/opensearch/index/mapper/DerivedFieldMapperQueryTests.java b/server/src/test/java/org/opensearch/index/mapper/DerivedFieldMapperQueryTests.java index c744f2592e24f..34652b8703034 100644 --- a/server/src/test/java/org/opensearch/index/mapper/DerivedFieldMapperQueryTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/DerivedFieldMapperQueryTests.java @@ -374,67 +374,67 @@ public void execute() { IndexSearcher searcher = new IndexSearcher(reader); Query query = QueryBuilders.termQuery("request_succeeded", "true").toQuery(queryShardContext); TopDocs topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); // IP Field Term Query scriptIndex[0] = 3; query = QueryBuilders.termQuery("client_ip", "192.168.0.0/16").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); scriptIndex[0] = 4; query = QueryBuilders.termsQuery("method", "DELETE", "PUT").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(3, topDocs.totalHits.value); + assertEquals(3, topDocs.totalHits.value()); query = QueryBuilders.termsQuery("method", "delete").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); query = QueryBuilders.termQuery("method", "delete").caseInsensitive(true).toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); // Range queries of types - date, long and double scriptIndex[0] = 2; query = QueryBuilders.rangeQuery("@timestamp").from("2024-03-20T14:20:50").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); scriptIndex[0] = 5; query = QueryBuilders.rangeQuery("request_size").from("4.1").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); scriptIndex[0] = 6; query = QueryBuilders.rangeQuery("duration").from("5800").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(6, topDocs.totalHits.value); + assertEquals(6, topDocs.totalHits.value()); scriptIndex[0] = 4; // Prefix Query query = QueryBuilders.prefixQuery("method", "DE").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); scriptIndex[0] = 4; query = QueryBuilders.wildcardQuery("method", "G*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // Regexp Query scriptIndex[0] = 4; query = QueryBuilders.regexpQuery("method", ".*LET.*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); // GeoPoint Query scriptIndex[0] = 7; query = geoShapeQuery("geopoint", new Rectangle(0.0, 55.0, 55.0, 0.0)).toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); } } } @@ -532,7 +532,7 @@ public void execute() { termQueryBuilder.caseInsensitive(true); Query query = termQueryBuilder.toQuery(queryShardContext); TopDocs topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // since last doc has a malformed json, if ignore_malformed isn't set or set as false, the query should fail termQueryBuilder = new TermQueryBuilder("object_field_without_ignored_malformed.keyword_field", "GET"); @@ -543,85 +543,85 @@ public void execute() { query = QueryBuilders.matchPhraseQuery("object_field.text_field", "document number 1").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); query = QueryBuilders.matchPhraseQuery("object_field.text_field", "document number 11").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); query = new MatchPhrasePrefixQueryBuilder("object_field.text_field", "document number").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); // Multi Phrase Query query = QueryBuilders.multiMatchQuery("GET", "object_field.nested_field.sub_field_1", "object_field.keyword_field") .type(MultiMatchQueryBuilder.Type.PHRASE) .toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // Range queries of types - date, long and double query = QueryBuilders.rangeQuery("object_field.date_field").from("2024-03-20T14:20:50").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.float_field").from("4.1").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.long_field").from("5800").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(6, topDocs.totalHits.value); + assertEquals(6, topDocs.totalHits.value()); // Prefix Query query = QueryBuilders.prefixQuery("object_field.keyword_field", "de").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); query = QueryBuilders.wildcardQuery("object_field.keyword_field", "g*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); QueryStringQueryParser queryParser = new QueryStringQueryParser(queryShardContext, "object_field.keyword_field"); queryParser.parse("GE?"); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // Regexp Query query = QueryBuilders.regexpQuery("object_field.keyword_field", ".*let.*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); // tested deep nested field query = QueryBuilders.regexpQuery("object_field.nested_field.sub_field_1", ".*let.*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); // Test nested array field query = QueryBuilders.rangeQuery("object_field.array_field").from("1").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(10, topDocs.totalHits.value); + assertEquals(10, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.array_field").from("3").to("6").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.array_field").from("9").to("9").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.array_field").from("10").to("12").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("object_field.array_field").from("31").to("50").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); // tested missing nested field query = QueryBuilders.regexpQuery("object_field.invalid_field.sub_field", ".*let.*").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); } } } @@ -774,7 +774,7 @@ public void execute() { termQueryBuilder.caseInsensitive(true); Query query = termQueryBuilder.toQuery(queryShardContext); TopDocs topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // test object_field_2 docsEvaluated[0] = 0; @@ -782,7 +782,7 @@ public void execute() { termQueryBuilder.caseInsensitive(true); query = termQueryBuilder.toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // since we have prefilter_field set to "raw_message", it should not evaluate all documents assertEquals(8, docsEvaluated[0]); @@ -801,20 +801,20 @@ public void execute() { // prefilter_field query = QueryBuilders.rangeQuery("invalid_object.date_field").from("2024-03-20T14:20:50").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); query = QueryBuilders.rangeQuery("long_prefilter_field_object.date_field") .from("2024-03-20T14:20:50") .toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(4, topDocs.totalHits.value); + assertEquals(4, topDocs.totalHits.value()); // test regular_derived_field docsEvaluated[0] = 0; scriptIndex[0] = 4; query = QueryBuilders.termQuery("regular_derived_field", "delete").caseInsensitive(true).toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); assertEquals(2, docsEvaluated[0]); // test regular_derived_field_without_prefilter_field @@ -824,7 +824,7 @@ public void execute() { .caseInsensitive(true) .toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(2, topDocs.totalHits.value); + assertEquals(2, topDocs.totalHits.value()); assertEquals(11, docsEvaluated[0]); } } @@ -950,7 +950,7 @@ public void execute() { termQueryBuilder.caseInsensitive(true); Query query = termQueryBuilder.toQuery(queryShardContext); TopDocs topDocs = searcher.search(query, 10); - assertEquals(7, topDocs.totalHits.value); + assertEquals(7, topDocs.totalHits.value()); // check if ignoreMalformed is set to false, the query fails on malformed values query = QueryBuilders.rangeQuery("object_field_3.keyword_field").from("2024-03-20T14:20:50").toQuery(queryShardContext); @@ -960,7 +960,7 @@ public void execute() { // check if ignoreMalformed is set to true, the query passes with 0 results query = QueryBuilders.rangeQuery("object_field_2.keyword_field").from("2024-03-20T14:20:50").toQuery(queryShardContext); topDocs = searcher.search(query, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); } } } diff --git a/server/src/test/java/org/opensearch/index/mapper/DerivedFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/DerivedFieldTypeTests.java index 7da8c9eb1efa0..8af8812ebf6ce 100644 --- a/server/src/test/java/org/opensearch/index/mapper/DerivedFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/DerivedFieldTypeTests.java @@ -14,7 +14,6 @@ import org.apache.lucene.document.KeywordField; import org.apache.lucene.document.LatLonPoint; import org.apache.lucene.document.LongField; -import org.apache.lucene.document.LongPoint; import org.apache.lucene.index.LeafReaderContext; import org.apache.lucene.index.memory.MemoryIndex; import org.apache.lucene.queries.spans.SpanMultiTermQueryWrapper; @@ -66,7 +65,7 @@ public void testBooleanType() { public void testDateType() { DerivedFieldType dft = createDerivedFieldType("date"); assertTrue(dft.getFieldMapper() instanceof DateFieldMapper); - assertTrue(dft.getIndexableFieldGenerator().apply(System.currentTimeMillis()) instanceof LongPoint); + assertTrue(dft.getIndexableFieldGenerator().apply(System.currentTimeMillis()) instanceof LongField); expectThrows(Exception.class, () -> dft.getIndexableFieldGenerator().apply("blah")); } @@ -83,7 +82,7 @@ public void testGeoPointType() { public void testIPType() { DerivedFieldType dft = createDerivedFieldType("ip"); assertTrue(dft.getFieldMapper() instanceof IpFieldMapper); - assertTrue(dft.getIndexableFieldGenerator().apply("127.0.0.1") instanceof InetAddressPoint); + assertTrue(dft.getIndexableFieldGenerator().apply("127.0.0.1") instanceof IpFieldMapper.InetAddressField); expectThrows(Exception.class, () -> dft.getIndexableFieldGenerator().apply("blah")); } diff --git a/server/src/test/java/org/opensearch/index/mapper/DocCountFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/DocCountFieldTypeTests.java index 4961736ea933c..74e45f1a30763 100644 --- a/server/src/test/java/org/opensearch/index/mapper/DocCountFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/DocCountFieldTypeTests.java @@ -31,7 +31,7 @@ */ package org.opensearch.index.mapper; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.opensearch.index.query.QueryShardException; import java.io.IOException; @@ -56,7 +56,7 @@ public void testRangeQuery() { public void testExistsQuery() { MappedFieldType ft = new DocCountFieldMapper.DocCountFieldType(); - assertTrue(ft.existsQuery(randomMockShardContext()) instanceof DocValuesFieldExistsQuery); + assertTrue(ft.existsQuery(randomMockShardContext()) instanceof FieldExistsQuery); } public void testFetchSourceValue() throws IOException { diff --git a/server/src/test/java/org/opensearch/index/mapper/DoubleIndexingDocTests.java b/server/src/test/java/org/opensearch/index/mapper/DoubleIndexingDocTests.java index 55847a15b577f..c8e07729aab90 100644 --- a/server/src/test/java/org/opensearch/index/mapper/DoubleIndexingDocTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/DoubleIndexingDocTests.java @@ -61,25 +61,25 @@ public void testDoubleIndexingSameDoc() throws Exception { }, reader -> { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(mapperService.fieldType("field1").termQuery("value1", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field2").termQuery("1", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field3").termQuery("1.1", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field4").termQuery("2010-01-01", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field5").termQuery("1", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field5").termQuery("2", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); topDocs = searcher.search(mapperService.fieldType("field5").termQuery("3", qsc), 10); - assertThat(topDocs.totalHits.value, equalTo(2L)); + assertThat(topDocs.totalHits.value(), equalTo(2L)); }); } } diff --git a/server/src/test/java/org/opensearch/index/mapper/FlatObjectFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/FlatObjectFieldTypeTests.java index 5b158b204b289..600f04edcd552 100644 --- a/server/src/test/java/org/opensearch/index/mapper/FlatObjectFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/FlatObjectFieldTypeTests.java @@ -42,6 +42,7 @@ import static org.opensearch.index.mapper.FlatObjectFieldMapper.VALUE_AND_PATH_SUFFIX; import static org.opensearch.index.mapper.FlatObjectFieldMapper.VALUE_SUFFIX; +import static org.apache.lucene.search.MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE; import static org.apache.lucene.search.MultiTermQuery.CONSTANT_SCORE_REWRITE; import static org.apache.lucene.search.MultiTermQuery.DOC_VALUES_REWRITE; @@ -619,8 +620,8 @@ public void testPrefixQuery() { true, false ); - Query expected = new PrefixQuery(new Term("field" + VALUE_SUFFIX, "foo"), CONSTANT_SCORE_REWRITE); - assertEquals(expected, ft.prefixQuery("foo", CONSTANT_SCORE_REWRITE, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); + Query expected = new PrefixQuery(new Term("field" + VALUE_SUFFIX, "foo"), CONSTANT_SCORE_BLENDED_REWRITE); + assertEquals(expected, ft.prefixQuery("foo", CONSTANT_SCORE_BLENDED_REWRITE, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } // test isSearchable=true, hasDocValues=false, mappedFieldTypeName!=null @@ -644,7 +645,7 @@ public void testPrefixQuery() { true ); Query expected = new PrefixQuery(new Term("field" + VALUE_SUFFIX, "field.foo"), DOC_VALUES_REWRITE); - assertEquals(expected, ft.prefixQuery("foo", CONSTANT_SCORE_REWRITE, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); + assertEquals(expected, ft.prefixQuery("foo", CONSTANT_SCORE_BLENDED_REWRITE, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } // test isSearchable=false, hasDocValues=true, mappedFieldTypeName!=null @@ -712,7 +713,7 @@ public void testRegexpQuery() { 0, RegexpQuery.DEFAULT_PROVIDER, 10, - CONSTANT_SCORE_REWRITE + CONSTANT_SCORE_BLENDED_REWRITE ), new RegexpQuery( new Term("field" + VALUE_SUFFIX, new BytesRef("field.foo")), @@ -741,7 +742,7 @@ public void testRegexpQuery() { 0, RegexpQuery.DEFAULT_PROVIDER, 10, - CONSTANT_SCORE_REWRITE + CONSTANT_SCORE_BLENDED_REWRITE ), new RegexpQuery( new Term("field" + VALUE_AND_PATH_SUFFIX, new BytesRef("field.field.field1=foo")), @@ -769,7 +770,7 @@ public void testRegexpQuery() { 0, RegexpQuery.DEFAULT_PROVIDER, 10, - CONSTANT_SCORE_REWRITE + CONSTANT_SCORE_BLENDED_REWRITE ); assertEquals(expected, ft.regexpQuery("foo", 0, 0, 10, null, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } @@ -788,7 +789,7 @@ public void testRegexpQuery() { 0, RegexpQuery.DEFAULT_PROVIDER, 10, - CONSTANT_SCORE_REWRITE + CONSTANT_SCORE_BLENDED_REWRITE ); assertEquals(expected, ft.regexpQuery("foo", 0, 0, 10, null, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } @@ -1136,13 +1137,7 @@ public void testRangeQuery() { true, true ); - Query dvQuery = new AutomatonQuery( - new Term("field" + VALUE_SUFFIX), - dvAutomaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - true, - DOC_VALUES_REWRITE - ); + Query dvQuery = new AutomatonQuery(new Term("field" + VALUE_SUFFIX), dvAutomaton, true, DOC_VALUES_REWRITE); Query expected = searchable == false ? dvQuery : (hasDocValue ? new IndexOrDocValuesQuery(indexQuery, dvQuery) : indexQuery); @@ -1201,19 +1196,8 @@ public void testRangeQuery() { true ); Automaton dvAutomaton = Operations.intersection(dvA1, dvA2); - Query indexQuery = new AutomatonQuery( - new Term("field" + VALUE_AND_PATH_SUFFIX), - termAutomaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - true - ); - Query dvQuery = new AutomatonQuery( - new Term("field" + VALUE_AND_PATH_SUFFIX), - dvAutomaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - true, - DOC_VALUES_REWRITE - ); + Query indexQuery = new AutomatonQuery(new Term("field" + VALUE_AND_PATH_SUFFIX), termAutomaton, true); + Query dvQuery = new AutomatonQuery(new Term("field" + VALUE_AND_PATH_SUFFIX), dvAutomaton, true, DOC_VALUES_REWRITE); Query expected = searchable == false ? dvQuery : (hasDocValue ? new IndexOrDocValuesQuery(indexQuery, dvQuery) : indexQuery); @@ -1245,7 +1229,7 @@ public void testWildcardQuery() { new WildcardQuery( new Term("field" + VALUE_SUFFIX, "foo*"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ), new WildcardQuery( new Term("field" + VALUE_SUFFIX, "field.foo*"), @@ -1268,7 +1252,7 @@ public void testWildcardQuery() { new WildcardQuery( new Term("field" + VALUE_AND_PATH_SUFFIX, "field.field1=foo*"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ), new WildcardQuery( new Term("field" + VALUE_AND_PATH_SUFFIX, "field.field.field1=foo*"), @@ -1290,7 +1274,7 @@ public void testWildcardQuery() { Query expected = new WildcardQuery( new Term("field" + VALUE_SUFFIX, "foo*"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ); assertEquals(expected, ft.wildcardQuery("foo*", null, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } @@ -1306,7 +1290,7 @@ public void testWildcardQuery() { Query expected = new WildcardQuery( new Term("field" + VALUE_AND_PATH_SUFFIX, "field.field1=foo*"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ); assertEquals(expected, ft.wildcardQuery("foo*", null, false, MOCK_QSC_ENABLE_INDEX_DOC_VALUES)); } diff --git a/server/src/test/java/org/opensearch/index/mapper/IdFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/IdFieldTypeTests.java index 3fb6be2203cd1..0f93f229dc0fa 100644 --- a/server/src/test/java/org/opensearch/index/mapper/IdFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/IdFieldTypeTests.java @@ -41,6 +41,8 @@ import org.opensearch.index.query.QueryShardContext; import org.opensearch.test.OpenSearchTestCase; +import java.util.List; + import org.mockito.Mockito; public class IdFieldTypeTests extends OpenSearchTestCase { @@ -72,10 +74,10 @@ public void testTermsQuery() { MappedFieldType ft = new IdFieldMapper.IdFieldType(() -> false); Query query = ft.termQuery("id", context); - assertEquals(new TermInSetQuery("_id", Uid.encodeId("id")), query); + assertEquals(new TermInSetQuery("_id", List.of(Uid.encodeId("id"))), query); query = ft.termQuery("id", context); - assertEquals(new TermInSetQuery("_id", Uid.encodeId("id")), query); + assertEquals(new TermInSetQuery("_id", List.of(Uid.encodeId("id"))), query); } public void testIsAggregatable() { diff --git a/server/src/test/java/org/opensearch/index/mapper/IgnoredFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/IgnoredFieldTypeTests.java index ef1530557d587..d93e14f603002 100644 --- a/server/src/test/java/org/opensearch/index/mapper/IgnoredFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/IgnoredFieldTypeTests.java @@ -48,7 +48,7 @@ public class IgnoredFieldTypeTests extends FieldTypeTestCase { public void testPrefixQuery() { MappedFieldType ft = IgnoredFieldMapper.IgnoredFieldType.INSTANCE; - Query expected = new PrefixQuery(new Term("_ignored", new BytesRef("foo*")), MultiTermQuery.CONSTANT_SCORE_REWRITE); + Query expected = new PrefixQuery(new Term("_ignored", new BytesRef("foo*")), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE); assertEquals(expected, ft.prefixQuery("foo*", null, MOCK_QSC)); OpenSearchException ee = expectThrows(OpenSearchException.class, () -> ft.prefixQuery("foo*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); @@ -68,7 +68,7 @@ public void testRegexpQuery() { 0, RegexpQuery.DEFAULT_PROVIDER, Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ); assertEquals(expected, ft.regexpQuery("foo?", 0, 0, 10, null, MOCK_QSC)); diff --git a/server/src/test/java/org/opensearch/index/mapper/IpFieldMapperTests.java b/server/src/test/java/org/opensearch/index/mapper/IpFieldMapperTests.java index 0f30e93f0622a..333f151997814 100644 --- a/server/src/test/java/org/opensearch/index/mapper/IpFieldMapperTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/IpFieldMapperTests.java @@ -84,16 +84,13 @@ public void testDefaults() throws Exception { ParsedDocument doc = mapper.parse(source(b -> b.field("field", "::1"))); IndexableField[] fields = doc.rootDoc().getFields("field"); - assertEquals(2, fields.length); - IndexableField pointField = fields[0]; - assertEquals(1, pointField.fieldType().pointIndexDimensionCount()); - assertEquals(16, pointField.fieldType().pointNumBytes()); - assertFalse(pointField.fieldType().stored()); - assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointField.binaryValue()); - IndexableField dvField = fields[1]; - assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType()); - assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), dvField.binaryValue()); - assertFalse(dvField.fieldType().stored()); + assertEquals(1, fields.length); + IndexableField pointFieldAndDVField = fields[0]; + assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount()); + assertEquals(16, pointFieldAndDVField.fieldType().pointNumBytes()); + assertFalse(pointFieldAndDVField.fieldType().stored()); + assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointFieldAndDVField.binaryValue()); + assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType()); } public void testNotIndexed() throws Exception { @@ -145,12 +142,11 @@ public void testStore() throws Exception { ParsedDocument doc = mapper.parse(source(b -> b.field("field", "::1"))); IndexableField[] fields = doc.rootDoc().getFields("field"); - assertEquals(3, fields.length); - IndexableField pointField = fields[0]; - assertEquals(1, pointField.fieldType().pointIndexDimensionCount()); - IndexableField dvField = fields[1]; - assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType()); - IndexableField storedField = fields[2]; + assertEquals(2, fields.length); + IndexableField pointFieldAndDVField = fields[0]; + assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount()); + assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType()); + IndexableField storedField = fields[1]; assertTrue(storedField.fieldType().stored()); assertEquals(new BytesRef(InetAddressPoint.encode(InetAddress.getByName("::1"))), storedField.binaryValue()); } @@ -190,16 +186,13 @@ public void testNullValue() throws IOException { doc = mapper.parse(source(b -> b.nullField("field"))); IndexableField[] fields = doc.rootDoc().getFields("field"); - assertEquals(2, fields.length); - IndexableField pointField = fields[0]; - assertEquals(1, pointField.fieldType().pointIndexDimensionCount()); - assertEquals(16, pointField.fieldType().pointNumBytes()); - assertFalse(pointField.fieldType().stored()); - assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointField.binaryValue()); - IndexableField dvField = fields[1]; - assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType()); - assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), dvField.binaryValue()); - assertFalse(dvField.fieldType().stored()); + assertEquals(1, fields.length); + IndexableField pointFieldAndDVField = fields[0]; + assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount()); + assertEquals(16, pointFieldAndDVField.fieldType().pointNumBytes()); + assertFalse(pointFieldAndDVField.fieldType().stored()); + assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointFieldAndDVField.binaryValue()); + assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType()); mapper = createDocumentMapper(fieldMapping(b -> { b.field("type", "ip"); diff --git a/server/src/test/java/org/opensearch/index/mapper/IpFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/IpFieldTypeTests.java index 243164c5fe8fb..ffb7bc5acea08 100644 --- a/server/src/test/java/org/opensearch/index/mapper/IpFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/IpFieldTypeTests.java @@ -210,7 +210,7 @@ public void testTermsQuery() { assertTrue(((ConstantScoreQuery) actual).getQuery() instanceof BooleanQuery); BooleanQuery bq = (BooleanQuery) ((ConstantScoreQuery) actual).getQuery(); assertEquals(2, bq.clauses().size()); - assertTrue(bq.clauses().stream().allMatch(c -> c.getOccur() == Occur.SHOULD)); + assertTrue(bq.clauses().stream().allMatch(c -> c.occur() == Occur.SHOULD)); } public void testDvOnlyTermsQuery() { diff --git a/server/src/test/java/org/opensearch/index/mapper/KeywordFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/KeywordFieldTypeTests.java index d52426c67d256..13ee47ba3f8af 100644 --- a/server/src/test/java/org/opensearch/index/mapper/KeywordFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/KeywordFieldTypeTests.java @@ -43,11 +43,10 @@ import org.apache.lucene.document.FieldType; import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.FuzzyQuery; import org.apache.lucene.search.IndexOrDocValuesQuery; import org.apache.lucene.search.MultiTermQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.RegexpQuery; import org.apache.lucene.search.TermInSetQuery; @@ -201,13 +200,13 @@ public void testTermsQuery() { public void testExistsQuery() { { KeywordFieldType ft = new KeywordFieldType("field"); - assertEquals(new DocValuesFieldExistsQuery("field"), ft.existsQuery(null)); + assertEquals(new FieldExistsQuery("field"), ft.existsQuery(null)); } { FieldType fieldType = new FieldType(); fieldType.setOmitNorms(false); KeywordFieldType ft = new KeywordFieldType("field", fieldType); - assertEquals(new NormsFieldExistsQuery("field"), ft.existsQuery(null)); + assertEquals(new FieldExistsQuery("field"), ft.existsQuery(null)); } { KeywordFieldType ft = new KeywordFieldType("field", true, false, Collections.emptyMap()); @@ -311,9 +310,12 @@ public void testFuzzyQuery() { ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, null, MOCK_QSC_ENABLE_INDEX_DOC_VALUES) ); - Query indexExpected = new FuzzyQuery(new Term("field", "foo"), 2, 1, 50, true); + Query indexExpected = new FuzzyQuery(new Term("field", "foo"), 2, 1, 50, true, MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE); MappedFieldType onlyIndexed = new KeywordFieldType("field", true, false, Collections.emptyMap()); - assertEquals(indexExpected, onlyIndexed.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC)); + assertEquals( + indexExpected, + onlyIndexed.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, MOCK_QSC) + ); Query dvExpected = new FuzzyQuery(new Term("field", "foo"), 2, 1, 50, true, MultiTermQuery.DOC_VALUES_REWRITE); MappedFieldType onlyDocValues = new KeywordFieldType("field", false, true, Collections.emptyMap()); @@ -334,7 +336,15 @@ public void testFuzzyQuery() { OpenSearchException ee = expectThrows( OpenSearchException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.AUTO, randomInt(10) + 1, randomInt(10) + 1, randomBoolean(), MOCK_QSC_DISALLOW_EXPENSIVE) + () -> ft.fuzzyQuery( + "foo", + Fuzziness.AUTO, + randomInt(10) + 1, + randomInt(10) + 1, + randomBoolean(), + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, + MOCK_QSC_DISALLOW_EXPENSIVE + ) ); assertEquals("[fuzzy] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", ee.getMessage()); } diff --git a/server/src/test/java/org/opensearch/index/mapper/StoredNumericValuesTests.java b/server/src/test/java/org/opensearch/index/mapper/StoredNumericValuesTests.java index cebd92a280556..07b64a9cdc58f 100644 --- a/server/src/test/java/org/opensearch/index/mapper/StoredNumericValuesTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/StoredNumericValuesTests.java @@ -158,7 +158,7 @@ public void testBytesAndNumericRepresentation() throws Exception { "field11" ); CustomFieldsVisitor fieldsVisitor = new CustomFieldsVisitor(fieldNames, false); - searcher.doc(0, fieldsVisitor); + searcher.storedFields().document(0, fieldsVisitor); fieldsVisitor.postProcess(mapperService::fieldType); assertThat(fieldsVisitor.fields().size(), equalTo(11)); diff --git a/server/src/test/java/org/opensearch/index/mapper/TextFieldTypeTests.java b/server/src/test/java/org/opensearch/index/mapper/TextFieldTypeTests.java index e672f94819541..130821df8dd56 100644 --- a/server/src/test/java/org/opensearch/index/mapper/TextFieldTypeTests.java +++ b/server/src/test/java/org/opensearch/index/mapper/TextFieldTypeTests.java @@ -37,6 +37,7 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.ConstantScoreQuery; import org.apache.lucene.search.FuzzyQuery; +import org.apache.lucene.search.MultiTermQuery; import org.apache.lucene.search.PrefixQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.RegexpQuery; @@ -147,20 +148,36 @@ public void testRegexpQuery() { public void testFuzzyQuery() { MappedFieldType ft = createFieldType(true); assertEquals( - new FuzzyQuery(new Term("field", "foo"), 2, 1, 50, true), - ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC) + new FuzzyQuery(new Term("field", "foo"), 2, 1, 50, true, CONSTANT_SCORE_BLENDED_REWRITE), + ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, MOCK_QSC) ); MappedFieldType unsearchable = createFieldType(false); IllegalArgumentException e = expectThrows( IllegalArgumentException.class, - () -> unsearchable.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC) + () -> unsearchable.fuzzyQuery( + "foo", + Fuzziness.fromEdits(2), + 1, + 50, + true, + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, + MOCK_QSC + ) ); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); OpenSearchException ee = expectThrows( OpenSearchException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.AUTO, randomInt(10) + 1, randomInt(10) + 1, randomBoolean(), MOCK_QSC_DISALLOW_EXPENSIVE) + () -> ft.fuzzyQuery( + "foo", + Fuzziness.AUTO, + randomInt(10) + 1, + randomInt(10) + 1, + randomBoolean(), + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE, + MOCK_QSC_DISALLOW_EXPENSIVE + ) ); assertEquals("[fuzzy] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", ee.getMessage()); } @@ -194,13 +211,7 @@ public void testIndexPrefixes() { Query expected = new ConstantScoreQuery( new BooleanQuery.Builder().add( - new AutomatonQuery( - new Term("field._index_prefix", "g*"), - automaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - false, - CONSTANT_SCORE_REWRITE - ), + new AutomatonQuery(new Term("field._index_prefix", "g*"), automaton, false, CONSTANT_SCORE_REWRITE), BooleanClause.Occur.SHOULD ).add(new TermQuery(new Term("field", "g")), BooleanClause.Occur.SHOULD).build() ); @@ -212,13 +223,7 @@ public void testIndexPrefixes() { expected = new ConstantScoreQuery( new BooleanQuery.Builder().add( - new AutomatonQuery( - new Term("field._index_prefix", "g*"), - automaton, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - false, - CONSTANT_SCORE_REWRITE - ), + new AutomatonQuery(new Term("field._index_prefix", "g*"), automaton, false, CONSTANT_SCORE_BLENDED_REWRITE), BooleanClause.Occur.SHOULD ).add(new TermQuery(new Term("field", "g")), BooleanClause.Occur.SHOULD).build() ); diff --git a/server/src/test/java/org/opensearch/index/query/BoolQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/BoolQueryBuilderTests.java index d0f26f3026789..a23dff39c6496 100644 --- a/server/src/test/java/org/opensearch/index/query/BoolQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/BoolQueryBuilderTests.java @@ -201,15 +201,15 @@ public void testMinShouldMatchFilterWithoutShouldClauses() throws Exception { assertThat(booleanQuery.getMinimumNumberShouldMatch(), equalTo(0)); assertThat(booleanQuery.clauses().size(), equalTo(1)); BooleanClause booleanClause = booleanQuery.clauses().get(0); - assertThat(booleanClause.getOccur(), equalTo(BooleanClause.Occur.FILTER)); - assertThat(booleanClause.getQuery(), instanceOf(BooleanQuery.class)); - BooleanQuery innerBooleanQuery = (BooleanQuery) booleanClause.getQuery(); + assertThat(booleanClause.occur(), equalTo(BooleanClause.Occur.FILTER)); + assertThat(booleanClause.query(), instanceOf(BooleanQuery.class)); + BooleanQuery innerBooleanQuery = (BooleanQuery) booleanClause.query(); // we didn't set minimum should match initially, there are no should clauses so it should be 0 assertThat(innerBooleanQuery.getMinimumNumberShouldMatch(), equalTo(0)); assertThat(innerBooleanQuery.clauses().size(), equalTo(1)); BooleanClause innerBooleanClause = innerBooleanQuery.clauses().get(0); - assertThat(innerBooleanClause.getOccur(), equalTo(BooleanClause.Occur.MUST)); - assertThat(innerBooleanClause.getQuery(), instanceOf(MatchAllDocsQuery.class)); + assertThat(innerBooleanClause.occur(), equalTo(BooleanClause.Occur.MUST)); + assertThat(innerBooleanClause.query(), instanceOf(MatchAllDocsQuery.class)); } public void testMinShouldMatchBiggerThanNumberOfShouldClauses() throws Exception { diff --git a/server/src/test/java/org/opensearch/index/query/DerivedFieldQueryTests.java b/server/src/test/java/org/opensearch/index/query/DerivedFieldQueryTests.java index bed2d22125810..e46b60dc460aa 100644 --- a/server/src/test/java/org/opensearch/index/query/DerivedFieldQueryTests.java +++ b/server/src/test/java/org/opensearch/index/query/DerivedFieldQueryTests.java @@ -106,7 +106,7 @@ public void execute() { iw.close(); IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(derivedFieldQuery, 10); - assertEquals(3, topDocs.totalHits.value); + assertEquals(3, topDocs.totalHits.value()); } } } @@ -177,7 +177,7 @@ public void execute() { ); searcher.search(derivedFieldQuery, 10); TopDocs topDocs = searcher.search(derivedFieldQuery, 10); - assertEquals(0, topDocs.totalHits.value); + assertEquals(0, topDocs.totalHits.value()); } } } diff --git a/server/src/test/java/org/opensearch/index/query/DisMaxQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/DisMaxQueryBuilderTests.java index cb0df38de5c02..425778118495d 100644 --- a/server/src/test/java/org/opensearch/index/query/DisMaxQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/DisMaxQueryBuilderTests.java @@ -106,7 +106,7 @@ public void testToQueryInnerPrefixQuery() throws Exception { + "}"; Query query = parseQuery(queryAsString).toQuery(createShardContext()); Query expected = new DisjunctionMaxQuery( - List.of(new BoostQuery(new PrefixQuery(new Term(TEXT_FIELD_NAME, "sh"), MultiTermQuery.CONSTANT_SCORE_REWRITE), 1.2f)), + List.of(new BoostQuery(new PrefixQuery(new Term(TEXT_FIELD_NAME, "sh"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), 1.2f)), 0 ); assertEquals(expected, query); diff --git a/server/src/test/java/org/opensearch/index/query/DistanceFeatureQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/DistanceFeatureQueryBuilderTests.java index 8489a65d26c91..d80c71d4e928d 100644 --- a/server/src/test/java/org/opensearch/index/query/DistanceFeatureQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/DistanceFeatureQueryBuilderTests.java @@ -33,7 +33,7 @@ package org.opensearch.index.query; import org.apache.lucene.document.LatLonPoint; -import org.apache.lucene.document.LongPoint; +import org.apache.lucene.document.LongField; import org.apache.lucene.search.Query; import org.opensearch.common.geo.GeoPoint; import org.opensearch.common.geo.GeoUtils; @@ -106,7 +106,7 @@ protected void doAssertLuceneQuery(DistanceFeatureQueryBuilder queryBuilder, Que } else { // NANOSECONDS pivotLong = pivotVal.getNanos(); } - expectedQuery = LongPoint.newDistanceFeatureQuery(fieldName, 1.0f, originLong, pivotLong); + expectedQuery = LongField.newDistanceFeatureQuery(fieldName, 1.0f, originLong, pivotLong); } assertEquals(expectedQuery, query); } diff --git a/server/src/test/java/org/opensearch/index/query/ExistsQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/ExistsQueryBuilderTests.java index c3b403e787c2c..910ae16ce3054 100644 --- a/server/src/test/java/org/opensearch/index/query/ExistsQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/ExistsQueryBuilderTests.java @@ -35,9 +35,8 @@ import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchNoDocsQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; import org.opensearch.test.AbstractQueryTestCase; @@ -92,15 +91,15 @@ protected void doAssertLuceneQuery(ExistsQueryBuilder queryBuilder, Query query, assertThat(booleanQuery.clauses().size(), equalTo(childFields.size())); for (int i = 0; i < childFields.size(); i++) { BooleanClause booleanClause = booleanQuery.clauses().get(i); - assertThat(booleanClause.getOccur(), equalTo(BooleanClause.Occur.SHOULD)); + assertThat(booleanClause.occur(), equalTo(BooleanClause.Occur.SHOULD)); } } else if (context.getMapperService().fieldType(field).hasDocValues()) { - assertThat(constantScoreQuery.getQuery(), instanceOf(DocValuesFieldExistsQuery.class)); - DocValuesFieldExistsQuery dvExistsQuery = (DocValuesFieldExistsQuery) constantScoreQuery.getQuery(); + assertThat(constantScoreQuery.getQuery(), instanceOf(FieldExistsQuery.class)); + FieldExistsQuery dvExistsQuery = (FieldExistsQuery) constantScoreQuery.getQuery(); assertEquals(field, dvExistsQuery.getField()); } else if (context.getMapperService().fieldType(field).getTextSearchInfo().hasNorms()) { - assertThat(constantScoreQuery.getQuery(), instanceOf(NormsFieldExistsQuery.class)); - NormsFieldExistsQuery normsExistsQuery = (NormsFieldExistsQuery) constantScoreQuery.getQuery(); + assertThat(constantScoreQuery.getQuery(), instanceOf(FieldExistsQuery.class)); + FieldExistsQuery normsExistsQuery = (FieldExistsQuery) constantScoreQuery.getQuery(); assertEquals(field, normsExistsQuery.getField()); } else { assertThat(constantScoreQuery.getQuery(), instanceOf(TermQuery.class)); @@ -115,7 +114,7 @@ protected void doAssertLuceneQuery(ExistsQueryBuilder queryBuilder, Query query, assertThat(booleanQuery.clauses().size(), equalTo(mappedFields.size())); for (int i = 0; i < mappedFields.size(); i++) { BooleanClause booleanClause = booleanQuery.clauses().get(i); - assertThat(booleanClause.getOccur(), equalTo(BooleanClause.Occur.SHOULD)); + assertThat(booleanClause.occur(), equalTo(BooleanClause.Occur.SHOULD)); } } } diff --git a/server/src/test/java/org/opensearch/index/query/IntervalQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/IntervalQueryBuilderTests.java index 660b8a3aa8994..6ed2af1ceabe1 100644 --- a/server/src/test/java/org/opensearch/index/query/IntervalQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/IntervalQueryBuilderTests.java @@ -36,9 +36,9 @@ import org.apache.lucene.queries.intervals.IntervalQuery; import org.apache.lucene.queries.intervals.Intervals; import org.apache.lucene.queries.intervals.IntervalsSource; -import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.FuzzyQuery; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.util.BytesRef; @@ -836,7 +836,7 @@ public void testWildcard() throws IOException { + TEXT_FIELD_NAME + "\": { " + "\"wildcard\" : { \"pattern\" : \"Te?m\", \"max_expansions\" : " - + (BooleanQuery.getMaxClauseCount() + 1) + + (IndexSearcher.getMaxClauseCount() + 1) + " } } } }"; expectThrows(IllegalArgumentException.class, () -> { IntervalQueryBuilder builder1 = (IntervalQueryBuilder) parseQuery(wildcard_over_max_expand_json); @@ -948,7 +948,7 @@ public void testRegexp() throws IOException { + TEXT_FIELD_NAME + "\": { " + "\"regexp\" : { \"pattern\" : \"te.m\", \"max_expansions\" : " - + (BooleanQuery.getMaxClauseCount() + 1) + + (IndexSearcher.getMaxClauseCount() + 1) + " } } } }"; expectThrows(IllegalArgumentException.class, () -> { IntervalQueryBuilder builder1 = (IntervalQueryBuilder) parseQuery(regexp_over_max_expand_json); diff --git a/server/src/test/java/org/opensearch/index/query/MatchBoolPrefixQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/MatchBoolPrefixQueryBuilderTests.java index 46a0192164817..7d45838455b12 100644 --- a/server/src/test/java/org/opensearch/index/query/MatchBoolPrefixQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/MatchBoolPrefixQueryBuilderTests.java @@ -61,8 +61,6 @@ import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.equalToIgnoringCase; -import static org.hamcrest.Matchers.hasProperty; -import static org.hamcrest.Matchers.hasSize; public class MatchBoolPrefixQueryBuilderTests extends AbstractQueryTestCase { @@ -126,7 +124,7 @@ protected void doAssertLuceneQuery(MatchBoolPrefixQueryBuilder queryBuilder, Que // all queries except the last should be TermQuery or SynonymQuery final Set allQueriesExceptLast = IntStream.range(0, booleanQuery.clauses().size() - 1) .mapToObj(booleanQuery.clauses()::get) - .map(BooleanClause::getQuery) + .map(BooleanClause::query) .collect(Collectors.toSet()); assertThat( allQueriesExceptLast, @@ -147,13 +145,13 @@ protected void doAssertLuceneQuery(MatchBoolPrefixQueryBuilder queryBuilder, Que }); // the last query should be PrefixQuery - final Query shouldBePrefixQuery = booleanQuery.clauses().get(booleanQuery.clauses().size() - 1).getQuery(); + final Query shouldBePrefixQuery = booleanQuery.clauses().get(booleanQuery.clauses().size() - 1).query(); assertThat(shouldBePrefixQuery, instanceOf(PrefixQuery.class)); if (queryBuilder.minimumShouldMatch() != null) { final int optionalClauses = (int) booleanQuery.clauses() .stream() - .filter(clause -> clause.getOccur() == BooleanClause.Occur.SHOULD) + .filter(clause -> clause.occur() == BooleanClause.Occur.SHOULD) .count(); final int expected = Queries.calculateMinShouldMatch(optionalClauses, queryBuilder.minimumShouldMatch()); assertThat(booleanQuery.getMinimumNumberShouldMatch(), equalTo(expected)); @@ -269,7 +267,7 @@ public void testAnalysis() throws Exception { asList( new TermQuery(new Term(TEXT_FIELD_NAME, "foo")), new TermQuery(new Term(TEXT_FIELD_NAME, "bar")), - new PrefixQuery(new Term(TEXT_FIELD_NAME, "baz"), MultiTermQuery.CONSTANT_SCORE_REWRITE) + new PrefixQuery(new Term(TEXT_FIELD_NAME, "baz"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE) ) ); } @@ -286,7 +284,7 @@ public void testAnalysisSynonym() throws Exception { new SynonymQuery.Builder(TEXT_FIELD_NAME).addTerm(new Term(TEXT_FIELD_NAME, "dogs")) .addTerm(new Term(TEXT_FIELD_NAME, "dog")) .build(), - new PrefixQuery(new Term(TEXT_FIELD_NAME, "red"), MultiTermQuery.CONSTANT_SCORE_REWRITE) + new PrefixQuery(new Term(TEXT_FIELD_NAME, "red"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE) ) ); } @@ -294,18 +292,18 @@ public void testAnalysisSynonym() throws Exception { public void testAnalysisSingleTerm() throws Exception { final MatchBoolPrefixQueryBuilder builder = new MatchBoolPrefixQueryBuilder(TEXT_FIELD_NAME, "foo"); final Query query = builder.toQuery(createShardContext()); - assertThat(query, equalTo(new PrefixQuery(new Term(TEXT_FIELD_NAME, "foo"), MultiTermQuery.CONSTANT_SCORE_REWRITE))); + assertThat(query, equalTo(new PrefixQuery(new Term(TEXT_FIELD_NAME, "foo"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE))); } private static void assertBooleanQuery(Query actual, List expectedClauseQueries) { - assertThat(actual, instanceOf(BooleanQuery.class)); + assertTrue(actual instanceof BooleanQuery); final BooleanQuery actualBooleanQuery = (BooleanQuery) actual; - assertThat(actualBooleanQuery.clauses(), hasSize(expectedClauseQueries.size())); - assertThat(actualBooleanQuery.clauses(), everyItem(hasProperty("occur", equalTo(BooleanClause.Occur.SHOULD)))); + assertEquals(expectedClauseQueries.size(), actualBooleanQuery.clauses().size()); for (int i = 0; i < actualBooleanQuery.clauses().size(); i++) { - final Query clauseQuery = actualBooleanQuery.clauses().get(i).getQuery(); - assertThat(clauseQuery, equalTo(expectedClauseQueries.get(i))); + BooleanClause actualClause = actualBooleanQuery.clauses().get(i); + assertEquals(BooleanClause.Occur.SHOULD, actualClause.occur()); + assertEquals(expectedClauseQueries.get(i), actualClause.query()); } } } diff --git a/server/src/test/java/org/opensearch/index/query/MatchQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/MatchQueryBuilderTests.java index 04dcacbd86ac5..157c8b8658f25 100644 --- a/server/src/test/java/org/opensearch/index/query/MatchQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/MatchQueryBuilderTests.java @@ -43,6 +43,7 @@ import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.FuzzyQuery; +import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.PhraseQuery; @@ -192,7 +193,7 @@ protected void doAssertLuceneQuery(MatchQueryBuilder queryBuilder, Query query, // calculate expected minimumShouldMatch value int optionalClauses = 0; for (BooleanClause c : bq.clauses()) { - if (c.getOccur() == BooleanClause.Occur.SHOULD) { + if (c.occur() == BooleanClause.Occur.SHOULD) { optionalClauses++; } } @@ -545,9 +546,9 @@ public void testAliasWithSynonyms() throws Exception { public void testMaxBooleanClause() { MatchQuery query = new MatchQuery(createShardContext()); query.setAnalyzer(new MockGraphAnalyzer(createGiantGraph(40))); - expectThrows(BooleanQuery.TooManyClauses.class, () -> query.parse(Type.PHRASE, TEXT_FIELD_NAME, "")); + expectThrows(IndexSearcher.TooManyClauses.class, () -> query.parse(Type.PHRASE, TEXT_FIELD_NAME, "")); query.setAnalyzer(new MockGraphAnalyzer(createGiantGraphMultiTerms())); - expectThrows(BooleanQuery.TooManyClauses.class, () -> query.parse(Type.PHRASE, TEXT_FIELD_NAME, "")); + expectThrows(IndexSearcher.TooManyClauses.class, () -> query.parse(Type.PHRASE, TEXT_FIELD_NAME, "")); } private static class MockGraphAnalyzer extends Analyzer { @@ -599,7 +600,7 @@ private static CannedBinaryTokenStream.BinaryToken[] createGiantGraphWithNoSide( } /** - * Creates a graph token stream with {@link BooleanQuery#getMaxClauseCount()} + * Creates a graph token stream with {@link IndexSearcher#getMaxClauseCount()} * expansions at the last position. **/ private static CannedBinaryTokenStream.BinaryToken[] createGiantGraphMultiTerms() { @@ -610,7 +611,7 @@ private static CannedBinaryTokenStream.BinaryToken[] createGiantGraphMultiTerms( tokens.add(new CannedBinaryTokenStream.BinaryToken(term1, 0, 2)); tokens.add(new CannedBinaryTokenStream.BinaryToken(term2, 1, 1)); tokens.add(new CannedBinaryTokenStream.BinaryToken(term2, 1, 1)); - for (int i = 0; i < BooleanQuery.getMaxClauseCount(); i++) { + for (int i = 0; i < IndexSearcher.getMaxClauseCount(); i++) { tokens.add(new CannedBinaryTokenStream.BinaryToken(term1, 0, 1)); } return tokens.toArray(new CannedBinaryTokenStream.BinaryToken[0]); diff --git a/server/src/test/java/org/opensearch/index/query/MoreLikeThisQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/MoreLikeThisQueryBuilderTests.java index b6449ef0332e1..081d580dcbd1e 100644 --- a/server/src/test/java/org/opensearch/index/query/MoreLikeThisQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/MoreLikeThisQueryBuilderTests.java @@ -276,8 +276,8 @@ protected void doAssertLuceneQuery(MoreLikeThisQueryBuilder queryBuilder, Query assertThat(query, instanceOf(BooleanQuery.class)); BooleanQuery booleanQuery = (BooleanQuery) query; for (BooleanClause booleanClause : booleanQuery) { - if (booleanClause.getQuery() instanceof MoreLikeThisQuery) { - MoreLikeThisQuery moreLikeThisQuery = (MoreLikeThisQuery) booleanClause.getQuery(); + if (booleanClause.query() instanceof MoreLikeThisQuery) { + MoreLikeThisQuery moreLikeThisQuery = (MoreLikeThisQuery) booleanClause.query(); assertThat(moreLikeThisQuery.getLikeFields().length, greaterThan(0)); } } diff --git a/server/src/test/java/org/opensearch/index/query/NestedQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/NestedQueryBuilderTests.java index c367d123402d4..c371f004295d6 100644 --- a/server/src/test/java/org/opensearch/index/query/NestedQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/NestedQueryBuilderTests.java @@ -490,10 +490,10 @@ public void testNestedDepthAllowed() throws Exception { OpenSearchToParentBlockJoinQuery blockJoinQuery = (OpenSearchToParentBlockJoinQuery) queryBuilder.toQuery(context); Optional childLeg = ((BooleanQuery) blockJoinQuery.getChildQuery()).clauses() .stream() - .filter(c -> c.getOccur() == BooleanClause.Occur.MUST) + .filter(c -> c.occur() == BooleanClause.Occur.MUST) .findFirst(); assertTrue(childLeg.isPresent()); - assertEquals(new MatchAllDocsQuery(), childLeg.get().getQuery()); + assertEquals(new MatchAllDocsQuery(), childLeg.get().query()); }; check.accept(createShardContext()); doWithDepth(randomIntBetween(1, 20), check); @@ -528,7 +528,7 @@ private void checkOnceNested(QueryShardContext ctx) throws Exception { assertEquals( "Can parse joins one by one without breaching depth limit", 2, - bool.clauses().stream().filter(c -> c.getQuery() instanceof OpenSearchToParentBlockJoinQuery).count() + bool.clauses().stream().filter(c -> c.query() instanceof OpenSearchToParentBlockJoinQuery).count() ); } } diff --git a/server/src/test/java/org/opensearch/index/query/QueryStringQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/QueryStringQueryBuilderTests.java index 6f295100d9e47..c7002b6683909 100644 --- a/server/src/test/java/org/opensearch/index/query/QueryStringQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/QueryStringQueryBuilderTests.java @@ -46,12 +46,12 @@ import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.ConstantScoreQuery; import org.apache.lucene.search.DisjunctionMaxQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.FuzzyQuery; import org.apache.lucene.search.IndexOrDocValuesQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.MultiTermQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.PhraseQuery; import org.apache.lucene.search.PrefixQuery; import org.apache.lucene.search.Query; @@ -599,13 +599,7 @@ public void testToQueryWildcardWithIndexedPrefixes() throws Exception { Automaton a = Operations.concatenate(Arrays.asList(Automata.makeChar('g'), Automata.makeAnyChar())); expectedQuery = new ConstantScoreQuery( new BooleanQuery.Builder().add( - new AutomatonQuery( - new Term("prefix_field._index_prefix", "g*"), - a, - Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - false, - MultiTermQuery.CONSTANT_SCORE_REWRITE - ), + new AutomatonQuery(new Term("prefix_field._index_prefix", "g*"), a, false, MultiTermQuery.CONSTANT_SCORE_REWRITE), Occur.SHOULD ).add(new TermQuery(new Term("prefix_field", "g")), Occur.SHOULD).build() ); @@ -792,7 +786,7 @@ public void testToQueryRegExpQueryTooComplex() throws Exception { TooComplexToDeterminizeException.class, () -> queryBuilder.toQuery(createShardContext()) ); - assertThat(e.getMessage(), containsString("Determinizing [ac]*")); + assertThat(e.getMessage(), containsString("Determinizing automaton")); assertThat(e.getMessage(), containsString("would require more than 10000 effort")); } @@ -818,7 +812,7 @@ public void testToQueryRegExpQueryMaxDeterminizedStatesParsing() throws Exceptio TooComplexToDeterminizeException.class, () -> queryBuilder.toQuery(createShardContext()) ); - assertThat(e.getMessage(), containsString("Determinizing [ac]*")); + assertThat(e.getMessage(), containsString("Determinizing automaton")); assertThat(e.getMessage(), containsString("would require more than 10 effort")); } @@ -989,10 +983,10 @@ public void testToQueryBooleanQueryMultipleBoosts() throws Exception { assertThat(query, instanceOf(BooleanQuery.class)); BooleanQuery booleanQuery = (BooleanQuery) query; assertThat(booleanQuery.getMinimumNumberShouldMatch(), equalTo(2)); - assertThat(booleanQuery.clauses().get(0).getOccur(), equalTo(BooleanClause.Occur.SHOULD)); - assertThat(booleanQuery.clauses().get(0).getQuery(), equalTo(new TermQuery(new Term(TEXT_FIELD_NAME, "foo")))); - assertThat(booleanQuery.clauses().get(1).getOccur(), equalTo(BooleanClause.Occur.SHOULD)); - assertThat(booleanQuery.clauses().get(1).getQuery(), equalTo(new TermQuery(new Term(TEXT_FIELD_NAME, "bar")))); + assertThat(booleanQuery.clauses().get(0).occur(), equalTo(BooleanClause.Occur.SHOULD)); + assertThat(booleanQuery.clauses().get(0).query(), equalTo(new TermQuery(new Term(TEXT_FIELD_NAME, "foo")))); + assertThat(booleanQuery.clauses().get(1).occur(), equalTo(BooleanClause.Occur.SHOULD)); + assertThat(booleanQuery.clauses().get(1).query(), equalTo(new TermQuery(new Term(TEXT_FIELD_NAME, "bar")))); } public void testToQueryPhraseQueryBoostAndSlop() throws IOException { @@ -1102,7 +1096,7 @@ public void testExistsFieldQuery() throws Exception { QueryStringQueryBuilder queryBuilder = new QueryStringQueryBuilder(TEXT_FIELD_NAME + ":*"); Query query = queryBuilder.toQuery(context); if ((context.getMapperService().fieldType(TEXT_FIELD_NAME).getTextSearchInfo().hasNorms())) { - assertThat(query, equalTo(new ConstantScoreQuery(new NormsFieldExistsQuery(TEXT_FIELD_NAME)))); + assertThat(query, equalTo(new ConstantScoreQuery(new FieldExistsQuery(TEXT_FIELD_NAME)))); } else { assertThat(query, equalTo(new ConstantScoreQuery(new TermQuery(new Term("_field_names", TEXT_FIELD_NAME))))); } @@ -1112,7 +1106,7 @@ public void testExistsFieldQuery() throws Exception { queryBuilder = new QueryStringQueryBuilder("_exists_:" + value); query = queryBuilder.toQuery(context); if ((context.getMapperService().fieldType(TEXT_FIELD_NAME).getTextSearchInfo().hasNorms())) { - assertThat(query, equalTo(new ConstantScoreQuery(new NormsFieldExistsQuery(TEXT_FIELD_NAME)))); + assertThat(query, equalTo(new ConstantScoreQuery(new FieldExistsQuery(TEXT_FIELD_NAME)))); } else { assertThat(query, equalTo(new ConstantScoreQuery(new TermQuery(new Term("_field_names", TEXT_FIELD_NAME))))); } @@ -1191,9 +1185,9 @@ public void testFromJson() throws IOException { public void testExpandedTerms() throws Exception { // Prefix Query query = new QueryStringQueryBuilder("aBc*").field(TEXT_FIELD_NAME).analyzer("whitespace").toQuery(createShardContext()); - assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "aBc"), MultiTermQuery.CONSTANT_SCORE_REWRITE), query); + assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "aBc"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), query); query = new QueryStringQueryBuilder("aBc*").field(TEXT_FIELD_NAME).analyzer("standard").toQuery(createShardContext()); - assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "abc"), MultiTermQuery.CONSTANT_SCORE_REWRITE), query); + assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "abc"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), query); // Wildcard query = new QueryStringQueryBuilder("aBc*D").field(TEXT_FIELD_NAME).analyzer("whitespace").toQuery(createShardContext()); @@ -1201,7 +1195,7 @@ public void testExpandedTerms() throws Exception { new WildcardQuery( new Term(TEXT_FIELD_NAME, "aBc*D"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ), query ); @@ -1210,7 +1204,7 @@ public void testExpandedTerms() throws Exception { new WildcardQuery( new Term(TEXT_FIELD_NAME, "abc*d"), Operations.DEFAULT_DETERMINIZE_WORK_LIMIT, - MultiTermQuery.CONSTANT_SCORE_REWRITE + MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE ), query ); @@ -1456,7 +1450,7 @@ public void testEnablePositionIncrement() throws Exception { public void testWithPrefixStopWords() throws Exception { Query query = new QueryStringQueryBuilder("the* quick fox").field(TEXT_FIELD_NAME).analyzer("stop").toQuery(createShardContext()); BooleanQuery expected = new BooleanQuery.Builder().add( - new PrefixQuery(new Term(TEXT_FIELD_NAME, "the"), MultiTermQuery.CONSTANT_SCORE_REWRITE), + new PrefixQuery(new Term(TEXT_FIELD_NAME, "the"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), Occur.SHOULD ) .add(new TermQuery(new Term(TEXT_FIELD_NAME, "quick")), Occur.SHOULD) @@ -1539,7 +1533,7 @@ public void testAnalyzedPrefix() throws Exception { .analyzer("standard") .analyzeWildcard(true) .toQuery(createShardContext()); - Query expected = new PrefixQuery(new Term(TEXT_FIELD_NAME, "quick"), MultiTermQuery.CONSTANT_SCORE_REWRITE); + Query expected = new PrefixQuery(new Term(TEXT_FIELD_NAME, "quick"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE); assertEquals(expected, query); } diff --git a/server/src/test/java/org/opensearch/index/query/RangeQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/RangeQueryBuilderTests.java index 79f1452db297b..34f4657bad100 100644 --- a/server/src/test/java/org/opensearch/index/query/RangeQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/RangeQueryBuilderTests.java @@ -37,10 +37,9 @@ import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.Term; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexOrDocValuesQuery; import org.apache.lucene.search.MatchNoDocsQuery; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.PointRangeQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; @@ -172,9 +171,9 @@ protected void doAssertLuceneQuery(RangeQueryBuilder queryBuilder, Query query, if (queryBuilder.from() == null && queryBuilder.to() == null) { final Query expectedQuery; if (context.getMapperService().fieldType(queryBuilder.fieldName()).hasDocValues()) { - expectedQuery = new ConstantScoreQuery(new DocValuesFieldExistsQuery(expectedFieldName)); + expectedQuery = new ConstantScoreQuery(new FieldExistsQuery(expectedFieldName)); } else if (context.getMapperService().fieldType(queryBuilder.fieldName()).getTextSearchInfo().hasNorms()) { - expectedQuery = new ConstantScoreQuery(new NormsFieldExistsQuery(expectedFieldName)); + expectedQuery = new ConstantScoreQuery(new FieldExistsQuery(expectedFieldName)); } else { expectedQuery = new ConstantScoreQuery(new TermQuery(new Term(FieldNamesFieldMapper.NAME, expectedFieldName))); } @@ -563,7 +562,7 @@ protected MappedFieldType.Relation getRelation(QueryRewriteContext queryRewriteC Query luceneQuery = rewrittenRange.toQuery(queryShardContext); final Query expectedQuery; if (queryShardContext.fieldMapper(query.fieldName()).hasDocValues()) { - expectedQuery = new ConstantScoreQuery(new DocValuesFieldExistsQuery(query.fieldName())); + expectedQuery = new ConstantScoreQuery(new FieldExistsQuery(query.fieldName())); } else { expectedQuery = new ConstantScoreQuery(new TermQuery(new Term(FieldNamesFieldMapper.NAME, query.fieldName()))); } diff --git a/server/src/test/java/org/opensearch/index/query/SimpleQueryStringBuilderTests.java b/server/src/test/java/org/opensearch/index/query/SimpleQueryStringBuilderTests.java index 7688772173b08..c838be336e844 100644 --- a/server/src/test/java/org/opensearch/index/query/SimpleQueryStringBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/SimpleQueryStringBuilderTests.java @@ -344,7 +344,7 @@ protected void doAssertLuceneQuery(SimpleQueryStringBuilder queryBuilder, Query private static int shouldClauses(BooleanQuery query) { int result = 0; for (BooleanClause c : query.clauses()) { - if (c.getOccur() == BooleanClause.Occur.SHOULD) { + if (c.occur() == BooleanClause.Occur.SHOULD) { result++; } } @@ -452,9 +452,9 @@ public void testMinimumShouldMatch() throws IOException { public void testExpandedTerms() throws Exception { // Prefix Query query = new SimpleQueryStringBuilder("aBc*").field(TEXT_FIELD_NAME).analyzer("whitespace").toQuery(createShardContext()); - assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "aBc"), MultiTermQuery.CONSTANT_SCORE_REWRITE), query); + assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "aBc"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), query); query = new SimpleQueryStringBuilder("aBc*").field(TEXT_FIELD_NAME).analyzer("standard").toQuery(createShardContext()); - assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "abc"), MultiTermQuery.CONSTANT_SCORE_REWRITE), query); + assertEquals(new PrefixQuery(new Term(TEXT_FIELD_NAME, "abc"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), query); // Fuzzy query = new SimpleQueryStringBuilder("aBc~1").field(TEXT_FIELD_NAME).analyzer("whitespace").toQuery(createShardContext()); @@ -703,7 +703,7 @@ public void testToFuzzyQuery() throws Exception { .fuzzyMaxExpansions(5) .fuzzyTranspositions(false) .toQuery(createShardContext()); - FuzzyQuery expected = new FuzzyQuery(new Term(TEXT_FIELD_NAME, "text"), 2, 2, 5, false); + FuzzyQuery expected = new FuzzyQuery(new Term(TEXT_FIELD_NAME, "text"), 2, 2, 5, false, FuzzyQuery.defaultRewriteMethod(5)); assertEquals(expected, query); } @@ -718,7 +718,7 @@ public void testLenientToPrefixQuery() throws Exception { "failed query, caused by Can only use prefix queries on keyword and text fields - not on [mapped_date] which is of type [date]" ) ); - expectedQueries.add(new PrefixQuery(new Term(TEXT_FIELD_NAME, "t"), MultiTermQuery.CONSTANT_SCORE_REWRITE)); + expectedQueries.add(new PrefixQuery(new Term(TEXT_FIELD_NAME, "t"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE)); DisjunctionMaxQuery expected = new DisjunctionMaxQuery(expectedQueries, 1.0f); assertEquals(expected, query); } @@ -771,7 +771,7 @@ public void testWithStopWords() throws Exception { public void testWithPrefixStopWords() throws Exception { Query query = new SimpleQueryStringBuilder("the* quick fox").field(TEXT_FIELD_NAME).analyzer("stop").toQuery(createShardContext()); BooleanQuery expected = new BooleanQuery.Builder().add( - new PrefixQuery(new Term(TEXT_FIELD_NAME, "the"), MultiTermQuery.CONSTANT_SCORE_REWRITE), + new PrefixQuery(new Term(TEXT_FIELD_NAME, "the"), MultiTermQuery.CONSTANT_SCORE_BLENDED_REWRITE), BooleanClause.Occur.SHOULD ) .add(new TermQuery(new Term(TEXT_FIELD_NAME, "quick")), BooleanClause.Occur.SHOULD) diff --git a/server/src/test/java/org/opensearch/index/query/SourceFieldMatchQueryTests.java b/server/src/test/java/org/opensearch/index/query/SourceFieldMatchQueryTests.java index 6af717a97b328..4cbaaaa175c7b 100644 --- a/server/src/test/java/org/opensearch/index/query/SourceFieldMatchQueryTests.java +++ b/server/src/test/java/org/opensearch/index/query/SourceFieldMatchQueryTests.java @@ -91,20 +91,20 @@ public void testAllPossibleScenarios() throws IOException { iw.close(); IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(matchBoth, 10); - assertEquals(topDocs.totalHits.value, 1); + assertEquals(topDocs.totalHits.value(), 1); assertEquals(topDocs.scoreDocs[0].doc, 0); topDocs = searcher.search(matchDelegate, 10); - assertEquals(topDocs.totalHits.value, 0); + assertEquals(topDocs.totalHits.value(), 0); topDocs = searcher.search(matchFilter, 10); - assertEquals(topDocs.totalHits.value, 0); + assertEquals(topDocs.totalHits.value(), 0); topDocs = searcher.search(matchNone, 10); - assertEquals(topDocs.totalHits.value, 0); + assertEquals(topDocs.totalHits.value(), 0); topDocs = searcher.search(matchMultipleDocs, 10); - assertEquals(topDocs.totalHits.value, 2); + assertEquals(topDocs.totalHits.value(), 2); // assert constant score for (ScoreDoc scoreDoc : topDocs.scoreDocs) { assertEquals(scoreDoc.score, 1.0, 0.00000000001); @@ -166,7 +166,7 @@ public void testMissingField() throws IOException { iw.close(); IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(matchDelegate, 10); - assertEquals(topDocs.totalHits.value, 0); + assertEquals(topDocs.totalHits.value(), 0); } } } diff --git a/server/src/test/java/org/opensearch/index/query/SpanMultiTermQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/SpanMultiTermQueryBuilderTests.java index b4abff118802e..31e6abbda0e70 100644 --- a/server/src/test/java/org/opensearch/index/query/SpanMultiTermQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/SpanMultiTermQueryBuilderTests.java @@ -42,7 +42,6 @@ import org.apache.lucene.queries.spans.SpanMultiTermQueryWrapper; import org.apache.lucene.queries.spans.SpanQuery; import org.apache.lucene.queries.spans.SpanTermQuery; -import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MultiTermQuery; @@ -237,7 +236,7 @@ public void testToQueryInnerTermQuery() throws IOException { assertThat(prefixQuery.getPrefix().text(), equalTo("f")); assertThat(wrapper.getRewriteMethod(), instanceOf(SpanBooleanQueryRewriteWithMaxClause.class)); SpanBooleanQueryRewriteWithMaxClause rewrite = (SpanBooleanQueryRewriteWithMaxClause) wrapper.getRewriteMethod(); - assertThat(rewrite.getMaxExpansions(), equalTo(BooleanQuery.getMaxClauseCount())); + assertThat(rewrite.getMaxExpansions(), equalTo(IndexSearcher.getMaxClauseCount())); assertTrue(rewrite.isHardLimit()); } } @@ -281,8 +280,8 @@ public void testTermExpansionExceptionOnSpanFailure() throws Exception { iw.addDocument(singleton(new TextField("body", "foo bar" + Integer.toString(i), Field.Store.NO))); } try (IndexReader reader = iw.getReader()) { - int origBoolMaxClauseCount = BooleanQuery.getMaxClauseCount(); - BooleanQuery.setMaxClauseCount(1); + int origBoolMaxClauseCount = IndexSearcher.getMaxClauseCount(); + IndexSearcher.setMaxClauseCount(1); try { QueryBuilder queryBuilder = new SpanMultiTermQueryBuilder(QueryBuilders.prefixQuery("body", "bar")); IndexSearcher searcher = new IndexSearcher(reader); @@ -290,7 +289,7 @@ public void testTermExpansionExceptionOnSpanFailure() throws Exception { RuntimeException exc = expectThrows(RuntimeException.class, () -> query.rewrite(searcher)); assertThat(exc.getMessage(), containsString("maxClauseCount")); } finally { - BooleanQuery.setMaxClauseCount(origBoolMaxClauseCount); + IndexSearcher.setMaxClauseCount(origBoolMaxClauseCount); } } } diff --git a/server/src/test/java/org/opensearch/index/query/TermsSetQueryBuilderTests.java b/server/src/test/java/org/opensearch/index/query/TermsSetQueryBuilderTests.java index 63ea71458f1e7..27e87d669a2eb 100644 --- a/server/src/test/java/org/opensearch/index/query/TermsSetQueryBuilderTests.java +++ b/server/src/test/java/org/opensearch/index/query/TermsSetQueryBuilderTests.java @@ -253,7 +253,7 @@ public void testDoToQuery() throws Exception { .doToQuery(context); IndexSearcher searcher = new IndexSearcher(ir); TopDocs topDocs = searcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs[0].doc, equalTo(1)); assertThat(topDocs.scoreDocs[1].doc, equalTo(3)); assertThat(topDocs.scoreDocs[2].doc, equalTo(4)); @@ -299,7 +299,7 @@ public void testDoToQuery_msmScriptField() throws Exception { .doToQuery(context); IndexSearcher searcher = new IndexSearcher(ir); TopDocs topDocs = searcher.search(query, 10, new Sort(SortField.FIELD_DOC)); - assertThat(topDocs.totalHits.value, equalTo(3L)); + assertThat(topDocs.totalHits.value(), equalTo(3L)); assertThat(topDocs.scoreDocs[0].doc, equalTo(0)); assertThat(topDocs.scoreDocs[1].doc, equalTo(2)); assertThat(topDocs.scoreDocs[2].doc, equalTo(4)); diff --git a/server/src/test/java/org/opensearch/index/replication/IndexLevelReplicationTests.java b/server/src/test/java/org/opensearch/index/replication/IndexLevelReplicationTests.java index ec1600094084a..afe306625b6bc 100644 --- a/server/src/test/java/org/opensearch/index/replication/IndexLevelReplicationTests.java +++ b/server/src/test/java/org/opensearch/index/replication/IndexLevelReplicationTests.java @@ -338,7 +338,7 @@ public void testConflictingOpsOnReplica() throws Exception { for (IndexShard shard : shards) { try (Engine.Searcher searcher = shard.acquireSearcher("test")) { TopDocs search = searcher.search(new TermQuery(new Term("f", "2")), 10); - assertEquals("shard " + shard.routingEntry() + " misses new version", 1, search.totalHits.value); + assertEquals("shard " + shard.routingEntry() + " misses new version", 1, search.totalHits.value()); } } } diff --git a/server/src/test/java/org/opensearch/index/search/nested/AbstractNumberNestedSortingTestCase.java b/server/src/test/java/org/opensearch/index/search/nested/AbstractNumberNestedSortingTestCase.java index 525c1303ab46b..8d5df54fa5031 100644 --- a/server/src/test/java/org/opensearch/index/search/nested/AbstractNumberNestedSortingTestCase.java +++ b/server/src/test/java/org/opensearch/index/search/nested/AbstractNumberNestedSortingTestCase.java @@ -248,7 +248,7 @@ public void testNestedSorting() throws Exception { Sort sort = new Sort(new SortField("field2", nestedComparatorSource)); TopFieldDocs topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(11)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(7)); @@ -263,7 +263,7 @@ public void testNestedSorting() throws Exception { sort = new Sort(new SortField("field2", nestedComparatorSource, true)); topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(28)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(13)); @@ -281,7 +281,7 @@ public void testNestedSorting() throws Exception { query = new ToParentBlockJoinQuery(new ConstantScoreQuery(childFilter), new QueryBitSetProducer(parentFilter), ScoreMode.None); sort = new Sort(new SortField("field2", nestedComparatorSource, true)); topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(6L)); + assertThat(topDocs.totalHits.value(), equalTo(6L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(23)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(12)); @@ -296,7 +296,7 @@ public void testNestedSorting() throws Exception { sort = new Sort(new SortField("field2", nestedComparatorSource)); topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(6L)); + assertThat(topDocs.totalHits.value(), equalTo(6L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(15)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(3)); @@ -312,7 +312,7 @@ public void testNestedSorting() throws Exception { nestedComparatorSource = createFieldComparator("field2", sortMode, 127, createNested(searcher, parentFilter, childFilter)); sort = new Sort(new SortField("field2", nestedComparatorSource, true)); topDocs = searcher.search(new TermQuery(new Term("__type", "parent")), 5, sort); - assertThat(topDocs.totalHits.value, equalTo(8L)); + assertThat(topDocs.totalHits.value(), equalTo(8L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(19)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(127)); @@ -328,7 +328,7 @@ public void testNestedSorting() throws Exception { nestedComparatorSource = createFieldComparator("field2", sortMode, -127, createNested(searcher, parentFilter, childFilter)); sort = new Sort(new SortField("field2", nestedComparatorSource)); topDocs = searcher.search(new TermQuery(new Term("__type", "parent")), 5, sort); - assertThat(topDocs.totalHits.value, equalTo(8L)); + assertThat(topDocs.totalHits.value(), equalTo(8L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(19)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(-127)); @@ -363,7 +363,7 @@ protected void assertAvgScoreMode(Query parentFilter, IndexSearcher searcher) th ); Sort sort = new Sort(new SortField("field2", nestedComparatorSource)); TopDocs topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(11)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(2)); diff --git a/server/src/test/java/org/opensearch/index/search/nested/DoubleNestedSortingTests.java b/server/src/test/java/org/opensearch/index/search/nested/DoubleNestedSortingTests.java index 427eae4589e63..1147d0d0afa84 100644 --- a/server/src/test/java/org/opensearch/index/search/nested/DoubleNestedSortingTests.java +++ b/server/src/test/java/org/opensearch/index/search/nested/DoubleNestedSortingTests.java @@ -96,7 +96,7 @@ protected void assertAvgScoreMode(Query parentFilter, IndexSearcher searcher) th ); Sort sort = new Sort(new SortField("field2", nestedComparatorSource)); TopDocs topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(11)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(2)); diff --git a/server/src/test/java/org/opensearch/index/search/nested/FloatNestedSortingTests.java b/server/src/test/java/org/opensearch/index/search/nested/FloatNestedSortingTests.java index eb04775ae28d8..f287a036e51e8 100644 --- a/server/src/test/java/org/opensearch/index/search/nested/FloatNestedSortingTests.java +++ b/server/src/test/java/org/opensearch/index/search/nested/FloatNestedSortingTests.java @@ -99,7 +99,7 @@ protected void assertAvgScoreMode( ); Sort sort = new Sort(new SortField("field2", nestedComparatorSource)); TopDocs topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(11)); assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(2)); diff --git a/server/src/test/java/org/opensearch/index/search/nested/NestedSortingTests.java b/server/src/test/java/org/opensearch/index/search/nested/NestedSortingTests.java index 8f1a9afa243a3..f50dcfde112f2 100644 --- a/server/src/test/java/org/opensearch/index/search/nested/NestedSortingTests.java +++ b/server/src/test/java/org/opensearch/index/search/nested/NestedSortingTests.java @@ -333,7 +333,7 @@ public void testNestedSorting() throws Exception { Sort sort = new Sort(new SortField("field2", nestedComparatorSource)); TopFieldDocs topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(3)); assertThat(((BytesRef) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).utf8ToString(), equalTo("a")); @@ -355,7 +355,7 @@ public void testNestedSorting() throws Exception { ); sort = new Sort(new SortField("field2", nestedComparatorSource, true)); topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(7L)); + assertThat(topDocs.totalHits.value(), equalTo(7L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(28)); assertThat(((BytesRef) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).utf8ToString(), equalTo("o")); @@ -381,7 +381,7 @@ public void testNestedSorting() throws Exception { query = new ToParentBlockJoinQuery(new ConstantScoreQuery(childFilter), new QueryBitSetProducer(parentFilter), ScoreMode.None); sort = new Sort(new SortField("field2", nestedComparatorSource, true)); topDocs = searcher.search(query, 5, sort); - assertThat(topDocs.totalHits.value, equalTo(6L)); + assertThat(topDocs.totalHits.value(), equalTo(6L)); assertThat(topDocs.scoreDocs.length, equalTo(5)); assertThat(topDocs.scoreDocs[0].doc, equalTo(23)); assertThat(((BytesRef) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).utf8ToString(), equalTo("m")); @@ -643,42 +643,42 @@ public void testMultiLevelNestedSorting() throws IOException { sortBuilder.setNestedSort(new NestedSortBuilder("chapters").setNestedSort(new NestedSortBuilder("chapters.paragraphs"))); QueryBuilder queryBuilder = new MatchAllQueryBuilder(); TopFieldDocs topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(5L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); + assertThat(topFields.totalHits.value(), equalTo(5L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("4")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(87)); - assertThat(searcher.doc(topFields.scoreDocs[2].doc).get("_id"), equalTo("1")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[2].doc).get("_id"), equalTo("1")); assertThat(((FieldDoc) topFields.scoreDocs[2]).fields[0], equalTo(234)); - assertThat(searcher.doc(topFields.scoreDocs[3].doc).get("_id"), equalTo("3")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[3].doc).get("_id"), equalTo("3")); assertThat(((FieldDoc) topFields.scoreDocs[3]).fields[0], equalTo(976)); - assertThat(searcher.doc(topFields.scoreDocs[4].doc).get("_id"), equalTo("5")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[4].doc).get("_id"), equalTo("5")); assertThat(((FieldDoc) topFields.scoreDocs[4]).fields[0], equalTo(Integer.MAX_VALUE)); // Specific genre { queryBuilder = new TermQueryBuilder("genre", "romance"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76)); queryBuilder = new TermQueryBuilder("genre", "science fiction"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(234)); queryBuilder = new TermQueryBuilder("genre", "horror"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976)); queryBuilder = new TermQueryBuilder("genre", "cooking"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87)); } @@ -687,16 +687,16 @@ public void testMultiLevelNestedSorting() throws IOException { sortBuilder.order(SortOrder.DESC); queryBuilder = new MatchAllQueryBuilder(); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(5L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); + assertThat(topFields.totalHits.value(), equalTo(5L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("1")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("1")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(849)); - assertThat(searcher.doc(topFields.scoreDocs[2].doc).get("_id"), equalTo("4")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[2].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[2]).fields[0], equalTo(180)); - assertThat(searcher.doc(topFields.scoreDocs[3].doc).get("_id"), equalTo("2")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[3].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[3]).fields[0], equalTo(76)); - assertThat(searcher.doc(topFields.scoreDocs[4].doc).get("_id"), equalTo("5")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[4].doc).get("_id"), equalTo("5")); assertThat(((FieldDoc) topFields.scoreDocs[4]).fields[0], equalTo(Integer.MIN_VALUE)); } @@ -704,26 +704,26 @@ public void testMultiLevelNestedSorting() throws IOException { { queryBuilder = new TermQueryBuilder("genre", "romance"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76)); queryBuilder = new TermQueryBuilder("genre", "science fiction"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(849)); queryBuilder = new TermQueryBuilder("genre", "horror"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976)); queryBuilder = new TermQueryBuilder("genre", "cooking"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(180)); } @@ -735,18 +735,18 @@ public void testMultiLevelNestedSorting() throws IOException { new NestedSortBuilder("chapters").setFilter(queryBuilder).setNestedSort(new NestedSortBuilder("chapters.paragraphs")) ); topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(2L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); + assertThat(topFields.totalHits.value(), equalTo(2L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("4")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(87)); sortBuilder.order(SortOrder.DESC); topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(2L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(2L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(76)); } @@ -763,18 +763,18 @@ public void testMultiLevelNestedSorting() throws IOException { ) ); topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(2L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(2L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(Integer.MAX_VALUE)); sortBuilder.order(SortOrder.DESC); topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(2L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(2L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87)); - assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); + assertThat(searcher.storedFields().document(topFields.scoreDocs[1].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(Integer.MIN_VALUE)); } @@ -788,26 +788,26 @@ public void testMultiLevelNestedSorting() throws IOException { queryBuilder = new TermQueryBuilder("genre", "romance"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("2")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76)); queryBuilder = new TermQueryBuilder("genre", "science fiction"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("1")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(Integer.MAX_VALUE)); queryBuilder = new TermQueryBuilder("genre", "horror"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("3")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(Integer.MAX_VALUE)); queryBuilder = new TermQueryBuilder("genre", "cooking"); topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher); - assertThat(topFields.totalHits.value, equalTo(1L)); - assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); + assertThat(topFields.totalHits.value(), equalTo(1L)); + assertThat(searcher.storedFields().document(topFields.scoreDocs[0].doc).get("_id"), equalTo("4")); assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87)); } diff --git a/server/src/test/java/org/opensearch/index/shard/IndexReaderWrapperTests.java b/server/src/test/java/org/opensearch/index/shard/IndexReaderWrapperTests.java index 4134f1bd91985..b9d54627dbc08 100644 --- a/server/src/test/java/org/opensearch/index/shard/IndexReaderWrapperTests.java +++ b/server/src/test/java/org/opensearch/index/shard/IndexReaderWrapperTests.java @@ -71,7 +71,7 @@ public void testReaderCloseListenerIsCalled() throws IOException { writer.addDocument(doc); DirectoryReader open = OpenSearchDirectoryReader.wrap(DirectoryReader.open(writer), new ShardId("foo", "_na_", 1)); IndexSearcher searcher = new IndexSearcher(open); - assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value); + assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value()); final AtomicInteger closeCalls = new AtomicInteger(0); CheckedFunction wrapper = reader -> new FieldMaskingReader( "field", @@ -100,7 +100,7 @@ public void testReaderCloseListenerIsCalled() throws IOException { } outerCount.incrementAndGet(); }); - assertEquals(0, wrap.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value); + assertEquals(0, wrap.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value()); wrap.close(); assertFalse("wrapped reader is closed", wrap.getIndexReader().tryIncRef()); assertEquals(sourceRefCount, open.getRefCount()); @@ -124,7 +124,7 @@ public void testIsCacheable() throws IOException { writer.addDocument(doc); DirectoryReader open = OpenSearchDirectoryReader.wrap(DirectoryReader.open(writer), new ShardId("foo", "_na_", 1)); IndexSearcher searcher = new IndexSearcher(open); - assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value); + assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value()); searcher.setSimilarity(iwc.getSimilarity()); final AtomicInteger closeCalls = new AtomicInteger(0); CheckedFunction wrapper = reader -> new FieldMaskingReader( @@ -170,7 +170,7 @@ public void testNoWrap() throws IOException { writer.addDocument(doc); DirectoryReader open = OpenSearchDirectoryReader.wrap(DirectoryReader.open(writer), new ShardId("foo", "_na_", 1)); IndexSearcher searcher = new IndexSearcher(open); - assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value); + assertEquals(1, searcher.search(new TermQuery(new Term("field", "doc")), 1).totalHits.value()); searcher.setSimilarity(iwc.getSimilarity()); CheckedFunction wrapper = directoryReader -> directoryReader; try ( diff --git a/server/src/test/java/org/opensearch/index/shard/IndexShardTests.java b/server/src/test/java/org/opensearch/index/shard/IndexShardTests.java index 535adfbff8dcc..7614a54da52bf 100644 --- a/server/src/test/java/org/opensearch/index/shard/IndexShardTests.java +++ b/server/src/test/java/org/opensearch/index/shard/IndexShardTests.java @@ -3069,9 +3069,9 @@ public void testReaderWrapperIsUsed() throws IOException { } try (Engine.Searcher searcher = shard.acquireSearcher("test")) { TopDocs search = searcher.search(new TermQuery(new Term("foo", "bar")), 10); - assertEquals(search.totalHits.value, 1); + assertEquals(search.totalHits.value(), 1); search = searcher.search(new TermQuery(new Term("foobar", "bar")), 10); - assertEquals(search.totalHits.value, 1); + assertEquals(search.totalHits.value(), 1); } CheckedFunction wrapper = reader -> new FieldMaskingReader("foo", reader); closeShards(shard); @@ -3093,9 +3093,9 @@ public void testReaderWrapperIsUsed() throws IOException { try (Engine.Searcher searcher = newShard.acquireSearcher("test")) { TopDocs search = searcher.search(new TermQuery(new Term("foo", "bar")), 10); - assertEquals(search.totalHits.value, 0); + assertEquals(search.totalHits.value(), 0); search = searcher.search(new TermQuery(new Term("foobar", "bar")), 10); - assertEquals(search.totalHits.value, 1); + assertEquals(search.totalHits.value(), 1); } try ( Engine.GetResult getResult = newShard.get(new Engine.Get(false, false, "1", new Term(IdFieldMapper.NAME, Uid.encodeId("1")))) diff --git a/server/src/test/java/org/opensearch/index/similarity/ScriptedSimilarityTests.java b/server/src/test/java/org/opensearch/index/similarity/ScriptedSimilarityTests.java index f4282b8221c05..b6291ce3bf4a6 100644 --- a/server/src/test/java/org/opensearch/index/similarity/ScriptedSimilarityTests.java +++ b/server/src/test/java/org/opensearch/index/similarity/ScriptedSimilarityTests.java @@ -167,7 +167,7 @@ public double execute( 3.2f ); TopDocs topDocs = searcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertTrue(called.get()); assertEquals(42, topDocs.scoreDocs[0].score, 0); w.close(); @@ -258,7 +258,7 @@ public double execute( searcher.setSimilarity(sim); Query query = new BoostQuery(new TermQuery(new Term("f", "foo")), 3.2f); TopDocs topDocs = searcher.search(query, 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertTrue(initCalled.get()); assertTrue(called.get()); assertEquals(42, topDocs.scoreDocs[0].score, 0); diff --git a/server/src/test/java/org/opensearch/index/store/FsDirectoryFactoryTests.java b/server/src/test/java/org/opensearch/index/store/FsDirectoryFactoryTests.java index 95113b7eeb370..6354fa04b0063 100644 --- a/server/src/test/java/org/opensearch/index/store/FsDirectoryFactoryTests.java +++ b/server/src/test/java/org/opensearch/index/store/FsDirectoryFactoryTests.java @@ -31,13 +31,11 @@ package org.opensearch.index.store; -import org.apache.lucene.store.AlreadyClosedException; import org.apache.lucene.store.Directory; import org.apache.lucene.store.IOContext; import org.apache.lucene.store.MMapDirectory; import org.apache.lucene.store.NIOFSDirectory; import org.apache.lucene.store.NoLockFactory; -import org.apache.lucene.store.SleepingLockWrapper; import org.apache.lucene.util.Constants; import org.opensearch.Version; import org.opensearch.cluster.metadata.IndexMetadata; @@ -49,13 +47,14 @@ import org.opensearch.index.shard.ShardPath; import org.opensearch.test.IndexSettingsModule; import org.opensearch.test.OpenSearchTestCase; -import org.hamcrest.Matchers; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.util.Arrays; import java.util.Locale; +import java.util.Set; +import java.util.function.BiPredicate; import static org.opensearch.test.store.MockFSDirectoryFactory.FILE_SYSTEM_BASED_STORE_TYPES; @@ -86,12 +85,6 @@ public void testPreload() throws IOException { assertTrue(hybridDirectory.useDelegate("foo.new")); assertFalse(hybridDirectory.useDelegate("foo.pos")); assertFalse(hybridDirectory.useDelegate("foo.pay")); - MMapDirectory delegate = hybridDirectory.getDelegate(); - assertThat(delegate, Matchers.instanceOf(FsDirectoryFactory.PreLoadMMapDirectory.class)); - FsDirectoryFactory.PreLoadMMapDirectory preLoadMMapDirectory = (FsDirectoryFactory.PreLoadMMapDirectory) delegate; - assertTrue(preLoadMMapDirectory.useDelegate("foo.dvd")); - assertTrue(preLoadMMapDirectory.useDelegate("foo.bar")); - assertFalse(preLoadMMapDirectory.useDelegate("foo.cfs")); } build = Settings.builder() .put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), IndexModule.Type.HYBRIDFS.name().toLowerCase(Locale.ROOT)) @@ -115,13 +108,6 @@ public void testPreload() throws IOException { assertFalse(hybridDirectory.useDelegate("foo.kdi")); assertFalse(hybridDirectory.useDelegate("foo.cfs")); assertFalse(hybridDirectory.useDelegate("foo.doc")); - MMapDirectory delegate = hybridDirectory.getDelegate(); - assertThat(delegate, Matchers.instanceOf(FsDirectoryFactory.PreLoadMMapDirectory.class)); - FsDirectoryFactory.PreLoadMMapDirectory preLoadMMapDirectory = (FsDirectoryFactory.PreLoadMMapDirectory) delegate; - assertTrue(preLoadMMapDirectory.useDelegate("foo.dvd")); - assertFalse(preLoadMMapDirectory.useDelegate("foo.bar")); - assertTrue(preLoadMMapDirectory.useDelegate("foo.cfs")); - assertTrue(preLoadMMapDirectory.useDelegate("foo.nvd")); } build = Settings.builder() .put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), IndexModule.Type.HYBRIDFS.name().toLowerCase(Locale.ROOT)) @@ -138,8 +124,6 @@ public void testPreload() throws IOException { assertTrue(hybridDirectory.useDelegate("foo.dvd")); assertTrue(hybridDirectory.useDelegate("foo.cfs")); assertTrue(hybridDirectory.useDelegate("foo.doc")); - MMapDirectory delegate = hybridDirectory.getDelegate(); - assertThat(delegate, Matchers.instanceOf(FsDirectoryFactory.PreLoadMMapDirectory.class)); } } @@ -151,41 +135,19 @@ private Directory newDirectory(Settings settings) throws IOException { return new FsDirectoryFactory().newDirectory(idxSettings, path); } - private void doTestPreload(String... preload) throws IOException { - Settings build = Settings.builder() - .put(IndexModule.INDEX_STORE_TYPE_SETTING.getKey(), "mmapfs") - .putList(IndexModule.INDEX_STORE_PRE_LOAD_SETTING.getKey(), preload) - .build(); - Directory directory = newDirectory(build); - try (Directory dir = directory) { - assertSame(dir, directory); // prevent warnings - assertFalse(directory instanceof SleepingLockWrapper); - if (preload.length == 0) { - assertTrue(directory.toString(), directory instanceof MMapDirectory); - assertFalse(((MMapDirectory) directory).getPreload()); - } else if (Arrays.asList(preload).contains("*")) { - assertTrue(directory.toString(), directory instanceof MMapDirectory); - assertTrue(((MMapDirectory) directory).getPreload()); - } else { - assertTrue(directory.toString(), directory instanceof FsDirectoryFactory.PreLoadMMapDirectory); - FsDirectoryFactory.PreLoadMMapDirectory preLoadMMapDirectory = (FsDirectoryFactory.PreLoadMMapDirectory) directory; - for (String ext : preload) { - assertTrue("ext: " + ext, preLoadMMapDirectory.useDelegate("foo." + ext)); - assertTrue("ext: " + ext, preLoadMMapDirectory.getDelegate().getPreload()); - } - assertFalse(preLoadMMapDirectory.useDelegate("XXX")); - assertFalse(preLoadMMapDirectory.getPreload()); - preLoadMMapDirectory.close(); - expectThrows( - AlreadyClosedException.class, - () -> preLoadMMapDirectory.getDelegate().openInput("foo.bar", IOContext.DEFAULT) - ); + private void doTestPreload(String... preload) { + Set preloadSet = Set.of(preload); + if (preload.length == 0) { + assertFalse(FsDirectoryFactory.createPreloadPredicate(preloadSet).test("file", null)); + } else if (Arrays.asList(preload).contains("*")) { + assertTrue(FsDirectoryFactory.createPreloadPredicate(preloadSet).test("file", null)); + } else { + BiPredicate preloadPredicate = FsDirectoryFactory.createPreloadPredicate(preloadSet); + for (String ext : preload) { + assertTrue("ext: " + ext, preloadPredicate.test("foo." + ext, null)); } + assertFalse(preloadPredicate.test("XXX", null)); } - expectThrows( - AlreadyClosedException.class, - () -> directory.openInput(randomBoolean() && preload.length != 0 ? "foo." + preload[0] : "foo.bar", IOContext.DEFAULT) - ); } public void testStoreDirectory() throws IOException { @@ -221,7 +183,7 @@ private void doTestStoreDirectory(Path tempDir, String typeSettingValue, IndexMo assertTrue(type + " " + directory.toString(), directory instanceof MMapDirectory); break; case FS: - if (Constants.JRE_IS_64BIT && MMapDirectory.UNMAP_SUPPORTED) { + if (Constants.JRE_IS_64BIT) { assertTrue(FsDirectoryFactory.isHybridFs(directory)); } else { assertTrue(directory.toString(), directory instanceof NIOFSDirectory); diff --git a/server/src/test/java/org/opensearch/index/store/RemoteDirectoryTests.java b/server/src/test/java/org/opensearch/index/store/RemoteDirectoryTests.java index ed79a2b0bd8e4..cc8c6085978a8 100644 --- a/server/src/test/java/org/opensearch/index/store/RemoteDirectoryTests.java +++ b/server/src/test/java/org/opensearch/index/store/RemoteDirectoryTests.java @@ -20,9 +20,7 @@ import org.opensearch.common.blobstore.BlobMetadata; import org.opensearch.common.blobstore.stream.write.WriteContext; import org.opensearch.common.blobstore.support.PlainBlobMetadata; -import org.opensearch.common.lucene.store.ByteArrayIndexInput; import org.opensearch.core.action.ActionListener; -import org.opensearch.index.store.remote.utils.BlockIOContext; import org.opensearch.test.OpenSearchTestCase; import org.junit.Before; @@ -94,7 +92,7 @@ public void testCopyFrom() throws IOException, InterruptedException { storeDirectory, filename, filename, - IOContext.READ, + IOContext.READONCE, () -> postUploadInvoked.set(true), new ActionListener<>() { @Override @@ -132,7 +130,7 @@ public void testCopyFromWithException() throws IOException, InterruptedException storeDirectory, filename, filename, - IOContext.READ, + IOContext.READONCE, () -> postUploadInvoked.set(true), new ActionListener<>() { @Override @@ -218,17 +216,6 @@ public void testOpenInput() throws IOException { assertTrue(indexInput instanceof RemoteIndexInput); assertEquals(100, indexInput.length()); verify(blobContainer).listBlobsByPrefixInSortedOrder("segment_1", 1, LEXICOGRAPHIC); - - BlockIOContext blockIOContextInvalidValues = new BlockIOContext(IOContext.DEFAULT, 10, 1000); - assertThrows(IllegalArgumentException.class, () -> remoteDirectory.openInput("segment_1", blockIOContextInvalidValues)); - - BlockIOContext blockIOContext = new BlockIOContext(IOContext.DEFAULT, 10, 50); - when(blobContainer.readBlob("segment_1", 10, 50)).thenReturn(mockInputStream); - byte[] bytes = new byte[(int) blockIOContext.getBlockSize()]; - when(mockInputStream.readAllBytes()).thenReturn(bytes); - indexInput = remoteDirectory.openInput("segment_1", blockIOContext); - assertTrue(indexInput instanceof ByteArrayIndexInput); - assertEquals(blockIOContext.getBlockSize(), indexInput.length()); } public void testOpenInputWithLength() throws IOException { diff --git a/server/src/test/java/org/opensearch/index/store/StoreTests.java b/server/src/test/java/org/opensearch/index/store/StoreTests.java index 3777753cb2de8..542f95a4894e2 100644 --- a/server/src/test/java/org/opensearch/index/store/StoreTests.java +++ b/server/src/test/java/org/opensearch/index/store/StoreTests.java @@ -69,6 +69,7 @@ import org.opensearch.common.SuppressForbidden; import org.opensearch.common.UUIDs; import org.opensearch.common.lucene.Lucene; +import org.opensearch.common.lucene.LuceneTests; import org.opensearch.common.settings.Settings; import org.opensearch.common.unit.TimeValue; import org.opensearch.common.util.FeatureFlags; @@ -1284,9 +1285,8 @@ public void testSegmentReplicationDiff() { @SuppressForbidden(reason = "sets the SEARCHABLE_SNAPSHOT_EXTENDED_COMPATIBILITY feature flag") public void testReadSegmentsFromOldIndices() throws Exception { int expectedIndexCreatedVersionMajor = SEARCHABLE_SNAPSHOT_EXTENDED_COMPATIBILITY_MINIMUM_VERSION.luceneVersion.major; - final String pathToTestIndex = "/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip"; Path tmp = createTempDir(); - TestUtil.unzip(getClass().getResourceAsStream(pathToTestIndex), tmp); + TestUtil.unzip(getClass().getResourceAsStream(LuceneTests.OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH), tmp); final ShardId shardId = new ShardId("index", "_na_", 1); Store store = null; @@ -1309,10 +1309,9 @@ public void testReadSegmentsFromOldIndices() throws Exception { } public void testReadSegmentsFromOldIndicesFailure() throws IOException { - final String pathToTestIndex = "/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip"; final ShardId shardId = new ShardId("index", "_na_", 1); Path tmp = createTempDir(); - TestUtil.unzip(getClass().getResourceAsStream(pathToTestIndex), tmp); + TestUtil.unzip(getClass().getResourceAsStream(LuceneTests.OLDER_VERSION_INDEX_ZIP_RELATIVE_PATH), tmp); IndexSettings indexSettings = IndexSettingsModule.newIndexSettings( "index", Settings.builder() diff --git a/server/src/test/java/org/opensearch/index/store/remote/file/OnDemandBlockSnapshotIndexInputTests.java b/server/src/test/java/org/opensearch/index/store/remote/file/OnDemandBlockSnapshotIndexInputTests.java index c7d0cc0c5b96e..5518b37d79dfd 100644 --- a/server/src/test/java/org/opensearch/index/store/remote/file/OnDemandBlockSnapshotIndexInputTests.java +++ b/server/src/test/java/org/opensearch/index/store/remote/file/OnDemandBlockSnapshotIndexInputTests.java @@ -170,7 +170,7 @@ private OnDemandBlockSnapshotIndexInput createOnDemandBlockSnapshotIndexInput(in doAnswer(invocation -> { BlobFetchRequest blobFetchRequest = invocation.getArgument(0); - return blobFetchRequest.getDirectory().openInput(blobFetchRequest.getFileName(), IOContext.READ); + return blobFetchRequest.getDirectory().openInput(blobFetchRequest.getFileName(), IOContext.READONCE); }).when(transferManager).fetchBlob(any()); FSDirectory directory = null; diff --git a/server/src/test/java/org/opensearch/index/store/remote/utils/TransferManagerRemoteDirectoryReaderTests.java b/server/src/test/java/org/opensearch/index/store/remote/utils/TransferManagerRemoteDirectoryReaderTests.java index e777a287bf10f..2d01bc1b7e881 100644 --- a/server/src/test/java/org/opensearch/index/store/remote/utils/TransferManagerRemoteDirectoryReaderTests.java +++ b/server/src/test/java/org/opensearch/index/store/remote/utils/TransferManagerRemoteDirectoryReaderTests.java @@ -17,6 +17,7 @@ import java.util.concurrent.CountDownLatch; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.doThrow; @@ -28,10 +29,12 @@ public class TransferManagerRemoteDirectoryReaderTests extends TransferManagerTe @Override protected void initializeTransferManager() throws IOException { remoteDirectory = mock(RemoteDirectory.class); - doAnswer(i -> new ByteArrayIndexInput("blob", createData())).when(remoteDirectory).openInput(eq("blob"), any()); + final byte[] data = createData(); + doAnswer(i -> new ByteArrayIndexInput("blob", data)).when(remoteDirectory) + .openBlockInput(eq("blob"), anyLong(), anyLong(), anyLong(), any()); transferManager = new TransferManager( (name, position, length) -> new InputStreamIndexInput( - remoteDirectory.openInput(name, new BlockIOContext(IOContext.DEFAULT, position, length)), + remoteDirectory.openBlockInput(name, position, length, data.length, IOContext.DEFAULT), length ), fileCache @@ -39,13 +42,14 @@ protected void initializeTransferManager() throws IOException { } protected void mockExceptionWhileReading() throws IOException { - doThrow(new IOException("Expected test exception")).when(remoteDirectory).openInput(eq("failure-blob"), any()); + doThrow(new IOException("Expected test exception")).when(remoteDirectory) + .openBlockInput(eq("failure-blob"), anyLong(), anyLong(), anyLong(), any()); } protected void mockWaitForLatchReader(CountDownLatch latch) throws IOException { doAnswer(i -> { latch.await(); return new ByteArrayIndexInput("blocking-blob", createData()); - }).when(remoteDirectory).openInput(eq("blocking-blob"), any()); + }).when(remoteDirectory).openBlockInput(eq("blocking-blob"), anyLong(), anyLong(), anyLong(), any()); } } diff --git a/server/src/test/java/org/opensearch/indices/IndicesQueryCacheTests.java b/server/src/test/java/org/opensearch/indices/IndicesQueryCacheTests.java index ba40343fb2130..9ec92d86d4f0b 100644 --- a/server/src/test/java/org/opensearch/indices/IndicesQueryCacheTests.java +++ b/server/src/test/java/org/opensearch/indices/IndicesQueryCacheTests.java @@ -93,8 +93,18 @@ public String toString(String field) { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - return new ConstantScoreScorer(this, score(), scoreMode, DocIdSetIterator.all(context.reader().maxDoc())); + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { + return new ScorerSupplier() { + @Override + public Scorer get(long l) throws IOException { + return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.all(context.reader().maxDoc())); + } + + @Override + public long cost() { + return 0; + } + }; } @Override @@ -403,12 +413,6 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio return weight.explain(context, doc); } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - scorerCalled = true; - return weight.scorer(context); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { scorerSupplierCalled = true; diff --git a/server/src/test/java/org/opensearch/indices/IndicesRequestCacheTests.java b/server/src/test/java/org/opensearch/indices/IndicesRequestCacheTests.java index e83ca247b6a1d..d60bbd62bdd13 100644 --- a/server/src/test/java/org/opensearch/indices/IndicesRequestCacheTests.java +++ b/server/src/test/java/org/opensearch/indices/IndicesRequestCacheTests.java @@ -1206,7 +1206,7 @@ public BytesReference get() { try (BytesStreamOutput out = new BytesStreamOutput()) { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(new TermQuery(new Term("id", Integer.toString(id))), 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); Document document = reader.storedFields().document(topDocs.scoreDocs[0].doc); out.writeString(document.get("value")); loadedFromCache = false; diff --git a/server/src/test/java/org/opensearch/indices/IndicesServiceCloseTests.java b/server/src/test/java/org/opensearch/indices/IndicesServiceCloseTests.java index 8a00cd2db21c9..12b4b92753ad9 100644 --- a/server/src/test/java/org/opensearch/indices/IndicesServiceCloseTests.java +++ b/server/src/test/java/org/opensearch/indices/IndicesServiceCloseTests.java @@ -41,6 +41,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.ScoreMode; import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.Weight; import org.opensearch.cluster.ClusterName; import org.opensearch.cluster.routing.allocation.DiskThresholdSettings; @@ -111,8 +112,18 @@ public String toString(String field) { public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { return new ConstantScoreWeight(this, boost) { @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - return new ConstantScoreScorer(this, score(), scoreMode, DocIdSetIterator.all(context.reader().maxDoc())); + public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { + return new ScorerSupplier() { + @Override + public Scorer get(long l) throws IOException { + return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.all(context.reader().maxDoc())); + } + + @Override + public long cost() { + return 0; + } + }; } @Override diff --git a/server/src/test/java/org/opensearch/lucene/grouping/CollapsingTopDocsCollectorTests.java b/server/src/test/java/org/opensearch/lucene/grouping/CollapsingTopDocsCollectorTests.java index f425423d1cd82..de1ff6afa903c 100644 --- a/server/src/test/java/org/opensearch/lucene/grouping/CollapsingTopDocsCollectorTests.java +++ b/server/src/test/java/org/opensearch/lucene/grouping/CollapsingTopDocsCollectorTests.java @@ -51,6 +51,7 @@ import org.apache.lucene.search.SortedNumericSortField; import org.apache.lucene.search.SortedSetSortField; import org.apache.lucene.search.TopFieldCollector; +import org.apache.lucene.search.TopFieldCollectorManager; import org.apache.lucene.search.TopFieldDocs; import org.apache.lucene.search.TotalHits; import org.apache.lucene.search.Weight; @@ -82,7 +83,11 @@ private static class SegmentSearcher extends IndexSearcher { } public void search(Weight weight, Collector collector) throws IOException { - search(ctx, weight, collector); + LeafReaderContextPartition[] partitions = new LeafReaderContextPartition[ctx.size()]; + for (int i = 0; i < partitions.length; i++) { + partitions[i] = LeafReaderContextPartition.createForEntireSegment(ctx.get(i)); + } + search(partitions, weight, collector); } @Override @@ -146,7 +151,7 @@ private > void assertSearchCollapse( collapsingCollector = CollapsingTopDocsCollector.createKeyword(collapseField.getField(), fieldType, sort, expectedNumGroups); } - TopFieldCollector topFieldCollector = TopFieldCollector.create(sort, totalHits, Integer.MAX_VALUE); + TopFieldCollector topFieldCollector = new TopFieldCollectorManager(sort, totalHits, null, Integer.MAX_VALUE, false).newCollector(); Query query = new MatchAllDocsQuery(); searcher.search(query, collapsingCollector); searcher.search(query, topFieldCollector); @@ -154,10 +159,10 @@ private > void assertSearchCollapse( TopFieldDocs topDocs = topFieldCollector.topDocs(); assertEquals(collapseField.getField(), collapseTopFieldDocs.field); assertEquals(expectedNumGroups, collapseTopFieldDocs.scoreDocs.length); - assertEquals(totalHits, collapseTopFieldDocs.totalHits.value); - assertEquals(TotalHits.Relation.EQUAL_TO, collapseTopFieldDocs.totalHits.relation); + assertEquals(totalHits, collapseTopFieldDocs.totalHits.value()); + assertEquals(TotalHits.Relation.EQUAL_TO, collapseTopFieldDocs.totalHits.relation()); assertEquals(totalHits, topDocs.scoreDocs.length); - assertEquals(totalHits, topDocs.totalHits.value); + assertEquals(totalHits, topDocs.totalHits.value()); Set seen = new HashSet<>(); // collapse field is the last sort diff --git a/server/src/test/java/org/opensearch/lucene/queries/BlendedTermQueryTests.java b/server/src/test/java/org/opensearch/lucene/queries/BlendedTermQueryTests.java index ca420bfbf8fbb..032cf6ccb910f 100644 --- a/server/src/test/java/org/opensearch/lucene/queries/BlendedTermQueryTests.java +++ b/server/src/test/java/org/opensearch/lucene/queries/BlendedTermQueryTests.java @@ -144,8 +144,8 @@ public void testDismaxQuery() throws IOException { Query rewrite = searcher.rewrite(query); assertThat(rewrite, instanceOf(BooleanQuery.class)); for (BooleanClause clause : (BooleanQuery) rewrite) { - assertThat(clause.getQuery(), instanceOf(TermQuery.class)); - TermQuery termQuery = (TermQuery) clause.getQuery(); + assertThat(clause.query(), instanceOf(TermQuery.class)); + TermQuery termQuery = (TermQuery) clause.query(); TermStates termStates = termQuery.getTermStates(); if (termQuery.getTerm().field().equals("unknown_field")) { assertThat(termStates.docFreq(), equalTo(0)); @@ -155,7 +155,7 @@ public void testDismaxQuery() throws IOException { assertThat(termStates.totalTermFreq(), greaterThan(0L)); } } - assertThat(searcher.search(query, 10).totalHits.value, equalTo((long) iters + username.length)); + assertThat(searcher.search(query, 10).totalHits.value(), equalTo((long) iters + username.length)); } { // test with an unknown field and an unknown term @@ -164,13 +164,13 @@ public void testDismaxQuery() throws IOException { Query rewrite = searcher.rewrite(query); assertThat(rewrite, instanceOf(BooleanQuery.class)); for (BooleanClause clause : (BooleanQuery) rewrite) { - assertThat(clause.getQuery(), instanceOf(TermQuery.class)); - TermQuery termQuery = (TermQuery) clause.getQuery(); + assertThat(clause.query(), instanceOf(TermQuery.class)); + TermQuery termQuery = (TermQuery) clause.query(); TermStates termStates = termQuery.getTermStates(); assertThat(termStates.docFreq(), equalTo(0)); assertThat(termStates.totalTermFreq(), equalTo(0L)); } - assertThat(searcher.search(query, 10).totalHits.value, equalTo(0L)); + assertThat(searcher.search(query, 10).totalHits.value(), equalTo(0L)); } { // test with an unknown field and a term that is present in only one field @@ -179,8 +179,8 @@ public void testDismaxQuery() throws IOException { Query rewrite = searcher.rewrite(query); assertThat(rewrite, instanceOf(BooleanQuery.class)); for (BooleanClause clause : (BooleanQuery) rewrite) { - assertThat(clause.getQuery(), instanceOf(TermQuery.class)); - TermQuery termQuery = (TermQuery) clause.getQuery(); + assertThat(clause.query(), instanceOf(TermQuery.class)); + TermQuery termQuery = (TermQuery) clause.query(); TermStates termStates = termQuery.getTermStates(); if (termQuery.getTerm().field().equals("username")) { assertThat(termStates.docFreq(), equalTo(1)); @@ -190,7 +190,7 @@ public void testDismaxQuery() throws IOException { assertThat(termStates.totalTermFreq(), equalTo(0L)); } } - assertThat(searcher.search(query, 10).totalHits.value, equalTo(1L)); + assertThat(searcher.search(query, 10).totalHits.value(), equalTo(1L)); } reader.close(); w.close(); diff --git a/server/src/test/java/org/opensearch/lucene/search/uhighlight/CustomUnifiedHighlighterTests.java b/server/src/test/java/org/opensearch/lucene/search/uhighlight/CustomUnifiedHighlighterTests.java index 1282b0152b57f..98b79e64991a8 100644 --- a/server/src/test/java/org/opensearch/lucene/search/uhighlight/CustomUnifiedHighlighterTests.java +++ b/server/src/test/java/org/opensearch/lucene/search/uhighlight/CustomUnifiedHighlighterTests.java @@ -102,7 +102,7 @@ private void assertHighlightOneDoc( IndexSearcher searcher = newSearcher(reader); iw.close(); TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), 1, Sort.INDEXORDER); - assertThat(topDocs.totalHits.value, equalTo(1L)); + assertThat(topDocs.totalHits.value(), equalTo(1L)); String rawValue = Strings.arrayToDelimitedString(inputs, String.valueOf(MULTIVAL_SEP_CHAR)); CustomUnifiedHighlighter highlighter = new CustomUnifiedHighlighter( searcher, diff --git a/server/src/test/java/org/opensearch/search/CreatePitSingleNodeTests.java b/server/src/test/java/org/opensearch/search/CreatePitSingleNodeTests.java index f54c5cfbacd46..9347af317f9fb 100644 --- a/server/src/test/java/org/opensearch/search/CreatePitSingleNodeTests.java +++ b/server/src/test/java/org/opensearch/search/CreatePitSingleNodeTests.java @@ -422,7 +422,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(matchAllQuery()) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(50L) ); @@ -433,7 +434,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "test")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(50L) ); assertThat( @@ -443,7 +445,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "test")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(50L) ); assertThat( @@ -453,7 +456,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "update")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(0L) ); assertThat( @@ -463,7 +467,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "update")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(0L) ); @@ -486,7 +491,7 @@ public void testPitAfterUpdateIndex() throws Exception { client().admin().indices().prepareRefresh().get(); assertThat( - client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(matchAllQuery()).get().getHits().getTotalHits().value(), Matchers.equalTo(50L) ); /* @@ -494,19 +499,19 @@ public void testPitAfterUpdateIndex() throws Exception { */ assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), Matchers.equalTo(0L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "test")).get().getHits().getTotalHits().value(), Matchers.equalTo(0L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), Matchers.equalTo(50L) ); assertThat( - client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value, + client().prepareSearch().setSize(0).setQuery(termQuery("message", "update")).get().getHits().getTotalHits().value(), Matchers.equalTo(50L) ); /* @@ -519,7 +524,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "test")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(50L) ); assertThat( @@ -529,7 +535,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "test")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(50L) ); assertThat( @@ -539,7 +546,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "update")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(0L) ); assertThat( @@ -549,7 +557,8 @@ public void testPitAfterUpdateIndex() throws Exception { .setQuery(termQuery("message", "update")) .get() .getHits() - .getTotalHits().value, + .getTotalHits() + .value(), Matchers.equalTo(0L) ); validatePitStats("test", 1, 0, 0); diff --git a/server/src/test/java/org/opensearch/search/GenericSearchExtBuilderTests.java b/server/src/test/java/org/opensearch/search/GenericSearchExtBuilderTests.java index 8fb1814962155..cc003f2ca0e57 100644 --- a/server/src/test/java/org/opensearch/search/GenericSearchExtBuilderTests.java +++ b/server/src/test/java/org/opensearch/search/GenericSearchExtBuilderTests.java @@ -162,8 +162,8 @@ public void testSerializationWithGenericSearchExtBuildersForSimpleValues() throw if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); @@ -186,8 +186,8 @@ public void testSerializationWithGenericSearchExtBuildersForMapValues() throws I if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); @@ -210,8 +210,8 @@ public void testSerializationWithGenericSearchExtBuildersForListValues() throws if (searchResponse.getHits().getTotalHits() == null) { assertNull(deserialized.getHits().getTotalHits()); } else { - assertEquals(searchResponse.getHits().getTotalHits().value, deserialized.getHits().getTotalHits().value); - assertEquals(searchResponse.getHits().getTotalHits().relation, deserialized.getHits().getTotalHits().relation); + assertEquals(searchResponse.getHits().getTotalHits().value(), deserialized.getHits().getTotalHits().value()); + assertEquals(searchResponse.getHits().getTotalHits().relation(), deserialized.getHits().getTotalHits().relation()); } assertEquals(searchResponse.getHits().getHits().length, deserialized.getHits().getHits().length); assertEquals(searchResponse.getNumReducePhases(), deserialized.getNumReducePhases()); diff --git a/server/src/test/java/org/opensearch/search/MultiValueModeTests.java b/server/src/test/java/org/opensearch/search/MultiValueModeTests.java index e011dd0bcf6c0..61f0761bca90e 100644 --- a/server/src/test/java/org/opensearch/search/MultiValueModeTests.java +++ b/server/src/test/java/org/opensearch/search/MultiValueModeTests.java @@ -748,7 +748,7 @@ public long nextOrd() { if (i < array[doc].length) { return array[doc][i++]; } else { - return NO_MORE_ORDS; + return NO_MORE_DOCS; } } @@ -883,7 +883,9 @@ private void verifySortedSet(Supplier supplier, int maxDoc) } int expected = -1; if (values.advanceExact(i)) { - for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = values.nextOrd()) { + int docValueCount = 0; + for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_DOCS + && docValueCount < values.docValueCount(); ord = values.nextOrd(), ++docValueCount) { if (expected == -1) { expected = (int) ord; } else { @@ -931,7 +933,10 @@ private void verifySortedSet( if (++count > maxChildren) { break; } - for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_ORDS; ord = values.nextOrd()) { + + int docValueCount = 0; + for (long ord = values.nextOrd(); ord != SortedSetDocValues.NO_MORE_DOCS + && docValueCount < values.docValueCount(); ord = values.nextOrd(), ++docValueCount) { if (expected == -1) { expected = (int) ord; } else { diff --git a/server/src/test/java/org/opensearch/search/SearchServiceTests.java b/server/src/test/java/org/opensearch/search/SearchServiceTests.java index 514e99a126267..8283c0585bd1e 100644 --- a/server/src/test/java/org/opensearch/search/SearchServiceTests.java +++ b/server/src/test/java/org/opensearch/search/SearchServiceTests.java @@ -1316,38 +1316,38 @@ public void testCreateSearchContext() throws IOException { public void testConcurrentSegmentSearchSearchContext() throws IOException { Object[][] scenarios = { // cluster setting, index setting, cluster mode setting, concurrent search enabled?, concurrent search executor null? - { null, null, null, false, true }, - { null, false, null, false, true }, - { null, true, null, true, false }, - { true, null, null, true, false }, - { true, false, null, false, true }, - { true, true, null, true, false }, - { false, null, null, false, true }, - { false, false, null, false, true }, - { false, true, null, true, false }, + { null, null, null, false }, + { null, false, null, false }, + { null, true, null, true }, + { true, null, null, true }, + { true, false, null, false }, + { true, true, null, true }, + { false, null, null, false }, + { false, false, null, false }, + { false, true, null, true }, // Adding cases with mode set to "none" - { null, null, "none", false, true }, - { true, true, "none", false, true }, - { false, false, "none", false, true }, - { true, false, "none", false, true }, - { false, true, "none", false, true }, + { null, null, "none", false }, + { true, true, "none", false }, + { false, false, "none", false }, + { true, false, "none", false }, + { false, true, "none", false }, // Adding cases with mode set to "all" - { null, null, "all", true, false }, - { true, true, "all", true, false }, - { false, false, "all", true, false }, - { true, false, "all", true, false }, - { false, true, "all", true, false }, + { null, null, "all", true }, + { true, true, "all", true }, + { false, false, "all", true }, + { true, false, "all", true }, + { false, true, "all", true }, // Adding cases with mode set to "auto" // auto mode concurrent search is false since request has no aggregation // however concurrentSearchExecutor will not be null - { null, null, "auto", false, false }, - { true, true, "auto", false, false }, - { false, false, "auto", false, false }, - { true, false, "auto", false, false }, - { false, true, "auto", false, false } }; + { null, null, "auto", false }, + { true, true, "auto", false }, + { false, false, "auto", false }, + { true, false, "auto", false }, + { false, true, "auto", false } }; String index = randomAlphaOfLengthBetween(5, 10).toLowerCase(Locale.ROOT); IndexService indexService = createIndex(index); @@ -1374,7 +1374,6 @@ public void testConcurrentSegmentSearchSearchContext() throws IOException { Boolean indexSetting = (Boolean) scenario[1]; String mode = (String) scenario[2]; Boolean concurrentSearchEnabled = (Boolean) scenario[3]; - Boolean concurrentSearchExecutorNull = (Boolean) scenario[4]; if (clusterSetting == null) { client().admin() @@ -1455,12 +1454,7 @@ public void testConcurrentSegmentSearchSearchContext() throws IOException { ); searchContext.evaluateRequestShouldUseConcurrentSearch(); assertEquals(concurrentSearchEnabled, searchContext.shouldUseConcurrentSearch()); - // verify executor nullability - if (!concurrentSearchExecutorNull) { - assertNotNull(searchContext.searcher().getExecutor()); - } else { - assertNull(searchContext.searcher().getExecutor()); - } + assertThat(searchContext.searcher().getTaskExecutor(), is(notNullValue())); } } // Cleanup @@ -1506,16 +1500,13 @@ public void testConcurrentSegmentSearchWithRandomizedModeSettings() throws IOExc // default to false in case mode setting is not set boolean concurrentSearchEnabled = false; - boolean nullExecutor = true; boolean aggregationSupportsConcurrent = randomBoolean(); if (indexMode != null) { concurrentSearchEnabled = !indexMode.equals("none") && aggregationSupportsConcurrent; - nullExecutor = indexMode.equals("none"); } else if (clusterMode != null) { concurrentSearchEnabled = !clusterMode.equals("none") && aggregationSupportsConcurrent; - nullExecutor = clusterMode.equals("none"); } // Set the cluster setting for mode @@ -1578,22 +1569,16 @@ public void testConcurrentSegmentSearchWithRandomizedModeSettings() throws IOExc searchContext.evaluateRequestShouldUseConcurrentSearch(); // check concurrentSearchenabled based on mode and supportedAggregation is computed correctly assertEquals(concurrentSearchEnabled, searchContext.shouldUseConcurrentSearch()); + assertThat(searchContext.searcher().getTaskExecutor(), is(notNullValue())); + } finally { + // Cleanup + client().admin() + .cluster() + .prepareUpdateSettings() + .setTransientSettings(Settings.builder().putNull(SearchService.CLUSTER_CONCURRENT_SEGMENT_SEARCH_MODE.getKey())) + .get(); - // Verify executor nullability based on mode - if (!nullExecutor) { - assertNotNull(searchContext.searcher().getExecutor()); - } else { - assertNull(searchContext.searcher().getExecutor()); - } } - - // Cleanup - client().admin() - .cluster() - .prepareUpdateSettings() - .setTransientSettings(Settings.builder().putNull(SearchService.CLUSTER_CONCURRENT_SEGMENT_SEARCH_MODE.getKey())) - .get(); - } /** @@ -1736,7 +1721,7 @@ public void testConcurrentSegmentSearchIsSetOnceDuringContextCreation() throws I searchContext.evaluateRequestShouldUseConcurrentSearch(); assertEquals(concurrentSearchSetting, searchContext.shouldUseConcurrentSearch()); // verify executor state in searcher - assertEquals(concurrentSearchSetting, (searchContext.searcher().getExecutor() != null)); + assertThat(searchContext.searcher().getTaskExecutor(), is(notNullValue())); // update cluster setting to flip the concurrent segment search state client().admin() @@ -1749,15 +1734,15 @@ public void testConcurrentSegmentSearchIsSetOnceDuringContextCreation() throws I // verify that concurrent segment search is still set to same expected value for the context assertEquals(concurrentSearchSetting, searchContext.shouldUseConcurrentSearch()); + } finally { + // Cleanup + client().admin() + .cluster() + .prepareUpdateSettings() + .setTransientSettings(Settings.builder().putNull(SearchService.CLUSTER_CONCURRENT_SEGMENT_SEARCH_SETTING.getKey())) + .get(); } } - - // Cleanup - client().admin() - .cluster() - .prepareUpdateSettings() - .setTransientSettings(Settings.builder().putNull(SearchService.CLUSTER_CONCURRENT_SEGMENT_SEARCH_SETTING.getKey())) - .get(); } /** diff --git a/server/src/test/java/org/opensearch/search/aggregations/MultiBucketCollectorTests.java b/server/src/test/java/org/opensearch/search/aggregations/MultiBucketCollectorTests.java index 8c94a9854ef10..1d899a3c61241 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/MultiBucketCollectorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/MultiBucketCollectorTests.java @@ -56,12 +56,6 @@ public class MultiBucketCollectorTests extends OpenSearchTestCase { private static class ScoreAndDoc extends Scorable { float score; - int doc = -1; - - @Override - public int docID() { - return doc; - } @Override public float score() { diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/ShardSizeTestCase.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/ShardSizeTestCase.java index eef7e4c45849d..0bf23bd3e2cad 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/ShardSizeTestCase.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/ShardSizeTestCase.java @@ -121,11 +121,11 @@ protected void indexData() throws Exception { SearchResponse resp = client().prepareSearch("idx").setRouting(routing1).setQuery(matchAllQuery()).get(); assertSearchResponse(resp); - long totalOnOne = resp.getHits().getTotalHits().value; + long totalOnOne = resp.getHits().getTotalHits().value(); assertThat(totalOnOne, is(15L)); resp = client().prepareSearch("idx").setRouting(routing2).setQuery(matchAllQuery()).get(); assertSearchResponse(resp); - long totalOnTwo = resp.getHits().getTotalHits().value; + long totalOnTwo = resp.getHits().getTotalHits().value(); assertThat(totalOnTwo, is(12L)); } diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java index 13a3d8145743b..1c3dd558706c1 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java @@ -34,7 +34,6 @@ import org.apache.lucene.document.LongPoint; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.TermQuery; @@ -90,7 +89,7 @@ public void testUnmappedFieldWithTerms() throws Exception { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder("name", Arrays.asList(new TermsValuesSourceBuilder("unmapped").field("unmapped"))), (result) -> { @@ -99,7 +98,7 @@ public void testUnmappedFieldWithTerms() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -114,7 +113,7 @@ public void testUnmappedFieldWithTerms() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -126,7 +125,7 @@ public void testUnmappedFieldWithTerms() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -141,7 +140,7 @@ public void testUnmappedFieldWithTerms() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -178,7 +177,7 @@ public void testUnmappedFieldWithHistogram() throws Exception { // just unmapped = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -188,7 +187,7 @@ public void testUnmappedFieldWithHistogram() throws Exception { ); // unmapped missing bucket = one result testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -204,7 +203,7 @@ public void testUnmappedFieldWithHistogram() throws Exception { // field + unmapped, no missing bucket = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -218,7 +217,7 @@ public void testUnmappedFieldWithHistogram() throws Exception { // field + unmapped with missing bucket = multiple results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -254,7 +253,7 @@ public void testUnmappedFieldWithDateHistogram() throws Exception { ); // just unmapped = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -266,7 +265,7 @@ public void testUnmappedFieldWithDateHistogram() throws Exception { ); // unmapped missing bucket = one result testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -286,7 +285,7 @@ public void testUnmappedFieldWithDateHistogram() throws Exception { // field + unmapped, no missing bucket = no results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -300,7 +299,7 @@ public void testUnmappedFieldWithDateHistogram() throws Exception { // field + unmapped with missing bucket = multiple results testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(mappedFieldName)), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(mappedFieldName)), dataset, () -> new CompositeAggregationBuilder( "name", @@ -336,7 +335,7 @@ public void testWithKeyword() throws Exception { createDocument("keyword", "c") ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); }, (result) -> { @@ -350,7 +349,7 @@ public void testWithKeyword() throws Exception { assertEquals(1L, result.getBuckets().get(2).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "a") @@ -415,7 +414,7 @@ public void testWithKeywordAndMissingBucket() throws Exception { assertEquals(1L, result.getBuckets().get(0).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").missingBucket(true); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", null) @@ -431,7 +430,7 @@ public void testWithKeywordAndMissingBucket() throws Exception { assertEquals(1L, result.getBuckets().get(2).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword") .missingBucket(true) .order(SortOrder.DESC); @@ -534,7 +533,7 @@ public void testWithKeywordMissingAfter() throws Exception { createDocument("keyword", "delta") ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); }, (result) -> { @@ -550,7 +549,7 @@ public void testWithKeywordMissingAfter() throws Exception { assertEquals(1L, result.getBuckets().get(3).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "car") @@ -566,7 +565,7 @@ public void testWithKeywordMissingAfter() throws Exception { assertEquals(1L, result.getBuckets().get(2).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").order(SortOrder.DESC); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "mar") @@ -594,7 +593,7 @@ public void testWithKeywordDesc() throws Exception { createDocument("keyword", "c") ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").order(SortOrder.DESC); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); }, (result) -> { @@ -608,7 +607,7 @@ public void testWithKeywordDesc() throws Exception { assertEquals(1L, result.getBuckets().get(0).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").order(SortOrder.DESC); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "c") @@ -635,7 +634,7 @@ public void testMultiValuedWithKeyword() throws Exception { ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); @@ -654,7 +653,7 @@ public void testMultiValuedWithKeyword() throws Exception { assertEquals(1L, result.getBuckets().get(4).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "b") @@ -684,7 +683,7 @@ public void testMultiValuedWithKeywordDesc() throws Exception { ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").order(SortOrder.DESC); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); @@ -703,7 +702,7 @@ public void testMultiValuedWithKeywordDesc() throws Exception { assertEquals(1L, result.getBuckets().get(0).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword").order(SortOrder.DESC); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "c") @@ -733,7 +732,7 @@ public void testWithKeywordAndLong() throws Exception { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -754,7 +753,7 @@ public void testWithKeywordAndLong() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -785,7 +784,7 @@ public void testWithKeywordAndLongDesc() throws Exception { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -809,7 +808,7 @@ public void testWithKeywordAndLongDesc() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -877,7 +876,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -898,7 +897,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { // keyword null bucket is last, long null bucket is last testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -919,7 +918,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { // keyword null bucket is last, long null bucket is first testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -940,7 +939,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { // asc, null bucket is last, search after non null value testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -958,7 +957,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { // desc, null bucket is last, search after non null value testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -981,7 +980,7 @@ public void testWithKeywordLongAndMissingBucket() throws Exception { // keyword null bucket is last, long null bucket is last testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1017,7 +1016,7 @@ public void testMultiValuedWithKeywordAndLong() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1050,7 +1049,7 @@ public void testMultiValuedWithKeywordAndLong() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1089,7 +1088,7 @@ public void testMultiValuedWithKeywordAndLongDesc() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1125,7 +1124,7 @@ public void testMultiValuedWithKeywordAndLongDesc() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1166,7 +1165,7 @@ public void testMultiValuedWithKeywordLongAndDouble() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1203,7 +1202,7 @@ public void testMultiValuedWithKeywordLongAndDouble() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1240,7 +1239,7 @@ public void testMultiValuedWithKeywordLongAndDouble() throws Exception { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1340,7 +1339,7 @@ public void testWithDateHistogram() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1364,7 +1363,7 @@ public void testWithDateHistogram() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T11:34:00"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1392,7 +1391,7 @@ public void testWithDateHistogram() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1424,7 +1423,7 @@ public void testWithDateHistogram() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1456,7 +1455,7 @@ public void testWithDateHistogram() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1498,7 +1497,7 @@ public void testWithDateTerms() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1536,7 +1535,7 @@ public void testWithDateHistogramAndFormat() throws IOException { createDocument("long", 4L) ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("date")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("date")), dataset, () -> { DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") .dateHistogramInterval(DateHistogramInterval.days(1)) .format("yyyy-MM-dd"); @@ -1552,7 +1551,7 @@ public void testWithDateHistogramAndFormat() throws IOException { assertEquals(2L, result.getBuckets().get(2).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("date")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("date")), dataset, () -> { DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") .dateHistogramInterval(DateHistogramInterval.days(1)) .format("yyyy-MM-dd"); @@ -1575,38 +1574,28 @@ public void testWithDateHistogramAndFormat() throws IOException { public void testThatDateHistogramFailsFormatAfter() throws IOException { OpenSearchParseException exc = expectThrows( OpenSearchParseException.class, - () -> testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("date")), - Collections.emptyList(), - () -> { - DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") - .dateHistogramInterval(DateHistogramInterval.days(1)) - .format("yyyy-MM-dd"); - return new CompositeAggregationBuilder("name", Collections.singletonList(histo)).aggregateAfter( - createAfterKey("date", "now") - ); - }, - (result) -> {} - ) + () -> testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("date")), Collections.emptyList(), () -> { + DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") + .dateHistogramInterval(DateHistogramInterval.days(1)) + .format("yyyy-MM-dd"); + return new CompositeAggregationBuilder("name", Collections.singletonList(histo)).aggregateAfter( + createAfterKey("date", "now") + ); + }, (result) -> {}) ); assertThat(exc.getCause(), instanceOf(IllegalArgumentException.class)); assertThat(exc.getCause().getMessage(), containsString("now() is not supported in [after] key")); exc = expectThrows( OpenSearchParseException.class, - () -> testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("date")), - Collections.emptyList(), - () -> { - DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") - .dateHistogramInterval(DateHistogramInterval.days(1)) - .format("yyyy-MM-dd"); - return new CompositeAggregationBuilder("name", Collections.singletonList(histo)).aggregateAfter( - createAfterKey("date", "1474329600000") - ); - }, - (result) -> {} - ) + () -> testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("date")), Collections.emptyList(), () -> { + DateHistogramValuesSourceBuilder histo = new DateHistogramValuesSourceBuilder("date").field("date") + .dateHistogramInterval(DateHistogramInterval.days(1)) + .format("yyyy-MM-dd"); + return new CompositeAggregationBuilder("name", Collections.singletonList(histo)).aggregateAfter( + createAfterKey("date", "1474329600000") + ); + }, (result) -> {}) ); assertThat(exc.getMessage(), containsString("failed to parse date field [1474329600000]")); assertWarnings("[interval] on [date_histogram] is deprecated, use [fixed_interval] or [calendar_interval] in the future."); @@ -1627,7 +1616,7 @@ public void testWithDateHistogramAndKeyword() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T09:00:34"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1661,7 +1650,7 @@ public void testWithDateHistogramAndKeyword() throws IOException { testSearchCase( Arrays.asList( new MatchAllDocsQuery(), - new DocValuesFieldExistsQuery("date"), + new FieldExistsQuery("date"), LongPoint.newRangeQuery("date", asLong("2016-09-20T11:34:00"), asLong("2017-10-20T06:09:24")) ), dataset, @@ -1700,7 +1689,7 @@ public void testWithKeywordAndHistogram() throws IOException { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("price")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("price")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1730,7 +1719,7 @@ public void testWithKeywordAndHistogram() throws IOException { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("price")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("price")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1772,7 +1761,7 @@ public void testWithHistogramAndKeyword() throws IOException { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -1804,7 +1793,7 @@ public void testWithHistogramAndKeyword() throws IOException { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("double")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("double")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -2073,7 +2062,7 @@ public void testWithKeywordAndDateHistogram() throws IOException { ) ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -2103,7 +2092,7 @@ public void testWithKeywordAndDateHistogram() throws IOException { ); testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> new CompositeAggregationBuilder( "name", @@ -2140,7 +2129,7 @@ public void testWithKeywordAndTopHits() throws Exception { createDocument("keyword", "c") ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).subAggregation( new TopHitsAggregationBuilder("top_hits").storedField("_none_") @@ -2152,22 +2141,22 @@ public void testWithKeywordAndTopHits() throws Exception { TopHits topHits = result.getBuckets().get(0).getAggregations().get("top_hits"); assertNotNull(topHits); assertEquals(topHits.getHits().getHits().length, 2); - assertEquals(topHits.getHits().getTotalHits().value, 2L); + assertEquals(topHits.getHits().getTotalHits().value(), 2L); assertEquals("{keyword=c}", result.getBuckets().get(1).getKeyAsString()); assertEquals(2L, result.getBuckets().get(1).getDocCount()); topHits = result.getBuckets().get(1).getAggregations().get("top_hits"); assertNotNull(topHits); assertEquals(topHits.getHits().getHits().length, 2); - assertEquals(topHits.getHits().getTotalHits().value, 2L); + assertEquals(topHits.getHits().getTotalHits().value(), 2L); assertEquals("{keyword=d}", result.getBuckets().get(2).getKeyAsString()); assertEquals(1L, result.getBuckets().get(2).getDocCount()); topHits = result.getBuckets().get(2).getAggregations().get("top_hits"); assertNotNull(topHits); assertEquals(topHits.getHits().getHits().length, 1); - assertEquals(topHits.getHits().getTotalHits().value, 1L); + assertEquals(topHits.getHits().getTotalHits().value(), 1L); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("keyword", "a") @@ -2179,13 +2168,13 @@ public void testWithKeywordAndTopHits() throws Exception { TopHits topHits = result.getBuckets().get(0).getAggregations().get("top_hits"); assertNotNull(topHits); assertEquals(topHits.getHits().getHits().length, 2); - assertEquals(topHits.getHits().getTotalHits().value, 2L); + assertEquals(topHits.getHits().getTotalHits().value(), 2L); assertEquals("{keyword=d}", result.getBuckets().get(1).getKeyAsString()); assertEquals(1L, result.getBuckets().get(1).getDocCount()); topHits = result.getBuckets().get(1).getAggregations().get("top_hits"); assertNotNull(topHits); assertEquals(topHits.getHits().getHits().length, 1); - assertEquals(topHits.getHits().getTotalHits().value, 1L); + assertEquals(topHits.getHits().getTotalHits().value(), 1L); }); } @@ -2193,7 +2182,7 @@ public void testWithTermsSubAggExecutionMode() throws Exception { // test with no bucket for (Aggregator.SubAggCollectionMode mode : Aggregator.SubAggCollectionMode.values()) { testSearchCase( - Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), + Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), Collections.singletonList(createDocument()), () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); @@ -2219,7 +2208,7 @@ public void testWithTermsSubAggExecutionMode() throws Exception { ) ); for (Aggregator.SubAggCollectionMode mode : Aggregator.SubAggCollectionMode.values()) { - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("keyword")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("keyword")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("keyword").field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).subAggregation( new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) @@ -2340,7 +2329,7 @@ private , V extends Comparable> void testRandomTerms( AtomicBoolean finish = new AtomicBoolean(false); int size = randomIntBetween(1, expected.size()); while (finish.get() == false) { - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery(field)), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery(field)), dataset, () -> { Map afterKey = null; if (seen.size() > 0) { afterKey = Collections.singletonMap(field, seen.get(seen.size() - 1)); @@ -2374,7 +2363,7 @@ public void testWithIP() throws Exception { createDocument("ip", InetAddress.getByName("192.168.0.1")) ) ); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("ip")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("ip")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("ip").field("ip"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)); }, (result) -> { @@ -2388,7 +2377,7 @@ public void testWithIP() throws Exception { assertEquals(2L, result.getBuckets().get(2).getDocCount()); }); - testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new DocValuesFieldExistsQuery("ip")), dataset, () -> { + testSearchCase(Arrays.asList(new MatchAllDocsQuery(), new FieldExistsQuery("ip")), dataset, () -> { TermsValuesSourceBuilder terms = new TermsValuesSourceBuilder("ip").field("ip"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)).aggregateAfter( Collections.singletonMap("ip", "::1") diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/SingleDimensionValuesSourceTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/SingleDimensionValuesSourceTests.java index 1b9c95b87d4c2..51729ed8f26a5 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/SingleDimensionValuesSourceTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/composite/SingleDimensionValuesSourceTests.java @@ -33,11 +33,12 @@ package org.opensearch.search.aggregations.bucket.composite; import org.apache.lucene.document.LongPoint; +import org.apache.lucene.index.CompositeReader; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.Term; import org.apache.lucene.search.BoostQuery; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexOrDocValuesQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.TermQuery; @@ -209,10 +210,8 @@ public void testNumericSorted() { new IndexOrDocValuesQuery(LongPoint.newRangeQuery("number", 0, 1), new MatchAllDocsQuery()) ) ); - assertNotNull(source.createSortedDocsProducerOrNull(reader, new DocValuesFieldExistsQuery("number"))); - assertNotNull( - source.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new DocValuesFieldExistsQuery("number"))) - ); + assertNotNull(source.createSortedDocsProducerOrNull(reader, new FieldExistsQuery("number"))); + assertNotNull(source.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new FieldExistsQuery("number")))); assertNotNull( source.createSortedDocsProducerOrNull( reader, @@ -235,12 +234,9 @@ public void testNumericSorted() { assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, new MatchAllDocsQuery())); assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, null)); assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, new TermQuery(new Term("keyword", "toto)")))); - assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, new DocValuesFieldExistsQuery("number"))); + assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, new FieldExistsQuery("number"))); assertNull( - sourceWithMissing.createSortedDocsProducerOrNull( - reader, - new ConstantScoreQuery(new DocValuesFieldExistsQuery("number")) - ) + sourceWithMissing.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new FieldExistsQuery("number"))) ); LongValuesSource sourceRev = new LongValuesSource( @@ -255,10 +251,8 @@ public void testNumericSorted() { -1 ); assertNull(sourceRev.createSortedDocsProducerOrNull(reader, null)); - assertNull(sourceRev.createSortedDocsProducerOrNull(reader, new DocValuesFieldExistsQuery("number"))); - assertNull( - sourceRev.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new DocValuesFieldExistsQuery("number"))) - ); + assertNull(sourceRev.createSortedDocsProducerOrNull(reader, new FieldExistsQuery("number"))); + assertNull(sourceRev.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new FieldExistsQuery("number")))); assertNull(sourceWithMissing.createSortedDocsProducerOrNull(reader, new TermQuery(new Term("keyword", "toto)")))); } else if (numberType == NumberFieldMapper.NumberType.HALF_FLOAT || numberType == NumberFieldMapper.NumberType.FLOAT @@ -275,11 +269,9 @@ public void testNumericSorted() { ); IndexReader reader = mockIndexReader(1, 1); assertNull(source.createSortedDocsProducerOrNull(reader, null)); - assertNull(source.createSortedDocsProducerOrNull(reader, new DocValuesFieldExistsQuery("number"))); + assertNull(source.createSortedDocsProducerOrNull(reader, new FieldExistsQuery("number"))); assertNull(source.createSortedDocsProducerOrNull(reader, new TermQuery(new Term("keyword", "toto)")))); - assertNull( - source.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new DocValuesFieldExistsQuery("number"))) - ); + assertNull(source.createSortedDocsProducerOrNull(reader, new ConstantScoreQuery(new FieldExistsQuery("number")))); } else { throw new AssertionError("missing type:" + numberType.typeName()); } @@ -288,7 +280,7 @@ public void testNumericSorted() { } private static IndexReader mockIndexReader(int maxDoc, int numDocs) { - IndexReader reader = mock(IndexReader.class); + CompositeReader reader = mock(CompositeReader.class); when(reader.hasDeletions()).thenReturn(maxDoc - numDocs > 0); when(reader.maxDoc()).thenReturn(maxDoc); when(reader.numDocs()).thenReturn(numDocs); diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregatorTests.java index 77e59375ef012..0f1bff17f8adf 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/range/BinaryRangeAggregatorTests.java @@ -66,7 +66,7 @@ public boolean advanceExact(int docID) { @Override public long nextOrd() { if (i == ords.length) { - return NO_MORE_ORDS; + return NO_MORE_DOCS; } return ords[i++]; } diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/IncludeExcludeTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/IncludeExcludeTests.java index 2356e2dc6d855..eee3f91d213af 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/IncludeExcludeTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/IncludeExcludeTests.java @@ -81,7 +81,7 @@ public boolean advanceExact(int docID) { @Override public long nextOrd() { if (consumed) { - return SortedSetDocValues.NO_MORE_ORDS; + return NO_MORE_DOCS; } else { consumed = true; return 0; diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java index 28cec4df5f7d7..6d66cfc24e558 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java @@ -41,7 +41,7 @@ import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; @@ -448,7 +448,7 @@ public void testWithNestedAggregations() throws IOException { InternalNested result = searchAndReduce( newIndexSearcher(indexReader), // match root document only - new DocValuesFieldExistsQuery(PRIMARY_TERM_NAME), + new FieldExistsQuery(PRIMARY_TERM_NAME), nested, fieldType ); @@ -492,7 +492,7 @@ public void testWithNestedScoringAggregations() throws IOException { () -> searchAndReduce( newIndexSearcher(indexReader), // match root document only - new DocValuesFieldExistsQuery(PRIMARY_TERM_NAME), + new FieldExistsQuery(PRIMARY_TERM_NAME), nested, fieldType ) @@ -509,7 +509,7 @@ public void testWithNestedScoringAggregations() throws IOException { InternalNested result = searchAndReduce( newIndexSearcher(indexReader), // match root document only - new DocValuesFieldExistsQuery(PRIMARY_TERM_NAME), + new FieldExistsQuery(PRIMARY_TERM_NAME), nested, fieldType ); @@ -520,7 +520,7 @@ public void testWithNestedScoringAggregations() throws IOException { InternalTopHits topHits = bucket.getAggregations().get("top_hits"); TotalHits hits = topHits.getHits().getTotalHits(); assertNotNull(hits); - assertThat(hits.value, equalTo(counter)); + assertThat(hits.value(), equalTo(counter)); assertThat(topHits.getHits().getMaxScore(), equalTo(Float.NaN)); counter += 1; } diff --git a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/TermsAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/TermsAggregatorTests.java index d97cfdf003600..bc22d5f6ef2e8 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/TermsAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/bucket/terms/TermsAggregatorTests.java @@ -46,7 +46,7 @@ import org.apache.lucene.index.IndexableField; import org.apache.lucene.index.NoMergePolicy; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.TotalHits; @@ -1385,7 +1385,7 @@ public void testWithNestedAggregations() throws IOException { InternalNested result = searchAndReduce( newSearcher(indexReader, false, true), // match root document only - new DocValuesFieldExistsQuery(PRIMARY_TERM_NAME), + new FieldExistsQuery(PRIMARY_TERM_NAME), nested, fieldType ); @@ -1399,7 +1399,7 @@ public void testWithNestedAggregations() throws IOException { InternalFilter result = searchAndReduce( newSearcher(indexReader, false, true), // match root document only - new DocValuesFieldExistsQuery(PRIMARY_TERM_NAME), + new FieldExistsQuery(PRIMARY_TERM_NAME), filter, fieldType ); @@ -1539,8 +1539,8 @@ private void assertNestedTopHitsScore(InternalMultiBucketAggregation terms int ptr = 9; for (MultiBucketsAggregation.Bucket bucket : terms.getBuckets()) { InternalTopHits topHits = bucket.getAggregations().get("top_hits"); - assertThat(topHits.getHits().getTotalHits().value, equalTo((long) ptr)); - assertEquals(TotalHits.Relation.EQUAL_TO, topHits.getHits().getTotalHits().relation); + assertThat(topHits.getHits().getTotalHits().value(), equalTo((long) ptr)); + assertEquals(TotalHits.Relation.EQUAL_TO, topHits.getHits().getTotalHits().relation()); if (withScore) { assertThat(topHits.getHits().getMaxScore(), equalTo(1f)); } else { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/AbstractGeoTestCase.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/AbstractGeoTestCase.java index 94cb4c7955a21..ee64172e771df 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/AbstractGeoTestCase.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/AbstractGeoTestCase.java @@ -272,7 +272,7 @@ public void setupSuiteScopeCluster() throws Exception { .setSize(5000) .get(); assertSearchResponse(response); - long totalHits = response.getHits().getTotalHits().value; + long totalHits = response.getHits().getTotalHits().value(); XContentBuilder builder = XContentFactory.jsonBuilder(); response.toXContent(builder, ToXContent.EMPTY_PARAMS); logger.info("Full high_card_idx Response Content:\n{ {} }", builder.toString()); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/AvgAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/AvgAggregatorTests.java index f8eab720f8d6c..bfe93286a564a 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/AvgAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/AvgAggregatorTests.java @@ -39,7 +39,7 @@ import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.MultiReader; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -167,7 +167,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 7))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 2))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 3))); @@ -178,7 +178,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 2))); iw.addDocument(singleton(new NumericDocValuesField("number", 3))); @@ -243,7 +243,7 @@ public void testSummationAccuracy() throws IOException { public void testUnmappedField() throws IOException { AvgAggregationBuilder aggregationBuilder = new AvgAggregationBuilder("_name").field("number"); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, avg -> { @@ -254,7 +254,7 @@ public void testUnmappedField() throws IOException { public void testUnmappedWithMissingField() throws IOException { AvgAggregationBuilder aggregationBuilder = new AvgAggregationBuilder("_name").field("number").missing(0L); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, avg -> { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/CardinalityAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/CardinalityAggregatorTests.java index b5dd27e37c332..060e06f7336b3 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/CardinalityAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/CardinalityAggregatorTests.java @@ -45,7 +45,7 @@ import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.index.NoMergePolicy; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -127,7 +127,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 7))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 1))); }, card -> { @@ -137,7 +137,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, card -> { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java index 4c0534c95a116..b4d779fc59951 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java @@ -39,7 +39,7 @@ import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -96,7 +96,7 @@ public void testNoDocs() throws IOException { public void testStringField() throws IOException { final String fieldName = "string"; MappedFieldType fieldType = new KeywordFieldMapper.KeywordFieldType(fieldName); - expectThrows(IllegalArgumentException.class, () -> testCase(new DocValuesFieldExistsQuery(fieldName), iw -> { + expectThrows(IllegalArgumentException.class, () -> testCase(new FieldExistsQuery(fieldName), iw -> { iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("bogus")))); iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("zwomp")))); iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("foobar")))); @@ -113,7 +113,7 @@ public void testRangeField() throws IOException { MappedFieldType fieldType = new RangeFieldMapper.RangeFieldType(fieldName, RangeType.DOUBLE); RangeFieldMapper.Range range = new RangeFieldMapper.Range(RangeType.DOUBLE, 1.0D, 5.0D, true, true); BytesRef encodedRange = RangeType.DOUBLE.encodeRanges(Collections.singleton(range)); - expectThrows(IllegalArgumentException.class, () -> testCase(new DocValuesFieldExistsQuery(fieldName), iw -> { + expectThrows(IllegalArgumentException.class, () -> testCase(new FieldExistsQuery(fieldName), iw -> { iw.addDocument(singleton(new BinaryDocValuesField(fieldName, encodedRange))); }, hdr -> {}, fieldType, fieldName)); } @@ -129,7 +129,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 60))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 40))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 20))); @@ -146,7 +146,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 60))); iw.addDocument(singleton(new NumericDocValuesField("number", 40))); iw.addDocument(singleton(new NumericDocValuesField("number", 20))); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/InternalTopHitsTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/InternalTopHitsTests.java index 01e259f84660d..230f50ad627b0 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/InternalTopHitsTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/InternalTopHitsTests.java @@ -211,8 +211,8 @@ protected void assertFromXContent(InternalTopHits aggregation, ParsedAggregation ParsedTopHits parsed = (ParsedTopHits) parsedAggregation; final SearchHits actualSearchHits = parsed.getHits(); - assertEquals(expectedSearchHits.getTotalHits().value, actualSearchHits.getTotalHits().value); - assertEquals(expectedSearchHits.getTotalHits().relation, actualSearchHits.getTotalHits().relation); + assertEquals(expectedSearchHits.getTotalHits().value(), actualSearchHits.getTotalHits().value()); + assertEquals(expectedSearchHits.getTotalHits().relation(), actualSearchHits.getTotalHits().relation()); assertEquals(expectedSearchHits.getMaxScore(), actualSearchHits.getMaxScore(), 0.0f); List expectedHits = Arrays.asList(expectedSearchHits.getHits()); @@ -276,8 +276,8 @@ protected void assertReduced(InternalTopHits reduced, List inpu TotalHits.Relation relation = TotalHits.Relation.EQUAL_TO; for (int input = 0; input < inputs.size(); input++) { SearchHits internalHits = inputs.get(input).getHits(); - totalHits += internalHits.getTotalHits().value; - if (internalHits.getTotalHits().relation == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { + totalHits += internalHits.getTotalHits().value(); + if (internalHits.getTotalHits().relation() == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { relation = TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO; } maxScore = max(maxScore, internalHits.getMaxScore()); @@ -401,7 +401,7 @@ protected InternalTopHits mutateInstance(InternalTopHits instance) { case 3: topDocs = new TopDocsAndMaxScore( new TopDocs( - new TotalHits(topDocs.topDocs.totalHits.value + between(1, 100), topDocs.topDocs.totalHits.relation), + new TotalHits(topDocs.topDocs.totalHits.value() + between(1, 100), topDocs.topDocs.totalHits.relation()), topDocs.topDocs.scoreDocs ), topDocs.maxScore + randomFloat() @@ -409,7 +409,7 @@ protected InternalTopHits mutateInstance(InternalTopHits instance) { break; case 4: TotalHits totalHits = new TotalHits( - searchHits.getTotalHits().value + between(1, 100), + searchHits.getTotalHits().value() + between(1, 100), randomFrom(TotalHits.Relation.values()) ); searchHits = new SearchHits(searchHits.getHits(), totalHits, searchHits.getMaxScore() + randomFloat()); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/MaxAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/MaxAggregatorTests.java index 3b965a51275b6..0c7a194fdbe25 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/MaxAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/MaxAggregatorTests.java @@ -50,7 +50,7 @@ import org.apache.lucene.index.NoMergePolicy; import org.apache.lucene.index.PointValues; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -204,7 +204,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 7))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 1))); }, max -> { @@ -214,7 +214,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, max -> { @@ -245,7 +245,7 @@ public void testQueryFiltersAll() throws IOException { public void testUnmappedField() throws IOException { MaxAggregationBuilder aggregationBuilder = new MaxAggregationBuilder("_name").field("number"); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, max -> { @@ -257,7 +257,7 @@ public void testUnmappedField() throws IOException { public void testUnmappedWithMissingField() throws IOException { MaxAggregationBuilder aggregationBuilder = new MaxAggregationBuilder("_name").field("number").missing(19L); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, max -> { @@ -286,7 +286,7 @@ public void testScript() throws IOException { AggregationBuilder aggregationBuilder = new MaxAggregationBuilder("_name").field("number") .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPT_NAME, Collections.emptyMap())); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery("number"), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number", 1))); }, max -> { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorTests.java index 794dd44b2db94..8779a07aa0c9b 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorTests.java @@ -36,7 +36,7 @@ import org.apache.lucene.document.NumericDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.IndexableField; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.tests.index.RandomIndexWriter; @@ -120,7 +120,7 @@ public void testNoMatchingField() throws IOException { public void testSomeMatchesSortedNumericDocValues() throws IOException { final int size = randomIntBetween(100, 1000); final List sample = new ArrayList<>(size); - testAggregation(new DocValuesFieldExistsQuery(FIELD_NAME), randomSample(size, point -> { + testAggregation(new FieldExistsQuery(FIELD_NAME), randomSample(size, point -> { sample.add(point); return singleton(new SortedNumericDocValuesField(FIELD_NAME, point)); }), agg -> { @@ -132,7 +132,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { public void testSomeMatchesNumericDocValues() throws IOException { final int size = randomIntBetween(100, 1000); final List sample = new ArrayList<>(size); - testAggregation(new DocValuesFieldExistsQuery(FIELD_NAME), randomSample(size, point -> { + testAggregation(new FieldExistsQuery(FIELD_NAME), randomSample(size, point -> { sample.add(point); return singleton(new NumericDocValuesField(FIELD_NAME, point)); }), agg -> { @@ -171,7 +171,7 @@ public void testUnmapped() throws IOException { FIELD_NAME ); - testAggregation(aggregationBuilder, new DocValuesFieldExistsQuery(FIELD_NAME), iw -> { + testAggregation(aggregationBuilder, new FieldExistsQuery(FIELD_NAME), iw -> { iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 7))); iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 1))); }, agg -> { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/MinAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/MinAggregatorTests.java index 69cd63e4b9e84..68042aba55d22 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/MinAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/MinAggregatorTests.java @@ -51,7 +51,7 @@ import org.apache.lucene.index.MultiReader; import org.apache.lucene.index.NoMergePolicy; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -226,7 +226,7 @@ public void testMatchesNumericDocValues() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 7))); iw.addDocument(singleton(new SortedNumericDocValuesField("number2", 2))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 3))); @@ -237,7 +237,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 7))); iw.addDocument(singleton(new NumericDocValuesField("number2", 2))); iw.addDocument(singleton(new NumericDocValuesField("number", 3))); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/SumAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/SumAggregatorTests.java index b029417999c6e..1eecd43aaee2f 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/SumAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/SumAggregatorTests.java @@ -41,7 +41,7 @@ import org.apache.lucene.index.IndexableField; import org.apache.lucene.index.MultiReader; import org.apache.lucene.index.Term; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -135,7 +135,7 @@ public void testNumericDocValues() throws IOException { } public void testSortedNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery(FIELD_NAME), iw -> { + testAggregation(new FieldExistsQuery(FIELD_NAME), iw -> { iw.addDocument(Arrays.asList(new SortedNumericDocValuesField(FIELD_NAME, 3), new SortedNumericDocValuesField(FIELD_NAME, 4))); iw.addDocument(Arrays.asList(new SortedNumericDocValuesField(FIELD_NAME, 3), new SortedNumericDocValuesField(FIELD_NAME, 4))); iw.addDocument(singleton(new SortedNumericDocValuesField(FIELD_NAME, 1))); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java index c5b1cdae1e7e2..161fa87e2190c 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java @@ -37,7 +37,7 @@ import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -94,7 +94,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("number", 8))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 5))); iw.addDocument(singleton(new SortedNumericDocValuesField("number", 3))); @@ -118,7 +118,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testCase(new DocValuesFieldExistsQuery("number"), iw -> { + testCase(new FieldExistsQuery("number"), iw -> { iw.addDocument(singleton(new NumericDocValuesField("number", 8))); iw.addDocument(singleton(new NumericDocValuesField("number", 5))); iw.addDocument(singleton(new NumericDocValuesField("number", 3))); diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/TopHitsAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/TopHitsAggregatorTests.java index c355f0078108b..5e23740b9614b 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/TopHitsAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/TopHitsAggregatorTests.java @@ -96,7 +96,7 @@ public void testTopLevel() throws Exception { result = testCase(query, topHits("_name")); } SearchHits searchHits = ((TopHits) result).getHits(); - assertEquals(3L, searchHits.getTotalHits().value); + assertEquals(3L, searchHits.getTotalHits().value()); assertEquals("3", searchHits.getAt(0).getId()); assertEquals("2", searchHits.getAt(1).getId()); assertEquals("1", searchHits.getAt(2).getId()); @@ -106,7 +106,7 @@ public void testTopLevel() throws Exception { public void testNoResults() throws Exception { TopHits result = (TopHits) testCase(new MatchNoDocsQuery(), topHits("_name").sort("string", SortOrder.DESC)); SearchHits searchHits = ((TopHits) result).getHits(); - assertEquals(0L, searchHits.getTotalHits().value); + assertEquals(0L, searchHits.getTotalHits().value()); assertFalse(AggregationInspectionHelper.hasValue(((InternalTopHits) result))); } @@ -130,27 +130,27 @@ public void testInsideTerms() throws Exception { // The "a" bucket TopHits hits = (TopHits) terms.getBucketByKey("a").getAggregations().get("top"); SearchHits searchHits = (hits).getHits(); - assertEquals(2L, searchHits.getTotalHits().value); + assertEquals(2L, searchHits.getTotalHits().value()); assertEquals("2", searchHits.getAt(0).getId()); assertEquals("1", searchHits.getAt(1).getId()); assertTrue(AggregationInspectionHelper.hasValue(((InternalTopHits) terms.getBucketByKey("a").getAggregations().get("top")))); // The "b" bucket searchHits = ((TopHits) terms.getBucketByKey("b").getAggregations().get("top")).getHits(); - assertEquals(2L, searchHits.getTotalHits().value); + assertEquals(2L, searchHits.getTotalHits().value()); assertEquals("3", searchHits.getAt(0).getId()); assertEquals("1", searchHits.getAt(1).getId()); assertTrue(AggregationInspectionHelper.hasValue(((InternalTopHits) terms.getBucketByKey("b").getAggregations().get("top")))); // The "c" bucket searchHits = ((TopHits) terms.getBucketByKey("c").getAggregations().get("top")).getHits(); - assertEquals(1L, searchHits.getTotalHits().value); + assertEquals(1L, searchHits.getTotalHits().value()); assertEquals("2", searchHits.getAt(0).getId()); assertTrue(AggregationInspectionHelper.hasValue(((InternalTopHits) terms.getBucketByKey("c").getAggregations().get("top")))); // The "d" bucket searchHits = ((TopHits) terms.getBucketByKey("d").getAggregations().get("top")).getHits(); - assertEquals(1L, searchHits.getTotalHits().value); + assertEquals(1L, searchHits.getTotalHits().value()); assertEquals("3", searchHits.getAt(0).getId()); assertTrue(AggregationInspectionHelper.hasValue(((InternalTopHits) terms.getBucketByKey("d").getAggregations().get("top")))); } @@ -225,7 +225,7 @@ public void testSetScorer() throws Exception { .build(); AggregationBuilder agg = AggregationBuilders.topHits("top_hits"); TopHits result = searchAndReduce(searcher, query, agg, STRING_FIELD_TYPE); - assertEquals(3, result.getHits().getTotalHits().value); + assertEquals(3, result.getHits().getTotalHits().value()); reader.close(); directory.close(); } diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/ValueCountAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/ValueCountAggregatorTests.java index 9babdbe78bf14..1e3512e0f9545 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/ValueCountAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/ValueCountAggregatorTests.java @@ -41,7 +41,7 @@ import org.apache.lucene.document.SortedDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.document.SortedSetDocValuesField; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.tests.index.RandomIndexWriter; @@ -180,7 +180,7 @@ public void testNoMatchingField() throws IOException { } public void testSomeMatchesSortedNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery(FIELD_NAME), ValueType.NUMERIC, iw -> { + testAggregation(new FieldExistsQuery(FIELD_NAME), ValueType.NUMERIC, iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("wrong_number", 7))); iw.addDocument(singleton(new SortedNumericDocValuesField(FIELD_NAME, 7))); iw.addDocument(singleton(new SortedNumericDocValuesField(FIELD_NAME, 1))); @@ -191,7 +191,7 @@ public void testSomeMatchesSortedNumericDocValues() throws IOException { } public void testSomeMatchesNumericDocValues() throws IOException { - testAggregation(new DocValuesFieldExistsQuery(FIELD_NAME), ValueType.NUMBER, iw -> { + testAggregation(new FieldExistsQuery(FIELD_NAME), ValueType.NUMBER, iw -> { iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 7))); iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 1))); }, count -> { diff --git a/server/src/test/java/org/opensearch/search/aggregations/metrics/WeightedAvgAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/metrics/WeightedAvgAggregatorTests.java index 8d5992e15d983..40290b5197e13 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/metrics/WeightedAvgAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/metrics/WeightedAvgAggregatorTests.java @@ -37,7 +37,7 @@ import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; @@ -168,7 +168,7 @@ public void testSomeMatchesNumericDocValues() throws IOException { MultiValuesSourceFieldConfig weightConfig = new MultiValuesSourceFieldConfig.Builder().setFieldName("weight_field").build(); WeightedAvgAggregationBuilder aggregationBuilder = new WeightedAvgAggregationBuilder("_name").value(valueConfig) .weight(weightConfig); - testCase(new DocValuesFieldExistsQuery("value_field"), aggregationBuilder, iw -> { + testCase(new FieldExistsQuery("value_field"), aggregationBuilder, iw -> { iw.addDocument(Arrays.asList(new NumericDocValuesField("value_field", 7), new SortedNumericDocValuesField("weight_field", 1))); iw.addDocument(Arrays.asList(new NumericDocValuesField("value_field", 2), new SortedNumericDocValuesField("weight_field", 1))); iw.addDocument(Arrays.asList(new NumericDocValuesField("value_field", 3), new SortedNumericDocValuesField("weight_field", 1))); diff --git a/server/src/test/java/org/opensearch/search/aggregations/startree/MetricAggregatorTests.java b/server/src/test/java/org/opensearch/search/aggregations/startree/MetricAggregatorTests.java index 05f48eb9243af..21a4155a53ee5 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/startree/MetricAggregatorTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/startree/MetricAggregatorTests.java @@ -13,7 +13,7 @@ import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.lucene.codecs.Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.document.Document; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.DirectoryReader; @@ -34,7 +34,7 @@ import org.opensearch.core.indices.breaker.NoneCircuitBreakerService; import org.opensearch.index.codec.composite.CompositeIndexFieldInfo; import org.opensearch.index.codec.composite.CompositeIndexReader; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.index.codec.composite912.datacube.startree.StarTreeDocValuesFormatTests; import org.opensearch.index.compositeindex.datacube.Dimension; import org.opensearch.index.compositeindex.datacube.Metric; @@ -108,7 +108,7 @@ protected Codec getCodec() { } catch (IOException e) { throw new RuntimeException(e); } - return new Composite912Codec(Lucene912Codec.Mode.BEST_SPEED, mapperService, testLogger); + return new Composite101Codec(Lucene101Codec.Mode.BEST_SPEED, mapperService, testLogger); } public void testStarTreeDocValues() throws IOException { diff --git a/server/src/test/java/org/opensearch/search/aggregations/startree/StarTreeFilterTests.java b/server/src/test/java/org/opensearch/search/aggregations/startree/StarTreeFilterTests.java index c1cb19b9576e4..eb891318dd087 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/startree/StarTreeFilterTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/startree/StarTreeFilterTests.java @@ -11,7 +11,7 @@ import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.lucene.codecs.Codec; -import org.apache.lucene.codecs.lucene912.Lucene912Codec; +import org.apache.lucene.codecs.lucene101.Lucene101Codec; import org.apache.lucene.document.Document; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.DirectoryReader; @@ -29,7 +29,7 @@ import org.opensearch.core.xcontent.XContentBuilder; import org.opensearch.index.codec.composite.CompositeIndexFieldInfo; import org.opensearch.index.codec.composite.CompositeIndexReader; -import org.opensearch.index.codec.composite.composite912.Composite912Codec; +import org.opensearch.index.codec.composite.composite101.Composite101Codec; import org.opensearch.index.codec.composite912.datacube.startree.StarTreeDocValuesFormatTests; import org.opensearch.index.compositeindex.datacube.MetricStat; import org.opensearch.index.compositeindex.datacube.startree.index.StarTreeValues; @@ -76,7 +76,7 @@ protected Codec getCodec(int maxLeafDoc, boolean skipStarNodeCreationForSDVDimen } catch (IOException e) { throw new RuntimeException(e); } - return new Composite912Codec(Lucene912Codec.Mode.BEST_SPEED, mapperService, testLogger); + return new Composite101Codec(Lucene101Codec.Mode.BEST_SPEED, mapperService, testLogger); } public void testStarTreeFilterWithNoDocsInSVDField() throws IOException { diff --git a/server/src/test/java/org/opensearch/search/aggregations/support/MissingValuesTests.java b/server/src/test/java/org/opensearch/search/aggregations/support/MissingValuesTests.java index c44e0fcca7584..a0a280c34579e 100644 --- a/server/src/test/java/org/opensearch/search/aggregations/support/MissingValuesTests.java +++ b/server/src/test/java/org/opensearch/search/aggregations/support/MissingValuesTests.java @@ -152,7 +152,7 @@ public long nextOrd() { if (i < ords[doc].length) { return ords[doc][i++]; } else { - return NO_MORE_ORDS; + return NO_MORE_DOCS; } } @@ -178,10 +178,10 @@ public int docValueCount() { for (int ord : ords[i]) { assertEquals(values[ord], withMissingReplaced.lookupOrd(withMissingReplaced.nextOrd())); } - assertEquals(SortedSetDocValues.NO_MORE_ORDS, withMissingReplaced.nextOrd()); + assertEquals(SortedSetDocValues.NO_MORE_DOCS, withMissingReplaced.nextOrd()); } else { assertEquals(missing, withMissingReplaced.lookupOrd(withMissingReplaced.nextOrd())); - assertEquals(SortedSetDocValues.NO_MORE_ORDS, withMissingReplaced.nextOrd()); + assertEquals(SortedSetDocValues.NO_MORE_DOCS, withMissingReplaced.nextOrd()); } } } diff --git a/server/src/test/java/org/opensearch/search/approximate/ApproximatePointRangeQueryTests.java b/server/src/test/java/org/opensearch/search/approximate/ApproximatePointRangeQueryTests.java index 4919cbc599892..5d9b523c35f82 100644 --- a/server/src/test/java/org/opensearch/search/approximate/ApproximatePointRangeQueryTests.java +++ b/server/src/test/java/org/opensearch/search/approximate/ApproximatePointRangeQueryTests.java @@ -191,11 +191,11 @@ protected String toString(int dimension, byte[] value) { IndexSearcher searcher = new IndexSearcher(reader); TopDocs topDocs = searcher.search(approximateQuery, 11000); - if (topDocs.totalHits.relation == Relation.EQUAL_TO) { - assertEquals(topDocs.totalHits.value, 11000); + if (topDocs.totalHits.relation() == Relation.EQUAL_TO) { + assertEquals(topDocs.totalHits.value(), 11000); } else { - assertTrue(11000 <= topDocs.totalHits.value); - assertTrue(maxHits >= topDocs.totalHits.value); + assertTrue(11000 <= topDocs.totalHits.value()); + assertTrue(maxHits >= topDocs.totalHits.value()); } } catch (IOException e) { throw new RuntimeException(e); diff --git a/server/src/test/java/org/opensearch/search/geo/GeoShapeQueryTests.java b/server/src/test/java/org/opensearch/search/geo/GeoShapeQueryTests.java index 1262ea4750a99..4f78d9166b414 100644 --- a/server/src/test/java/org/opensearch/search/geo/GeoShapeQueryTests.java +++ b/server/src/test/java/org/opensearch/search/geo/GeoShapeQueryTests.java @@ -913,19 +913,19 @@ public void testDistanceQuery() throws Exception { SearchResponse response = client().prepareSearch("test_distance") .setQuery(QueryBuilders.geoShapeQuery("geo", circleBuilder.buildGeometry()).relation(ShapeRelation.WITHIN)) .get(); - assertEquals(2, response.getHits().getTotalHits().value); + assertEquals(2, response.getHits().getTotalHits().value()); response = client().prepareSearch("test_distance") .setQuery(QueryBuilders.geoShapeQuery("geo", circleBuilder.buildGeometry()).relation(ShapeRelation.INTERSECTS)) .get(); - assertEquals(2, response.getHits().getTotalHits().value); + assertEquals(2, response.getHits().getTotalHits().value()); response = client().prepareSearch("test_distance") .setQuery(QueryBuilders.geoShapeQuery("geo", circleBuilder.buildGeometry()).relation(ShapeRelation.DISJOINT)) .get(); - assertEquals(2, response.getHits().getTotalHits().value); + assertEquals(2, response.getHits().getTotalHits().value()); response = client().prepareSearch("test_distance") .setQuery(QueryBuilders.geoShapeQuery("geo", circleBuilder.buildGeometry()).relation(ShapeRelation.CONTAINS)) .get(); - assertEquals(0, response.getHits().getTotalHits().value); + assertEquals(0, response.getHits().getTotalHits().value()); } public void testIndexRectangleSpanningDateLine() throws Exception { diff --git a/server/src/test/java/org/opensearch/search/internal/ContextIndexSearcherTests.java b/server/src/test/java/org/opensearch/search/internal/ContextIndexSearcherTests.java index 606c2512a3d58..96c5d56ff008c 100644 --- a/server/src/test/java/org/opensearch/search/internal/ContextIndexSearcherTests.java +++ b/server/src/test/java/org/opensearch/search/internal/ContextIndexSearcherTests.java @@ -49,7 +49,6 @@ import org.apache.lucene.index.Term; import org.apache.lucene.index.TermsEnum; import org.apache.lucene.search.BoostQuery; -import org.apache.lucene.search.BulkScorer; import org.apache.lucene.search.ConstantScoreQuery; import org.apache.lucene.search.DocIdSetIterator; import org.apache.lucene.search.Explanation; @@ -60,7 +59,7 @@ import org.apache.lucene.search.QueryVisitor; import org.apache.lucene.search.Scorable; import org.apache.lucene.search.ScoreMode; -import org.apache.lucene.search.Scorer; +import org.apache.lucene.search.ScorerSupplier; import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.Weight; @@ -304,7 +303,7 @@ public void onRemoval(ShardId shardId, Accountable accountable) { assertEquals(1, searcher.count(new CreateScorerOnceQuery(new MatchAllDocsQuery()))); TopDocs topDocs = searcher.search(new BoostQuery(new ConstantScoreQuery(new TermQuery(new Term("foo", "bar"))), 3f), 1); - assertEquals(1, topDocs.totalHits.value); + assertEquals(1, topDocs.totalHits.value()); assertEquals(1, topDocs.scoreDocs.length); assertEquals(3f, topDocs.scoreDocs[0].score, 0); @@ -348,7 +347,7 @@ public void testSlicesInternal() throws Exception { // 2 slices will be created since max segment per slice of 5 will be reached assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { - assertEquals(5, slices[i].leaves.length); + assertEquals(5, slices[i].partitions.length); } // Case 2: Verify the slice count when custom max slice computation is used @@ -359,9 +358,9 @@ public void testSlicesInternal() throws Exception { assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { if (i < 2) { - assertEquals(3, slices[i].leaves.length); + assertEquals(3, slices[i].partitions.length); } else { - assertEquals(2, slices[i].leaves.length); + assertEquals(2, slices[i].partitions.length); } } } @@ -419,9 +418,9 @@ public void testGetSlicesWithNonNullExecutorButCSDisabled() throws Exception { assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { if (i < 2) { - assertEquals(3, slices[i].leaves.length); + assertEquals(3, slices[i].partitions.length); } else { - assertEquals(2, slices[i].leaves.length); + assertEquals(2, slices[i].partitions.length); } } } @@ -559,15 +558,8 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio } @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - assertTrue(seenLeaves.add(context.reader().getCoreCacheHelper().getKey())); - return weight.scorer(context); - } - - @Override - public BulkScorer bulkScorer(LeafReaderContext context) throws IOException { - assertTrue(seenLeaves.add(context.reader().getCoreCacheHelper().getKey())); - return weight.bulkScorer(context); + public ScorerSupplier scorerSupplier(LeafReaderContext leafReaderContext) throws IOException { + return weight.scorerSupplier(leafReaderContext); } @Override diff --git a/server/src/test/java/org/opensearch/search/internal/MaxTargetSliceSupplierTests.java b/server/src/test/java/org/opensearch/search/internal/MaxTargetSliceSupplierTests.java index 2684cf901f080..f14670f187958 100644 --- a/server/src/test/java/org/opensearch/search/internal/MaxTargetSliceSupplierTests.java +++ b/server/src/test/java/org/opensearch/search/internal/MaxTargetSliceSupplierTests.java @@ -25,7 +25,7 @@ public void testSliceCountGreaterThanLeafCount() throws Exception { // verify slice count is same as leaf count assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { - assertEquals(1, slices[i].leaves.length); + assertEquals(1, slices[i].partitions.length); } } @@ -37,7 +37,7 @@ public void testSingleSliceWithMultipleLeaves() throws Exception { int leafCount = randomIntBetween(1, 10); IndexSearcher.LeafSlice[] slices = MaxTargetSliceSupplier.getSlices(getLeaves(leafCount), 1); assertEquals(1, slices.length); - assertEquals(leafCount, slices[0].leaves.length); + assertEquals(leafCount, slices[0].partitions.length); } public void testSliceCountLessThanLeafCount() throws Exception { @@ -51,7 +51,7 @@ public void testSliceCountLessThanLeafCount() throws Exception { assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { - assertEquals(expectedLeavesPerSlice, slices[i].leaves.length); + assertEquals(expectedLeavesPerSlice, slices[i].partitions.length); } // Case 2: test with first 2 slice more leaves than others @@ -63,9 +63,9 @@ public void testSliceCountLessThanLeafCount() throws Exception { assertEquals(expectedSliceCount, slices.length); for (int i = 0; i < expectedSliceCount; ++i) { if (i < 2) { - assertEquals(expectedLeavesInFirst2Slice, slices[i].leaves.length); + assertEquals(expectedLeavesInFirst2Slice, slices[i].partitions.length); } else { - assertEquals(expectedLeavesInOtherSlice, slices[i].leaves.length); + assertEquals(expectedLeavesInOtherSlice, slices[i].partitions.length); } } } diff --git a/server/src/test/java/org/opensearch/search/lookup/LeafFieldsLookupTests.java b/server/src/test/java/org/opensearch/search/lookup/LeafFieldsLookupTests.java index 8c4b8ad6d1776..d2ee314cf5b5c 100644 --- a/server/src/test/java/org/opensearch/search/lookup/LeafFieldsLookupTests.java +++ b/server/src/test/java/org/opensearch/search/lookup/LeafFieldsLookupTests.java @@ -31,6 +31,7 @@ package org.opensearch.search.lookup; +import org.apache.lucene.index.DocValuesSkipIndexType; import org.apache.lucene.index.DocValuesType; import org.apache.lucene.index.FieldInfo; import org.apache.lucene.index.IndexOptions; @@ -77,6 +78,7 @@ public void setUp() throws Exception { true, IndexOptions.NONE, DocValuesType.NONE, + DocValuesSkipIndexType.NONE, -1, Collections.emptyMap(), 0, diff --git a/server/src/test/java/org/opensearch/search/profile/query/ProfileScorerTests.java b/server/src/test/java/org/opensearch/search/profile/query/ProfileScorerTests.java index 5b2cb3e17fb38..a4ff92ec556e9 100644 --- a/server/src/test/java/org/opensearch/search/profile/query/ProfileScorerTests.java +++ b/server/src/test/java/org/opensearch/search/profile/query/ProfileScorerTests.java @@ -51,7 +51,7 @@ private static class FakeScorer extends Scorer { public float maxScore, minCompetitiveScore; protected FakeScorer(Weight weight) { - super(weight); + super(); } @Override @@ -85,8 +85,7 @@ public void testPropagateMinCompetitiveScore() throws IOException { Weight weight = query.createWeight(new IndexSearcher(new MultiReader()), ScoreMode.TOP_SCORES, 1f); FakeScorer fakeScorer = new FakeScorer(weight); QueryProfileBreakdown profile = new QueryProfileBreakdown(); - ProfileWeight profileWeight = new ProfileWeight(query, weight, profile); - ProfileScorer profileScorer = new ProfileScorer(profileWeight, fakeScorer, profile); + ProfileScorer profileScorer = new ProfileScorer(fakeScorer, profile); profileScorer.setMinCompetitiveScore(0.42f); assertEquals(0.42f, fakeScorer.minCompetitiveScore, 0f); } @@ -96,8 +95,7 @@ public void testPropagateMaxScore() throws IOException { Weight weight = query.createWeight(new IndexSearcher(new MultiReader()), ScoreMode.TOP_SCORES, 1f); FakeScorer fakeScorer = new FakeScorer(weight); QueryProfileBreakdown profile = new QueryProfileBreakdown(); - ProfileWeight profileWeight = new ProfileWeight(query, weight, profile); - ProfileScorer profileScorer = new ProfileScorer(profileWeight, fakeScorer, profile); + ProfileScorer profileScorer = new ProfileScorer(fakeScorer, profile); profileScorer.setMinCompetitiveScore(0.42f); fakeScorer.maxScore = 42f; assertEquals(42f, profileScorer.getMaxScore(DocIdSetIterator.NO_MORE_DOCS), 0f); diff --git a/server/src/test/java/org/opensearch/search/profile/query/QueryProfilerTests.java b/server/src/test/java/org/opensearch/search/profile/query/QueryProfilerTests.java index 3a7c711d324c4..59469a77e90fc 100644 --- a/server/src/test/java/org/opensearch/search/profile/query/QueryProfilerTests.java +++ b/server/src/test/java/org/opensearch/search/profile/query/QueryProfilerTests.java @@ -437,11 +437,6 @@ public Explanation explain(LeafReaderContext context, int doc) throws IOExceptio throw new UnsupportedOperationException(); } - @Override - public Scorer scorer(LeafReaderContext context) throws IOException { - throw new UnsupportedOperationException(); - } - @Override public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException { return new ScorerSupplier() { diff --git a/server/src/test/java/org/opensearch/search/query/QueryPhaseTests.java b/server/src/test/java/org/opensearch/search/query/QueryPhaseTests.java index 84057ab1a1b15..20861e146d93d 100644 --- a/server/src/test/java/org/opensearch/search/query/QueryPhaseTests.java +++ b/server/src/test/java/org/opensearch/search/query/QueryPhaseTests.java @@ -58,12 +58,13 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.Collector; import org.apache.lucene.search.ConstantScoreQuery; -import org.apache.lucene.search.DocValuesFieldExistsQuery; import org.apache.lucene.search.FieldComparator; import org.apache.lucene.search.FieldDoc; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.FilterCollector; import org.apache.lucene.search.FilterLeafCollector; import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.IndexSearcher.LeafReaderContextPartition; import org.apache.lucene.search.LeafCollector; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; @@ -191,7 +192,7 @@ private void countTestCase(Query query, IndexReader reader, boolean shouldCollec ContextIndexSearcher countSearcher = shouldCollectCount ? newContextSearcher(reader, executor) : newEarlyTerminationContextSearcher(reader, 0, executor); - assertEquals(countSearcher.count(query), context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(countSearcher.count(query), context.queryResult().topDocs().topDocs.totalHits.value()); } private void countTestCase(boolean withDeletions) throws Exception { @@ -225,11 +226,11 @@ private void countTestCase(boolean withDeletions) throws Exception { Query matchAllCsq = new ConstantScoreQuery(matchAll); Query tq = new TermQuery(new Term("foo", "bar")); Query tCsq = new ConstantScoreQuery(tq); - Query dvfeq = new DocValuesFieldExistsQuery("foo"); - Query dvfeq_points = new DocValuesFieldExistsQuery("latLonDVField"); + Query dvfeq = new FieldExistsQuery("foo"); + Query dvfeq_points = new FieldExistsQuery("latLonDVField"); Query dvfeqCsq = new ConstantScoreQuery(dvfeq); // field with doc-values but not indexed will need to collect - Query dvOnlyfeq = new DocValuesFieldExistsQuery("docValuesOnlyField"); + Query dvOnlyfeq = new FieldExistsQuery("docValuesOnlyField"); BooleanQuery bq = new BooleanQuery.Builder().add(matchAll, Occur.SHOULD).add(tq, Occur.MUST).build(); countTestCase(matchAll, reader, false, false); @@ -270,12 +271,12 @@ public void testPostFilterDisablesCountOptimization() throws Exception { context.parsedQuery(new ParsedQuery(new MatchAllDocsQuery())); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); context.setSearcher(newContextSearcher(reader, executor)); context.parsedPostFilter(new ParsedQuery(new MatchNoDocsQuery())); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value()); reader.close(); dir.close(); } @@ -303,7 +304,7 @@ public void testTerminateAfterWithFilter() throws Exception { for (int i = 0; i < 10; i++) { context.parsedPostFilter(new ParsedQuery(new TermQuery(new Term("foo", Integer.toString(i))))); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); } reader.close(); @@ -325,12 +326,12 @@ public void testMinScoreDisablesCountOptimization() throws Exception { context.setSize(0); context.setTask(new SearchShardTask(123L, "", "", "", null, Collections.emptyMap())); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); context.minimumScore(100); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value); - assertEquals(TotalHits.Relation.EQUAL_TO, context.queryResult().topDocs().topDocs.totalHits.relation); + assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value()); + assertEquals(TotalHits.Relation.EQUAL_TO, context.queryResult().topDocs().topDocs.totalHits.relation()); reader.close(); dir.close(); } @@ -380,15 +381,15 @@ public void testInOrderScrollOptimization() throws Exception { context.setSize(size); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertNull(context.queryResult().terminatedEarly()); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); context.setSearcher(newEarlyTerminationContextSearcher(reader, size, executor)); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0].doc, greaterThanOrEqualTo(size)); reader.close(); dir.close(); @@ -426,7 +427,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.queryCollectorManagers().put(TotalHitCountCollector.class, manager); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertFalse(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(10)); assertThat(manager.getTotalHits(), equalTo(numDocs)); } @@ -436,7 +437,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.setSize(1); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); // Do not expect an exact match when terminate_after is used in conjunction to size = 0 as an optimization introduced by @@ -446,7 +447,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); assertThat( - context.queryResult().topDocs().topDocs.totalHits.value, + context.queryResult().topDocs().topDocs.totalHits.value(), allOf(greaterThanOrEqualTo(1L), lessThanOrEqualTo((long) numDocs)) ); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(0)); @@ -456,7 +457,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.setSize(1); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); } { @@ -467,7 +468,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.parsedQuery(new ParsedQuery(bq)); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); context.setSize(0); @@ -475,7 +476,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); assertThat( - context.queryResult().topDocs().topDocs.totalHits.value, + context.queryResult().topDocs().topDocs.totalHits.value(), allOf(greaterThanOrEqualTo(1L), lessThanOrEqualTo((long) numDocs)) ); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(0)); @@ -486,7 +487,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.queryCollectorManagers().put(TotalHitCountCollector.class, manager); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(manager.getTotalHits(), equalTo(1)); context.queryCollectorManagers().clear(); @@ -498,7 +499,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); assertThat( - context.queryResult().topDocs().topDocs.totalHits.value, + context.queryResult().topDocs().topDocs.totalHits.value(), allOf(greaterThanOrEqualTo(1L), lessThanOrEqualTo((long) numDocs)) ); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(0)); @@ -515,10 +516,10 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); if (trackTotalHits == -1) { - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(0L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(0L)); } else { assertThat( - context.queryResult().topDocs().topDocs.totalHits.value, + context.queryResult().topDocs().topDocs.totalHits.value(), allOf(greaterThanOrEqualTo(Math.min(trackTotalHits, 10L)), lessThanOrEqualTo((long) numDocs)) ); } @@ -539,9 +540,9 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); if (trackTotalHits == -1) { - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(0L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(0L)); } else { - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(7L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(7L)); } assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(7)); } @@ -576,7 +577,7 @@ public void testIndexSortingEarlyTermination() throws Exception { context.sort(new SortAndFormats(sort, new DocValueFormat[] { DocValueFormat.RAW })); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0], instanceOf(FieldDoc.class)); FieldDoc fieldDoc = (FieldDoc) context.queryResult().topDocs().topDocs.scoreDocs[0]; @@ -586,7 +587,7 @@ public void testIndexSortingEarlyTermination() throws Exception { context.parsedPostFilter(new ParsedQuery(new MinDocQuery(1))); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertNull(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(numDocs - 1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(numDocs - 1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0], instanceOf(FieldDoc.class)); assertThat(fieldDoc.fields[0], anyOf(equalTo(1), equalTo(2))); @@ -596,7 +597,7 @@ public void testIndexSortingEarlyTermination() throws Exception { context.queryCollectorManagers().put(TotalHitCountCollector.class, manager); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertNull(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0], instanceOf(FieldDoc.class)); assertThat(fieldDoc.fields[0], anyOf(equalTo(1), equalTo(2))); @@ -659,19 +660,19 @@ public void testIndexSortScrollOptimization() throws Exception { context.sort(searchSortAndFormat); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertNull(context.queryResult().terminatedEarly()); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); int sizeMinus1 = context.queryResult().topDocs().topDocs.scoreDocs.length - 1; FieldDoc lastDoc = (FieldDoc) context.queryResult().topDocs().topDocs.scoreDocs[sizeMinus1]; context.setSearcher(newEarlyTerminationContextSearcher(reader, 10, executor)); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertNull(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); FieldDoc firstDoc = (FieldDoc) context.queryResult().topDocs().topDocs.scoreDocs[0]; for (int i = 0; i < searchSortAndFormat.sort.getSort().length; i++) { @SuppressWarnings("unchecked") @@ -721,17 +722,17 @@ public void testDisableTopScoreCollection() throws Exception { TopDocsCollectorContext topDocsContext = TopDocsCollectorContext.createTopDocsCollectorContext(context, false); assertEquals(topDocsContext.create(null).scoreMode(), org.apache.lucene.search.ScoreMode.COMPLETE); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value); - assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation, TotalHits.Relation.EQUAL_TO); + assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value()); + assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation(), TotalHits.Relation.EQUAL_TO); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(3)); context.sort(new SortAndFormats(new Sort(new SortField("other", SortField.Type.INT)), new DocValueFormat[] { DocValueFormat.RAW })); topDocsContext = TopDocsCollectorContext.createTopDocsCollectorContext(context, false); assertEquals(topDocsContext.create(null).scoreMode(), org.apache.lucene.search.ScoreMode.TOP_DOCS); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value()); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(3)); - assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation, TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO); + assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation(), TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO); reader.close(); dir.close(); @@ -874,8 +875,8 @@ public void testEnhanceSortOnNumeric() throws Exception { assertSortResults(topDocs, (long) numDocs, false); final TotalHits totalHits = topDocs.totalHits; - assertEquals(TotalHits.Relation.EQUAL_TO, totalHits.relation); - assertEquals(numDocs, totalHits.value); + assertEquals(TotalHits.Relation.EQUAL_TO, totalHits.relation()); + assertEquals(numDocs, totalHits.value()); } reader.close(); @@ -939,10 +940,10 @@ public void testMaxScoreQueryVisitor() { // assert score docs are in order and their number is as expected private void assertSortResults(TopDocs topDocs, long expectedNumDocs, boolean isDoubleSort) { - if (topDocs.totalHits.relation == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { - assertThat(topDocs.totalHits.value, lessThanOrEqualTo(expectedNumDocs)); + if (topDocs.totalHits.relation() == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) { + assertThat(topDocs.totalHits.value(), lessThanOrEqualTo(expectedNumDocs)); } else { - assertEquals(topDocs.totalHits.value, expectedNumDocs); + assertEquals(topDocs.totalHits.value(), expectedNumDocs); } long cur1, cur2; long prev1 = Long.MIN_VALUE; @@ -988,7 +989,7 @@ public void testMinScore() throws Exception { context.trackTotalHitsUpTo(5); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); - assertEquals(10, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(10, context.queryResult().topDocs().topDocs.totalHits.value()); reader.close(); dir.close(); @@ -1027,19 +1028,19 @@ public void testMaxScore() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(1, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, greaterThanOrEqualTo(6L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), greaterThanOrEqualTo(6L)); context.sort(new SortAndFormats(sort, new DocValueFormat[] { DocValueFormat.RAW })); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(1, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, greaterThanOrEqualTo(6L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), greaterThanOrEqualTo(6L)); context.trackScores(false); QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(1, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, greaterThanOrEqualTo(6L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), greaterThanOrEqualTo(6L)); reader.close(); dir.close(); @@ -1078,7 +1079,7 @@ public void testCollapseQuerySearchResults() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(2, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs, instanceOf(CollapseTopFieldDocs.class)); CollapseTopFieldDocs topDocs = (CollapseTopFieldDocs) context.queryResult().topDocs().topDocs; @@ -1090,7 +1091,7 @@ public void testCollapseQuerySearchResults() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(2, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs, instanceOf(CollapseTopFieldDocs.class)); topDocs = (CollapseTopFieldDocs) context.queryResult().topDocs().topDocs; @@ -1102,7 +1103,7 @@ public void testCollapseQuerySearchResults() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult(), queryPhaseSearcher); assertTrue(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(2, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs, instanceOf(CollapseTopFieldDocs.class)); topDocs = (CollapseTopFieldDocs) context.queryResult().topDocs().topDocs; @@ -1273,9 +1274,9 @@ private static ContextIndexSearcher newEarlyTerminationContextSearcher(IndexRead ) { @Override - public void search(List leaves, Weight weight, Collector collector) throws IOException { + protected void search(LeafReaderContextPartition[] partitions, Weight weight, Collector collector) throws IOException { final Collector in = new AssertingEarlyTerminationFilterCollector(collector, size); - super.search(leaves, weight, in); + super.search(partitions, weight, in); } }; } diff --git a/server/src/test/java/org/opensearch/search/query/QueryProfilePhaseTests.java b/server/src/test/java/org/opensearch/search/query/QueryProfilePhaseTests.java index a44b654e462f5..6c52e9f0aed9a 100644 --- a/server/src/test/java/org/opensearch/search/query/QueryProfilePhaseTests.java +++ b/server/src/test/java/org/opensearch/search/query/QueryProfilePhaseTests.java @@ -32,6 +32,7 @@ import org.apache.lucene.search.FilterCollector; import org.apache.lucene.search.FilterLeafCollector; import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.IndexSearcher.LeafReaderContextPartition; import org.apache.lucene.search.LeafCollector; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.MatchNoDocsQuery; @@ -164,7 +165,7 @@ public void testPostFilterDisablesCountOptimization() throws Exception { context.parsedQuery(new ParsedQuery(new MatchAllDocsQuery())); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery // see: https://github.com/apache/lucene/pull/672 assertProfileData(context, "ConstantScoreQuery", query -> { @@ -189,7 +190,7 @@ public void testPostFilterDisablesCountOptimization() throws Exception { context.setSearcher(newContextSearcher(reader, executor)); context.parsedPostFilter(new ParsedQuery(new MatchNoDocsQuery())); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value()); assertProfileData(context, collector -> { assertThat(collector.getReason(), equalTo("search_post_filter")); assertThat(collector.getTime(), greaterThan(0L)); @@ -255,7 +256,7 @@ public void testTerminateAfterWithFilter() throws Exception { for (int i = 0; i < 10; i++) { context.parsedPostFilter(new ParsedQuery(new TermQuery(new Term("foo", Integer.toString(i))))); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertProfileData(context, collector -> { assertThat(collector.getReason(), equalTo("search_post_filter")); @@ -322,7 +323,7 @@ public void testMinScoreDisablesCountOptimization() throws Exception { context.setSize(0); context.setTask(new SearchShardTask(123L, "", "", "", null, Collections.emptyMap())); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(1, context.queryResult().topDocs().topDocs.totalHits.value()); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery // see: https://github.com/apache/lucene/pull/672 assertProfileData(context, "ConstantScoreQuery", query -> { @@ -346,8 +347,8 @@ public void testMinScoreDisablesCountOptimization() throws Exception { context.minimumScore(100); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value); - assertEquals(TotalHits.Relation.EQUAL_TO, context.queryResult().topDocs().topDocs.totalHits.relation); + assertEquals(0, context.queryResult().topDocs().topDocs.totalHits.value()); + assertEquals(TotalHits.Relation.EQUAL_TO, context.queryResult().topDocs().topDocs.totalHits.relation()); assertProfileData(context, "MatchAllDocsQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), greaterThanOrEqualTo(1L)); @@ -416,10 +417,10 @@ public void testInOrderScrollOptimization() throws Exception { context.setSize(size); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertNull(context.queryResult().terminatedEarly()); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); assertProfileData(context, "ConstantScoreQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), equalTo(0L)); @@ -441,8 +442,8 @@ public void testInOrderScrollOptimization() throws Exception { context.setSearcher(newEarlyTerminationContextSearcher(reader, size, executor)); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0].doc, greaterThanOrEqualTo(size)); assertProfileData(context, "ConstantScoreQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); @@ -498,7 +499,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.setSize(1); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertProfileData(context, "MatchAllDocsQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); @@ -539,7 +540,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.setSize(0); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(0)); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery // see: https://github.com/apache/lucene/pull/672 @@ -576,7 +577,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.setSize(1); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertProfileData(context, "MatchAllDocsQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); @@ -622,7 +623,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.parsedQuery(new ParsedQuery(bq)); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertProfileData(context, "BooleanQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); @@ -684,7 +685,7 @@ public void testTerminateAfterEarlyTermination() throws Exception { context.parsedQuery(new ParsedQuery(bq)); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(0)); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery @@ -751,9 +752,9 @@ public void testTerminateAfterEarlyTermination() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertTrue(context.queryResult().terminatedEarly()); if (trackTotalHits == -1) { - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(0L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(0L)); } else { - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(7L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(7L)); } assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(7)); assertProfileData(context, "BooleanQuery", query -> { @@ -867,7 +868,7 @@ public void testIndexSortingEarlyTermination() throws Exception { context.sort(new SortAndFormats(sort, new DocValueFormat[] { DocValueFormat.RAW })); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0], instanceOf(FieldDoc.class)); FieldDoc fieldDoc = (FieldDoc) context.queryResult().topDocs().topDocs.scoreDocs[0]; @@ -897,7 +898,7 @@ public void testIndexSortingEarlyTermination() throws Exception { context.parsedPostFilter(new ParsedQuery(new MinDocQuery(1))); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertNull(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo(numDocs - 1L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo(numDocs - 1L)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(1)); assertThat(context.queryResult().topDocs().topDocs.scoreDocs[0], instanceOf(FieldDoc.class)); assertThat(fieldDoc.fields[0], anyOf(equalTo(1), equalTo(2))); @@ -1035,10 +1036,10 @@ public void testIndexSortScrollOptimization() throws Exception { context.sort(searchSortAndFormat); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertNull(context.queryResult().terminatedEarly()); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery // see: https://github.com/apache/lucene/pull/672 assertProfileData(context, "ConstantScoreQuery", query -> { @@ -1066,9 +1067,9 @@ public void testIndexSortScrollOptimization() throws Exception { context.setSearcher(newEarlyTerminationContextSearcher(reader, 10, executor)); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertNull(context.queryResult().terminatedEarly()); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.terminateAfter(), equalTo(0)); - assertThat(context.queryResult().getTotalHits().value, equalTo((long) numDocs)); + assertThat(context.queryResult().getTotalHits().value(), equalTo((long) numDocs)); assertProfileData(context, "ConstantScoreQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), equalTo(0L)); @@ -1144,8 +1145,8 @@ public void testDisableTopScoreCollection() throws Exception { TopDocsCollectorContext topDocsContext = TopDocsCollectorContext.createTopDocsCollectorContext(context, false); assertEquals(topDocsContext.create(null).scoreMode(), org.apache.lucene.search.ScoreMode.COMPLETE); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value); - assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation, TotalHits.Relation.EQUAL_TO); + assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value()); + assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation(), TotalHits.Relation.EQUAL_TO); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(3)); assertProfileData(context, "SpanNearQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); @@ -1178,9 +1179,9 @@ public void testDisableTopScoreCollection() throws Exception { topDocsContext = TopDocsCollectorContext.createTopDocsCollectorContext(context, false); assertEquals(topDocsContext.create(null).scoreMode(), org.apache.lucene.search.ScoreMode.TOP_DOCS); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(numDocs / 2, context.queryResult().topDocs().topDocs.totalHits.value()); assertThat(context.queryResult().topDocs().topDocs.scoreDocs.length, equalTo(3)); - assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation, TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO); + assertEquals(context.queryResult().topDocs().topDocs.totalHits.relation(), TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO); // IndexSearcher#rewrite optimizes by rewriting non-scoring queries to ConstantScoreQuery // see: https://github.com/apache/lucene/pull/672 assertProfileData(context, "ConstantScoreQuery", query -> { @@ -1233,7 +1234,7 @@ public void testMinScore() throws Exception { context.trackTotalHitsUpTo(5); QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); - assertEquals(10, context.queryResult().topDocs().topDocs.totalHits.value); + assertEquals(10, context.queryResult().topDocs().topDocs.totalHits.value()); assertProfileData(context, "BooleanQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), greaterThan(0L)); @@ -1318,7 +1319,7 @@ public void testMaxScore() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(1, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, greaterThanOrEqualTo(6L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), greaterThanOrEqualTo(6L)); assertProfileData(context, "BooleanQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), greaterThan(0L)); @@ -1361,7 +1362,7 @@ public void testMaxScore() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(1, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, greaterThanOrEqualTo(6L)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), greaterThanOrEqualTo(6L)); assertProfileData(context, "BooleanQuery", query -> { assertThat(query.getTimeBreakdown().keySet(), not(empty())); assertThat(query.getTimeBreakdown().get("score"), greaterThan(0L)); @@ -1437,7 +1438,7 @@ public void testCollapseQuerySearchResults() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(2, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs, instanceOf(CollapseTopFieldDocs.class)); assertProfileData(context, "TermQuery", query -> { @@ -1482,7 +1483,7 @@ public void testCollapseQuerySearchResults() throws Exception { QueryPhase.executeInternal(context.withCleanQueryResult().withProfilers(), queryPhaseSearcher); assertFalse(Float.isNaN(context.queryResult().getMaxScore())); assertEquals(2, context.queryResult().topDocs().topDocs.scoreDocs.length); - assertThat(context.queryResult().topDocs().topDocs.totalHits.value, equalTo((long) numDocs)); + assertThat(context.queryResult().topDocs().topDocs.totalHits.value(), equalTo((long) numDocs)); assertThat(context.queryResult().topDocs().topDocs, instanceOf(CollapseTopFieldDocs.class)); assertProfileData(context, "TermQuery", query -> { @@ -1712,9 +1713,9 @@ private static ContextIndexSearcher newEarlyTerminationContextSearcher(IndexRead ) { @Override - public void search(List leaves, Weight weight, Collector collector) throws IOException { + protected void search(LeafReaderContextPartition[] partitions, Weight weight, Collector collector) throws IOException { final Collector in = new AssertingEarlyTerminationFilterCollector(collector, size); - super.search(leaves, weight, in); + super.search(partitions, weight, in); } }; } diff --git a/server/src/test/java/org/opensearch/search/sort/BucketedSortForFloatsTests.java b/server/src/test/java/org/opensearch/search/sort/BucketedSortForFloatsTests.java index 5cf3cc1d46a05..574564bb37767 100644 --- a/server/src/test/java/org/opensearch/search/sort/BucketedSortForFloatsTests.java +++ b/server/src/test/java/org/opensearch/search/sort/BucketedSortForFloatsTests.java @@ -144,18 +144,12 @@ public void testScorer() throws IOException { } private class MockScorable extends Scorable { - private int doc; private float score; @Override public float score() throws IOException { return score; } - - @Override - public int docID() { - return doc; - } } /** diff --git a/server/src/test/java/org/opensearch/search/suggest/completion/CategoryContextMappingTests.java b/server/src/test/java/org/opensearch/search/suggest/completion/CategoryContextMappingTests.java index ff85a912552fc..09a66a2cfcd91 100644 --- a/server/src/test/java/org/opensearch/search/suggest/completion/CategoryContextMappingTests.java +++ b/server/src/test/java/org/opensearch/search/suggest/completion/CategoryContextMappingTests.java @@ -799,7 +799,9 @@ public void testParsingContextFromDocument() throws Exception { ParseContext.Document document = new ParseContext.Document(); KeywordFieldMapper.KeywordFieldType keyword = new KeywordFieldMapper.KeywordFieldType("category"); - document.add(new KeywordFieldMapper.KeywordField(keyword.name(), new BytesRef("category1"), new FieldType())); + FieldType keywordFieldType = new FieldType(); + keywordFieldType.setStored(true); + document.add(new KeywordFieldMapper.KeywordField(keyword.name(), new BytesRef("category1"), keywordFieldType)); // Ignore doc values document.add(new SortedSetDocValuesField(keyword.name(), new BytesRef("category1"))); Set context = mapping.parseContext(document); diff --git a/server/src/test/java/org/opensearch/search/suggest/phrase/DirectCandidateGeneratorTests.java b/server/src/test/java/org/opensearch/search/suggest/phrase/DirectCandidateGeneratorTests.java index 25ee8f95b8acb..e123a671652b5 100644 --- a/server/src/test/java/org/opensearch/search/suggest/phrase/DirectCandidateGeneratorTests.java +++ b/server/src/test/java/org/opensearch/search/suggest/phrase/DirectCandidateGeneratorTests.java @@ -234,8 +234,8 @@ public void testFrequencyThreshold() throws Exception { ) ); assertThat(candidateSet.candidates.length, equalTo(1)); - assertThat(candidateSet.candidates[0].termStats.docFreq, equalTo(numDocs - 1)); - assertThat(candidateSet.candidates[0].termStats.totalTermFreq, equalTo((long) numDocs - 1)); + assertThat(candidateSet.candidates[0].termStats.docFreq(), equalTo(numDocs - 1)); + assertThat(candidateSet.candidates[0].termStats.totalTermFreq(), equalTo((long) numDocs - 1)); // test that it doesn't overflow assertThat(generator.thresholdTermFrequency(Integer.MAX_VALUE), equalTo(Integer.MAX_VALUE)); @@ -250,8 +250,8 @@ public void testFrequencyThreshold() throws Exception { ) ); assertThat(candidateSet.candidates.length, equalTo(1)); - assertThat(candidateSet.candidates[0].termStats.docFreq, equalTo(numDocs - 1)); - assertThat(candidateSet.candidates[0].termStats.totalTermFreq, equalTo((long) numDocs - 1)); + assertThat(candidateSet.candidates[0].termStats.docFreq(), equalTo(numDocs - 1)); + assertThat(candidateSet.candidates[0].termStats.totalTermFreq(), equalTo((long) numDocs - 1)); // test that it doesn't overflow assertThat(generator.thresholdTermFrequency(Integer.MAX_VALUE), equalTo(Integer.MAX_VALUE)); diff --git a/server/src/test/java/org/opensearch/snapshots/RestoreServiceIntegTests.java b/server/src/test/java/org/opensearch/snapshots/RestoreServiceIntegTests.java index 92da980d70f34..bfda27c97ca4c 100644 --- a/server/src/test/java/org/opensearch/snapshots/RestoreServiceIntegTests.java +++ b/server/src/test/java/org/opensearch/snapshots/RestoreServiceIntegTests.java @@ -284,9 +284,9 @@ public void testRestoreWithRename() throws Exception { isSearchDone.await(waitInSeconds, TimeUnit.SECONDS); - assertEquals(documents, Objects.requireNonNull(searchIndexResponseListener.result().getHits().getTotalHits()).value); + assertEquals(documents, Objects.requireNonNull(searchIndexResponseListener.result().getHits().getTotalHits()).value()); if (this.includeAlias) { - assertEquals(documents, Objects.requireNonNull(searchAliasResponseListener.result().getHits().getTotalHits()).value); + assertEquals(documents, Objects.requireNonNull(searchAliasResponseListener.result().getHits().getTotalHits()).value()); } } } diff --git a/server/src/test/java/org/opensearch/snapshots/SnapshotResiliencyTests.java b/server/src/test/java/org/opensearch/snapshots/SnapshotResiliencyTests.java index d21282ff0441f..697ec511b54f2 100644 --- a/server/src/test/java/org/opensearch/snapshots/SnapshotResiliencyTests.java +++ b/server/src/test/java/org/opensearch/snapshots/SnapshotResiliencyTests.java @@ -422,7 +422,7 @@ public void testSuccessfulSnapshotAndRestore() { final AtomicBoolean documentCountVerified = new AtomicBoolean(); continueOrDie(searchResponseListener, r -> { - assertEquals(documents, Objects.requireNonNull(r.getHits().getTotalHits()).value); + assertEquals(documents, Objects.requireNonNull(r.getHits().getTotalHits()).value()); documentCountVerified.set(true); }); @@ -1041,11 +1041,11 @@ public void testConcurrentSnapshotRestoreAndDeleteOther() { assertEquals( documentsFirstSnapshot + documentsSecondSnapshot, - Objects.requireNonNull(searchIndexResponseListener.result().getHits().getTotalHits()).value + Objects.requireNonNull(searchIndexResponseListener.result().getHits().getTotalHits()).value() ); assertEquals( documentsFirstSnapshot + documentsSecondSnapshot, - Objects.requireNonNull(searchAliasResponseListener.result().getHits().getTotalHits()).value + Objects.requireNonNull(searchAliasResponseListener.result().getHits().getTotalHits()).value() ); assertThat(deleteSnapshotStepListener.result().isAcknowledged(), is(true)); assertThat(restoreSnapshotResponseListener.result().getRestoreInfo().failedShards(), is(0)); @@ -1405,7 +1405,7 @@ public void testSuccessfulSnapshotWithConcurrentDynamicMappingUpdates() { final AtomicBoolean documentCountVerified = new AtomicBoolean(); continueOrDie(searchResponseStepListener, r -> { - final long hitCount = r.getHits().getTotalHits().value; + final long hitCount = r.getHits().getTotalHits().value(); assertThat( "Documents were restored but the restored index mapping was older than some documents and misses some of their fields", (int) hitCount, diff --git a/server/src/test/resources/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip b/server/src/test/resources/indices/bwc/es-6.3.0/testIndex-es-6.3.0.zip deleted file mode 100644 index db86a76153b25b599dd5ba0a37042b1f44592e27..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2467 zcmZ{mc|26>AIHbomnI^z#MqY@F&M?z`MH)t80%PO#+oc+@5)Y5GE9sy&DhsML=oB7 z<{}|=DZ5DBOehL>s^5LJyzc$uJkN8U^LoF|>-%}W=X^d^=8Q~y004ju@Y*%rPVGjQ z=rH~61_J=VLZ97ELQr0qhIXH%p>8XSxw{=ZOd#e4Y~(z~sGmP@%m$ z^=WHtzpdu!?QAdTJ6eA?zF+uN>7p!DFHRIcY=~1Hq$a10#9LkMl%p~4^jd9vunJ>P zjE=&{b8BS7ZAXd5%1Wg`1qbG%T>1@_0EJkPLc2&wB~zCdB-x3V=?NMXeo1gVO0mCi z?RSSw8&#;d2*{)2w@x2o!SI4<;!tX$n!e@y6lbG6$0AP!I9Jma!!c?Zb+Xfs?GIwC z9*V|U0c1h-Ix7RLeyX@`DkEu|2>{qV{6odqKkUsdrcI8K1usUlu^a(sCea4!eXAuH zHXzSv72K9X#Zr2vQwc)Gj;FTkG-nz+x5o+;#zYgwQ=f|{sHY-}?%p#QDov{@s4}%X ze5wQ&q7PR~C2QK+T^YBp8XFa?J@f)rRd2)Ex{$YX@YlhfuD6e5aD|72w8zo^CBg14 z7UkG1c|7K05Ij7r9kY>p`}#rT=FB}w;ab;S5iXuJnbc?}sD<%0VQh=1XN*HO$DMbe zFblo-@KF3#WDJQ?TUGPyUVBr`zMkPir(tS7(&vJ2fL8tfP@ z+|qxf#TEA;5P6yW7%Bl@^Bz??+^x4b7H8IU`{O`>&1&}YCutE=dyEAmPtd~5o0Da7 zNY|7i1YxWqHLW)zzY+De)^h0iTHFYMn6YHdEQ?0js!@aO`Veg5)I|Lpan=g>Q=Blb zOSVH=6j;ksq$EVV`|7X}d!B$Q6=h_M4y^yc5ukITZnYT`uzlw8qb5(W<_VWi+>|md zrCyyG4fp-r@+UnEY%$4APPF;*RhIde3ykpv60EGi&YKSG;Tj3WCU-%KxPWtK=Mn*} zO!nnAztz9}tCn#UTO`zvR$(t=zc_u1p|Oi6D4|t$JH9s)%spOx2%s>#E~UYx0wf1@ zz5;?s;tVhpvy1y_URpK4XEw}p-yGIS32Q)cUfICyM~sXZ7p<(AAYOp_7XsU&{2VQY zkvV6%Kq>lK);!aPQg$yUL&V3WlbrRU4YG~SJs!u-M(jtrMM#bDuChmjCmXo z3UJU6QEW5(F|@N%%W1EX@%SP~6&~T6WkX`ux;f-K}$(i6Czo z5!E^OSg?GbsN|`f-UKk`Dmy&LDc{@VWFz%HXvIF)gRb+pljfz^f0gRliSG7B5J- zBCq;pFN1IV^tG_`!RB7H_Dgu6CFzN4I!>^1Cu)grJ=Ehtc9kMrk0 zW0}zTGC{fz1kf+$Z(e}){b~j4->l#QH;G8jCnDzaa&t!h#Rd|4rs4^LmjwlJr`#DJ zWuHAkFMH6;2kyyiZ*TR~8@1Jsx8grbY-f&7j}=k1MJpVvgS{?^mKTzf$@;B#DtUB` zGzDb3G3Y8*zDY~fy4gnz54dEy{c5~#c@L1?Zsj`QNuBl{9{hn`62M8MEfJBG$B7u^ zgdIn7IR{_zwGL76cRJ%6%pzc=Mb$QBgd0N7WcJNx^-KeihES-vxg;+muOt@B>(vy= z*tU4r)*fs}VxQxK4jBR;jf3w?wacmZDnZp{OQ)3DqEuY+uW4jynG!!bJ`bFtfB>#)M8``QU$V`6adb(G7!6-iRhFLRz_7Wvm6ux zO6GKh0Qp@qnb{N)ftNzU5lOC@lsrV24>&vDfya%AAQ5teC=O0pVwAiiZdrNxPA*vl z=Z+Hp0cRUL0YS>9h`Kou^Cb8&F2`mC*T@3v6kn1L>Ghg}*E*ViARFiClI3R0ZoP0s zY!>OUD9VGkIfh*FuqUH(#5}DaTrj!gubvn6WKlnBo1h?>TfLRZ8&Nw~SydK$O3gIRj9X@xL?3KV(V#m({mK@+/data/nodes/0/indices//0/index`. @@ -16,7 +16,6 @@ curl -X PUT -H 'Content-Type: application/json' 'localhost:9200/testindex?pretty "number_of_replicas": 0 }, "mappings": { - "testData": { "properties": { "id": { "type": "keyword" }, "isTestData": { "type": "boolean" }, @@ -35,11 +34,10 @@ curl -X PUT -H 'Content-Type: application/json' 'localhost:9200/testindex?pretty "testDate": { "type": "date" }, "testLocation": { "type": "geo_point"} } - } } }' -curl -X POST "localhost:9200/testindex/testData/?pretty" -H 'Content-Type: application/json' -d' +curl -X POST "localhost:9200/testindex/_doc/1" -H 'Content-Type: application/json' -d' { "id": "testData1", "isTestData": true, @@ -54,4 +52,7 @@ curl -X POST "localhost:9200/testindex/testData/?pretty" -H 'Content-Type: appli "testLocation": "48.553532,-113.022881" } ' + +curl -X POST "localhost:9200/testindex/_flush" -H 'Content-Type: application/json' -d'{}' ``` +Flushing is essential to commit the changes to ensure tests don't fail complaining there are no docs in segment. diff --git a/server/src/test/resources/indices/bwc/os-1.3.0/testIndex-os-1.3.0.zip b/server/src/test/resources/indices/bwc/os-1.3.0/testIndex-os-1.3.0.zip new file mode 100644 index 0000000000000000000000000000000000000000..f751d349dc943185f6d6c101bbecce3edbd286d1 GIT binary patch literal 2899 zcma);c{J4PAIHaxJ+g}#Tx3ZhSA?#zB{G)5FjE;zWh){(A$yjIG2>zwl66Exlw~9u zYhzbbL_@=E<_d8eC4Q5Bzu(C{zxv%jp6_|S=X{^nIp5dwJfF`S1_iN;0ssJRKz|7C z%$szZ-x#d788!fb6Ced}(NMqO;qCzE0f3}RFM56*!C`v%>h3@itPuC~?+=Sfrg-4ouca$QR9hZPwOV)h0(%UJp! zOz=QO8eUl4Lj-az-O{rLz;?N67b+Er7g-tOXf`*!pK)Ije@_mnYwKHTF)02$(pYRCubA;cS@~FuE=Mex zr<7Mr7yd<0*65mHo2=kI;0+nwQcAVw+VW( zHkA?)P)SrQgJ+p3qJRk}wig$kwYd}^CtGfvdnDJR)WMw@-5@XhORjx@DGGuZk1MsA z;{+!hBZ`ME_a6A;v#ja|XQUEp0(Y1VXToiiSd9Ld$yp$#f+yY4?F=*UW~-7nb}zg! zRYopX?T%9QnvHseTtaHmsq zZiOoWc|C4Lr}jkfM(A^{hw7;CY~Yex=$Ju;S42(jNP3l?bk)U^QSR+h?J5PrazIBC zB6?~&O{|tyqnxUFf*$r_B(pM+Q7^Wbo~_CEY9QwPoE{-Z8x+^Y=fY*L-YVFb(~_b^ zzIRe2h)jywgHHq9c$;BAg6To56uz9~*p)Gk2T{cf*& zmwtxv*EQY6RRY(O;gClz2Kq-%tGyh2AnYl6305-U>(F(`%+K~&&C4gbp)(9E_%!eO zlaDjbrLYWL^5X=C)M>{rPbZuonXI?sw`NH-L}X178YcIbCxVYAl=c3KS4PjhW`NSErv;1hx=YwM{a zUivaZoBOICUH1$A>j-_1P+z{T(pxC9q!v-5E2GWJIl`T^cZ+SmNaM>y$e0%Y!# zo%P!zFWG`*l?_ABJ+CWxRi{2I?2n!=a=^y>JiZCAZSC>xF|Lhzs7&anlo$e+;fA|H zUtaxl-g_d)JD=LeX|}LUA1MH!oIqNw0qbi8!H3Y1527~Fo@$!W!wL0P-CH5kz1qOK zmdpS;ZmXP5XO9|feOI{VMPFd}hA0^ib*!4cc(`KJxb&IXXs{Jxo>9%~l?@|04(pjh z1dGD4>LxOVK{>SH^{9CruPo=e52uS4Vx>q>UcSY+MSg5-RXO6?%Gq#vv-xSkX@yk{ z9vsiBPgX5UbHh>OtHUk>lTv-pOpd#y@EsU@~t0b{BB<&~TL|_s0UwdT0yE9-cCJM%*jX*#}>6HQm_07m7T(I52nq z)X*oLr>)NSmEKR5N|V?wyZKraxf%*GW7#A z4Qqd9EWje|NK?!wHzjc1BAzZ85Emq#qd*g12IoQYbMZ)|1Q?%}uX(=5Ru8PC zT5?z9+RA|aX&s6irj;sJ@RfQhT~{_tn^P1cnm1G`ovdp*G z?8`&h{?-(=n$Oj(umXf0ODr!-Y@oZRultptK$j!`&+ee@H>-A($B!R3x^3NDw|gaD z!fe3r87j~HqzMl6|206v5TLvt(gDsz-XB{z&S$)ng!CMAoBrA&=evE(VB63XN? zwwO$!5;oVEqoWkjy3S#8YA+#~S>#_r>eq}HZC}I4zc+j{MIkYZ$FXDzX*`Qb9q(k4 zB2R$|(G)6nv)Cfp8ugG-T3vr&#uc=1aG?`Eai{m>(=|hN%(@2uS7|Xxz$X?qkV8At zcv+eNJE?1D^6$r2X`un$LGJ24eittPOd1Tz29yE)D7#7g7g`$ob^4zq$MG%6{Vm1) zowYmLe9uy4ZR`$G~dJ#0}VW)_%tOk@)Xt<8 literal 0 HcmV?d00001 diff --git a/test/fixtures/azure-fixture/Dockerfile b/test/fixtures/azure-fixture/Dockerfile index 074e1caded134..67c072b3ca02c 100644 --- a/test/fixtures/azure-fixture/Dockerfile +++ b/test/fixtures/azure-fixture/Dockerfile @@ -1,5 +1,5 @@ -FROM ubuntu:18.04 +FROM ubuntu:22.04 RUN apt-get update -qqy -RUN apt-get install -qqy openjdk-11-jre-headless +RUN apt-get install -qqy openjdk-21-jre-headless ENTRYPOINT exec java -classpath "/fixture/shared/*" fixture.azure.AzureHttpFixture 0.0.0.0 8091 container EXPOSE 8091 diff --git a/test/fixtures/gcs-fixture/Dockerfile b/test/fixtures/gcs-fixture/Dockerfile index ffb1d8b19dfcd..0da51addeaaab 100644 --- a/test/fixtures/gcs-fixture/Dockerfile +++ b/test/fixtures/gcs-fixture/Dockerfile @@ -1,7 +1,7 @@ -FROM ubuntu:18.04 +FROM ubuntu:22.04 RUN apt-get update -qqy -RUN apt-get install -qqy openjdk-11-jre-headless +RUN apt-get install -qqy openjdk-21-jre-headless ARG port ARG bucket diff --git a/test/fixtures/s3-fixture/Dockerfile.eks b/test/fixtures/s3-fixture/Dockerfile.eks index d03960472a6a8..beba3fa0a106a 100644 --- a/test/fixtures/s3-fixture/Dockerfile.eks +++ b/test/fixtures/s3-fixture/Dockerfile.eks @@ -1,7 +1,7 @@ -FROM ubuntu:18.04 +FROM ubuntu:22.04 RUN apt-get update -qqy -RUN apt-get install -qqy openjdk-11-jre-headless +RUN apt-get install -qqy openjdk-21-jre-headless ARG fixtureClass ARG port diff --git a/test/framework/src/main/java/org/opensearch/index/mapper/MapperTestCase.java b/test/framework/src/main/java/org/opensearch/index/mapper/MapperTestCase.java index 01a4005255f29..825074d881001 100644 --- a/test/framework/src/main/java/org/opensearch/index/mapper/MapperTestCase.java +++ b/test/framework/src/main/java/org/opensearch/index/mapper/MapperTestCase.java @@ -37,9 +37,8 @@ import org.apache.lucene.index.IndexableField; import org.apache.lucene.index.IndexableFieldType; import org.apache.lucene.index.LeafReaderContext; -import org.apache.lucene.search.DocValuesFieldExistsQuery; +import org.apache.lucene.search.FieldExistsQuery; import org.apache.lucene.search.IndexSearcher; -import org.apache.lucene.search.NormsFieldExistsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; import org.opensearch.common.CheckedConsumer; @@ -113,14 +112,14 @@ protected void assertExistsQuery(MapperService mapperService) throws IOException protected void assertExistsQuery(MappedFieldType fieldType, Query query, ParseContext.Document fields) { if (fieldType.hasDocValues()) { - assertThat(query, instanceOf(DocValuesFieldExistsQuery.class)); - DocValuesFieldExistsQuery fieldExistsQuery = (DocValuesFieldExistsQuery) query; + assertThat(query, instanceOf(FieldExistsQuery.class)); + FieldExistsQuery fieldExistsQuery = (FieldExistsQuery) query; assertEquals("field", fieldExistsQuery.getField()); assertDocValuesField(fields, "field"); assertNoFieldNamesField(fields); } else if (fieldType.getTextSearchInfo().hasNorms()) { - assertThat(query, instanceOf(NormsFieldExistsQuery.class)); - NormsFieldExistsQuery normsFieldExistsQuery = (NormsFieldExistsQuery) query; + assertThat(query, instanceOf(FieldExistsQuery.class)); + FieldExistsQuery normsFieldExistsQuery = (FieldExistsQuery) query; assertEquals("field", normsFieldExistsQuery.getField()); assertHasNorms(fields, "field"); assertNoDocValuesField(fields, "field"); diff --git a/test/framework/src/main/java/org/opensearch/indices/analysis/AnalysisFactoryTestCase.java b/test/framework/src/main/java/org/opensearch/indices/analysis/AnalysisFactoryTestCase.java index ca23f67215f3d..6e1b48926e200 100644 --- a/test/framework/src/main/java/org/opensearch/indices/analysis/AnalysisFactoryTestCase.java +++ b/test/framework/src/main/java/org/opensearch/indices/analysis/AnalysisFactoryTestCase.java @@ -225,6 +225,7 @@ public abstract class AnalysisFactoryTestCase extends OpenSearchTestCase { // https://github.com/apache/lucene/pull/12915 .put("japanesehiraganauppercase", Void.class) .put("japanesekatakanauppercase", Void.class) + .put("romaniannormalization", Void.class) .immutableMap(); static final Map> KNOWN_CHARFILTERS = new MapBuilder>() diff --git a/test/framework/src/main/java/org/opensearch/repositories/blobstore/OpenSearchBlobStoreRepositoryIntegTestCase.java b/test/framework/src/main/java/org/opensearch/repositories/blobstore/OpenSearchBlobStoreRepositoryIntegTestCase.java index 4e0c9270cd627..136cb53823257 100644 --- a/test/framework/src/main/java/org/opensearch/repositories/blobstore/OpenSearchBlobStoreRepositoryIntegTestCase.java +++ b/test/framework/src/main/java/org/opensearch/repositories/blobstore/OpenSearchBlobStoreRepositoryIntegTestCase.java @@ -347,7 +347,7 @@ public void testSnapshotAndRestore() throws Exception { logger.info("--> add random documents to {}", index); addRandomDocuments(index, randomIntBetween(10, 1000)); } else { - int docCount = (int) client().prepareSearch(index).setSize(0).get().getHits().getTotalHits().value; + int docCount = (int) client().prepareSearch(index).setSize(0).get().getHits().getTotalHits().value(); int deleteCount = randomIntBetween(1, docCount); logger.info("--> delete {} random documents from {}", deleteCount, index); for (int i = 0; i < deleteCount; i++) { @@ -415,7 +415,7 @@ public void testMultipleSnapshotAndRollback() throws Exception { addRandomDocuments(indexName, docCount); } // Check number of documents in this iteration - docCounts[i] = (int) client().prepareSearch(indexName).setSize(0).get().getHits().getTotalHits().value; + docCounts[i] = (int) client().prepareSearch(indexName).setSize(0).get().getHits().getTotalHits().value(); logger.info("--> create snapshot {}:{} with {} documents", repoName, snapshotName + "-" + i, docCounts[i]); assertSuccessfulSnapshot( client().admin() diff --git a/test/framework/src/main/java/org/opensearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/opensearch/search/aggregations/AggregatorTestCase.java index 27142b298db52..ca7646c072e70 100644 --- a/test/framework/src/main/java/org/opensearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/opensearch/search/aggregations/AggregatorTestCase.java @@ -49,6 +49,7 @@ import org.apache.lucene.sandbox.document.HalfFloatPoint; import org.apache.lucene.search.Collector; import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.IndexSearcher.LeafReaderContextPartition; import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.QueryCache; @@ -681,8 +682,10 @@ protected A searchAndReduc final int size = compCTX.leaves().size(); final ShardSearcher[] subSearchers = new ShardSearcher[size]; for (int searcherIDX = 0; searcherIDX < subSearchers.length; searcherIDX++) { - final LeafReaderContext leave = compCTX.leaves().get(searcherIDX); - subSearchers[searcherIDX] = new ShardSearcher(leave, compCTX); + final LeafReaderContextPartition partition = LeafReaderContextPartition.createForEntireSegment( + compCTX.leaves().get(searcherIDX) + ); + subSearchers[searcherIDX] = new ShardSearcher(partition, compCTX); } for (ShardSearcher subSearcher : subSearchers) { MultiBucketConsumer shardBucketConsumer = new MultiBucketConsumer( @@ -845,20 +848,20 @@ protected IndexReader wrapDirectoryReader(DirectoryReader reader) throws IOExcep } private static class ShardSearcher extends IndexSearcher { - private final List ctx; + private final LeafReaderContextPartition[] partitions; - ShardSearcher(LeafReaderContext ctx, IndexReaderContext parent) { + ShardSearcher(LeafReaderContextPartition partition, IndexReaderContext parent) { super(parent); - this.ctx = Collections.singletonList(ctx); + this.partitions = new LeafReaderContextPartition[] { partition }; } public void search(Weight weight, Collector collector) throws IOException { - search(ctx, weight, collector); + search(partitions, weight, collector); } @Override public String toString() { - return "ShardSearcher(" + ctx.get(0) + ")"; + return "ShardSearcher(" + partitions[0].ctx + ")"; } } diff --git a/test/framework/src/main/java/org/opensearch/snapshots/AbstractSnapshotIntegTestCase.java b/test/framework/src/main/java/org/opensearch/snapshots/AbstractSnapshotIntegTestCase.java index 9bf725766069c..7db9875387500 100644 --- a/test/framework/src/main/java/org/opensearch/snapshots/AbstractSnapshotIntegTestCase.java +++ b/test/framework/src/main/java/org/opensearch/snapshots/AbstractSnapshotIntegTestCase.java @@ -566,7 +566,7 @@ protected Settings.Builder snapshotRepoSettingsForShallowCopy(Path path) { protected long getCountForIndex(String indexName) { return client().search( new SearchRequest(new SearchRequest(indexName).source(new SearchSourceBuilder().size(0).trackTotalHits(true))) - ).actionGet().getHits().getTotalHits().value; + ).actionGet().getHits().getTotalHits().value(); } protected void assertDocCount(String index, long count) { diff --git a/test/framework/src/main/java/org/opensearch/test/CorruptionUtils.java b/test/framework/src/main/java/org/opensearch/test/CorruptionUtils.java index 8e551a51a2094..12817ec1ab6cd 100644 --- a/test/framework/src/main/java/org/opensearch/test/CorruptionUtils.java +++ b/test/framework/src/main/java/org/opensearch/test/CorruptionUtils.java @@ -96,7 +96,7 @@ public static void corruptFile(Random random, Path... files) throws IOException long checksumAfterCorruption; long actualChecksumAfterCorruption; - try (ChecksumIndexInput input = dir.openChecksumInput(fileToCorrupt.getFileName().toString(), IOContext.READONCE)) { + try (ChecksumIndexInput input = dir.openChecksumInput(fileToCorrupt.getFileName().toString())) { assertThat(input.getFilePointer(), is(0L)); input.seek(input.length() - CodecUtil.footerLength()); checksumAfterCorruption = input.getChecksum(); diff --git a/test/framework/src/main/java/org/opensearch/test/OpenSearchIntegTestCase.java b/test/framework/src/main/java/org/opensearch/test/OpenSearchIntegTestCase.java index 1c26ea4ca2c91..cb93622ab8728 100644 --- a/test/framework/src/main/java/org/opensearch/test/OpenSearchIntegTestCase.java +++ b/test/framework/src/main/java/org/opensearch/test/OpenSearchIntegTestCase.java @@ -846,9 +846,11 @@ public void waitNoPendingTasksOnAll() throws Exception { /** Ensures the result counts are as expected, and logs the results if different */ public void assertResultsAndLogOnFailure(long expectedResults, SearchResponse searchResponse) { final TotalHits totalHits = searchResponse.getHits().getTotalHits(); - if (totalHits.value != expectedResults || totalHits.relation != TotalHits.Relation.EQUAL_TO) { + if (totalHits.value() != expectedResults || totalHits.relation() != TotalHits.Relation.EQUAL_TO) { StringBuilder sb = new StringBuilder("search result contains ["); - String value = Long.toString(totalHits.value) + (totalHits.relation == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO ? "+" : ""); + String value = Long.toString(totalHits.value()) + (totalHits.relation() == TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO + ? "+" + : ""); sb.append(value).append("] results. expected [").append(expectedResults).append("]"); String failMsg = sb.toString(); for (SearchHit hit : searchResponse.getHits().getHits()) { @@ -1043,7 +1045,8 @@ public void waitForDocs(final long numDocs, final BackgroundIndexer indexer) thr .setQuery(matchAllQuery()) .get() .getHits() - .getTotalHits().value; + .getTotalHits() + .value(); if (count == lastKnownCount) { // no progress - try to refresh for the next time diff --git a/test/framework/src/main/java/org/opensearch/test/TestSearchContext.java b/test/framework/src/main/java/org/opensearch/test/TestSearchContext.java index 09a72dcdc3641..eec890e289468 100644 --- a/test/framework/src/main/java/org/opensearch/test/TestSearchContext.java +++ b/test/framework/src/main/java/org/opensearch/test/TestSearchContext.java @@ -172,7 +172,7 @@ public TestSearchContext( this.indexShard = indexShard; this.queryShardContext = queryShardContext; this.searcher = searcher; - this.concurrentSegmentSearchEnabled = searcher != null && (searcher.getExecutor() != null); + this.concurrentSegmentSearchEnabled = searcher != null; /* executor is always set */ this.maxSliceCount = randomIntBetween(0, 2); this.scrollContext = scrollContext; } diff --git a/test/framework/src/main/java/org/opensearch/test/engine/ThrowingLeafReaderWrapper.java b/test/framework/src/main/java/org/opensearch/test/engine/ThrowingLeafReaderWrapper.java index d23c0df9ba1c2..f3ae94d39a9f1 100644 --- a/test/framework/src/main/java/org/opensearch/test/engine/ThrowingLeafReaderWrapper.java +++ b/test/framework/src/main/java/org/opensearch/test/engine/ThrowingLeafReaderWrapper.java @@ -108,13 +108,6 @@ public Terms terms(String field) throws IOException { return terms; } - @Override - public Fields getTermVectors(int docID) throws IOException { - Fields fields = super.getTermVectors(docID); - thrower.maybeThrow(Flags.TermVectors); - return fields == null ? null : new ThrowingFields(fields, thrower); - } - /** * Wraps a Fields but with additional asserts */ diff --git a/test/framework/src/main/java/org/opensearch/test/hamcrest/OpenSearchAssertions.java b/test/framework/src/main/java/org/opensearch/test/hamcrest/OpenSearchAssertions.java index 650558aaa97a6..3c145c2bd4e74 100644 --- a/test/framework/src/main/java/org/opensearch/test/hamcrest/OpenSearchAssertions.java +++ b/test/framework/src/main/java/org/opensearch/test/hamcrest/OpenSearchAssertions.java @@ -299,14 +299,16 @@ public static void assertOrderedSearchHits(SearchResponse searchResponse, String public static void assertHitCount(SearchResponse countResponse, long expectedHitCount) { final TotalHits totalHits = countResponse.getHits().getTotalHits(); - if (totalHits.relation != TotalHits.Relation.EQUAL_TO || totalHits.value != expectedHitCount) { + if (totalHits.relation() != TotalHits.Relation.EQUAL_TO || totalHits.value() != expectedHitCount) { fail("Count is " + totalHits + " but " + expectedHitCount + " was expected. " + formatShardStatus(countResponse)); } } public static void assertHitCount(SearchResponse countResponse, long minHitCount, long maxHitCount) { final TotalHits totalHits = countResponse.getHits().getTotalHits(); - if (!(totalHits.relation == TotalHits.Relation.EQUAL_TO && totalHits.value >= minHitCount && totalHits.value <= maxHitCount)) { + if (!(totalHits.relation() == TotalHits.Relation.EQUAL_TO + && totalHits.value() >= minHitCount + && totalHits.value() <= maxHitCount)) { fail( "Count is " + totalHits @@ -344,7 +346,7 @@ public static void assertFourthHit(SearchResponse searchResponse, Matcher matcher) { assertThat(number, greaterThan(0)); assertThat("SearchHit number must be greater than 0", number, greaterThan(0)); - assertThat(searchResponse.getHits().getTotalHits().value, greaterThanOrEqualTo((long) number)); + assertThat(searchResponse.getHits().getTotalHits().value(), greaterThanOrEqualTo((long) number)); assertThat(searchResponse.getHits().getAt(number - 1), matcher); } @@ -560,8 +562,8 @@ public static T assertBooleanSubQuery(Query query, Class su assertThat(query, instanceOf(BooleanQuery.class)); BooleanQuery q = (BooleanQuery) query; assertThat(q.clauses().size(), greaterThan(i)); - assertThat(q.clauses().get(i).getQuery(), instanceOf(subqueryType)); - return subqueryType.cast(q.clauses().get(i).getQuery()); + assertThat(q.clauses().get(i).query(), instanceOf(subqueryType)); + return subqueryType.cast(q.clauses().get(i).query()); } /** diff --git a/test/logger-usage/src/main/java/org/opensearch/test/loggerusage/OpenSearchLoggerUsageChecker.java b/test/logger-usage/src/main/java/org/opensearch/test/loggerusage/OpenSearchLoggerUsageChecker.java index d28134a492541..52d3dfff48974 100644 --- a/test/logger-usage/src/main/java/org/opensearch/test/loggerusage/OpenSearchLoggerUsageChecker.java +++ b/test/logger-usage/src/main/java/org/opensearch/test/loggerusage/OpenSearchLoggerUsageChecker.java @@ -201,7 +201,7 @@ private static class ClassChecker extends ClassVisitor { private final Predicate methodsToCheck; ClassChecker(Consumer wrongUsageCallback, Predicate methodsToCheck) { - super(Opcodes.ASM7); + super(Opcodes.ASM9); this.wrongUsageCallback = wrongUsageCallback; this.methodsToCheck = methodsToCheck; }