Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Auto Suggest] PPL & SQL Value Suggestion #8275

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
bf6a69b
basic value suggestion + other stuff
paulstn Sep 18, 2024
ab915ad
allow for more than one value suggestion provider
paulstn Sep 20, 2024
462d067
allow for parenthesis suggestions
paulstn Sep 20, 2024
da5c691
ui settings for suggestion and limit
paulstn Sep 20, 2024
4d39d57
pass in dataset for mds support
paulstn Sep 21, 2024
5335744
update sql with table name and spaces after insertion
paulstn Sep 23, 2024
11618a4
fix values for sql
paulstn Sep 23, 2024
dd164a2
Merge branch 'main' into autosuggest-value-suggestion
paulstn Sep 23, 2024
a9ef4be
Merge branch 'main' into autosuggest-value-suggestion
paulstn Oct 9, 2024
1cd082f
update http type
Oct 9, 2024
7df418f
update http type
Oct 9, 2024
4c20321
remove dataset parameter and use services instead
Oct 9, 2024
e337a19
remove commented code
Oct 9, 2024
ec5c9d5
Changeset file for PR #8275 created/updated
opensearch-changeset-bot[bot] Oct 9, 2024
ef744b6
fix util tests
Oct 9, 2024
3381377
Merge branch 'main' into autosuggest-value-suggestion
paulstn Oct 21, 2024
5d86402
remove sql grammar changes
paulstn Oct 29, 2024
660d6f9
remove ppl grammar changes
paulstn Oct 29, 2024
eceb926
modify sql to prefer rule predicate and manually suggest fields, oper…
paulstn Oct 30, 2024
410a10d
add plus minus back to numbers in grammar
paulstn Oct 30, 2024
51c8d26
temporary sql rerun and constrain system implemented
paulstn Nov 5, 2024
d11356d
Merge branch 'main' into autosuggest-value-suggestion
paulstn Nov 5, 2024
41ebfb4
allow for ID in SELECT while preventing dup table rule
paulstn Nov 6, 2024
7697696
include commas
paulstn Nov 11, 2024
8f2ef80
debug console logs
paulstn Nov 11, 2024
4b8fd28
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
paulstn Nov 19, 2024
3298f6e
redo generated
paulstn Nov 19, 2024
10cee8f
made various changes to ppl and sql autocomplete
paulstn Nov 21, 2024
ba51dd5
Merge branch 'main' into autosuggest-value-suggestion
paulstn Nov 21, 2024
d6d9638
manual suggestions for IN operator workflow
paulstn Nov 23, 2024
edca257
redo custom sql predicate logic w/ better WS handling
paulstn Nov 23, 2024
c13234a
clean up predicate rule and keep rerun as is
paulstn Nov 24, 2024
9a0eeb9
handle test diffs
paulstn Nov 24, 2024
503f348
Merge branch 'main' into autosuggest-value-suggestion
paulstn Dec 18, 2024
87d81f6
put commas and parens as keywords, not operators
paulstn Nov 24, 2024
1423892
test fetchFieldSuggestions
paulstn Dec 18, 2024
de9f1f7
move dataset defining within fetchColumnValues and remove multitable …
paulstn Dec 18, 2024
0a6fe78
rename fetchFields to formatFields
paulstn Dec 18, 2024
3e593af
add value formatter and change formatters to take in suggestion list …
paulstn Dec 18, 2024
01744c6
improve readability by only using spread operator in final code compl…
paulstn Dec 18, 2024
680fdda
modify some util funcs to be more generic
paulstn Dec 18, 2024
9aa9902
tests for format values to suggestions
paulstn Dec 18, 2024
ba5f3a5
test fetchColumnValues
paulstn Dec 19, 2024
5da1216
catch value fetching errors separately outside of other suggestions
paulstn Dec 19, 2024
2a4419f
sql preferred rules column name tests
paulstn Dec 19, 2024
c174874
sql preferred rules all predicate rule tests
paulstn Dec 19, 2024
b91a1e7
basic sql code completion tests
paulstn Dec 19, 2024
ca9e79f
rewritten + better + more sql code completion tests
paulstn Dec 19, 2024
603a1d9
ppl code completion end to end tests
paulstn Dec 19, 2024
d854f59
basic opensearch ppl autocomplete object tests w/o preferred rules
paulstn Dec 19, 2024
77ba770
Merge branch 'main' into autosuggest-value-suggestion
paulstn Dec 19, 2024
470a184
skip aggregate test, requires new rerun system
paulstn Dec 24, 2024
92d7061
pr comments incl. basic visited rules tests
paulstn Dec 27, 2024
e97c32e
update utils types to be generic instead of 'any'
paulstn Jan 6, 2025
0a0ae50
move suggestion importance to constants
paulstn Jan 6, 2025
9d4cd1f
Merge branch 'main' into autosuggest-value-suggestion
paulstn Jan 6, 2025
65de34e
remove value suggestion providor changes
paulstn Jan 6, 2025
00b02e9
take in dataset type to only send agg queries for index patterns and …
paulstn Jan 8, 2025
7dd9475
Merge branch 'main' into autosuggest-value-suggestion
paulstn Jan 9, 2025
911c52b
fix name issue with merge
paulstn Jan 9, 2025
161038a
fix merge again
paulstn Jan 9, 2025
ad431b4
remove jest reference
paulstn Jan 9, 2025
25c06cd
sql predicate pref rule field name expansion
paulstn Jan 14, 2025
26782e2
ppl pref rule field name expansion
paulstn Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelogs/fragments/8275.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- Autocomplete Value Suggestion ([#8275](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/8275))
2 changes: 2 additions & 0 deletions src/plugins/data/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ export const UI_SETTINGS = {
FILTERS_PINNED_BY_DEFAULT: 'filters:pinnedByDefault',
FILTERS_EDITOR_SUGGEST_VALUES: 'filterEditor:suggestValues',
QUERY_ENHANCEMENTS_ENABLED: 'query:enhancements:enabled',
QUERY_ENHANCEMENTS_SUGGEST_VALUES: 'query:enhancements:suggestValues',
QUERY_ENHANCEMENTS_SUGGEST_VALUES_LIMIT: 'query:enhancements:suggestValuesLimit',
QUERY_DATAFRAME_HYDRATION_STRATEGY: 'query:dataframe:hydrationStrategy',
SEARCH_QUERY_LANGUAGE_BLOCKLIST: 'search:queryLanguageBlocklist',
NEW_HOME_PAGE: 'home:useNewHomePage',
Expand Down
168 changes: 1 addition & 167 deletions src/plugins/data/public/antlr/dql/code_completion.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,176 +6,12 @@
import { monaco } from '@osd/monaco';
import { getSuggestions } from './code_completion';
import { DataPublicPluginStart, IDataPluginServices } from '../../types';
import { IndexPattern } from '../../index_patterns';
import { testingIndex } from '../shared/constants';

/**
* Constants
*/

const testingIndex = ({
title: 'opensearch_dashboards_sample_data_flights',
fields: [
{
count: 0,
name: 'Carrier',
displayName: 'Carrier',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 2,
name: 'DestCityName',
displayName: 'DestCityName',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'DestCountry',
displayName: 'DestCountry',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'DestWeather',
displayName: 'DestWeather',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'DistanceMiles',
displayName: 'DistanceMiles',
type: 'number',
esTypes: ['float'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'FlightDelay',
displayName: 'FlightDelay',
type: 'boolean',
esTypes: ['boolean'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'FlightNum',
displayName: 'FlightNum',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: 'OriginWeather',
displayName: 'OriginWeather',
type: 'string',
esTypes: ['keyword'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: true,
subType: undefined,
},
{
count: 0,
name: '_id',
displayName: '_id',
type: 'string',
esTypes: ['_id'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: false,
subType: undefined,
},
{
count: 0,
name: '_index',
displayName: '_index',
type: 'string',
esTypes: ['_index'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: false,
subType: undefined,
},
{
count: 0,
name: '_score',
displayName: '_score',
type: 'number',
scripted: false,
searchable: false,
aggregatable: false,
readFromDocValues: false,
subType: undefined,
},
{
count: 0,
name: '_source',
displayName: '_source',
type: '_source',
esTypes: ['_source'],
scripted: false,
searchable: false,
aggregatable: false,
readFromDocValues: false,
subType: undefined,
},
{
count: 0,
name: '_type',
displayName: '_type',
type: 'string',
esTypes: ['_type'],
scripted: false,
searchable: true,
aggregatable: true,
readFromDocValues: false,
subType: undefined,
},
],
getFieldByName: jest.fn((fieldName: string) => {
return testingIndex.fields.find((field) => field.name === fieldName);
}),
} as unknown) as IndexPattern;

const booleanOperatorSuggestions = [
{ text: 'or', type: 11, detail: 'Operator', insertText: 'or ' },
{ text: 'and', type: 11, detail: 'Operator', insertText: 'and ' },
Expand Down Expand Up @@ -225,8 +61,6 @@ const allCarrierValueSuggestions = [
{ text: 'OpenSearch-Air', type: 13, detail: 'Value', insertText: '"OpenSearch-Air" ' },
];

const carrierWithNotSuggestions = allCarrierValueSuggestions.concat(notOperatorSuggestion);

const logCarrierValueSuggestion = [
{ text: 'Logstash Airways', type: 13, detail: 'Value', insertText: '"Logstash Airways" ' },
];
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/data/public/antlr/dql/code_completion.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import { IndexPattern, IndexPatternField } from '../../index_patterns';
import { QuerySuggestion, QuerySuggestionGetFnArgs } from '../../autocomplete';
import { DQLParserVisitor } from './.generated/DQLParserVisitor';
import { IDataPluginServices } from '../..';
import { fetchFieldSuggestions } from '../shared/utils';
import { formatFieldsToSuggestions } from '../shared/utils';
import { SuggestionItemDetailsTags } from '../shared/constants';

const findCursorIndex = (
Expand Down Expand Up @@ -128,7 +128,7 @@ export const getSuggestions = async ({
// check to see if field rule is a candidate. if so, suggest field names
if (candidates.rules.has(DQLParser.RULE_field)) {
completions.push(
...fetchFieldSuggestions(indexPattern, (field: string) => {
...formatFieldsToSuggestions(indexPattern, (field: string) => {
const indexField = indexPattern.getFieldByName(field);
if (indexField && ['boolean', 'string'].includes(indexField.type)) {
return `${field} : `;
Expand Down

Large diffs are not rendered by default.

Loading
Loading