Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: implement an Unleash AI service #8408

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@
"murmurhash3js": "^3.0.1",
"mustache": "^4.1.0",
"nodemailer": "^6.9.9",
"openai": "^4.67.3",
"openapi-types": "^12.1.3",
"owasp-password-strength-test": "^1.3.0",
"parse-database-url": "^0.3.0",
Expand Down
85 changes: 85 additions & 0 deletions src/lib/features/ai/ai-service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import type { ChatCompletionMessageParam } from 'openai/resources/chat/completions';
import type { ChatCompletionRunner } from 'openai/resources/beta/chat/completions';
import OpenAI from 'openai';
import type {
IAuthRequest,
IUnleashConfig,
IUnleashServices,
Logger,
} from '../../server-impl';
import type FeatureToggleService from '../feature-toggle/feature-toggle-service';
import type { FeatureSearchService } from '../feature-search/feature-search-service';
import { createFlag } from './tools/create-flag';
import { toggleFlag } from './tools/toggle-flag';
import { searchFlag } from './tools/search-flag';

export class AIService {
private config: IUnleashConfig;

private logger: Logger;

private client: OpenAI | undefined;

private featureService: FeatureToggleService;

private featureSearchService: FeatureSearchService;

constructor(
config: IUnleashConfig,
{
featureToggleService,
featureSearchService,
}: Pick<
IUnleashServices,
'featureToggleService' | 'featureSearchService'
>,
) {
this.config = config;
this.logger = config.getLogger('features/ai/ai-service.ts');
this.featureService = featureToggleService;
this.featureSearchService = featureSearchService;
}

getClient(): OpenAI {
if (this.client) {
return this.client;
}

const apiKey = this.config.openAIAPIKey;
if (!apiKey) {
throw new Error(
'Unleash AI is unavailable: Missing OpenAI API key',
);
}

this.client = new OpenAI({ apiKey });
return this.client;
}

chat(
messages: ChatCompletionMessageParam[],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the intent here? Does the chat history live on the UI at the moment? Can't quite see how this works behind a load balancer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're still on the early stage of this project where we're mostly just migrating code from the hackathon branch and polishing it a bit. So for now the chat history still lives in the UI. We have another task for later that moves this ownership to the server.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same vein, I'm pretty sure our toggle-flag tool is not respecting change requests currently, which is another task we have for later.

req: IAuthRequest,
): ChatCompletionRunner {
const client = this.getClient();

return client.beta.chat.completions.runTools({
model: 'gpt-4o-mini',
messages,
tools: [
createFlag({
featureService: this.featureService,
auditUser: req.audit,
}),
toggleFlag({
featureService: this.featureService,
auditUser: req.audit,
user: req.user,
}),
searchFlag({
featureSearchService: this.featureSearchService,
user: req.user,
}),
],
});
}
}
66 changes: 66 additions & 0 deletions src/lib/features/ai/tools/create-flag.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import type { RunnableToolFunctionWithParse } from 'openai/lib/RunnableFunction';
import type { IAuditUser } from '../../../types';
import type FeatureToggleService from '../../feature-toggle/feature-toggle-service';

type createFlagOperationalParams = {
featureService: FeatureToggleService;
auditUser: IAuditUser;
};

type createFlagParams = {
project: string;
name: string;
description?: string;
};

type createFlagFunctionParams = createFlagOperationalParams & createFlagParams;

const createFlagFunction = async ({
featureService,
auditUser,
project,
name,
description,
}: createFlagFunctionParams) => {
try {
const response = await featureService.createFeatureToggle(
project,
{ name, description },
auditUser,
);

return response;
} catch (error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the try/catch here for? Debugging?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything can go wrong when calling this method, so this allows us to forward either the success response or the error message back to the assistant, allowing it to handle both scenarios gracefully.

return error;
}
};

export const createFlag = ({
featureService,
auditUser,
}: createFlagOperationalParams): RunnableToolFunctionWithParse<createFlagParams> => ({
type: 'function',
function: {
function: (params: createFlagParams) =>
createFlagFunction({ ...params, featureService, auditUser }),
name: 'createFlag',
description:
'Create a feature flag by name and project. Optionally supply a description',
parse: JSON.parse,
parameters: {
type: 'object',
properties: {
project: {
type: 'string',
description: 'The project in which to create the flag',
},
name: { type: 'string', description: 'The name of the flag' },
description: {
type: 'string',
description: 'The description of the flag',
},
},
required: ['project', 'name'],
},
},
});
161 changes: 161 additions & 0 deletions src/lib/features/ai/tools/search-flag.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import type { RunnableToolFunctionWithParse } from 'openai/lib/RunnableFunction';
import type { IUser } from '../../../types';
import type { FeatureSearchService } from '../../feature-search/feature-search-service';
import { normalizeQueryParams } from '../../feature-search/search-utils';

type searchFlagOperationalParams = {
featureSearchService: FeatureSearchService;
user: IUser;
};

type searchFlagParams = {
query?: string;
project?: string;
type?: string;
tag?: string;
segment?: string;
createdAt?: string;
state?: string;
status?: string[];
};

type searchFlagFunctionParams = searchFlagOperationalParams & searchFlagParams;

const searchFlagFunction = async ({
featureSearchService,
user,
query,
project,
type,
tag,
segment,
createdAt,
state,
status,
}: searchFlagFunctionParams) => {
try {
const userId = user.id;
const {
normalizedQuery,
normalizedSortOrder,
normalizedOffset,
normalizedLimit,
} = normalizeQueryParams(
{ query },
{
limitDefault: 50,
maxLimit: 100,
},
);
const normalizedStatus = status
?.map((tag) => tag.split(':'))
.filter(
(tag) =>
tag.length === 2 &&
['enabled', 'disabled'].includes(tag[1]),
);

const response = await featureSearchService.search({
searchParams: normalizedQuery,
project,
type,
userId,
tag,
segment,
state,
createdAt,
status: normalizedStatus,
offset: normalizedOffset,
limit: normalizedLimit,
sortOrder: normalizedSortOrder,
favoritesFirst: true,
});

return response;
} catch (error) {
return error;
}
};

export const searchFlag = ({
featureSearchService,
user,
}: searchFlagOperationalParams): RunnableToolFunctionWithParse<searchFlagParams> => ({
type: 'function',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty cool! How well does this work in practice? Does OpenAI use all of these terms effectively?

Copy link
Member Author

@nunogois nunogois Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, based on the hackathon, this worked pretty well, so it should! We can always fine tune if needed.

If we're specifically talking about this search-flag tool I'm not sure because I haven't tried it yet 😄 It's a new tool to replace the old get-flag one. Good news is that we'll find out soon enough whether this works like we expect it to!

function: {
function: (params: searchFlagParams) =>
searchFlagFunction({ ...params, featureSearchService, user }),
name: 'searchFlag',
description:
'Search a feature flag by name, project, environment, and enabled status',
parse: JSON.parse,
parameters: {
type: 'object',
properties: {
query: {
type: 'string',
description: 'Search by query',
examples: ['feature_a'],
},
project: {
type: 'string',
description:
'Search by project. Operators such as IS, IS_NOT, IS_ANY_OF, IS_NONE_OF are supported',
pattern:
'^(IS|IS_NOT|IS_ANY_OF|IS_NONE_OF):(.*?)(,([a-zA-Z0-9_]+))*$',
examples: ['IS:default'],
},
state: {
type: 'string',
description:
'Search by state (e.g., active/stale). Operators such as IS, IS_NOT, IS_ANY_OF, IS_NONE_OF are supported',
pattern:
'^(IS|IS_NOT|IS_ANY_OF|IS_NONE_OF):(.*?)(,([a-zA-Z0-9_]+))*$',
examples: ['IS:active'],
},
type: {
type: 'string',
description:
'Search by type. Operators such as IS, IS_NOT, IS_ANY_OF, IS_NONE_OF are supported',
pattern:
'^(IS|IS_NOT|IS_ANY_OF|IS_NONE_OF):(.*?)(,([a-zA-Z0-9_]+))*$',
examples: ['IS:release'],
},
tag: {
type: 'string',
description:
'Search by tags. Tags must specify a type and a value, joined with a colon. Operators such as INCLUDE, DO_NOT_INCLUDE, INCLUDE_ALL_OF, INCLUDE_ANY_OF, EXCLUDE_IF_ANY_OF, EXCLUDE_ALL are supported',
pattern:
'^(INCLUDE|DO_NOT_INCLUDE|INCLUDE_ALL_OF|INCLUDE_ANY_OF|EXCLUDE_IF_ANY_OF|EXCLUDE_ALL):([^:,]+:.+?)(,\\s*[^:,]+:.+?)*$',
examples: ['INCLUDE:simple:my_tag'],
},
segment: {
type: 'string',
description:
'Search by segments. Operators such as INCLUDE, DO_NOT_INCLUDE, INCLUDE_ALL_OF, INCLUDE_ANY_OF, EXCLUDE_IF_ANY_OF, EXCLUDE_ALL are supported',
pattern:
'^(INCLUDE|DO_NOT_INCLUDE|INCLUDE_ALL_OF|INCLUDE_ANY_OF|EXCLUDE_IF_ANY_OF|EXCLUDE_ALL):(.*?)(,([a-zA-Z0-9_]+))*$',
examples: ['INCLUDE:pro-users'],
},
createdAt: {
type: 'string',
description:
'Search by creation date. Operators such as IS_BEFORE or IS_ON_OR_AFTER are supported',
pattern:
'^(IS_BEFORE|IS_ON_OR_AFTER):\\d{4}-\\d{2}-\\d{2}$',
examples: ['IS_ON_OR_AFTER:2023-01-28'],
},
status: {
type: 'array',
items: {
type: 'string',
description:
'Search by environment status. The environment and status are joined by a colon.',
examples: ['production:enabled'],
},
},
},
required: [],
},
},
});
Loading
Loading