[editor] Add Output Error Rendering #803
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jan 6, 2024
rholinshead
requested review from
saqadri,
suyoglastmileai,
Ankush-lastmile,
jonathanlastmileai and
rossdanlm
as code owners
added 5 commits
January 8, 2024 10:34
# [editor] ErrorBoundary Renderer for PromptInput With the ability to set arbitrary JSON for the prompt input, if we have a PromptSchema which states what the input should look like, we should be able to do some basic validation that the input at least matches the general type (for now, compare 'string' PromptInput vs object -- in the future, we could do some basic validation against the schema). If there is any error with rendering the schema-based rendering for the input, we can use ErrorBoundary to fallback to an error message informing the user that the format is invalid and to toggle to JSON editor to fix. <img width="1083" alt="Screenshot 2024-01-05 at 11 01 07 PM" src="https://github.com/lastmile-ai/aiconfig/assets/5060851/d0a817f5-b85b-4501-9ca5-28b324061ed4"> Toggling to the JSON editor will clear the error state and try to render the correct Schema renderer when toggling back. Note that in dev mode, these errors are still propagated to a top-level error screen which you can dismiss to show the fallback. In prod, that won't happen and will just show the fallback and an error in the console. We will do the same for the SettingsRenderer in a subsequent PR ## Testing: - Updated `OpenAIChatModelParserPromptSchema` input to match that of `OpenAIChatVisionModelParserPromptSchema` while the input is still string. See the error fallback - Toggle to JSON editor and back, ensure error fallback still shows - Toggle to JSON editor and update to valid input ``` { "attachments": [ { "data": "https://s3.amazonaws.com/files.uploads.lastmileai.com/uploads/cldxsqbel0000qs8owp8mkd0z/2023_12_1_21_23_24/942/Screenshot 2023-11-28 at 11.11.25 AM.png", "mime_type": "image/png" }, { "data": "https://s3.amazonaws.com/files.uploads.lastmileai.com/uploads/cldxsqbel0000qs8owp8mkd0z/2023_12_1_21_23_24/8325/Screenshot 2023-11-28 at 1.51.52 PM.png", "mime_type": "image/png" } ], "data": "What do these images show?" } ``` and see proper schema rendering https://github.com/lastmile-ai/aiconfig/assets/5060851/1343a201-e5eb-46a0-a7f5-a4bdb228d120
# [editor] ErrorBoundary Renderer for ModelSettings With the ability to set arbitrary JSON for the model settings, if we have a PromptSchema which states what the setting types should be, the settings renderer will assume those types match the config settings JSON. If the JSON doesn't actually match the schema types, it's possible for the settings renderer to throw an error. If that happens, we can use an ErrorBoundary to fallback to an error message informing the user that the format is invalid and to toggle the JSON editor to fix. <img width="510" alt="Screenshot 2024-01-06 at 3 56 50 PM" src="https://github.com/lastmile-ai/aiconfig/assets/5060851/1eec492b-d342-4a24-ae27-a83441c827cd"> Toggling to the JSON editor will clear the error state and try to render the correct Schema renderer when toggling back. Note that in dev mode, these errors are still propagated to a top-level error screen which you can dismiss to show the fallback. In prod, that won't happen and will just show the fallback and an error in the console. ## Testing: - Use JSON editor for model settings to set frequency_penalty to a string; expected value is a number so the settings renderer will error. See the error fallback - Toggle to JSON editor and back, ensure error fallback still shows - Toggle to JSON editor and update to valid settings and then toggle back to see proper schema rendering https://github.com/lastmile-ai/aiconfig/assets/5060851/060ddb20-129d-450e-97ab-cf2939269f90
# [editor][easy] Add 'model' to relevant Prompt Schemas There is still some investigation needed to know how to handle model vs parser nicely in the editor UX. For now, let's just expose the 'model' option in the settings for those parsers that support it to ensure the usage of those parsers won't be blocked.
# [editor][easy] Center Running Spinner The 'running' spinner for the RunPromptButton is currently not aligned anymore. This fixes it to keep it centered: <img width="61" alt="Screenshot 2024-01-06 at 4 15 42 PM" src="https://github.com/lastmile-ai/aiconfig/assets/5060851/4d39298d-de72-4119-998e-ea2cddbe4f94"> <img width="499" alt="Screenshot 2024-01-06 at 4 15 48 PM" src="https://github.com/lastmile-ai/aiconfig/assets/5060851/5d9da258-6508-4fe1-90e0-db7f2b33093c">
# [editor] Add Output Error Rendering Adding some basic rendering for Error output types if they're ever added to any configs. Also, propagate server run errors through the client as (client-only) Error outputs to display for the relevant prompt: ## Testing: - Raise an exception in the api/run method on the server and ensure it propagates to the output <img width="992" alt="Screenshot 2024-01-06 at 4 33 55 PM" src="https://github.com/lastmile-ai/aiconfig/assets/5060851/93309f64-4ce5-47dd-b553-c6fb44daaca0">
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[editor] Add Output Error Rendering
[editor] Add Output Error Rendering
Adding some basic rendering for Error output types if they're ever added to any configs. Also, propagate server run errors through the client as (client-only) Error outputs to display for the relevant prompt:
Testing:
Stack created with Sapling. Best reviewed with ReviewStack.