Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate post text from hand-written notes #323

Open
4 tasks
jeffpaul opened this issue Jan 7, 2022 · 4 comments
Open
4 tasks

Generate post text from hand-written notes #323

jeffpaul opened this issue Jan 7, 2022 · 4 comments
Assignees
Labels
help wanted Extra attention is needed needs:engineering This requires engineering to resolve. needs:ux This requires user experience to resolve.

Comments

@jeffpaul
Copy link
Member

jeffpaul commented Jan 7, 2022

Is your enhancement related to a problem? Please describe.
Paper Website does a nice job of generating website content from hand-written notes/journals. This seems similarly feasible via open source using various OCR API's available and our friend WordPress, though we'll want to work through the UX to make this as graceful as possible given the WP mobile - to - ClassifAI handoff that'll need to happen.

gif.mp4

Describe the solution you'd like

  • UX screen flow from image capture/upload (assuming mobile and desktop uploads) to post title and paragraph block generated content
  • v1 build the desktop upload - to - post content generation flow
  • v2 integrate WP iOS/Android mobile upload (maybe mobile web as well?)
  • screenshots / gif showing feature in action for readme & classifaiplugin.com microsite

Designs
UX needs roughly noted above.

Describe alternatives you've considered
If we're going to aim to build this feature, the only alternative I can surmise is to try and roll this directly into the WP mobile apps which... I don't know you tell me if that's more optimal?

Additional context
Source idea, props @dimensionmedia.

@jeffpaul jeffpaul added type:enhancement help wanted Extra attention is needed needs:ux This requires user experience to resolve. needs:engineering This requires engineering to resolve. labels Jan 7, 2022
@jeffpaul jeffpaul added this to the Future Release milestone Jan 7, 2022
@jeffpaul jeffpaul moved this to Incoming in Open Source Practice Jan 7, 2022
@jeffpaul jeffpaul moved this from Incoming to To Do in Open Source Practice Jan 7, 2022
@jeffpaul
Copy link
Member Author

Sharing an in-progress UX flow of an initial / MVP attempt at building this into ClassifAI (admitting that there will be many other possible user flows, but wanting to at least get a basic / most expected flow attempted first to gather feedback before enhancing the user flows further): https://miro.com/app/board/uXjVO1fP4jE=/?share_link_id=639841463424

@jeffpaul
Copy link
Member Author

Props to Jennifer Hicks and Ayanda Mbonambi for the UX research, analysis, and lowfi mockups.

@jeffpaul
Copy link
Member Author

In discussing the base UX case to consider supporting here with @fabiankaegy, we came to the conclusion that having a simple WP Admin page (potentially exposed on the side menu, TBD) that captures images (1-n), sends those out to Azure for processing, and then adds that as text within paragraph blocks in a post/page. The expectation is that this page will be accessed via mobile web on a mobile device such that photos on said device can be uploaded to this WP Admin page. The open question is how best (at least in this base case, initial released version) to get a link to that WP Admin page to a users mobile device (as we're expecting that ClassifAI will have been installed/activated or updated from that users' desktop device and not from mobile).

Keep in mind that we're not trying to support ALL the various users flows that could happen, but a single user flow to gather feedback on the viability of this functionality before expanding user flows.

@jeffpaul
Copy link
Member Author

In discussing this with @dkotter, we could also investigate whether we can inject an icon/link into the WP mobile web view to trigger the image capture flow but neither of us were immediately sure if that was (1) possible and if so (2) how to go about doing that.

@Sidsector9 Sidsector9 self-assigned this Jan 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed needs:engineering This requires engineering to resolve. needs:ux This requires user experience to resolve.
Projects
Status: To Do
Development

No branches or pull requests

2 participants