Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New External API Extraction Feature #379

Draft
wants to merge 65 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
6dbdc32
Added proxy retriever for external open_clip feature
lucaro Dec 11, 2022
1997b98
Added LevenshteinScoringTextRetriever
lucaro Dec 17, 2022
afc5fe5
Changed query string encoding in LevenshteinScoringTextRetriever
lucaro Dec 17, 2022
2cc9b81
Changed query string encoding in LevenshteinScoringTextRetriever again
lucaro Dec 17, 2022
875fe63
Added a way to specify the host of the external clip text instance vi…
sauterl Dec 19, 2022
60a5f6a
Merge remote-tracking branch 'origin/main' into VBS23
Spiess Apr 21, 2023
a71b74d
refactored AudioSegmentExporter
faberf Jun 8, 2023
168b480
Added isExtractable and refactored AudioSpectogramExporter as an example
faberf Jun 8, 2023
f589f94
added the correct file extension and dataurlprefix
faberf Jun 8, 2023
d2c1c12
- starting work on external api feature
silvanheller May 15, 2023
94f0c8f
updating with development stuff & adding model key
silvanheller May 16, 2023
0017dc9
Initial Implementation
faberf May 17, 2023
c8d6950
implemented client to python endpoint
faberf May 17, 2023
b544368
working commit, refactored external api support
faberf Jun 1, 2023
761d8ae
fixing getRows() which had infinite recursion
silvanheller May 17, 2023
58b953f
adding logging for faulty queries
silvanheller May 22, 2023
89e0fcd
adding logging for faulty queries
silvanheller May 22, 2023
539cdc5
moving logger to trace
silvanheller May 22, 2023
55a147b
clarifying documentation
silvanheller May 30, 2023
546b23c
first cleanup
silvanheller Jun 5, 2023
7d27038
working commit for rewrite of external-api-extraction
faberf Jun 8, 2023
2dee77f
Fixed windows-path issues in extraction item container for live-extra…
sauterl Jun 13, 2023
63f1f5c
Added Documentation
faberf Jun 13, 2023
a90e507
Merge branch 'main' into refactoring_exporters
faberf Jun 13, 2023
c95a02f
- starting work on external api feature
silvanheller May 15, 2023
99cb177
updating with development stuff & adding model key
silvanheller May 16, 2023
5589e5a
Initial Implementation
faberf May 17, 2023
89efaa9
implemented client to python endpoint
faberf May 17, 2023
c3e0f26
working commit, refactored external api support
faberf Jun 1, 2023
396a7cc
fixing getRows() which had infinite recursion
silvanheller May 17, 2023
37ff7ee
adding logging for faulty queries
silvanheller May 22, 2023
e3873b0
adding logging for faulty queries
silvanheller May 22, 2023
21fec92
moving logger to trace
silvanheller May 22, 2023
4649563
clarifying documentation
silvanheller May 30, 2023
fabc2ee
first cleanup
silvanheller Jun 5, 2023
1c5bed8
working commit for rewrite of external-api-extraction
faberf Jun 8, 2023
0b21390
Merge branch 'external-api-extraction' of github.com:vitrivr/cineast …
faberf Jun 13, 2023
c16433b
fixed message template
faberf Aug 23, 2023
78b06cc
templates are now stripped
faberf Sep 11, 2023
8987321
added FESVectorFeature
faberf Sep 11, 2023
d6d4c3f
removed alpha channel from video frames
faberf Sep 11, 2023
ca26e6e
Merged dev
lucaro Oct 8, 2023
6424616
Merge remote-tracking branch 'origin/main' into VBS24
lucaro Oct 8, 2023
8c85619
First sketch of handler for vector lookup
lucaro Oct 8, 2023
fb97e26
gitflow-feature-stash: externalclip
net-cscience-raphael Oct 26, 2023
bae673c
conda environment
net-cscience-raphael Oct 26, 2023
6d29951
added extraction for external clip
net-cscience-raphael Oct 26, 2023
823da7a
updated ffmeg due to access violations on some windows systems
net-cscience-raphael Oct 26, 2023
99adc01
Merge branch 'feature/externalclip' into external-api-extraction
net-cscience-raphael Nov 27, 2023
5280763
changed feature vector
net-cscience-raphael Nov 27, 2023
443c1ec
Added lighting and normal mapping to the lwjgl renderer (#386)
net-cscience-raphael Feb 26, 2024
78764d5
Ads exception handling for textures in 3d model
net-cscience-raphael Mar 4, 2024
ccab60f
ads external open ml clip for 3d models
net-cscience-raphael Mar 4, 2024
043acd3
ads error handling loading shaders
net-cscience-raphael Mar 4, 2024
850f4f0
Added depth to parent folder name generator
net-cscience-raphael Mar 5, 2024
3713e92
Added timeout to render process
net-cscience-raphael Mar 6, 2024
2276f9e
logging
net-cscience-raphael Mar 11, 2024
543e1fc
Added TimeLimitedFunction to perform a timeout if loading with extern…
net-cscience-raphael Mar 12, 2024
ae927e1
Ads documentation to TimeLimitedFunction
net-cscience-raphael Mar 12, 2024
d0fa44f
Changed scene render datatype from u_int to int
net-cscience-raphael Mar 25, 2024
2e8731c
and back do unsigned
net-cscience-raphael Mar 25, 2024
c582799
3d-debug non cachable error from lassimp ibrary
net-cscience-raphael Jun 12, 2024
42242c7
fixes the crash on runtime
net-cscience-raphael Jun 17, 2024
26bd3ff
Fault tolerance for model inconsitency
net-cscience-raphael Jul 1, 2024
436fbef
Added logging to time limited function
net-cscience-raphael Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,7 @@ private float[] embedModel(IModel model) throws IOException, InterruptedExceptio

private float[] embedVideo(List<MultiImage> frames) throws IOException, InterruptedException {
var images = frames.stream().map(image -> image.getBufferedImage()).collect(Collectors.toList());
images = images.stream().map(img -> new BufferedImage(img.getWidth(), img.getHeight(), BufferedImage.TYPE_INT_RGB)).collect(Collectors.toList());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are overwriting the previously generated images with empty images here, that is almost certainly not what you want.

var encodings = embedMultipleImages(images);

// Sum
Expand Down