context shifting with images

[lobsterchan27]

context shifting works when you send the same image but is it possible to not have to reprocess the entire prompt when its the same other than the image? using the same instance of kobold with images resets the kvcache context shifting whenever a new image is used. sending the new image while keeping the old one doesnt work either multiple images get mixed together to the model

thanks.

[LostRuins]

Is the image used for LLAVA? Because that DOES change the context, llava 1.5 images get converted into (approx) 500 tokens, and llava 1.6 converts into about 1.6k tokens.

[lobsterchan27]

yes im using generate api endpoint. passing images as base64 string. im guessing the image is being directly inserted into the prompt at the beginning. but other than that there is an issue if you add more than 1 image. its not properly recognizing that they are separate images. the details of more than 1 image get mixed.

[LostRuins]

yes, that is a known issue unfortunately