Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor face extraction and alignment logic; enhance bounding box transformation and facial landmark adjustments #1414

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jogendra-india
Copy link

@jogendra-india jogendra-india commented Dec 30, 2024

Tickets

#1413

What has been done

In this PR, I have modified the code so that bounding box around image also gets rotated on image rotation during alignment.

Here are some result:
Original Image I used (left rotated):
image_rotated

Here is the earlier aligned image extracted (plotted aligned_sub_img from extract_face() function that is return of align_img_wrt_eyes function)
Earlier aligned image

After the change in PR, this is how it looks now:
Final aligned image after change

Tested the performance on random images to identify the same person using embeddings from the image below.
Original image

Image 1:
Image 1

Image 2:
Image 2

Image 3:
Image 3

Here are the results:

Cosine Similarity Comparison Table

Metric Image 1 (Original) Image 1 (After Change) Image 2 (Original) Image 2 (After Change) Image 3 (Original) Image 3 (After Change)
Facenet 0.63 0.65 0.78 0.75 0.48 0.41
ArcFace 0.45 0.39 0.47 0.42 0.29 0.19
VGG-Face 0.40 0.39 0.46 0.42 0.64 0.73
Facenet512 0.69 0.63 0.66 0.58 0.85 0.90

Observations:

  1. Image 1 shows a slight improvement in Facenet (0.63 → 0.65) but a slight drop in other metrics.
  2. Image 2 maintains overall performance but slightly decreases across all metrics after the change.
  3. Image 3 (left-rotated) demonstrates better alignment with VGG-Face (0.64 → 0.73) and Facenet512 (0.85 → 0.90), indicating improved handling of rotations.

@serengil Not entirely sure if this has resulted in significant improvements. While the alignment issue appears to be resolved, there may still be other areas requiring adjustments. I had anticipated cosine similarity improvements across all models, but the results seem mixed.

I would appreciate it if you could thoroughly test this with your expertise and recommend any additional changes that might be necessary.

Thank you!

How to test

make lint && make test

…ansformation and facial landmark adjustments
@jogendra-india jogendra-india marked this pull request as ready for review December 30, 2024 18:41
@serengil
Copy link
Owner

Hmmm, the scale of these changes seems risky given the minimal or negative impact. It is not worth the trouble.

@jogendra-india
Copy link
Author

Hmmm, the scale of these changes seems risky given the minimal or negative impact. It is not worth the trouble.

Agreed! Alternatively, we could limit this change to cases where the eyes are positioned vertically, such as in rotated images. However, this might be over-engineering. I'll revisit this later in the week if I come up with other adjustments. How does it look on your end with your test images?

@serengil
Copy link
Owner

seems it passed the unit tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants