Support kCMPixelFormat_32BGRA and add github action for running tests #1

darioalessandro · 2024-12-19T14:15:37Z

Summary

This pull request introduces a new function buf_bgra_to_rgb that converts image data from BGRA (Blue, Green, Red, Alpha) format to RGB format. The
function now supports the specific pixel format kCMPixelFormat_32BGRA, enhancing its versatility for handling images in Apple's Core Media framework.

Key Changes

Support for kCMPixelFormat_32BGRA: Added functionality to process images with this specific pixel format, expanding the library's capability to
handle diverse image types.
Input Validation: Enhanced checks to ensure width and height are even numbers, crucial for formats requiring even dimensions.
Buffer Size Verification: Ensured correct buffer lengths based on given resolution, critical for accurate data processing.
Efficient Pixel Conversion: Implemented conversion of BGRA pixel data to RGB format using chunks_exact for efficient handling.

New Features

Pixel Format Compatibility: The function now recognizes and processes the kCMPixelFormat_32BGRA pixel format, aligning with Core Media
standards.
Dimension Check: Verifies that width and height are even numbers, ensuring compatibility with image formats that require even
dimensions.
Efficient Buffer Handling: Processes data in chunks of four bytes, minimizing overhead for improved performance.

Bug Fixes

Resolved potential issues related to incorrect buffer sizes through validation checks.
Ensured proper handling of alpha values (ignored in conversion).

Known Issues

Function assumes correct output buffer sizing; mismatches may cause runtime errors.
Performance optimization opportunities exist for very large images, though current implementation is efficient.

Documentation Improvements

Added comments explaining each step for clarity and ease of understanding.
Updated API documentation to reflect support for kCMPixelFormat_32BGRA.

Testing

Comprehensive test cases include checks for invalid dimensions, incorrect buffer sizes, and compatibility with kCMPixelFormat_32BGRA.
Additional tests ensure functionality and robustness under various scenarios.

Integration

Function seamlessly integrates into the existing image processing pipeline, enhancing the library's capabilities without disrupting current workflows.

This implementation strengthens the image processing library by adding support for the kCMPixelFormat_32BGRA pixel format while maintaining efficiency and safety.

… it really is

darioalessandro · 2024-12-19T14:59:47Z

nokhwa-core/src/pixel_format.rs

@@ -405,9 +496,10 @@ impl FormatDecoder for LumaAFormat {
 /// let image: ImageBuffer<Rgb<u8>, Vec<u8>> = buffer.to_image::<YuyvFormat>();
 /// ```
 #[derive(Copy, Clone, Debug, Default, Hash, Ord, PartialOrd, Eq, PartialEq)]
-pub struct YuyvFormat;
+pub struct I420Format;


This was never YUYV it was always meant to be i420, this is a breaking change but it is the right thing to do

darioalessandro · 2024-12-26T12:50:01Z

.github/workflows/examples.yml

@@ -29,4 +29,4 @@ jobs:
      - name: Cargo Check 
        run: |
          cd examples/threaded-capture
-          cargo check


darioalessandro · 2024-12-26T12:50:32Z

nokhwa-core/src/buffer.rs

-            2 => Mat::new_rows_cols_with_data::<GA8>(resolution.width() as i32, resolution.height() as i32, cast_slice(data)),
-            3 => Mat::new_rows_cols_with_data::<RGB8>(resolution.width() as i32, resolution.height() as i32, cast_slice(data)),
-            4 => Mat::new_rows_cols_with_data::<RGBA8>(resolution.width() as i32, resolution.height() as i32, cast_slice(data)),
+            1 => Mat::new_rows_cols_with_data::<G8>(


this is just cargo fmt

nokhwa-core/src/buffer.rs

darioalessandro · 2024-12-26T12:51:23Z

nokhwa-core/src/pixel_format.rs

@@ -77,6 +77,16 @@ impl FormatDecoder for RgbFormat {
                .collect()),
            FrameFormat::RAWRGB => Ok(data.to_vec()),
            FrameFormat::NV12 => nv12_to_rgb(resolution, data, false),
+            FrameFormat::BGRA => {


this is the meat of the change

darioalessandro · 2024-12-26T12:51:59Z

nokhwa-core/tests/assets/README.md

@@ -0,0 +1,38 @@
+
+# Assets


Documenting how I added the test images and the license for the pyramid picture

darioalessandro · 2025-02-09T00:22:07Z

examples/threaded-capture/src/main.rs

@@ -32,7 +32,7 @@ fn main() {

    let format = RequestedFormat::new::<RgbFormat>(RequestedFormatType::AbsoluteHighestFrameRate);

-    let first_camera = cameras.first().unwrap();


This is not needed, remove

nokhwa-core/src/buffer.rs

darioalessandro · 2025-02-09T00:25:29Z

nokhwa-core/src/pixel_format.rs

 /// let image: ImageBuffer<Rgb<u8>, Vec<u8>> = buffer.to_image::<YuyvFormat>();
 /// ```
 #[derive(Copy, Clone, Debug, Default, Hash, Ord, PartialOrd, Eq, PartialEq)]
 pub struct YuyvFormat;

 impl FormatDecoder for YuyvFormat {
+    // YUV 4:2:0 planar colors. but we need to change the image crate to use this format


This should be a TODO

darioalessandro · 2025-02-09T00:26:25Z

nokhwa-core/src/pixel_format.rs

 }

+/// Converts an image in YUYV format to I420 (YUV 4:2:0) format.


Ensure that this is a proper rust doc comment

darioalessandro · 2025-02-09T00:27:26Z

nokhwa-core/src/types.rs

@@ -1548,21 +1557,21 @@ pub fn buf_mjpeg_to_rgb(data: &[u8], dest: &mut [u8], rgba: bool) -> Result<(),
        });
    }

-    jpeg_decompress.read_scanlines_into::<u8>(dest).map_err(|why| {
-        NokhwaError::ProcessFrameError {
+    jpeg_decompress


cargo fmt

darioalessandro · 2025-02-09T00:27:43Z

nokhwa-core/src/types.rs

        return Err(NokhwaError::ProcessFrameError {
            src: FrameFormat::NV12,
            destination: "RGB".to_string(),
-            error: "bad input buffer size".to_string(),
+            error: format!("bad input buffer size, expected {} but got {}", expected_len, data.len()),


Add better error

darioalessandro · 2025-02-09T00:28:09Z

nokhwa-core/src/types.rs

+        let b = chunk[0];
+        let g = chunk[1];
+        let r = chunk[2];
+        // let _a = chunk[3]; // Alpha is ignored for RGB


remove commented code

darioalessandro added 4 commits December 16, 2024 07:00

add bgra

8c12b38

not ready for primetime but renameb yuyv to i420 because that is what…

7eb5e9d

… it really is

save

5584868

save

fec4d19

darioalessandro changed the title ~~[DO NOT REVIEW YET][WIP] Rename Yuyv to i420~~ [DO NOT REVIEW YET][WIP] Rename Yuyv to i420 and support kCMPixelFormat_32BGRA Dec 19, 2024

save

3b0a2fe

darioalessandro commented Dec 19, 2024

View reviewed changes

darioalessandro added 8 commits December 19, 2024 10:12

add bgra to linux bindings

aad27f7

checkin test assets

a701a6e

adding assets with license

c46d077

add tests to ci

8a2f4fc

added better tests

ed9d6b5

remove unused imports

1c9385b

move nokhwa ci to a different file

9552a9c

run fmt

ac3fbfb

darioalessandro changed the title ~~[DO NOT REVIEW YET][WIP] Rename Yuyv to i420 and support kCMPixelFormat_32BGRA~~ [DO NOT REVIEW YET][WIP] support kCMPixelFormat_32BGRA Dec 26, 2024

darioalessandro changed the title ~~[DO NOT REVIEW YET][WIP] support kCMPixelFormat_32BGRA~~ Support kCMPixelFormat_32BGRA and add github action for running tests Dec 26, 2024

darioalessandro commented Dec 26, 2024

View reviewed changes

nokhwa-core/src/buffer.rs Outdated Show resolved Hide resolved

darioalessandro commented Dec 26, 2024

View reviewed changes

darioalessandro added 2 commits December 26, 2024 08:06

revert unrelated ci change

074f3da

add better logging for bad buffer size

72dba49

darioalessandro commented Feb 9, 2025

View reviewed changes

nokhwa-core/src/buffer.rs Outdated Show resolved Hide resolved

darioalessandro commented Feb 9, 2025

View reviewed changes

darioalessandro added 3 commits February 8, 2025 19:29

revert buffer changes back to 0.10c

dd3a4bb

revert threaded-capture demo changes

7ee7d03

enhance error handling

719b7cc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support kCMPixelFormat_32BGRA and add github action for running tests #1

Support kCMPixelFormat_32BGRA and add github action for running tests #1

darioalessandro commented Dec 19, 2024 •

edited

Loading

darioalessandro Dec 19, 2024 •

edited

Loading

darioalessandro Dec 26, 2024

darioalessandro Dec 26, 2024

darioalessandro Dec 26, 2024

darioalessandro Dec 26, 2024

darioalessandro Feb 9, 2025

darioalessandro Feb 9, 2025

darioalessandro Feb 9, 2025

darioalessandro Feb 9, 2025

darioalessandro Feb 9, 2025

darioalessandro Feb 9, 2025

		@@ -32,7 +32,7 @@ fn main() {

		let format = RequestedFormat::new::<RgbFormat>(RequestedFormatType::AbsoluteHighestFrameRate);

		let first_camera = cameras.first().unwrap();

		}

		/// Converts an image in YUYV format to I420 (YUV 4:2:0) format.

Support kCMPixelFormat_32BGRA and add github action for running tests #1

Are you sure you want to change the base?

Support kCMPixelFormat_32BGRA and add github action for running tests #1

Conversation

darioalessandro commented Dec 19, 2024 • edited Loading

Summary

Key Changes

New Features

Bug Fixes

Known Issues

Documentation Improvements

Testing

Integration

darioalessandro Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darioalessandro commented Dec 19, 2024 •

edited

Loading

darioalessandro Dec 19, 2024 •

edited

Loading