Feature Request: Thoughts on CustomModel template class?

[Jordan-Pierce]

What are your thoughts on having a sort of CustomModel template class that would mostly serve as an example for others to follow such that it would allow them to create their own model to be used with YOLO-Patch-Based-Inference, instead of using a ultralytics-provided model (yolo, fastsam, rtdetr)?

This library uses ultralytics models, but only for making detections in one line of code. But if someone wanted to use their own model, all they really need to do is wrap it in a custom class than ensures that it has the following:

from ultralytics.engine import Results

class CustomModel:
    model.names = ...
    ...
    def predict( batch, imgsz, conf, iou, classes, verbose, extra_args):
        # Make predictions
        # Process predictions
        results = Results(...)    # processed predictions, in ultralytics Results format 
        return results

This would expand the use of the library for others who cannot use ultralytics due to commercial licenses (though, I think technically the dependency of Results still means that it cannot be used by those folks, but one thing at a time I guess).

For reference, this is similar to supervision's implementation of SAHI, where they are model agnostic, and only need the results come out as supervision.Detections.

[Koldim2001]

Hi. Yes, the idea is really great!

At the moment, I’m not entirely sure how to implement something like this and write documentation for it. I think it would be worth creating a separate class in /patched_yolo_infer/nodes/ called CustomModel.py.

I’d be really grateful if you could help with creating this and assist with writing documentation at the end of the file: examples/example_patch_based_inference.ipynb, explaining how to create a custom model and run it through the library. If anything, I’d be happy to help and provide guidance as well.

[Jordan-Pierce]

Sounds good, I'll look into it. Cheers.

[Jordan-Pierce]

@Koldim2001 I ended up going a different route for my needs, so unsure if the above feature request really applies (or is needed). Essentially I broke up some of the methods in MakeCropsDetectThem and injected a few methods such that it allows the user to run each process individually, and therefore can provide the predictions from a model made externally:

element_crops = MakeCropsDetectThem(
    image=img,
    model=YOLO("yolov8n.pt"),
    segment=False,
    show_crops=False,
    shape_x=300,
    shape_y=300,
    overlap_x=50,
    overlap_y=50,
    conf=0.5,
    iou=0.7,
    batch_inference=True,
    show_processing_status=True,
)

# Perform the crops
element_crops.make_crops()
# Get the crops, make preedictions 
results = model.predict(element_crops.get_crops())
# Detect objects
element_crops.detect_them(results)
# Combine the detections
result = CombineDetections(element_crops, nms_threshold=0.05)

The model in this example is still provides a ultralytics Results object, but it shows that as long as your model returns that object (i.e., convert the output), you can use your own model, whatever it is. Not sure if this something you want to add, so closing for now.