model.yolact_edge

Description

🎭 Instance segmentation model for real-time inference

class Node(config=None, **kwargs)[source]

Initializes and uses YolactEdge to infer from an image frame

The YolactEdge node is capable of detecting objects from 80 categories. The table of object categories can be found here.

Inputs

img (numpy.ndarray): A NumPy array of shape \((height, width, channels)\) containing the image data in BGR format.

Outputs

bboxes (numpy.ndarray): A NumPy array of shape \((N, 4)\) containing normalized bounding box coordinates of \(N\) detected objects. Each bounding box is represented as \((x_1, y_1, x_2, y_2)\) where \((x_1, y_1)\) is the top-left corner and \((x_2, y_2)\) is the bottom-right corner. The order corresponds to bbox_labels and bbox_scores.

bbox_labels (numpy.ndarray): A NumPy array of shape \((N)\) containing strings representing the labels of detected objects. The order corresponds to bboxes and bbox_scores.

bbox_scores (numpy.ndarray): A NumPy array of shape \((N)\) containing confidence scores \([0, 1]\) of detected objects. The order corresponds to bboxes and bbox_labels.

masks (numpy.ndarray): A NumPy array of shape \((N, H, W)\) containing \(N\) detected binarized masks where \(H\) and \(W\) are the height and width of the masks. The order corresponds to bbox_labels.

Configs

  • model_type (str) – (str): {“r101-fpn”, “r50-fpn”, “mobilenetv2”}, default=”r50-fpn”.

  • weights_parent_dir (Optional[str]) – default = null.
    Change the parent directory where weights will be stored by replacing null with an absolute path to the desired directory.

  • input_size (int) – default = 550.
    Input image resolution of the YolactEdge model.

  • detect (List[Union[int, string]]) – default=[0].
    List of object class names or IDs to be detected. To detect all classes, refer to the tech note.

  • max_num_detections – (int): default=100.
    Maximum number of detections per image, for all classes.

  • iou_threshold (float) – [0, 1], default = 0.5.
    Overlapping bounding boxes with Intersection over Union (IoU) above the threshold will be discarded.

  • score_threshold (float) – [0, 1], default = 0.2.
    Bounding boxes with confidence score (product of objectness score and classification score) below the threshold will be discarded.

References

YolactEdge: Real-time Instance Segmentation on the Edge https://arxiv.org/abs/2012.12259

Inference code and model weights: https://github.com/haotian-liu/yolact_edge