{"id":81895,"date":"2023-07-21T14:11:22","date_gmt":"2023-07-21T06:11:22","guid":{"rendered":"https:\/\/www.seeedstudio.com\/blog\/?p=81895"},"modified":"2023-09-08T10:00:32","modified_gmt":"2023-09-08T02:00:32","slug":"what-is-fast-sam-how-to-segment-anything-on-the-edge","status":"publish","type":"post","link":"https:\/\/www.seeedstudio.com\/blog\/2023\/07\/21\/what-is-fast-sam-how-to-segment-anything-on-the-edge\/","title":{"rendered":"What is FastSAM: How to Segment Anything on the Edge?"},"content":{"rendered":"\n

FastSAM was born to tackle the issue of the substantial computational resource requirements associated with heavy Transformer(ViT) models. By harnessing the power of the lightweight CNN model<\/strong>, FastSAM becomes a more effective method rather than SAM, capable of segmenting any object in an image under the guidance of user interactive prompts, all in real-time<\/strong> and much faster!<\/p>\n\n\n\n

Introducing SAM and Beyond<\/h2>\n\n\n\n

Compared with traditional dataset labeling and collection methods, which require lots of resources and time to retrain the model once the dataset has been changed, SAM stands out as a fully automatic cued image segmentation model that can cut out almost anything from an image. Being trained on 11 million images and 1.1 billion segmentation masks, SAM is capable of generating or retrieving masks for specific objects in an image based on points\/text prompts.<\/p>\n\n\n\n

FastSAM goes beyond that. It converts the task into an instance segmentation task, using only 1\/50 of the SA-1B dataset<\/a> published by Meta AI, and finally achieves 50 times faster<\/strong> runtimes while achieving comparable accuracy to SAM.<\/p>\n\n\n

\n
\"\"
Image from Fast Segment Anything paper<\/a><\/figcaption><\/figure><\/div>\n\n\n

Two Stages for FastSAM<\/h2>\n\n\n\n

Break down FastSAM into two stages: All-instance Segmentation (AIS) and Prompt-guided Selection (PGS). The first one is the basic method to segment all objects or regions in one image, and the second one is task-oriented post-processing, helping to identify the specific object of interest.<\/p>\n\n\n\n

ALL-instance Segmentation<\/h3>\n\n\n\n

The first stage uses YOLOv8-seg<\/strong> to produce the segmentation masks<\/strong> of all instances in the image.<\/p>\n\n\n\n

YOLOv8\u2019s backbone network and neck module substitute YOLOv5\u2019s C3 module with the C2f module. The updated Head module embraces a decoupled structure, separating classification and detection heads, and shifts from Anchor-Based to Anchor-Free.<\/p>\n\n\n

\n
\"\"
Segment all objects in the view<\/figcaption><\/figure><\/div>\n\n\n

YOLOv8-seg applies YOLACT principles for instance segmentation. It begins with feature extraction from an image via a backbone network and the Feature Pyramid Network (FPN), integrating diverse-size features. The output consists of the detection and segmentation branches.<\/p>\n\n\n\n

Prompt-guided Selection<\/h3>\n\n\n\n

In the second stage, it outputs the region-of-interest<\/strong> corresponding to the prompt. Here are three methods:<\/p>\n\n\n

\n
\"\"
Image from Fast Segment Anything paper<\/a><\/figcaption><\/figure><\/div>\n\n\n