Image segmentation task#

An image segmentation task is a process of isolating a region of an image by delineating the object’s shape. The bounds used for delineating are more specific than bounding boxes used in detection. That’s why you will be using the bounds in the form of a polygon, which requires more annotation effort than classification or detection. Once you outline the object’s shape, the region will be highlighted and will provide image-by-pixel details since an image is a collection of various pixels. By segmenting the image, you group the pixels that have similar attributes. This level of detail allows for an in-depth image analysis.

Semantic vs Instance Segmentation#

Instance segmentation detects and delineates each distinct object of interest in an image while semantic segmentation groups parts of an image that belong to the same entity. To illustrate the difference, take a look at the example below:

Instance Segmentation	Semantic Segmentation

In instance segmentation, each individual object is distinguished and separated based on its unique features, allowing for the identification and counting of separate entities, such as multiple people in an image. In contrast, semantic segmentation classifies all similar objects as a single entity, e.g., recognizing them collectively as humans, without differentiating between individuals.

Segmentation vs classification and object detection#

Analyzing segmented regions in an image at a granular level is a segmentation’s hallmark that distinguishes it from other computer vision tasks such as classification and detection. Classification focuses on seeing what is in the picture in general, detection centers on classifying and localizing the objects in an image. Segmentation goes a step further and examines an isolated region on a pixel level.

The table below presents object features each task can examine:

Classification	Detection	Segmentation
Class	Class	Class
	Location	Location
		Shape

The more effort (information) we put in, the more knowledge the model can learn. Hence, the annotation effort in supervised computer vision tasks is the lowest starting with:

Classification
Detection
Segmentation

Examples of leveraging image segmentation#

The image segmentation task can solve a myriad of real-life problems. For a better understanding of what you can do with this task, we compiled a list of things you can use it for:

Medical Image Processing: analysis of individual cells, synapses, nuclei, vesicles, measurement of tissue volumes
Photography: a video enhancement app automatically applying color correction to particular areas, such as the lips and eyes of a person in the image