Handling the World in Code: A Simple Attempt at Image Segmentation

Since I’ve already established the fundamentals in Seeing the World in Code, this is going to be a relatively short article about how you can use the RGB and HSV channels. We will use the image I used in White Balancing to get the labels out of the bottles in the image:

Let us try the RGB channel first:

from skimage.io import imread, imshow
img = imread('img1.jpg')
# RGB Channels
img_r = img[:,:,0]
img_g = img[:,:,1]
img_b = img[:,:,2]

# Segmenting the image through their RGB channels
res_b = ((img_r < 101) & (img_g < 101) & (img_b >=101))/255

red_b = img[:,:,0]*res_b
green_b = img[:,:,1]*res_b
blue_b = img[:,:,2]*res_b
img_masked_b = np.dstack((red_b,green_b,blue_b))
imshow(img_masked_b);

Unfortunately, segmenting through the RGB channel is very specific to a color. Should we widen the segmentation, we will end up including many colors as well that pass the condition. Let’s try and see what we’ll come up with using the HSV channels:

from skimage.color import rgb2hsv
import matplotlib.pyplot as plt
img_hsv = rgb2hsv(img)
fig, ax = plt.subplots(1, 3, figsize=(12,4))
ax[0].imshow(img_hsv[:,:,0], cmap='hsv')
ax[0].set_title('Hue')
ax[1].imshow(img_hsv[:,:,1], cmap='gray')
ax[1].set_title('Saturation')
ax[2].imshow(img_hsv[:,:,2], cmap='gray')
ax[2].set_title('Value');
fig.colorbar(ax[0].imshow(img_hsv[:,:,0], cmap='hsv'), ax=ax.ravel().tolist())

The saturation channel looks like a good start given that the values of the label are almost of the same saturation levels. And to try and filter everything else out, we also use the hue channel:

res_b = (img_hsv[:,:,0] > 0.65) & (img_hsv[:,:,1] > 0.5)
red_b = img[:,:,0]*res_b
green_b = img[:,:,1]*res_b
blue_b = img[:,:,2]*res_b
img_masked_b = np.dstack((red_b,green_b,blue_b))
imshow(img_masked_b);

Unfortunately, as with many real-world pictures, it’s a lot more complicated than what we’d hope. Thus, many other parts of the image was included. This may be easily solved by using blob detection. But that’s a topic for another article.

Needless to say, we’ve established our fundamentals and knowing that the values are just in arrays, we are able to manipulate them as we please.