Hey folks!! Do you remember the good old days of Harry Potter? Do you remember Harry Potter’s magic wand and his Invisiblity Cloak? Wasn’t it amazing? Have you ever wanted to wear that cloak? Have you ever wondered how life would be if you had the superpower of being invisible?
If your answers to all these questions is a big Yessssss…, then my friend, you are at the right spot!! By the end of this article, you can surely experience being invisible — obviously not in real life!
In this article, we will create this magical experience using an image processing technique called Color detection and segmentation.
In order to run this code, you need a cloth of same color and no other color should be visible into that cloth. We are taking the red cloth. If you are taking some other cloth, the code will remain the same but with minute changes.
Now how are we actually going to create an Invisibility Cloak?
This technique is opposite to the Green Screening. In green screening, we remove background but here we will remove the foreground frame.
1. Capture and store the background frame [ This will be done for some seconds ]
2. Detect the red coloured cloth using colour detection and segmentation algorithm.
3. Segment out the red coloured cloth by generating a mask. [ used in code ]
4. Generate the final augmented output to create a magical effect.
We’ll be using three main libraries of python. “cv2” for video capturing, frame reading, pixel management, color management of the video, “NumPy” which is used to do mathematical manipulation with the pixels of the video, flipping the order of matrices. and “time” for delays.
import cv2 import numpy as np import time
Recording and caching the background
We are replacing the red colored pixels with the background pixels to create the invisible effect in the video. For doing this, we have to store the background image for each frame.
def recordBackground(): print(""" Recording Background ..................... """) cap = cv2.VideoCapture(0) time.sleep(1) background=0 for i in range(30): ret,background = cap.read() cap.release() cv2.destroyAllWindows() background = np.flip(background,axis=1) print(""" Background recorded !! """) return background
Firstly, I’ve created a function for recording background. Then I stored the captured video in a variable name cap. Then, I read each frame of video and store it in the background variable which was initially zero. Then we flip the background values in the x-axis and return the value of the background.
Next, I’ll be defining a function to get invisible.
def getInvisible(): background=recordBackground() print(""" Get ready to become invisible ..................... """) cap = cv2.VideoCapture(0) while(cap.isOpened()): ret, img = cap.read() img = np.flip(img,axis=1) # Converting image to HSV color space. hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) value = (35, 35) blurred = cv2.GaussianBlur(hsv, value,0) lower_red = np.array([0,120,70]) upper_red = np.array([10,255,255]) mask1 = cv2.inRange(hsv,lower_red,upper_red) lower_red = np.array([170,120,70]) upper_red = np.array([180,255,255]) mask2 = cv2.inRange(hsv,lower_red,upper_red) # Addition of the two masks to generate the final mask. mask = mask1+mask2 mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((5,5),np.uint8)) # Replacing pixels corresponding to cloak with the background pixels. img[np.where(mask==255)] = background[np.where(mask==255)] cv2.imshow('Display',img) k = cv2.waitKey(1) if k == 27: cap.release() break cv2.destroyAllWindows() if __name__ == '__main__': getInvisible()
In this function, I’ll get the background value from the recordbackground() function. Then I’ll loop over the frames and convert the color of the image from BGR to HSV color format. I’m converting the RGB (red-blue-green) to HSV(hue-saturation-value) because RGB values are highly sensitive to illumination. After the conversion of RGB to HSV, it is time to specify the range of color to detect red color in the video.
Then, I’ll use GaussianBlur function to remove noises and distortion from the video frames. Then I had created two masks for the different shades of red and then I added them to generate the final mask. Then, I’ll be replacing pixels of mask with the pixels of background.
Boom!!! You are a magician!!!
Once you run the code wait for a few seconds, let the webcam capture the background image then you can come inside the frame with your magical Cloak and be invisible 😉
Hurray!! You are a magician now 😉