Enhance images with super resolution using OpenCV involves using advanced techniques to upscale low-resolution images and improve their quality. OpenCV is a powerful computer vision library that provides various tools and algorithms for image processing tasks. When combined with deep learning techniques, it becomes possible to achieve state-of-the-art super-resolution results.
In this project we will compare the traditional method for enhancing the resolution of images with various deep learning pretrained models.
You can watch the video-based tutorial with a step-by-step explanation down below.
Download Modules
We will download the deep learning modules that we will use in this project
# links to download models
https://github.com/Saafke/EDSR_Tensorflow/blob/master/models/
https://github.com/fannymonori/TF-LapSRN/blob/master/export/LapSRN_x8.pb In this project we have used two different deep learning models EDSR and LapSRN which enhances the resolution of the low quality images
Using the above given links we can download this models
In this project we will see how this models are used to enhance the resolution of low quality images and compare these with the traditional method
Import Modules
import cv2
from cv2 import dnn_superrescv2 - OpenCV module
Enhancing images with super resolution using EDSR Model
EDSR (Enhanced Deep Super-Resolution) is a popular deep learning model used for single-image super-resolution tasks. It aims to generate a high-resolution image from a low-resolution input image.
EDSR is known for its simplicity and effectiveness in achieving state-of-the-art performance in super-resolution tasks
Now let us see the steps involved in enhancing the resolution of low quality images using EDSR deep learning model
First we will initialize the super resolution object
# initialize super resolution object
sr = dnn_superres.DnnSuperResImpl_create()
# read the model
path = 'EDSR_x4.pb'
sr.readModel(path)
# set the model and scale
sr.setModel('edsr', 4)First we will create an instance of the DnnSuperResImpl class, which will be used to perform super-resolution using deep neural networks
Next the readModel() method is used to load the pre-trained EDSR model from the specified path. The model is usually stored in a .pb file format
Next set the loaded EDSR model as the active model for super-resolution. The first argument is the model type, which in this case is 'edsr'. The second argument specifies the scaling factor, which determines the degree of upscaling
After executing these steps, the sr object can be used to perform super-resolution using the EDSR model
Next step is optional if you have cuda support then you can execute the below code to set the preferable backend and target else it will use the CPU
# if you have cuda support
sr.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
sr.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)setPreferableBackend() sets the preferable backend for the EDSR model to CUDA. By setting the backend to CUDA, the EDSR model can leverage the power of NVIDIA GPUs for faster inference.
setPreferableTarget() sets the preferable target for the EDSR model to CUDA
By setting both the backend and target to CUDA, you ensure that the EDSR model utilizes the GPU's parallel processing capabilities, which can significantly speed up the super-resolution process compared to using only the CPU
Next we will have to feed the image for which you want to upscale the resolution
# load the image
image = cv2.imread('test.png')This will load the given image
Next we will upscale the image using initialized EDSR model
# upsample the image
upscaled = sr.upsample(image)
# save the upscaled image
cv2.imwrite('upscaled_test.png', upscaled)upsample() method of the sr object is used to perform super-resolution on the input image. The method takes the low-resolution image as input and returns the corresponding high-resolution image obtained through the EDSR model.
We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('upscaled_test.png') and the image array (upscaled) as inputs and writes the image to the specified file path.
Next we will upscale the same image using traditional bicubic model
# traditional method - bicubic
bicubic = cv2.resize(image, (upscaled.shape[1], upscaled.shape[0]), interpolation=cv2.INTER_CUBIC)
# save the image
cv2.imwrite('bicubic_test.png', bicubic)cv2.resize() function performs bicubic interpolation on the low-resolution image and resizes the image to the specified dimensions of the upscaled image using bicubic interpolation
We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('bicubic_test.png') and the image array (bicubic) as inputs and writes the image to the specified file path
Next we will compare the upscaled image from both models
This is the low resolution input image
The resolution of this image is 232 X 155
This is the upscaled image obtained using EDSR model
The resolution of this image is 928 X 620
We can see that the resolution of original image is enhanced by 4 times and the quality is also retrieved.
This is the upscaled image obtained using traditional method
The resolution of this image is 928 X 620 but the quality of the image is not up to the mark.
The traditional model has increased the resolution of image but you can clearly see that there is some kind of distortion all over the image . On the other hand, we can see that the EDSR model has increased the resolution of image by retrieving the high quality.
Enhance images with super resolution using LapSRN Model
LapSRN (Laplacian Pyramid Super-Resolution Network) is a deep learning model used for single-image super-resolution tasks.
It is designed to generate high-resolution images from low-resolution inputs using a laplacian pyramid-based approach.
LapSRN is known for its ability to produce high-quality super-resolved images while maintaining fine details
The EDSR model which we used earlier upscales the images up to 4X times whereas the LapSRN model can upscale the image up to 8X times
Now let us see the steps involved in enhancing the resolution of low quality images using LapSRN deep learning model. The process is similar to what you saw for EDSR model
First we will initialize the super resolution object
# initialize super resolution object
sr = dnn_superres.DnnSuperResImpl_create()
# read the model
path = 'LapSRN_x8.pb'
sr.readModel(path)
# set the model and scale
sr.setModel('lapsrn', 8)First we will create an instance of the DnnSuperResImpl class, which will be used to perform super-resolution using deep neural networks
Next the readModel() method is used to load the pre-trained LapSRN model from the specified path. The model is usually stored in a .pb file format
Next set the loaded LapSRN model as the active model for super-resolution. The first parameter is the model type, which in this case is 'lapsrn'. The second parameter, 8, specifies the scale factor, indicating that the model is trained to perform 8x super-resolution
After executing these steps, the sr object can be used to perform super-resolution using the LapSRN model
As mentioned earlier the next step is optional if you have cuda support then you can execute the below code to set the preferable backend and target else it will use the CPU
# if you have cuda support
sr.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
sr.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)Next we will have to feed the image for which you want to upscale the resolution
# load the image
image = cv2.imread('test.png')This will load the given image
Next we will upscale the image using initialized LapSRN model
# upsample the image
upscaled = sr.upsample(image)
# save the upscaled image
cv2.imwrite('upscaled_test_lapsrn.png', upscaled)Similar to EDSR here also upsample() method of the sr object is used to perform super-resolution on the input image. The method takes the low-resolution image as input and returns the corresponding high-resolution image obtained through the LapSRN model.
We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('upscaled_test_lapsrn.png') and the image array (upscaled) as inputs and writes the image to the specified file path.
Next we will upscale the same image using traditional bicubic model
# traditional method - bicubic
bicubic = cv2.resize(image, (upscaled.shape[1], upscaled.shape[0]), interpolation=cv2.INTER_CUBIC)
# save the image
cv2.imwrite('bicubic_test_8x.png', bicubic)Here as well we have used the same traditional bicubic method to enhance the resolution of the image
We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('bicubic_test_8x.png') and the image array (bicubic) as inputs and writes the image to the specified file path
Next we will compare the upscaled image from both models
We have provided same input image that was used for EDSR model
The resolution of this image is 232 X 155
This is the upscaled image obtained using LapSRN model
The resolution of this image is 1856 X 1240
We can see that the resolution of original image is enhanced by 8 times and the quality is also retrieved.
This is the upscaled image obtained using traditional method
The resolution of this image is 1856 X 1240 but the quality of the image is not up to the mark.
The traditional model has increased the resolution of image but you can clearly see that there is some kind of distortion all over the image . On the other hand, we can see that the LapSRN model has increased the resolution of image up to 8X times with minor distortion on image.
Final Thoughts
The bicubic method is a simple and fast technique but often leads to blurry results with limited improvement in image quality whereas deep learning-based methods like EDSR and LapSRN have shown superior performance in terms of generating high-quality super-resolved images. They can produce sharper details and preserve more texture information compared to the bicubic method
Bicubic interpolation is computationally lightweight and can quickly upscale an image. On the other hand, EDSR and LapSRN involve training deep neural networks and can be computationally intensive, requiring substantial computational resources and time for training. However, during the inference phase, where the models are used to upscale images, the computational complexity of EDSR and LapSRN can vary depending on the model architecture and implementation
The traditional bicubic method tends to struggle with higher upscaling factors, producing blurrier and less satisfying results. EDSR and LapSRN, being deep learning-based methods, have been designed to handle larger upscaling factors. While EDSR performs well for a wide range of upscaling factors, LapSRN's progressive approach specifically addresses the challenge of generating high-resolution images with better results for larger upscaling factors
In conclusion, deep learning-based image enhancement methods have shown great potential in improving image quality and generating visually appealing enhancements. Continued research and development in this field will likely lead to further advancements and more sophisticated techniques for enhancing images in the future
In this project tutorial we have seen that the traditional bicubic method is a simple and fast interpolation technique that can provide satisfactory results for modest upscaling factors. However, EDSR and LapSRN, as deep learning-based super-resolution methods, offer superior performance in terms of generating sharper, visually appealing images with enhanced details, particularly for larger upscaling factors. However, they come with increased computational complexity and the need for training on specific datasets.
Get the project notebook from here
Thanks for reading the article!!!
Check out more project videos from the YouTube channel Hackers Realm