top of page
  • Writer's pictureHackers Realm

Enhance Your Images with Super Resolution | OpenCV | Deep Learning | Project Tutorial | Python

Updated: May 24, 2023

Enhance images with super resolution using OpenCV involves using advanced techniques to upscale low-resolution images and improve their quality. OpenCV is a powerful computer vision library that provides various tools and algorithms for image processing tasks. When combined with deep learning techniques, it becomes possible to achieve state-of-the-art super-resolution results.

enhance your images using deep super resolution opencv
Enhance your images using Deep Super Resolution with OpenCV

In this project we will compare the traditional method for enhancing the resolution of images with various deep learning pretrained models.


You can watch the video-based tutorial with a step-by-step explanation down below.


Download Modules


We will download the deep learning modules that we will use in this project

  • In this project we have used two different deep learning models EDSR and LapSRN which enhances the resolution of the low quality images

  • Using the above given links we can download this models

  • In this project we will see how this models are used to enhance the resolution of low quality images and compare these with the traditional method


Import Modules


import cv2
from cv2 import dnn_superres
  • cv2 - OpenCV module


Enhancing images with super resolution using EDSR Model


  • EDSR (Enhanced Deep Super-Resolution) is a popular deep learning model used for single-image super-resolution tasks. It aims to generate a high-resolution image from a low-resolution input image.

  • EDSR is known for its simplicity and effectiveness in achieving state-of-the-art performance in super-resolution tasks

  • Now let us see the steps involved in enhancing the resolution of low quality images using EDSR deep learning model


First we will initialize the super resolution object

# initialize super resolution object
sr = dnn_superres.DnnSuperResImpl_create()

# read the model
path = 'EDSR_x4.pb'
sr.readModel(path)

# set the model and scale
sr.setModel('edsr', 4)
  • First we will create an instance of the DnnSuperResImpl class, which will be used to perform super-resolution using deep neural networks

  • Next the readModel() method is used to load the pre-trained EDSR model from the specified path. The model is usually stored in a .pb file format

  • Next set the loaded EDSR model as the active model for super-resolution. The first argument is the model type, which in this case is 'edsr'. The second argument specifies the scaling factor, which determines the degree of upscaling

  • After executing these steps, the sr object can be used to perform super-resolution using the EDSR model


Next step is optional if you have cuda support then you can execute the below code to set the preferable backend and target else it will use the CPU

# if you have cuda support
sr.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
sr.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)
  • setPreferableBackend() sets the preferable backend for the EDSR model to CUDA. By setting the backend to CUDA, the EDSR model can leverage the power of NVIDIA GPUs for faster inference.

  • setPreferableTarget() sets the preferable target for the EDSR model to CUDA

  • By setting both the backend and target to CUDA, you ensure that the EDSR model utilizes the GPU's parallel processing capabilities, which can significantly speed up the super-resolution process compared to using only the CPU


Next we will have to feed the image for which you want to upscale the resolution

# load the image
image = cv2.imread('test.png')
  • This will load the given image


Next we will upscale the image using initialized EDSR model

# upsample the image
upscaled = sr.upsample(image)
# save the upscaled image
cv2.imwrite('upscaled_test.png', upscaled)
  • upsample() method of the sr object is used to perform super-resolution on the input image. The method takes the low-resolution image as input and returns the corresponding high-resolution image obtained through the EDSR model.

  • We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('upscaled_test.png') and the image array (upscaled) as inputs and writes the image to the specified file path.


Next we will upscale the same image using traditional bicubic model

# traditional method - bicubic
bicubic = cv2.resize(image, (upscaled.shape[1], upscaled.shape[0]), interpolation=cv2.INTER_CUBIC)
# save the image
cv2.imwrite('bicubic_test.png', bicubic)
  • cv2.resize() function performs bicubic interpolation on the low-resolution image and resizes the image to the specified dimensions of the upscaled image using bicubic interpolation

  • We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('bicubic_test.png') and the image array (bicubic) as inputs and writes the image to the specified file path


Next we will compare the upscaled image from both models


original low resolution image for model input
Original low resolution image

  • This is the low resolution input image

  • The resolution of this image is 232 X 155


enhanced high resolution image using EDSR
Enhanced highresolution image using EDSR

  • This is the upscaled image obtained using EDSR model

  • The resolution of this image is 928 X 620

  • We can see that the resolution of original image is enhanced by 4 times and the quality is also retrieved.


increased resolution output image for bicubic model
Increased resolution output image for bicubic method

  • This is the upscaled image obtained using traditional method

  • The resolution of this image is 928 X 620 but the quality of the image is not up to the mark.

The traditional model has increased the resolution of image but you can clearly see that there is some kind of distortion all over the image . On the other hand, we can see that the EDSR model has increased the resolution of image by retrieving the high quality.



Enhance images with super resolution using LapSRN Model


  • LapSRN (Laplacian Pyramid Super-Resolution Network) is a deep learning model used for single-image super-resolution tasks.

  • It is designed to generate high-resolution images from low-resolution inputs using a laplacian pyramid-based approach.

  • LapSRN is known for its ability to produce high-quality super-resolved images while maintaining fine details

  • The EDSR model which we used earlier upscales the images up to 4X times whereas the LapSRN model can upscale the image up to 8X times

  • Now let us see the steps involved in enhancing the resolution of low quality images using LapSRN deep learning model. The process is similar to what you saw for EDSR model


First we will initialize the super resolution object


# initialize super resolution object
sr = dnn_superres.DnnSuperResImpl_create()

# read the model
path = 'LapSRN_x8.pb'
sr.readModel(path)

# set the model and scale
sr.setModel('lapsrn', 8)
  • First we will create an instance of the DnnSuperResImpl class, which will be used to perform super-resolution using deep neural networks

  • Next the readModel() method is used to load the pre-trained LapSRN model from the specified path. The model is usually stored in a .pb file format

  • Next set the loaded LapSRN model as the active model for super-resolution. The first parameter is the model type, which in this case is 'lapsrn'. The second parameter, 8, specifies the scale factor, indicating that the model is trained to perform 8x super-resolution

  • After executing these steps, the sr object can be used to perform super-resolution using the LapSRN model


As mentioned earlier the next step is optional if you have cuda support then you can execute the below code to set the preferable backend and target else it will use the CPU

# if you have cuda support
sr.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
sr.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

Next we will have to feed the image for which you want to upscale the resolution

# load the image
image = cv2.imread('test.png')
  • This will load the given image


Next we will upscale the image using initialized LapSRN model

# upsample the image
upscaled = sr.upsample(image)
# save the upscaled image
cv2.imwrite('upscaled_test_lapsrn.png', upscaled)
  • Similar to EDSR here also upsample() method of the sr object is used to perform super-resolution on the input image. The method takes the low-resolution image as input and returns the corresponding high-resolution image obtained through the LapSRN model.

  • We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('upscaled_test_lapsrn.png') and the image array (upscaled) as inputs and writes the image to the specified file path.


Next we will upscale the same image using traditional bicubic model

# traditional method - bicubic
bicubic = cv2.resize(image, (upscaled.shape[1], upscaled.shape[0]), interpolation=cv2.INTER_CUBIC)
# save the image
cv2.imwrite('bicubic_test_8x.png', bicubic)
  • Here as well we have used the same traditional bicubic method to enhance the resolution of the image

  • We will save the image obtained using imwrite() function from OpenCV. This function takes the filename ('bicubic_test_8x.png') and the image array (bicubic) as inputs and writes the image to the specified file path



Next we will compare the upscaled image from both models


original low resolution image for LapSRN
Original low resolution image
  • We have provided same input image that was used for EDSR model

  • The resolution of this image is 232 X 155


Super resolution image from LapSRN model
Super resolution image from LapSRN
  • This is the upscaled image obtained using LapSRN model

  • The resolution of this image is 1856 X 1240

  • We can see that the resolution of original image is enhanced by 8 times and the quality is also retrieved.


  • This is the upscaled image obtained using traditional method

  • The resolution of this image is 1856 X 1240 but the quality of the image is not up to the mark.

The traditional model has increased the resolution of image but you can clearly see that there is some kind of distortion all over the image . On the other hand, we can see that the LapSRN model has increased the resolution of image up to 8X times with minor distortion on image.



Final Thoughts

  • The bicubic method is a simple and fast technique but often leads to blurry results with limited improvement in image quality whereas deep learning-based methods like EDSR and LapSRN have shown superior performance in terms of generating high-quality super-resolved images. They can produce sharper details and preserve more texture information compared to the bicubic method

  • Bicubic interpolation is computationally lightweight and can quickly upscale an image. On the other hand, EDSR and LapSRN involve training deep neural networks and can be computationally intensive, requiring substantial computational resources and time for training. However, during the inference phase, where the models are used to upscale images, the computational complexity of EDSR and LapSRN can vary depending on the model architecture and implementation

  • The traditional bicubic method tends to struggle with higher upscaling factors, producing blurrier and less satisfying results. EDSR and LapSRN, being deep learning-based methods, have been designed to handle larger upscaling factors. While EDSR performs well for a wide range of upscaling factors, LapSRN's progressive approach specifically addresses the challenge of generating high-resolution images with better results for larger upscaling factors

  • In conclusion, deep learning-based image enhancement methods have shown great potential in improving image quality and generating visually appealing enhancements. Continued research and development in this field will likely lead to further advancements and more sophisticated techniques for enhancing images in the future

In this project tutorial we have seen that the traditional bicubic method is a simple and fast interpolation technique that can provide satisfactory results for modest upscaling factors. However, EDSR and LapSRN, as deep learning-based super-resolution methods, offer superior performance in terms of generating sharper, visually appealing images with enhanced details, particularly for larger upscaling factors. However, they come with increased computational complexity and the need for training on specific datasets.


Get the project notebook from here


Thanks for reading the article!!!


Check out more project videos from the YouTube channel Hackers Realm


Comments


bottom of page