Integrating OpenCV in Qt GUI applications

OpenCV is, hands down, the best Computer Vision library out there. If you want to create multi-platform computer vision applications, OpenCV is the way to go. The thing is, it doesn't provide you with the tools to create rich UIs, other than the "basic" controls which will make your app look like it never left the research environment it was created on.

The best multi-platform UI library is Qt. Wouldn't it be nice to use QT to create the UIs and OpenCV to crunch all the data? The problem is, neither Qt nor OpenCV provide tools for smooth interaction. How do you display an OpenCV image in a Qt widget? Not possible, at least not in an easy way.

Let's see how we can fix this. The idea is to have a Qt QWidget class (the base class for all the UI elements in Qt) which is able to display OpenCV images. Qt has the ability to display objects of the type QImage via the QPainter::drawImage() method. Of course, a QImage is no OpenCV image...

Luckily the QImage class has support for in-memory construction, that is, you can assign a QImage to an existing image stored in the memory. But, unluckily, QImage only supports a set of image formats which are not OpenCV's defaults. By default, OpenCV stores color images in memory using the BGR byte ordering instead of RGB which is the format supported by QImage. Also, if you want to display grayscale images, QImage doesn't support regular, 8-bit grayscale images.

For these reasons, some conversion needs to be done prior to displaying, so we can assign a OpenCV image buffer to a QImage.

With this in mind, let's build our QWidget class that displays OpenCV images. It will store internally a QImage which will point to a converted OpenCV image, The QWidget should be able to update the image, so we can display video on the widget itself.

Here's the code. Store in cvimagewidget.h.

#pragma once
#include <QWidget>
#include <QImage>
#include <QPainter>
#include <opencv2/opencv.hpp>

class CVImageWidget : public QWidget
{
    Q_OBJECT
public:
    explicit CVImageWidget(QWidget *parent = 0) : QWidget(parent) {}

    QSize sizeHint() const { return _qimage.size(); }
    QSize minimumSizeHint() const { return _qimage.size(); }

public slots:
    
    void showImage(const cv::Mat& image) {
        // Convert the image to the RGB888 format
        switch (image.type()) {
        case CV_8UC1:
            cvtColor(image, _tmp, CV_GRAY2RGB);
            break;
        case CV_8UC3:
            cvtColor(image, _tmp, CV_BGR2RGB);
            break;
        }

        // QImage needs the data to be stored continuously in memory
        assert(_tmp.isContinuous());
        // Assign OpenCV's image buffer to the QImage. Note that the bytesPerLine parameter
        // (http://qt-project.org/doc/qt-4.8/qimage.html#QImage-6) is 3*width because each pixel
        // has three bytes.
        _qimage = QImage(_tmp.data, _tmp.cols, _tmp.rows, _tmp.cols*3, QImage::Format_RGB888);

        this->setFixedSize(image.cols, image.rows);

        repaint();
    }

protected:
    void paintEvent(QPaintEvent* /*event*/) {
        // Display the image
        QPainter painter(this);
        painter.drawImage(QPoint(0,0), _qimage);
        painter.end();
    }
    
    QImage _qimage;
    cv::Mat _tmp;
};

Now let's try this baby. Put this on main.cpp:

#include <cvimagewidget.h>

#include <QDialog>
#include <QApplication>
#include <QMainWindow>

int main(int argc, char** argv) {
    QApplication app(argc, argv);
    QMainWindow window;
    
    // Create the image widget
    CVImageWidget* imageWidget = new CVImageWidget();
    window.setCentralWidget(imageWidget);
    
    // Load an image
    cv::Mat image = cv::imread("somepicture.jpg", true);
    imageWidget->showImage(image);
    
    window.show();
    
    return app.exec();
}

Compile using qmake and you should see the image in all its glory inside a Qt widget. Also note that if you perform several calls to CVImageWidget::showImage(image), the widget will be refreshed on each call, so you can easily display video frames in the widget in real time.