sábado, 19 de mayo de 2012

Integrating OpenCV in Qt GUI applications

OpenCV is, hands down, the best Computer Vision library out there. If you want to create multi-platform computer vision applications, OpenCV is the way to go. The thing is, it doesn't provide you with the tools to create rich UIs, other than the "basic" controls which will make your app look like it never left the research environment it was created on.

The best multi-platform UI library is Qt. Wouldn't it be nice to use QT to create the UIs and OpenCV to crunch all the data? The problem is, neither Qt nor OpenCV provide tools for smooth interaction. How do you display an OpenCV image in a Qt widget? Not possible, at least not in an easy way.

Let's see how we can fix this. The idea is to have a Qt QWidget class (the base class for all the UI elements in Qt) which is able to display OpenCV images. Qt has the ability to display objects of the type QImage via the QPainter::drawImage() method. Of course, a QImage is no OpenCV image...

Luckily the QImage class has support for in-memory construction, that is, you can assign a QImage to an existing image stored in the memory. But, unluckily, QImage only supports a set of image formats which are not OpenCV's defaults. By default, OpenCV stores color images in memory using the BGR byte ordering instead of RGB which is the format supported by QImage. Also, if you want to display grayscale images, QImage doesn't support regular, 8-bit grayscale images.

For these reasons, some conversion needs to be done prior to displaying, so we can assign a OpenCV image buffer to a QImage.

With this in mind, let's build our QWidget class that displays OpenCV images. It will store internally a QImage which will point to a converted OpenCV image, The QWidget should be able to update the image, so we can display video on the widget itself.

Here's the code. Store in cvimagewidget.h.

#pragma once
#include <QWidget>
#include <QImage>
#include <QPainter>
#include <opencv2/opencv.hpp>

class CVImageWidget : public QWidget
    explicit CVImageWidget(QWidget *parent = 0) : QWidget(parent) {}

    QSize sizeHint() const { return _qimage.size(); }
    QSize minimumSizeHint() const { return _qimage.size(); }

public slots:
    void showImage(const cv::Mat& image) {
        // Convert the image to the RGB888 format
        switch (image.type()) {
        case CV_8UC1:
            cvtColor(image, _tmp, CV_GRAY2RGB);
        case CV_8UC3:
            cvtColor(image, _tmp, CV_BGR2RGB);

        // QImage needs the data to be stored continuously in memory
        // Assign OpenCV's image buffer to the QImage. Note that the bytesPerLine parameter
        // (http://qt-project.org/doc/qt-4.8/qimage.html#QImage-6) is 3*width because each pixel
        // has three bytes.
        _qimage = QImage(_tmp.data, _tmp.cols, _tmp.rows, _tmp.cols*3, QImage::Format_RGB888);

        this->setFixedSize(image.cols, image.rows);


    void paintEvent(QPaintEvent* /*event*/) {
        // Display the image
        QPainter painter(this);
        painter.drawImage(QPoint(0,0), _qimage);
    QImage _qimage;
    cv::Mat _tmp;

Now let's try this baby. Put this on main.cpp:

#include <cvimagewidget.h>

#include <QDialog>
#include <QApplication>
#include <QMainWindow>

int main(int argc, char** argv) {
    QApplication app(argc, argv);
    QMainWindow window;
    // Create the image widget
    CVImageWidget* imageWidget = new CVImageWidget();
    // Load an image
    cv::Mat image = cv::imread("somepicture.jpg", true);
    return app.exec();

Compile using qmake and you should see the image in all its glory inside a Qt widget. Also note that if you perform several calls to CVImageWidget::showImage(image), the widget will be refreshed on each call, so you can easily display video frames in the widget in real time.

viernes, 18 de mayo de 2012

Installing OpenCL in Ubuntu 12.04

OpenCL is the new standard for "heterogeneous computing". This is a fancy way of saying that it enables us to write code that will seamlessly run both in CPU and GPU without any changes, unlike for example CUDA which only runs on GPUs.

Now, my laptop doesn't have any GPU (who has the money for that??), but I want to use OpenCL for some computations. Luckily, Intel provides an implementation of the OpenCL standard which runs on any CPU, not just the Intel ones, without the need of installing additional and proprietary Linux drivers.

The SDK provided by Intel comes in an RPM format so it is not Ubuntu-friendly. Thankfully, it is possible to convert the RPM to a .deb package and, with some additional tweaking, having a working OpenCL installation. Most of the following instructions were extracted from the ~mhr3 blog so credit goes for them.

OK, now, for the actual install:
  1. Download the Intel SDK for OpenCL Applications from Intel's web site (http://software.intel.com/en-us/articles/vcsource-tools-opencl-sdk/). The download options are not easy to spot in the messy page, they are on the top right bo. After downloading, you will end with a .tgz file with an RPM inside (crazy, I know). In my case, the file is named intel_sdk_for_ocl_applications_2012_x64.tgz.
  2. Extract the RPM from the .tgz file:
    $ tar zxvf intel_sdk_for_ocl_applications_2012_x64.tgz

    This will extract the intel_ocl_sdk_2012_x64.rpm file.
  3. Convert the RPM file to .deb format and install:
    $ sudo apt-get install -y rpm alien libnuma1    # In case you don't have these packages
    $ fakeroot alien --to-deb intel_ocl_sdk_2012_x64.rpm
    $ sudo dpkg -i intel-ocl-sdk_2.0-31361_amd64.deb
  4. Now the SDK and libraries will be installed to /usr/lib64, while Ubuntu expects them to be in /usr/lib. No problem, just make a symlink and update the library cache:
    $ sudo ln -s /usr/lib64/libOpenCL.so /usr/lib/libOpenCL.so
    $ sudo ldconfig
That's it! OpenCL should be installed now. Let's try with a test program to see the device capabilities.

 #include <iostream>  
 #include <CL/cl.hpp>  
 #include <boost/foreach.hpp>
 int main(int, char**) {  
     std::vector<cl::Platform> platforms;  
     BOOST_FOREACH(cl::Platform platform, platforms) {  
         std::cout << "Platform: " << platform.getInfo<CL_PLATFORM_NAME>() << std::endl;  
         std::vector<cl::Device> devices;  
         platform.getDevices(CL_DEVICE_TYPE_GPU | CL_DEVICE_TYPE_CPU, &devices);  
         BOOST_FOREACH(cl::Device device, devices) {  
             std::cout << "Device: " << device.getInfo<CL_DEVICE_TYPE>();  
             std::cout << " (" << CL_DEVICE_TYPE_GPU << " means GPU, " << CL_DEVICE_TYPE_CPU << " means CPU)" << std::endl;  

Save this snippet into opencl.cpp, compile and run:
$ g++ opencl.cpp -lOpenCL -o opencl && ./opencl
Platform: Intel(R) OpenCL
Device: 2 (4 means GPU, 2 means CPU)
This shows that this particular machine has one OpenCL-capable Intel CPU.

Coming up: Making your life easier and using PyOpenCL for your computations.