build qt with ssl support

If you does not provide any parameter when configuring qt, the qt binaries you build will not support ssl. When you build a project that needs ssl, you will get the following error:

error: variable 'QSslConfiguration conf' has initializer but incomplete type
     QSslConfiguration conf = request.sslConfiguration();

class QNetworkRequest' has no member named 'sslConfiguration'
error: 'QSslSocket' has not been declared
error: 'class QNetworkRequest' has no member named 'setSslConfiguration'
error: aggregate 'QSslError error' has incomplete type and cannot be defined

That is because the ssl related functions such as sslConfiguration() (in C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\access\qnetworkrequest.cpp)are conditional compiled into Qt5Network.dll based on the marco QT_NO_SSL, i.e., only when QT_NO_SSL is undefined are the functions compiled into Qt5Network.dll. While QT_NO_SSL is indeed defined in C:\build\qtbase\src\network\qtnetwork-config.h.

I’ve been wondering why we should configure before compiling. This example gives an answer. A software such as Qt source code may be compiled into different binaries using conditional compiling instructions based on the environment. The configuring process checks the current environment and produces macros or different values of macros. Then in the actual building stage, the macros or different values of macros will guide the compiler to generate different pieces of code. In this example, the network configure results such as QT_NO_SSL are saved in C:\build\qtbase\src\network\qtnetwork-config.h. The source code that uses the configure results will include this header, e.g., C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\kernel\qtnetworkglobal.h will include the header file:

#include <QtCore/qglobal.h>
#include <QtNetwork/qtnetwork-config.h>

Note that <QtNetwork/qtnetwork-config.h> refers to C:\build\qtbase\include\QtNetwork\qtnetwork-config.h which includes  “../../src/network/qtnetwork-config.h”, i.e., C:\build\qtbase\src\network\qtnetwork-config.h. C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\access\qnetworkrequest.cpp includes C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\access\qnetworkrequest.h which includes C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\kernel\qtnetworkglobal.h (C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\kernel\kernel.pri adds the directory to INCLUDEPATH) which includes C:\build\qtbase\src\network\qtnetwork-config.h in the end. That explains why the built Qt5Network.dll does not support ssl.

The explanation of the above compiling errors is likewise. In the project that uses QNetworkRequest, we will include <QNetworkRequest> which is C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\include\QtNetwork\QNetworkRequest, which includes C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\include\QtNetwork\qnetworkrequest.h which includes C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\access\qnetworkrequest.h which includes QtNetwork/qtnetworkglobal.h which is C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\include\QtNetwork\qtnetworkglobal.h which includes C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\kernel\qtnetworkglobal.h which includes QtNetwork/qtnetwork-config.h which is C:\build\qtbase\include\QtNetwork\qtnetwork-config.h which includes C:\build\qtbase\src\network/qtnetwork-config.h which contains the QT_NO_SSL definition, thus the class declaration for QNetworkRequest(in   C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\access\qnetworkrequest.h) will not include the prototype of sslConfiguration() and setSslConfiguration(), which leads the compiling errors.

If you build your project using a qt prebuild such as mingw73_64 accompanied by the qt installation package, all include files are in C:\Qt\Qt5.12.1\5.12.1\mingw73_64\include\QtNetwork\(QNetworkRequest, qnetworkrequest.h, qtnetworkglobal.h, qtnetwork-config.h). Note that this qtnetwork-config.h was generated by configuring on the machine that produces the prebuild qt, not your computer. There is no QT_NO_SSL definition in this version of qtnetwork-config.h which indicates the machine that produces the prebuilt qt supports ssl.

It is time to talk about how qmake generates the QT_NO_SSL definition in C:\build\qtbase\src\network\qtnetwork-config.h. When qmake configures the network module, it parses C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\configure.json. The ssl related features in that configure.json are:

"openssl": {
        "label": "OpenSSL",
        "enable": "false",
        "condition": "features.openssl-runtime || features.openssl-linked",
        "output": [
            "privateFeature",
            { "type": "publicQtConfig", "condition": "!features.openssl-linked" },
            { "type": "define", "negative": true, "name": "QT_NO_OPENSSL" }
        ]
    },
    "openssl-runtime": {
        "autoDetect": "!config.winrt && !config.wasm",
        "enable": "input.openssl == 'yes' || input.openssl == 'runtime'",
        "disable": "input.openssl == 'no' || input.openssl == 'linked' || input.ssl == 'no'",
        "condition": "!features.securetransport && libs.openssl_headers"
    },
    "openssl-linked": {
        "label": "  Qt directly linked to OpenSSL",
        "autoDetect": false,
        "enable": "input.openssl == 'linked'",
        "condition": "!features.securetransport && libs.openssl",
        "output": [
            "privateFeature",
            { "type": "define", "name": "QT_LINKED_OPENSSL" }
        ]
    },
    "securetransport": {
        "label": "SecureTransport",
        "disable": "input.securetransport == 'no' || input.ssl == 'no'",
        "condition": "config.darwin && (input.openssl == '' || input.openssl == 'no')",
        "output": [
            "privateFeature",
            { "type": "define", "name": "QT_SECURETRANSPORT" }
        ]
    },
    "ssl": {
        "label": "SSL",
        "condition": "config.winrt || features.securetransport || features.openssl",
        "output": [ "publicFeature", "feature" ]
    },

You can see the ssl feature is available if we configure winrt or the securetransport feature is available or the openssl feature is available. On my Windows desktop, both winrt and securetransport are unavailable. So we see if openssl is available. The openssl feature is available if the openssl-runtime feature or the openssl-linked feature is available. Since we did not provide -openssl or -ssl option, enable/disable of openssl-runtime are evaluated to false, autoDetect of openssl-runtime is evaluated to true, so qmake further checks the condition. Since  securetransport is unavailable(we did not configure darwin) and libs.openssl_headers is false, the final result is openssl-runtime is unavailable. As to the openssl-linked feature, since enable is false and autoDetect is  false, the openssl-linked  is checked as unavailable, and the condition is even not checked. Now both  openssl-runtime and openssl-linked are checked to be false so openssl is unavailable. The qtConfOutput_feature function will put “#define QT_NO_SSL” in  C:\build\qtbase\src\network\qtnetwork-config.h(publicHeader).

Now, I will explain to you why libs.openssl_headers is evaluated to false in the feature openssl_runtime. This involves the parsing of the openssl_headers lib under the libraries key in C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network/configure.json.

"openssl_headers": {
     "label": "OpenSSL Headers",
     "export": "openssl",
     "test": {
         "tail": [
             "#if !defined(OPENSSL_VERSION_NUMBER) || OPENSSL_VERSION_NUMBER-0 < 0x10000000L",
             "#  error OpenSSL >= 1.0.0 is required",
             "#endif",
             "#if OPENSSL_VERSION_NUMBER-0 >= 0x10002000L && !defined(OPENSSL_NO_EC) && !defined(SSL_CTRL_SET_CURVES)",
             "#  error OpenSSL was reported as >= 1.0.2 but is missing required features, possibly it's libressl which is unsupported",
             "#endif"
         ]
     },
     "headers": [ "openssl/ssl.h", "openssl/opensslv.h" ],
     "sources": [
         {
             "comment": "placeholder for OPENSSL_PATH",
             "libs": ""
         }
     ]
 },

 

The evaluation is done through qtConfEvaluateSingleExpression->qtConfHandleLibrary(openssl_headers)->qtConfLibrary_inline(config.qtbase_network.libraries.openssl_headers.sources.0,config.qtbase_network.libraries.openssl_headers)(since the first source in sources has no type key, its type is set to inline by default.)->qtConfResolveAllLibs(check the existence of libs in a source)->qtConfResolvePathIncs(config.qtbase_network.libraries.openssl_headers.sources.0.includedir,””,config.qtbase_network.libraries.openssl_headers). qtConfResolvePathIncs will call qtConfGetTestIncludes to read the content of config.qtbase_network.libraries.openssl_headers.headers from configure.json( “openssl/ssl.h”, “openssl/opensslv.h” ), call qtConfFindInPathList to search every header file in directories listed in $$QMAKE_DEFAULT_INCDIRS. If it cannot find any of the header files, it will return false. This is the case because qt installation package does not include openssl, and the openssl headers do not exist in the QMAKE_DEFAULT_INCDIRS:C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/include/c++ C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/include/c++/x86_64-w64-mingw32 C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/include/c++/backward C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/include C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/include-fixed C:/Qt/Qt5.12.1/Tools/mingw730_64/x86_64-w64-mingw32/include. Although I installed openssl later but I did not install openssl under C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0/ so qmake won’t find the openssl headers in those directories. so qtConfLibrary_inline returns false. After checking the existence of lib’s binaries and headers, qmake will see if the current lib has a test subkey. If it has a test subkey, call qtConfTest_compile to do the test for the source. Only if  qtConfTest_compile  succeeds is the source accepted.  The qtConfTest_compile  function creates a directory openssl_headers under C:\build\qtbase\config.tests\, and generated some files in that directory: main.cpp, openssl_headers.pro. The file content is gotten from the value of the test subkey and some builtin content. Then the little test project to test openssl_headers is qmaked and built. Only when the test project can be successfully built is the source considered to be accepted. The commands to qmake and build the test project is:

cd /d C:\build\qtbase\config.tests\openssl_headers && C:\build\qtbase\bin\qmake.exe "CONFIG -= qt debug_and_release app_bundle lib_bundle" "CONFIG += shared warn_off console single_arch" "QMAKE_USE += openssl_headers" "QMAKE_LIBS_OPENSSL_HEADERS = " "QMAKE_INCDIR_OPENSSL_HEADERS = C:\\OpenSSL-Win64\\include"  C:/build/qtbase/config.tests/openssl_headers

set MAKEFLAGS=& mingw32-make

 

After checking all sources under openssl_headers/sources/ key(for openssl_headers, there is only one source), none succeeds, so qtConfHandleLibrary sets config.qtbase_network.libraries.openssl_headers.result to false, and qtConfEvaluateSingleExpression returns the value of config.qtbase_network.libraries.openssl_headers.result(false). If any source is accepted, qtConfHandleLibrary  returns true and won’t check the remaining sources of the lib.

Note that the only source in sources,i.e.,sources[0] has an empty libs, qtConfResolveAllLibs has no lib to check so returns true. Also note that if there were multiple sources in sources, qmake would check the same headers(libraries/openssl_headers/headers) during checking each source. If there is no  libraries/openssl_headers/headers, qmake won’t bother to check it, i.e., qtConfResolvePathIncs returns true. Apart from what are in QMAKE_DEFAULT_INCDIRS, you can provide extra include dirs for qmake to check the header files in libraries/openssl_headers/headers. For example, you can run “configure.bat OPENSSL_INCDIR=d:\myprogrammingnotes.com”, then qmake will look for openssl/ssl.h, openssl/opensslv.h in d:\myprogrammingnotes.com. If both header files are found, libs.openssl_headers is evaluated to true. You can also add more paths to search using the OPENSSL_PREFIX parameter. For example, if you run “configure.bat OPENSSL_INCDIR=d:\myprogrammingnotes.com OPENSSL_PREFIX=d:\myopenssl”, qmake will search the headers in both d:\myprogrammingnotes.com and d:\myopenssl\include in addition to those in QMAKE_DEFAULT_INCDIRS. The OPENSSL_PREFIX argument also provides extra path, i.e., d:\myopenssl\lib, for searching the lib(if any). You may wonder why I use OPENSSL_INCDIR and OPENSSL_PREFIX instead of OPENSSL_HEADERS_INCDIR and OPENSSL_HEADERS_PREFIX. That is because the alias, i.e., what is called externally, of the lib openssl_headers is openssl as specified in configure.json(“export”: “openssl”).

The value of the subkey libraries/openssl_headers/headers is currently [ “openssl/ssl.h”, “openssl/opensslv.h” ], but it can be more complex structure such as [{condition:xxx,headers:[yyy,zzz]},{condition:xxxx,headers:[yyyy,zzzz]}], qmake will only check those headers whose condition is evaluated to true.

From the above analysis, we know that each source in sources is mainly used to check whether the provided libs in that source exist. The check for the header existence is just a part-time job. In fact the libs under the libraries key can have relationship that is similar to class inheritance. For example, the lib openssl inherits from openssl_headers:

"openssl": {
    "label": "OpenSSL",
    "test": {
        "inherit": "openssl_headers",
        "main": "SSL_free(SSL_new(0));"
    },
    "sources": [
        { "type": "openssl" },
        {
            "libs": "-lssleay32 -llibeay32",
            "condition": "config.win32"
        },
        {
            "libs": "-llibssl -llibcrypto",
            "condition": "config.msvc"
        },
        {
            "libs": "-lssl -lcrypto",
            "condition": "!config.msvc"
        }
    ]
}

The sources of openssl are used to check the existence of shared libs while the check of the existence of lib headers is done by its parent lib: openssl_headers. Specifically, for each source of openssl, after checking whether the libs exist, qmake will check if the header files listed in its parent lib’s header(openssl_headers/headers) exist.

Ok, we’ve analyzed the rubbish code of qmake that is aimed at setting up technical barriers. We’re now ready to figure out a way to enable the ssl support to build qt.

First, we can use the option “-openssl yes” or “-openssl runtime” to let enable be true for the feature openssl-runtime, then we use OPENSSL_INCDIR=C:\OpenSSL-Win64\include to let the condition be true for openssl-runtime. Now the openssl-runtime feature is available so the ssl feature is available and QT_NO_SSL won’t be emitted to  C:\build\qtbase\src\network\qtnetwork-config.h. Thus, ssl related code such as QSslSocket will be compiled without problem. The commands are:

configure.bat -openssl yes OPENSSL_INCDIR=C:\OpenSSL-Win64\include
or,
configure.bat -openssl runtime OPENSSL_INCDIR=C:\OpenSSL-Win64\include

You may find this method to enable ssl only needs the openssl header files, not openssl libs or dlls. In fact no code is copied from openssl libs to Qt5Network.dll. You can build Qt5Network.dll without openssl libs/dlls. The load of  Qt5Network.dll is also not dependent on the load of openssl dll. Your project using Qt5Network.dll does not need the openssl libs/dll as well if it does not use ssl functions. But if your project uses ssl functions, you need to copy the openssl dlls(libcrypto-version-x64.dll,libssl-version-x64.dll) to your app’s installation directory,otherwise, your app won’t work correctly although it can be started. How is it possible? Well, if you use this so called runtime openssl method and call a function that uses functions provided by openssl dlls such as QNetworkRequest::sslConfiguration(), the function will call QSslSocketPrivate::ensureInitialized() to load openssl dlls (::LoadLibrary) and resolve the openssl functions (::GetProcAddress) before calling the openssl functions such as OPENSSL_init_ssl. These openssl functions are wrapped by qt as q_OPENSSL_init_ssl. Because qt source code do not use the openssl function names directly, it does not need to link against the openssl libs to resolve the names. In fact, the openssl functions take pointers as their parameters and return value. It actually has no need of the openssl headers. Just forward declaring the openssl types of structures is enough to build Qt5Network.dll. They detect  and include openssl headers such as openssl/ssl.h just because they are too lazy to write the forward declarations into their code. The result is you must get the openssl headers to build qt with ssl support, while the ideal situation is that you do not need any openssl stuff to build qt with ssl support, and only need the two openssl dlls to run your app that is linked against the ssl enabled qt libs. You may have used the function QSslSocket::supportsSsl() to check the ssl support in runtime. This function also tries to load the openssl dlls and returns true if succeeds.

The second method to add ssl support for qt build is to link the openssl lib to Qt5Network.dll. To do that, you should use the “-openssl linked” option. Then you should gareentee libs.openssl is evaluated to true so the openssl-linked feature is available, and openssl/ssl features are then available too. There are quite a few ways to let libs.openssl be true as there are multiple sources under openssl/sources. Source 0 indicates you can create an environment variable: OPENSSL_LIBS whose content is “-llibssl” or “-llibcrypto”, or “-llibssl -llibcrypto”. qmake will check if the libs specified in OPENSSL_LIBS exist. For libssl, it will be expanded to liblibssl.dll.a liblibssl.a libssl.dll.a libssl.a libssl.lib. If one of these files is found in QMAKE_DEFAULT_LIBDIRS:C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc/x86_64-w64-mingw32/7.3.0 C:/Qt/Qt5.12.1/Tools/mingw730_64/lib/gcc C:/Qt/Qt5.12.1/Tools/mingw730_64/x86_64-w64-mingw32/lib C:/Qt/Qt5.12.1/Tools/mingw730_64/lib, qtConfLibrary_openssl returns true. It is not likely that you installed openssl to those paths. To let qmake find the openssl libs, you need to use the -L option(lpaths) to add extra search paths (EXTRA_LIBDIR) during configure. Unlike qtConfLibrary_inline, qtConfLibrary_openssl  does not check the exisistence of header files nor create QMAKE_INCDIR_OPENSSL_HEADERS = C:\\OpenSSL-Win64\\include(as done for openssl_headers where your input variable OPENSSL_INCDIR is copied to config.qtbase_network.libraries.openssl_headers.sources.0.includedir in qtConfResolvePathIncs, and further copied to QMAKE_INCDIR_OPENSSL_HEADERS in qtConfLibraryArgs, which is used to generate the qmake argument for building the test project of openssl_headers) . But libraries.openssl.test has an inherit subkey whose content is “openssl_headers”, which means openssl’s test program’s source code will include the headers gotten from libraries.openssl_headers.headers. This will lead to the failure of building openssl’s test program as the openssl headers cannot be found. The cue is not using OPENSSL_INCDIR  but the -I option(C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\configure.json, commandline/prefix/I) which goes to EXTRA_INCLUDEPATH which goes to the qmake argument “INCLUDEPATH +=…” in qtConfTest_compile, which makes the build of openssl’s project successful. So the final configure command is:

configure.bat -openssl linked -L C:\OpenSSL-Win64\lib -I C:\OpenSSL-Win64\include

The -L option and the -I option will be saved in c:\build\qtbase\mkspecs\qmodule.pri as:

EXTRA_INCLUDEPATH += C:\\OpenSSL-Win64\\include
EXTRA_LIBDIR += C:\\OpenSSL-Win64\\lib

When qmaking C:\Qt\Qt5.12.1\5.12.1\Src\qtbase\src\network\network.pro, the EXTRA_INCLUDEPATH will go to the INCPATH Makefile variable and the EXTRA_LIBDIR will go to the LIBS Makefile variable.

The built Qt5Network.dll will depend on the openssl dll   libcrypto-version-x64.dll,libssl-version-x64.dll, and won’t be loaded without the two openssl dlls. You app that links against Qt5Network.dll  cannot be started without libcrypto-version-x64.dll,libssl-version-x64.dll, even it does not use any ssl function.

Now, we look at other sources of libraries/openssl/sources. The second source is for old version of openssl which names its dll after libeay32.dll and ssleay32.dll. The third source is for msvc compiler instead of mingw. The last source lists libs as “-lssl -lcrypto”. The lib ssl will be expanded to for .a libs: libssl.dll.a libssl.a ssl.dll.a ssl.a, and one .lib lib:ssl.lib.Unfortunately, none of these libs can be found in my openssl installation. I have to change the libs of the last source from “-lssl -lcrypto” to “-llibssl -llibcrypto” to make it work.   To let qmake find these libs, you also need to provide the “-L C:\OpenSSL-Win64\lib” option. To succefully build the openssl’s test project, you also need to provide the include directory of openssl to avoid the openssl/ssl.h not found errors. Here, you have more options to choose than testing the first source because the last source is processd by qtConfLibrary_inline rather than qtConfLibrary_openssl as for the first source. qtConfLibrary_inline calls qtConfResolvePathIncs to resolve the include paths so you can use the “OPENSSL_INCDIR=C:\OpenSSL-Win64\include”. Of course, you can also use the “-I C:\OpenSSL-Win64\include” option as for the first source. So the final configure command is:

configure.bat -openssl linked -L C:\OpenSSL-Win64\lib -I C:\OpenSSL-Win64\include
or,

configure.bat -openssl linked -L C:\OpenSSL-Win64\lib OPENSSL_INCDIR=C:\OpenSSL-Win64\include

No need to set OPENSSL_LIBS envoroment variable. Just modify the libs of the fourth(last) source of libraries/openssl/sources.

A subtle difference between using source 0 and source 3 is using source 0 will write EXTRA_INCLUDEPATH and EXTRA_LIBDIR to c:\build\qtbase\mkspecs\qmodule.pri, but using source 3 with the OPENSSL_INCDIR qmake argument only writes EXTRA_LIBDIR to c:\build\qtbase\mkspecs\qmodule.pri but writes “QMAKE_INCDIR_OPENSSL = C:\\OpenSSL-Win64\\include” to C:\build\qtbase\src\network\qtnetwork-config.pri. Either way can garentee the openssl headers can be found when building Qt5Network.dll.

We have talked about how to build qt with openssl. In summary, you should use one of the following commands to configure before compiling qt with openssl.

  • configure.bat -openssl yes OPENSSL_INCDIR=C:\OpenSSL-Win64\include
  • configure.bat -openssl runtime OPENSSL_INCDIR=C:\OpenSSL-Win64\include
  • set OPENSSL_LIBS=”-llibssl -llibcrypto”,  configure.bat -openssl linked -L C:\OpenSSL-Win64\lib -I C:\OpenSSL-Win64\include
  • change source in configure.json to “-llibssl -llibcrypto”, configure.bat -openssl linked -L C:\OpenSSL-Win64\lib -I C:\OpenSSL-Win64\include
  • change source in configure.json to “-llibssl -llibcrypto”, configure.bat -openssl linked -L C:\OpenSSL-Win64\lib OPENSSL_INCDIR=C:\OpenSSL-Win64\include

The first two configure commands will let Qt5Network.dll load openssl dlls dynamically at runtime, while the last 3 configure commands will make Qt5Network.dll load openssl dlls at start-up time.

 P.S.: for old versions of qt such as qt 5.3, there is no “-openssl runtime” or “-openssl yes” option.  I do not know what version of qt this article is talking about, but if you use the -openssl-runtime option when configuring, it will complain “Unknown option -openssl-runtime”. The openssl logic is in qtbase\src\network\ssl\ssl.pri. Specifically, you should use the following configure commands to add openssl support to your qt build. To use runtime openssl:

configure.bat -openssl -I C:\OpenSSL-Win64\include

To link to openssl at building time(statically or dynamically):

configure.bat -openssl -openssl-linked -I C:\OpenSSL-Win64\include -L C:\OpenSSL-Win64\lib OPENSSL_LIBS="-llibssl -llibcrypto"

Note that , to use runtime openssl, do not include the -L and OPENSSL_LIBS options when configuring, otherwise, you will meet problems when linking to another version of openssl in your own project.

If you like my content, please consider buying me a coffee. Buy me a coffeeBuy me a coffee Thank you for your support!
Posted in

Leave a Reply