yet another qt application deployment problem

Deploying a Qt application always causes problems for Qt users. This is why there are so many complaints and appeal to add a command to deploy program automatically. But Qt team won’t do this,  for unknown reason.Instead, they composed many articles (e.g., this and this)about this issue and suggest you read them and try to understand them all. Unfortunately, even you read all their articles carefully, you still can not get your  application run as a standalone. Often the problem appears as you can run your program in Qt creator but can not run it outside Qt Creator, or your application can be executed on the development machine but can not be executed on target machine that qt has not been  installed.We know clearly two reasons for this: the environment variables are not set correctly, and/or some dlls your application depends on are not included in the application package. But resolving the two problem is very hard if you are not a savvy developer.
Let’s consider the environment variable problem first. If the application runs well under Qt creator but can not run outside Qt Creator, you may have got the environment variable problem. QT Creator creates some environment variables itself, automatically. If you do not create those environment variables outside QT Creator, you program is very likely to stop running. But how to see the environment variables Qt Creator creates? If you click the menu item Tools/Options to check if environment variables are set there, you’ll be disappointed.

This is a major design defect of Qt Creator: it scatters the settings everywhere making it very difficult to find where a particular setting is(if you’ve read  my post about how to find the current compiler Qt creator is using, you should agree my opinion). In fact, the environment variables Qt creator set up to run your program can be found by clicking the “project” button on left bar, then clicking “Build & Run”/Run.

You can find “Base environment for this run configuration” under the “Run Environment” section.There you can choose “Build Environment”,”System Environment”, or “Clean Environment” for your program. The “System Environment” is what you can find outside Qt creator. i.e., the environment of the Explorer, while in the “Build Environment”, Qt creator creates many new environment variables of its own. That is the cause your application cannot run outside Qt creator. To let your application execute outside Qt creator, you should duplicate some environment variables such as PATH in Build environment to the Explorer.

The second reason for unsuccessful deployment is missing dlls. You just cannot know exactly what dlls are used by your program. You application may link to dlls in compiling time, or it loads some dlls at runtime. All articles teaching you how to deploy a Qt application will tell you to use Dependency Walker to find the dependent dlls. However, this tool is outdated/obsoleted. The old version (regardless of 32bit version or 64 bit versin)on its official website will freeze loading program in modern Windows(such as Windows 10). I found this post that claims the updated version of Dependency Walker is contained in Windows Driver Kit(WDK), but actually it has already been dropped from that package. The hint in the comments of that post led me download the old WDK(WDK for Windows 8),and Dependency Walker did exist in that package, but unfortunately did not work, either. I used Dependency Walker in old version of Windows and it did work. Even you can make Dependency Walker to load your program, you need to know how to use this piece of software so you can get the correct set of dependent dlls. If you load your program by clicking the File/Open menu item, you can find only part of dlls your application depends on. Those are dlls that are linked to your program in compiling time. But Qt program may load other dlls after running. To check whole set of dependent dlls, you should use the profiling functionality(Profile/Start Profiling) of Dependency Walker.

This function actually runs your program and prints the loaded dlls on the fly. Of course, to make sure your program can be run normally, you should prepare the environment variables beforehand. I’ve not figured out a way to let Dependency Walker to create needed environment variables(like what Qt Creator does) to run a program.

Since Dependency Walker does not work in latest Windows, you may resort to other software to check dlls your program is loading. For me,as a C++ programmer, why not write some code to get the loaded dlls myself? I copied this code into my to-be-checked program and made some improvement(mainly related to convert TCHAR* to QString and get the id of current process)o it can work well under Qt.

void BrowserWindow::handleShowDllsTriggered()
{


    DWORD processID=GetCurrentProcessId();
    HMODULE hMods[1024];
    HANDLE hProcess;
    DWORD cbNeeded;
    unsigned int i;

    // Print the process identifier.

    printf( "\nProcess ID: %u\n", processID );

    // Get a handle to the process.

    hProcess = OpenProcess( PROCESS_QUERY_INFORMATION |
                            PROCESS_VM_READ,
                            FALSE, processID );
    if (NULL == hProcess)
        return;

   // Get a list of all the modules in this process.

    if( EnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded))
    {
        for ( i = 0; i < (cbNeeded / sizeof(HMODULE)); i++ )
        {
            TCHAR szModName[MAX_PATH];

            // Get the full path to the module's file.

            if ( GetModuleFileNameEx( hProcess, hMods[i], szModName,
                                      sizeof(szModName) / sizeof(TCHAR)))
            {
                // Print the module name and handle value.

               // _tprintf( TEXT("\t%s (0x%08X)\n"), szModName, hMods[i] );
              //  _tprintf( TEXT("\t%s \n"), szModName );
                QString s=QString::fromWCharArray(szModName);
                qDebug()<<s;
    
            }
        }
    }

    // Release the handle to the process.

    CloseHandle( hProcess );
}

 

Whenever I want to know the current loaded dlls, I execute this piece of code, and it prints all dlls with their full path names. This works great! I only need to copy all Qt/compiler related dlls to my application package to accomplish the deployment.

But you should pay attention where to copy those dlls. Simply putting them in the same folder as your main program may not work. Some dlls should be put in sub-folders in your package directory. I really do not know exactly what sub-folders I should create for which dlls. Fortunately, there is a program in the Qt installation directory called windeployqt.exe, which intends to be used to deploy an application. As you know, Qt team seems not to want its customers to deploy their applications too easily, they creates this malfunctioning tool. If you use windeployqt.exe to deploy your Qt app, most likely it won’t work. It may work if your target machine has the same architecture as your build machine since winqtdelpoy.exe will copy all dlls from the directory(such as C:\qt5100\5.10.0\msvc2017_64\) in the Qt installation directory that corresponds to the architecture of the build machine.But it does not work if the target machine runs, for example, a 32-bit OS. So why I mention this tool if it does not work? Well, I use this tool to create sub-folders in my packaging directory and get dependent dll names even the dll versions are wrong. Then I use my own code(talked earlier) to pick the correct version of dlls and copy them to replace the wrong versions. In short, windeployqt.exe gives us what to copy and where to copy, while my code does the actual copy. Here is my improved version of code:

void BrowserWindow::handleShowDllsTriggered()
{

    DWORD processID=GetCurrentProcessId();
    HMODULE hMods[1024];
    HANDLE hProcess;
    DWORD cbNeeded;
    unsigned int i;

    // Print the process identifier.

    printf( "\nProcess ID: %u\n", processID );

    // Get a handle to the process.

    hProcess = OpenProcess( PROCESS_QUERY_INFORMATION |
                            PROCESS_VM_READ,
                            FALSE, processID );
    if (NULL == hProcess)
        return;

   // Get a list of all the modules in this process.

    if( EnumProcessModules(hProcess, hMods, sizeof(hMods), &cbNeeded))
    {
        for ( i = 0; i < (cbNeeded / sizeof(HMODULE)); i++ )
        {
            TCHAR szModName[MAX_PATH];

            // Get the full path to the module's file.

            if ( GetModuleFileNameEx( hProcess, hMods[i], szModName,
                                      sizeof(szModName) / sizeof(TCHAR)))
            {
                // Print the module name and handle value.

               // _tprintf( TEXT("\t%s (0x%08X)\n"), szModName, hMods[i] );
              //  _tprintf( TEXT("\t%s \n"), szModName );
                QString s=QString::fromWCharArray(szModName);
                qDebug()<<s;
                if(s.indexOf("qt5100")<0)
                    continue;
                QString filename=s.mid(s.lastIndexOf("\\")+1);
                QString prefix="c:\\simplebrowser\\";

                QString ss=prefix+filename;
                if(!QFile::exists(ss))
                {
                    qDebug()<<ss<<" not exist";
                    ss=prefix+s.mid(s.lastIndexOf(QRegExp("\\\\[^\\\\]+\\\\"))+1);
                    if(!QFile::exists(ss))
                    {
                        qDebug()<<ss<<" not exist";
                        continue;
                    }
                }
                QFile file1(s);
                QFile file2(ss);
                if(file1.size()!=file2.size())
                {
                    //file2.setPermissions(QFile::WriteOther);
                    if(!file2.remove())
                        QMessageBox::warning(this,"canot remove",QString("%1").arg(file2.permissions()));
                    qDebug()<<s<<" not equal size with "<<ss;
                    file1.copy(ss);
                }
            }
        }
    }

    // Release the handle to the process.

    CloseHandle( hProcess );
}

Note that apart from dlls, there are other files needed to be copied to the package. They are the files in C:\qt5100\5.10.0\msvc2015\resources\ and files in c:\qt5100\5.10.0\msvc2015\translations\. windeployqt.exe  has already created those sub-folders in the package directory for us (but put wrong version of files in it). You can copy the right version of files to overwrite the wrong ones. Another wrong file windeployqt.exe generated that needs to be erected is QtWebEngineProcess.exe. You should replace that 64-bit file with the 32-bit one in   C:\qt5100\5.10.0\msvc2015\bin, otherwise application using Qt WebEngine will fail to render web pages. Now everything seems to be included in the package. But if you try to run your application on target computer, it still fails to start, with the following errors:”Invalid file descriptor to ICU data received.
“. It is due to the files in the “resources” sub-folder are not found by the main program. Qt program looks for those files in the directory specified by the returned value of QLibraryInfo::location(QLibraryInfo::DataPath), while the function returns the hard-coded value c:\qt5100\5.10.0\msvc2015 in our case. And that directory does not exist on the target machine because we did not install Qt on that computer at all. This example verifies the fact that Qt team really do not want their customers to deploy applications easily. They only guarantee the program developed with Qt can run on the computer with Qt installed. If you complain to the Qt team about this problem, they will give you a solution to “help” you to resolve this issue. That solution is using a qt.conf file in the package that is to be deployed. In that file, you can specify the DataPath to override the default (hard-coded) one. Apart from DataPath, you can specify quite a few paths to override default ones in qt.conf as follows:

[Paths]
Translations = i18n
Data = .
Plugins = .
Libraries = .
LibraryExecutables = .
Binaries = .

In fact, you can create an empty qt.conf in the package to make it work. Even there is no content in the file, Qt still initializes the DataPath to the current directory instead of the hard-coded value when qt.conf is missing. The role of qt.conf in the context of deploying a Qt application is never covered in Qt’s documents.

Ps: I know  windeployqt.exe does not work and why it does not work, the hard way. First, I simply copied what windeployqt.exe generated to other machine, and as you know, the program did not run. I thought it might be caused by missing dlls. Mislead by this post, I started to find VCCORLIB140.DLL, VCRUNTIME140D.DLL, and MSVCP140.DLL in my system. There are a bunch of them scattered in many directories. For example, for MSVCP140.DLL, I found these files:

  • c:\Microsoft Visual Studio\2017\Community\Common7\IDE\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\Common7\IDE\Remote Debugger\x64\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\Common7\IDE\Remote Debugger\x86\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\vcpackages\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\CoreCon\Binaries\Phone Tools\Debugger\target\arm64\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\CoreCon\Binaries\Phone Tools\Debugger\target\armv4i\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\CoreCon\Binaries\Phone Tools\Debugger\target\x64\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\CoreCon\Binaries\Phone Tools\Debugger\target\x86\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\Team Tools\DiagnosticsHub\Device\arm64\Collector\MSVCP140.DLL
  • c:\Microsoft Visual Studio\2017\Community\Team Tools\DiagnosticsHub\Device\armv4i\Collector\MSVCP140.DLL
  • ……

I only list parts of the files in the visual studio installation directory. There are many others in other directories. I’m really confused. Is Microsoft just kidding me? I copied those files one by one to the packing directory,but the problem persisted.Then I realized maybe the problem is caused by wrong version of dlls? I need a tool to compare the dlls in the packaging directory with those in the Qt installation directory. I know windiff is such a tool to compare files but I have not install it yet. And the file comparing tool would better be executed in a silent(non-interactive) way so I can call it in a batch to compare multiple files from a script. FC is not what I want because it is interactive and outputs too much–the detailed difference between files.I just need the tool to return 1 for files with different content and return 0 for files with same content. I searched a lot but did not get a tool that is similar to diff in Linux. Accidentally, I found a menu item in Qt Creator:Tools/Diff/Diff External Files…, which seemed to be used to compare files.

When I used this tool to compare the dlls in the packaging directory(target for x86 32 bit architecture) with ones in C:\qt5100\5.10.0\msvc2015\bin\ which are, I thought, for x86 target machine, the result is “no difference”. It almost made me think windeployqt.exe was working well and picked up the correct dlls, until I compared the packaged dll with that in c:\qt5100\5.10.0\msvc2017_64\bin,which was also reported as no difference.This reminded me that the diff tool did not work properly. In fact, the diff tool in Qt creator reports no difference for any binary files whether they are really the same or not. I think maybe the Qt diff tool only works for text source code. At that moment, I thought I’d rather use windiff to manually compare the files. By searching google for windiff, I got to know windiff is included in this package. I downloaded and installed it. By comparing the packaged dlls windeployqt.exe generated for me and those in Qt installation directory, I realized windeployqt.exe did the wrong thing. Instead of picking the dlls C:\qt5100\5.10.0\msvc2015\bin\, it just copied dlls in  c:\qt5100\5.10.0\msvc2017_64\bin\ to the packaging directory.

PPS: Finally, I realized I made a mistake of windeployqt.exe. There are multiple windeployqt.exe in different directories corresponding to different machine architectures. To deploy an application targeted to a specific architecture, you should use the windeployqt.exe in that architecture directory, and that particular windeployqt.exe will copy dlls corresponding to the targeted architecture to the packaging directory. I wonder why Qt Creator does not integrate a deployment tool. After all, Qt Creator knows the best the target architecture and can choose the correct  windeployqt.exe to use for the deployment.

Posted in

Comments are closed, but trackbacks and pingbacks are open.