PE file format

Unlike science, technique is easy to be followed, cracked, hacked, copied, plagiarized, stolen, even surpassed by others so it need some barriers. Why are there so many evil words associated with technique, not science? If you have some psychological knowledge, you may hear the word reflection.  Yes, it is the reflection of those who are afraid of being plagiarized, stolen, or surpassed.

You may find it is difficult to read/memorize/understand the PE file format(you may argue that you don’t need to memorize the format, you can look up in the specification on demand, but trust me, if you want to be proficient in system coding, you need to memorize a lot of details of the specification, just as you have to memorize the pronunciation of many words if you want to communicate fluently with others and you don’t bother to look up in a dictionary on demand). That is because it’s one of the elaborately designed technique barriers like the Apartment concept in COM.

Setting up  a technique barrier is not an easy thing, sometimes it’s harder than the technique itself. The first criterion for a good technique barriers is DO NOT set up too many barriers; otherwise, when you look around, you’ll find nobody wants to play with you. Second, the barrier must be transparent to those who are far way from it because on the bottom line, technique barrier is neither ethical nor political correctness. It makes people disgusting when seeing it. However, it must be strong enough to protect you if somebody wants to pass it to beat you. PE file format is such a technique barrier. Few people except system coders realize its existence. Even fewer people understand deeply about it so only some genius coders can touch the core of Windows. In contrast, Qt’s technique barrier strategy does not satisfy the two criteria.  Qt wants its qmake build system( for the .pro file) to be a tech barrier. But because all Qt programmers need to play with the .pro file,  the tech barrier either makes them disgusting or lures them to overcome the barrier. In the end, Qt gives up the qmake system and switches to the CMakefile. Qt also wants to make the openssl component as a barrier (if you ever make a serious software using Qt, you must know what I’m saying) to use their software but this barrier is not strong enough.

Specifically, how to set up a technique barrier? This mainly attributes to the document guys. I once thought the main task of the technical writers in a company is to deliver accurate, understandable and easy to follow documents for the company’s products, but it turns out I was too naive. Why do you feel it difficult to read this PE file format document?

  • Create many similar terminologies so that it is hard for you to pick up a particular one from your memory and hard to memorize their correspondence to the entities they refer to. In that article, there are many terminologies  tagged with xxx header: File Headers,Section Headers,PE header,Microsoft COFF Header,COFF File Header,Optional Header,PE file header,COFF object file header,Archive member header,Import header. You need a few more heads to handle with these headers.
  • Use different terminologies to refer to the same entity such as

    The import info/export info/base relocations/resource info are actually import section/export section/reloc section/resource section.

  • Use a new terminology that is never explained such as the “Image Pages” in the above example.
  • Use the same terminology to refer to different entities
  • Use similar terminologies to refer to similar but different entities and do not explicitly disclose their difference such as import table/import address table/import directory table/import name table/.idata section/Import Lookup Table/Hint Name Table.
  • Use a terminology that refers to nothing, e.g, Import Table. Although it gives a link after the term, click the link you’ll find nothing about “Import table”, only Directory Table/Import Lookup Table/Hint-Name Table there.
  • Give incomplete information such as the layout of the .idata section:

    ,in which the IAT is missing.

  • Provide wrong information. e.g., “An ordinal number is used as an index into the export address table.”. In fact, it should be “The value of an ordinal number minus the OrdinalBase is used as an index into the export address table.”

But I’d say the space left for the doc guys to play tricks is becoming smaller. After all, a big company needs to face the complains of a large group of customers. For those who need not face end customers directly, they are high in playing the word game. If you ever read RFCs/international standards, you must know what I mean. A common phenomenon is every word in the document is correct in spell, every sentence in the document is correct in grammar, but you just can not catch what the document is saying.
Okay, let’s return to today’s topic: PE file format. As said, as a technology, it’s doomed to be learned, sooner or later.

From a top view, a PE file consists of a header, a section table and some sections. (The term section table they invent to confuse you does not mean a collection of sections but a collection of section headers each of which describes a section after the section table.) However, the methods to locate the components within the header/section table, and the individual sections are different. The locations of components in the header/section table are specified in the PE specification, i.e., what components follow what other components. But the PE specification does not specify the locations of the sections. A section does not necessarily  follow the section table immediately(which means there may be holes between the last byte of the section table and the first byte of the first section). A section does not necessarily  follow the previous section immediately(which means  there may be holes between the last byte of previous section and the first byte of current section). This means after your parser parses the header/section table/a section, it should not take it for granted that the next byte is the beginning of next section. Your parse needs to look for some fields in the section table to locate a section in the PE file and get its size.

The .edata section stores the export information and typically has the following layout:

Export Directory Table

Export Address Table

If the .dll supports exporting by name, the .edata has more data in it:

Export Directory Table

Export Address Table

Export Name Pointer Table

Export Ordinal Table

Export Name Table

Note that since the components in the .edata section, like the content on other sections, are located by the pointers in other structures(such as the location of the Export Directory Table is specified in the first entry of the Data Directories of the Optional Header which actually describes the address of the first byte of the all the export tables such as the export directory table, export address table, export name pointer table, etc. and the sum-up size of these tables), there may be holes between the components, and they are not necessarily  arranged in the above order. Outside all these components, there may be other content in the .edata section.

The Export Directory Table has pointers to the Export Address Table, Export Name Pointer Table, and Export Ordinal Table, but no pointer to the Export Name Table. In fact, there is no pointer elsewhere to the Export Name Table. The Export Name Table is just a combination of null-terminated strings. The Export Address Table stores the pointers to every exported functions, but it is possible that not all  Export Address Table  entries points to some function, some entries may have the NULL value, some entries may point to string like “Otherdll.afunction” elsewhere in the .edata section but not in any of the mentioned structures. The Export Name Pointer Table stores the pointers to the strings (exported function names) in the Export Name Table. Although the strings in the Export Name Table may be unordered, the pointers to them are sorted in the Export Name Pointer Table according to the lexical order of pointed strings, not the values of the pointers themselves. The Export Ordinal Table contains the indexes(not ordinals) of exported by name functions in the Export Address Table.

For every exported function, there is a slot in the Export Address Table recording its RVA(address). The index of the slot plus the value of the OrdinalBase field in the  Export Directory Table is called the ordinal of the function.

The following is the export table for a dll named mydll.dll which has one export function _Z2Sumii. The table is 0x45 bytes(the blue area).

the following is its interpretation:

 

We can see the dll name is not in any part of the above mentioned structure.

The .idata section stores the import information which typically has the following layout:

  • Directory Table
    Null Directory Entry
  • DLL1 Import Lookup Table
    Null
  • DLL2 Import Lookup Table
    Null
  • DLL3 Import Lookup Table
    Null
  • Hint-Name Table
  • IAT for dll1
    Null
  • IAT for Dll2
    Null
  • IAT for Dlls
    Null

The first byte of the Directory Table(Import Directory Table) is not necessarily the first byte of the .idata section because the structure is not located by the specification but the second entry of the Data Directories of the Optional Header. Unlike Export Directory Table which has only one entry, the Import Directory Table is an array of multiple entries. Each entry describes an imported dll including a pointer to the dll name(residing in the .idata but outside any structure mentioned above), a pointer to the ILT(Import Lookup Table), a pointer to the IAT(Import Address Table), etc. Hint-Name Table is just a collection of all the imported function names(null-terminated) for all the imported dlls, so despite there may be multiple imported dlls, there is only one Hint-Name Table.

If the current image imports all functions by their names, the ILT tables only contains pointers to function names in the Hint-Name Table. Each ILT table contains the pointers to the imported function names for one dll. On disk, the IAT has the same content as the ILT. When loading into memory by the image loader, the IAT will be populated with the actual addresses of the imported functions. The image loader gets a dll name from the Import Directory Table, gets imported function names from the IAT for the dll and Hint-Name Table, loads the dll into memory, search the function names in the  Export Name Pointer Table/Export Name Table, gets the index of the function in the Export Name Pointer Table, get the value of the entry at the same index of the Export Ordinal Table, uses that value as the index to get the corresponding entry of the  Export Address Table, and the value of that entry is the actual address of the imported function so fill it into the IAT entry for the function. In the whole process, we do not use the ordinal of function or the OrdinalBase field in the imported dll.

If current image imports some function by ordinal, the IAT entry for the function would be its ordinal(on disk) and the image loader subtracts the OrdinalBase got from the imported dll from the ordinal to get an index, and uses the index to get the actual function  address from the   Export Address Table of the imported dll ,then fill the function address into the entry of the IAT for the imported function.

Because the .idata section is usually a read-only section, the image loader must temporarily modify the section attribute to read/write then completes the population of the IAT, and then modify the section attribute back to read-only.

This is how the addresses in the IAT are used. When building a dll(e.g. myprogrammingnotes.dll), an import lib for the dll(e.g. myprogrammingnotes.lib) is generated. For an exported function myfun, two symbols(myfun, __imp_myfun) and a piece of code(jmp [__imp_myfun]) are produced in  myprogrammingnotes.lib.  The myfun is the address of the code “jmp [__imp_myfun]” while the __imp_myfun is the address of the entry of the IAT for the exported function. In your app that links to myprogrammingnotes.lib, if you write the code as:

void myfun();

....

myfun();

Then “myfun();” is compiled to “call myfun”. The symbol “myfun” matches with the “myfun” symbol in  myprogrammingnotes.lib so the executed machine codes are:

call myfun
myfun:
   jmp [__imp_myfun]

If you write the code as:

 

__declspec(dllimport) void myfun();

....

myfun();

Then “myfun();” is compiled to “call [__imp_myfun]”. So the executed machine code is:

 

call [__imp_myfun]

 

There are many offsets in the PE file, most of which  are RVAs, which means the linker producing the PE file mainly concerns about the memory address instead of the position in the file. But there are still some data that are the offset (to the beginning of the file) in the file.

At file offset 0x3c, there is an offset (to the beginning of the PE file) which is the offset of the PE signature(“PE\0\0”) . This signature separates the MS-DOS stub and the real PE header which is shared with .obj file as the coff file header(20 bytes). Which follows the coff header is the optional header which only exists in image not .obj. All kinds of images share some common fields in the optional header, but windows has its unique fields in the optional header. The AddressOfEntryPoint, BaseOfCode , BaseOfData   in the common option header are RVAs. The ImageBase in the Windows specific optional header is neither a RVA nor a file offset. It is a fixed number indicating a memory address. The VirtualAddress (va1) in the data directory is obvious RVA. The VirtualAddress (va2) in the section header is RVA too while the PointerToRawData in the sector header is the offset to the beginning of the file. va1 is not necessarily equal to va2. va2 is SectionAlignment-aligned while v1 is not necessarily SectionAlignment-aligned. The VirtualSize in section headers is not necessarily a multiple of SectionAlignment. The size in data directories is not necessarily a multiple of SectionAlignment.  The PointerToRawData in section headers must be FileAlignment-aligned so every section occupies a multiple of FileAlignment bytes on the disk even it does not have that many real data. This is also indicated by the SizeOfRawData in section headers, which must be a multiple of  FileAlignment. You must rely on the VirtualSize not the SizeOfRawData in the section header to decide the real number of data in the section. For a section that has little content, you’ll find the VirtualSize is less than SizeOfRawData and the bytes  after VirtualSize till the beginning of next section in the file are filled by zeros. Only VirtualSize bytes are read into memory. In the case where VirtualSize is bigger than SizeOfRawData, the last VirtualSize-SizeOfRawData bytes for the section are filled with 0s in memory. The pointers in export directory table,export address table, export name table are all RVAs. The pointers in import directory table, import lookup table, import address table are all RVAs.

 

 

Did you like this?
Tip admin with Cryptocurrency

Donate Bitcoin to admin

Scan to Donate Bitcoin to admin
Scan the QR code or copy the address below into your wallet to send some bitcoin:

Donate Bitcoin Cash to admin

Scan to Donate Bitcoin Cash to admin
Scan the QR code or copy the address below into your wallet to send bitcoin:

Donate Ethereum to admin

Scan to Donate Ethereum to admin
Scan the QR code or copy the address below into your wallet to send some Ether:

Donate Litecoin to admin

Scan to Donate Litecoin to admin
Scan the QR code or copy the address below into your wallet to send some Litecoin:

Donate Monero to admin

Scan to Donate Monero to admin
Scan the QR code or copy the address below into your wallet to send some Monero:

Donate ZCash to admin

Scan to Donate ZCash to admin
Scan the QR code or copy the address below into your wallet to send some ZCash:

Leave a Reply