Category Archives: WinAPI

Postings about WinAPI

Available hotfixes for VC2005-SP1 / VC2008 RTM / VC2008-SP1

There are already several hotfixes available for VC2005 SP1 and VC2008 RTM. Here is a short overview:

Updated: 2008-11-06
Updated: 2009-03-09
Updated: 2009-03-13
Updated: 2009-03-18
Updated: 2009-05-08
Updated: 2009-05-13
Updated: 2009-05-23
Updated: 2009-10-06

MFCNext / VS2008 VC++ Feature Pack is RTM

The feature Pack for VC++ 2008 is now available for download.

It is some kind of confusing:
You can download the feature Pack for the englisch version only (ENU)!

If you have installed a localized version of VS (e.g. german), you must wait until the release of SP1. Hopefully the ENU-SP1 will also include the Feature Pack…

If you have a non-english OS the installation will fail! You need to change the system-language and formats to “en-us” and re-run the installation… this is a typically US bug…

Also one note: If you want to use the new Ribbon UI (Office 2007 UI, also known as Fluent UI), you must agree to the Office 2007 license!

Here are the links:
Download-Site
Direct download link
Redistributable package (x86)
Docu: MFC Feature Pack
Docu: TR1 Extensions

Just upgrade to VS2008! Even without an existing license!

According to the Microsoft Website Visual Studio 2008 Pricing there is no need to buy the full product. You can always buy the upgrade for VS2008 Standard or Professional if you have:

  • An earlier version of Microsoft Visual Studio
  • Any other developer tool (including free developer tools, such as Visual Studio Express Editions or Eclipse)

That looks interesting! So, download the express edition, then you can buy the upgrade version of VS2008!

I just made a copy of the page to be sure, that if MS removes this sentense in the future, I have a copy to witness:
VisualStudio2008Pricing_2008-04-04.xps

Unhandled exceptions in VC8 and above… for x86 and x64

Starting with VC8 (VS2005) it is not possible to catch all unhandled exceptions with your installed exception-filter (via SetUnhandledExceptionFilter). In some situations, the CRT forces the call of WER (Windows Error Reporting). One situation (/GS buffer overruns) can never be catched in your filter. If you want to catch all exception inside your filter, you need to intercept calls to SetUnhandledExceptionFilter to prevent the removing of all previous installed filters.

See also my previous post about this problem:
“SetUnhandledExceptionFilter” and VC8″

Here is the code to “hook” the SetUnhandledExceptionFilter which works now for x86 and x64:

#include 
#include 
#include 

#if defined _M_X64 || defined _M_IX86
LPTOP_LEVEL_EXCEPTION_FILTER WINAPI 
  MyDummySetUnhandledExceptionFilter(
  LPTOP_LEVEL_EXCEPTION_FILTER lpTopLevelExceptionFilter)
{
  return NULL;
}
#else
#error "This code works only for x86 and x64!"
#endif

BOOL PreventSetUnhandledExceptionFilter()
{
  HMODULE hKernel32 = LoadLibrary(_T("kernel32.dll"));
  if (hKernel32 == NULL) return FALSE;
  void *pOrgEntry = GetProcAddress(hKernel32, 
    "SetUnhandledExceptionFilter");
  if(pOrgEntry == NULL) return FALSE;

  DWORD dwOldProtect = 0;
  SIZE_T jmpSize = 5;
#ifdef _M_X64
  jmpSize = 13;
#endif
  BOOL bProt = VirtualProtect(pOrgEntry, jmpSize, 
    PAGE_EXECUTE_READWRITE, &dwOldProtect);
  BYTE newJump[20];
  void *pNewFunc = &MyDummySetUnhandledExceptionFilter;
#ifdef _M_IX86
  DWORD dwOrgEntryAddr = (DWORD) pOrgEntry;
  dwOrgEntryAddr += jmpSize; // add 5 for 5 op-codes for jmp rel32
  DWORD dwNewEntryAddr = (DWORD) pNewFunc;
  DWORD dwRelativeAddr = dwNewEntryAddr - dwOrgEntryAddr;
  // JMP rel32: Jump near, relative, displacement relative to next instruction.
  newJump[0] = 0xE9;  // JMP rel32
  memcpy(&newJump[1], &dwRelativeAddr, sizeof(pNewFunc));
#elif _M_X64
  // We must use R10 or R11, because these are "scratch" registers 
  // which need not to be preserved accross function calls
  // For more info see: Register Usage for x64 64-Bit
  // http://msdn.microsoft.com/en-us/library/ms794547.aspx
  // Thanks to Matthew Smith!!!
  newJump[0] = 0x49;  // MOV R11, ...
  newJump[1] = 0xBB;  // ...
  memcpy(&newJump[2], &pNewFunc, sizeof (pNewFunc));
  //pCur += sizeof (ULONG_PTR);
  newJump[10] = 0x41;  // JMP R11, ...
  newJump[11] = 0xFF;  // ...
  newJump[12] = 0xE3;  // ...
#endif
  SIZE_T bytesWritten;
  BOOL bRet = WriteProcessMemory(GetCurrentProcess(),
    pOrgEntry, newJump, jmpSize, &bytesWritten);

  if (bProt != FALSE)
  {
    DWORD dwBuf;
    VirtualProtect(pOrgEntry, jmpSize, dwOldProtect, &dwBuf);
  }
  return bRet;
}

LONG WINAPI MyUnhandledExceptionFilter(
struct _EXCEPTION_POINTERS *lpTopLevelExceptionFilter)
{
  // TODO: MiniDumpWriteDump
  FatalAppExit(0, _T("Unhandled Exception occured!"));
  return EXCEPTION_CONTINUE_SEARCH;
}

int _tmain()
{
  SetUnhandledExceptionFilter(MyUnhandledExceptionFilter);
  BOOL bRet = PreventSetUnhandledExceptionFilter();
  _tprintf(_T("Prevented: %d"), bRet);

  abort();  // force Dr.Watson in release!
}

Screencast: API Logging with the Application Compatibility Layer

Hi all!

Today I did a screencast on how to use the application compatibility layer (and especially the APILogger shim) to do API logging.

This is a very interesting screencast if you want to log and display API calls of other programs in a very nice and simple way.

So enjoy (14:59 min):
http://www.kalmbach-software.de/screencasts/UsingAPILogger

Here are also the links which are shown in the presentation:

Screencast: Static link to the C-Runtime to prevent vcredist and overcome “Application configuration” problems

Here is a short screencast which shows how to adjust the project settings to link against the static C-runtime (CRT).

This prevents that you must ship the CRT-DLLs with your application. Your application just works after copying the exe to the target computer.

This is a “german” screencast:

http://www.kalmbach-software.de/screencasts/VC2008EE-StaticLinkCRT/

HowTo: Correctly read reparse data in Vista

In many newsgroups/forums you will see the following approach to read the reparse data (for example the original directory of a junction):

This works very well under XP (but only if you are an admin). It does not work under Vista if UAC is enabled (even if you are admin). It only works if the process was started evelated (as real admin)!

But if you go to the command prompt and type “dir c:\ /AL”, you will see that in the command prompt the display is always possible, even if you are not an admin:

C:\>dir /AL 
 Volume in drive C has no label. 
 Volume Serial Number is B424-9F82              

 Directory of C:\ 
11/02/2006  02:00 PM         Documents and Settings [C:\Users] 
               0 File(s)              0 bytes 
               1 Dir(s)  57,193,873,408 bytes free

So there must be a better way to read the reparse data.

First I tried to find an other function to get the data. But this had no success… then I remembered my research about “Shims” which are part of XP and later. These shims can be used to “fix” application compatibilities in news OSs. To activate a shim for an appliction you can use the “Application Compatibility Toolkit” from Microsoft. And also I remembered an Shim namaned “APILogger”. Ok, then I copied “cmd.exe” to a temp-dir and renamned it; created and installed the sdb-File with the APILogger for this new “testcmd.exe”; and executed the “dir c:\ /AL” command. Afterwards I could easily find the correct approach to get the reparse data (in a later blogpost I will explain how to use the APILogger-shim).

The key point is: You must open the directory only with “FILE_READ_EA” access!

Here is now the full working source-code for reading reparse-data:

#include "stdafx.h"
#include 

typedef struct _REPARSE_DATA_BUFFER {
  ULONG  ReparseTag;
  USHORT  ReparseDataLength;
  USHORT  Reserved;
  union {
    struct {
      USHORT  SubstituteNameOffset;
      USHORT  SubstituteNameLength;
      USHORT  PrintNameOffset;
      USHORT  PrintNameLength;
      ULONG   Flags; // it seems that the docu is missing this entry (at least 2008-03-07)
      WCHAR  PathBuffer[1];
      } SymbolicLinkReparseBuffer;
    struct {
      USHORT  SubstituteNameOffset;
      USHORT  SubstituteNameLength;
      USHORT  PrintNameOffset;
      USHORT  PrintNameLength;
      WCHAR  PathBuffer[1];
      } MountPointReparseBuffer;
    struct {
      UCHAR  DataBuffer[1];
    } GenericReparseBuffer;
  };
} REPARSE_DATA_BUFFER, *PREPARSE_DATA_BUFFER;

#define REPARSE_DATA_BUFFER_HEADER_SIZE  FIELD_OFFSET(REPARSE_DATA_BUFFER, GenericReparseBuffer)

#define MAXIMUM_REPARSE_DATA_BUFFER_SIZE  ( 16 * 1024 )

int _tmain(int argc, _TCHAR* argv[])
{
  HANDLE hFile;
  LPCTSTR szMyFile = _T("C:\\Documents and Settings");  // Mount-Point (JUNCTION)
  //LPCTSTR szMyFile = _T("C:\\Users\\All Users");  // Symbolic-Link (SYMLINKD)

  hFile = CreateFile(szMyFile, FILE_READ_EA, FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT, NULL);
  if (hFile == INVALID_HANDLE_VALUE)
  {
    _tprintf(_T("Could not open dir '%s'; error: %d\n"), szMyFile, GetLastError());
    return 1;
  }

  // Allocate the reparse data structure
  DWORD dwBufSize = MAXIMUM_REPARSE_DATA_BUFFER_SIZE;
  REPARSE_DATA_BUFFER* rdata;
  rdata = (REPARSE_DATA_BUFFER*) malloc(dwBufSize);

  // Query the reparse data
  DWORD dwRetLen;
  BOOL bRet = DeviceIoControl(hFile, FSCTL_GET_REPARSE_POINT, NULL, 0, rdata, dwBufSize, &dwRetLen, NULL);
  if (bRet == FALSE)
  {
    _tprintf(_T("DeviceIoControl failed with error: %d\n"), GetLastError());
    CloseHandle(hFile);
    return 1;
  }
  CloseHandle(hFile);

  if (IsReparseTagMicrosoft(rdata->ReparseTag))
  {
    if (rdata->ReparseTag == IO_REPARSE_TAG_SYMLINK)
    {
      printf("Symbolic-Link\n");
      size_t slen = rdata->SymbolicLinkReparseBuffer.SubstituteNameLength / sizeof(WCHAR);
      WCHAR *szSubName = new WCHAR[slen+1];
      wcsncpy_s(szSubName, slen+1, &rdata->SymbolicLinkReparseBuffer.PathBuffer[rdata->SymbolicLinkReparseBuffer.SubstituteNameOffset / sizeof(WCHAR)], slen);
      szSubName[slen] = 0;
      printf("SubstitutionName (len: %d): '%S'\n", rdata->SymbolicLinkReparseBuffer.SubstituteNameLength, szSubName);
      delete [] szSubName;

      size_t plen = rdata->SymbolicLinkReparseBuffer.PrintNameLength / sizeof(WCHAR);
      WCHAR *szPrintName = new WCHAR[plen+1];
      wcsncpy_s(szPrintName, plen+1, &rdata->SymbolicLinkReparseBuffer.PathBuffer[rdata->SymbolicLinkReparseBuffer.PrintNameOffset / sizeof(WCHAR)], plen);
      szPrintName[plen] = 0;
      printf("PrintName (len: %d): '%S'\n", rdata->SymbolicLinkReparseBuffer.PrintNameLength, szPrintName);
      delete [] szPrintName;
    }
    else if (rdata->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT)
    {
      printf("Mount-Point\n");
      size_t slen = rdata->MountPointReparseBuffer.SubstituteNameLength / sizeof(WCHAR);
      WCHAR *szSubName = new WCHAR[slen+1];
      wcsncpy_s(szSubName, slen+1, &rdata->MountPointReparseBuffer.PathBuffer[rdata->MountPointReparseBuffer.SubstituteNameOffset / sizeof(WCHAR)], slen);
      szSubName[slen] = 0;
      printf("SubstitutionName (len: %d): '%S'\n", rdata->MountPointReparseBuffer.SubstituteNameLength, szSubName);
      delete [] szSubName;

      size_t plen = rdata->MountPointReparseBuffer.PrintNameLength / sizeof(WCHAR);
      WCHAR *szPrintName = new WCHAR[plen+1];
      wcsncpy_s(szPrintName, plen+1, &rdata->MountPointReparseBuffer.PathBuffer[rdata->MountPointReparseBuffer.PrintNameOffset / sizeof(WCHAR)], plen);
      szPrintName[plen] = 0;
      printf("PrintName (len: %d): '%S'\n", rdata->MountPointReparseBuffer.PrintNameLength, szPrintName);
      delete [] szPrintName;
    }
    else
    {
      printf("No Mount-Point or Symblic-Link...\n");
    }
  }
  else
  {
    _tprintf(_T("Not a Microsoft-reparse point - could not query data!\n"));
  }
  free(rdata);
  return 0;
}

ADD (2008-03-07): Thanks to the comment of Sergey, I now changed the sample code to correctly display JUNCTIONs (mount points) and SYMLINKD (symbolic-links). It seems that the documentation of REPARSE_DATA_BUFFER is incorrect, because the missed a field named “Flags”. I changed the structure to the correct definition which can be found in the DDK header file “ntifs.h”.

marshal_as library in VC2008

The VC++ team added a (simple) marshal_as library in VC2008. But this library only supports simple datatypes like

  • String^ to char* / wchar_t* / BSTR / bstr_t / CComBSTR / std::string / std::wstring / CString<char> / CString<wchar_t>  and vice versa
  • IntPtr to HANDLE  and vice/versa

So, the support only contains “Strings” and “Handle”. But this are the most commonly needed scenarios for marshaling.
The usage of some of the string marshaling is shown here:
http://www.c-plusplus.de/forum/viewtopic-var-p-is-1455833.html#1455833

An overview of the build-in marshaling can be found here: Overview of Marshaling in C++

There is a website (http://www.marshal-as.net/), in which you can look-up additional marshaling implementations like:

Hopefully many, more will follow…
If you have an implementation of some kind of “marshal_as”, just put a comment to Kate Gregory

Local user-mode dumps on Vista SP1 and W2k8

Starting with Vista, MS introduced the new Windows Error Reporting API (WER) and removed Dr. Watson from the system. The WER-System now directly sends by default the data to WinQual. They also changed the behaviour, that a minidump is only generated if the WinQual-Server requests this. So by default there is no way to get to a local minidump if you are not registered at WinQual (which costs 400$/year).

Several ISVs complained about this issue. And now MS added a new feature in VIsta SP1 and W2k8 to allow the creationg of local minidump regardless of the WER setting. FOr more info see:

Collecting User-Mode Dumps

Smallest application size for win32 console application

Several times someone asked the question: “Why is a simple console application 52 KB (CRT statically linked) or 7 KB (CRT as DLL) in size?”

The answer is: Because it uses the CRT 😉

Just to show you, that you can have smaller applications, here is (from my point of view) the smallest “usefull” application (Hello world):

// Filename: smallest.cpp 
#include 
#pragma comment(linker, "/entry:entry") 

void entry() 
{ 
  TCHAR szText[] = TEXT("Hello world\n"); 
  DWORD dwWritten; 
  WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), szText, lstrlen(szText), &dwWritten, NULL); 
}

You must compile and link this file with:
cl /c /O1 /GS- smallest.cpp
link /subsystem:console smallest.obj kernel32.lib

Then you get an EXE file with 2048 Bytes.

You can even reduce it more if you merge the sections together and specify an alignment for the linker (but then program will then not run on Win98;) ):
cl /c /O1 /GS- smallest.cpp
link /subsystem:console /MERGE:.rdata=.text /ALIGN:16 smallest.obj kernel32.lib

Then you get an EXE file with 688 Bytes (VS2008) / 768 Bytes (VS2010).

So the question is: why does the CRT need so much space?

Most of the space is needed for “security reasons”. That is also the reason why I needed the specify the “/GS-” compiler switch to reduce the exe size. If you enable the buffer security checks, it triggers the compiler to use several functions and structures, which must be provided by the CRT.
And if you use the CRT, it also provides the default entry point and initializes all kind of stuff like calling constructors of static/global classes and others.
These are the main reasons why the CRT uses so much memory.

Another think is: if you want to use floating-point values, you also need the CRT. The reason is not very obvious for me… but the compiler is triggering the symbol “_fltused” to be linked with the image. If you provide your own “implementation” of this variable, it seems to work; but I don’t understand why the compiler is needing this variable. This variable must be set to a “magic” value (0x9875); I don’t know where this value comse from (expect it is defined in the CRT sources).

If you want to compile these samples from within the VS (2008) IDE, then you need to change the following setings from the default “Win32 Console Application”:

  • Switch to “Release” configuration
  • C/C++|Code Generation|Buffer Security Check: No (/GS-)
  • Linker|Debugging|Generate Debug Info: No 
  • Linker|Advanced|Entry Point: entry
  • Linker|Advanced|Randomize Base Address: Disable Image Randomization (/DYNAMICBASE:NO)
  • Linker|Advanced|Fixed Base Address: Image must be loaded at a fixed address (/FIXED)
  • Manifest Tool|Input and Output|Embed Manifest: No

And then you have your 2KB application from within the IDE 😉

A new team is now improving MSDN Help!

Since VS2002 the new help system was really bad. And it got even worser in every release of visual studio. The slowest version was starting with VS2005 and VS2008. Now MS has heard us!

April announced on her weblog, that MS has now set up a new team to “redesign” the MSDN Help system:

Help is Getting Help – The Long Overdue Makeover for Visual Studio and MSDN Help

Really great news! Hopefully this project will be successful!

MFC Update: MFC feature pack for VS2008 now available als Beta!

Yesterday MS announced the public availability of the MFCNext feature pack for Visual Studio 2008. This feature pack will contain all the “good stuff” which was already shown at TechEd-Developer in Barcelona.

You can download and get more infos about this feature pack from the following sites:

VCBlog post:
http://blogs.msdn.com/vcblog/archive/2008/01/07/mfc-beta-now-available.aspx

Download link:
http://www.microsoft.com/downloads/details.aspx?FamilyId=D466226B-8DAB-445F-A7B4-448B326C48E7&displaylang=en

Documentation:
http://www.microsoft.com/downloads/details.aspx?FamilyId=0D805D4E-2DC2-47C7-8818-A9F59DE4CD9B&displaylang=en

Breaking changes in the VC2008 CRT

Starting with VC9 (VS2008; compiler version 15) Microsoft removed some global variables (_osplatform, _osver, _winmajor, _winminor, _winver) and the corrsponding functions (_get_xxx). This will affect several applications.

Also the VC9 CRT DLLs are no longer dependend on the system msvcrt.dll (this is really an improvement).

Here is a list of Breaking Changes (CRT)

MFCUpdate: MFC now has a “Visual Manager” and will add “Office 2007 Ribbon” controls! And many more features!!!

Really great news for MFC developers!

Microsoft announced a hugh investment in MFC which is currently being finished and available in March 2008. It will be an add-on for VS2008. Next month there will be an Release Candidate (RC) available.

Here is just a small overview:

If you just recompile with VS2008, then you have the vista-common-controls (for example the new OpenFileDIalog; of course, you can also use the old-style dialog, if you want!).

If you change some small parts of your code, you can use the new visual manager. Several styles are currently available and you can also write your own visual manager to implement your own style! And the best: You can change the style during runtime!
Also the menu icons will be automatically displayed by looking at the toolbar icons!

Also by adding some code, you can use the Office 2007 Ribbon Controls! Customizing the Toolbar by the user! (of course, everything is highly customizable by the programmer, you always can disable these features). Commands-Routing and all other stuff are not changed! Just add some code and then you has then new styles.

Also Docking is now for free aviable in MFC (DockingPane)!

MS also made a strong commitment to native code. If Office will have a new UI, MFC will also be implementing this style in the next version.

And the best: All the UI controls are supported starting with Windows 2000 and just uses native code (so no need to have Office2007 or Vista to enable these features)!!!

If you changing the Theme in the UI-Settings of the OS, it depends on what Visual Manager have used. The “Windows-Visual-Manager” will follow the OS settings of the OS.

I have made some pictures from the session, you can download it here: http://blog.kalmbachnet.de/files/TechEd2007_MFCNext.zip (39 MB). Enjoy!

Just some small (funny) notes at the end:

  • Expression Blend is written in MFC! The main frame is MFC and only the content is WPF 😉
  • Most controls are “ported” from BGCSoft into MFC!
  • For Ribbon, there is currently no designer available, it must be done in code.
  • The VC++ team will put a video on Channel9 in the next hours/days, so you can also see this imporvements:
    http://channel9.msdn.com/Showforum.aspx?forumid=14&tagid=17

TechEd-Developer Session: VC++ and Windows Vista: A Natural Fit

Kate Gregory showed that C++/CLI is the best and sometimes the only (allowed) language to access all the Vista features.
In short:

  • Only in VC++ you can access all Vista features
  • Some Vista features are easier to access from VC++
  • Some Vista features are only accessible from VC++ 😉
  • Some Vista features must be written in native code (like PropertyHandler, ThumbnailHandler)
  • C++/CLI is a “natural” and “must have” tool to fully develop for Windows Vista
  • C++/CLI is the best language for InterOp

The first impressions about TechEd Developer / Great news about C++

Maybe my exüectatiosn were too high… in general I would say that most of the sessions do not go deep. Mostly they only touch the surface…

Of course, there are also some exceptions: All C++ sessions are realy great!

The first session was “What’s new in Visual C++ 2008”. Just a short overview:

  • Vista only: In the Project Properties under “Manifest” you now can explicite set the UAC-Level (asInvoker, highestAvailability, requireAdministrator); so no need to specify this “by hand”; if you upgrade your project it will by default generate an manifest with UAC set to “asInvoker”, so you do not face virtualization problems under Vista
  • Class designer for C++; but only for viewing; there is currently now two-way direction for editing in the class designer and reflection this changes in he code; but better than nothing…
  • Vista-only: Randomizing the base address; this allows vista to use a random base address at startup; this helps to prevent code-injection into the process; this is also on by default for all MS DLLs. Also you can mark your program that it can deal with “data execution prevention /NXCOMPAT)
  • Multiprocessor build for cl.exe! This is also great news! It will allow to speed up the build process even if you just have one project (in VS2005 multi processor support was only possible if you had many solutioons which did not depend on each other)
  • Incremental build for managed code
  • Many areas with performance improvements in VS2005, like: Load solution performance, editor/IntelliSense responsiveness

The second session was “Best practices for Native-Managed InterOp in VC++ 2008 ; also a short summary:

  • Introduction to STL.NET (STL which can be used with managed types); these system is faster than the .NET collections 😉
  • Marshaling Library (marshal_as<to> (from) ); this is a very exciting feature and can be extended with only implementations

So today, we can be look forward to see the two sessions about C++/CLI with Vista and MFC-Updates!

Stay tuned…