Unicode on Downlevel Windows (IUC 18) Unicode Across Windows Michael S. Kaplan Trigeminal Software, Inc. Cathy A. Wissink Microsoft.

Unicode on Downlevel Windows (IUC 18)

UnicodeUnicodeAcrossAcross

WindowsWindows

Michael S. KaplanMichael S. KaplanTrigeminal Software, Trigeminal Software, Inc.Inc.

Cathy A. WissinkCathy A. WissinkMicrosoftMicrosoft


Why Unicode in your Why Unicode in your application?application? World wide EXEWorld wide EXE Decrease localization costsDecrease localization costs Superior multilingual supportSuperior multilingual support Better integration with the NT Better integration with the NT

platformplatform Support for new languages on Support for new languages on

Windows 2000, Windows XP, and Windows 2000, Windows XP, and beyondbeyond


Why not Unicode in your apps?Why not Unicode in your apps?(note that none of these are true any more; some were never true!)(note that none of these are true any more; some were never true!)

One code page at a time is One code page at a time is enough, right?enough, right?

Every major language has Every major language has at at leastleast one version of Windows one version of Windows which supports itwhich supports it

Visual Studio does not support Visual Studio does not support UnicodeUnicode

There is no support for Unicode There is no support for Unicode when you are on Win95/98/MEwhen you are on Win95/98/ME


The old workaroundsThe old workarounds Build two versions of the applicationBuild two versions of the application

Multiple code bases or build types to maintainMultiple code bases or build types to maintain Larger downloads/setupsLarger downloads/setups Harder to isolate problems in codeHarder to isolate problems in code

Always do ANSI applicationsAlways do ANSI applications No multilingual supportNo multilingual support Poorer performance on NT platformsPoorer performance on NT platforms No support for Unicode only languagesNo support for Unicode only languages

Create your own layer (based on the 4/99 MSJ Create your own layer (based on the 4/99 MSJ article from Microsoft)article from Microsoft)

http://www.microsoft.com/msj/0499/multilangUnicode/multilangunicode.htmhttp://www.microsoft.com/msj/0499/multilangUnicode/multilangunicode.htm A lot of work required!A lot of work required!


Bumps in the roadBumps in the road

Microsoft assumed that the Microsoft assumed that the migration to NT would happen migration to NT would happen more quickly and that ISVs would more quickly and that ISVs would make that happenmake that happen

Developers needed to support the Developers needed to support the consumer applicationsconsumer applications

Microsoft did not have a consistent Microsoft did not have a consistent Unicode story for all platforms (and Unicode story for all platforms (and needed one)needed one)

Something had to give....Something had to give....


The MicrosoftThe Microsoft®® Layer Layer for Unicode™ on for Unicode™ on

WindowsWindows®® 95/98/ME Systems95/98/ME Systems


What is MSLU?What is MSLU?

The The MMICROICROSSOFT OFT LLayer for ayer for UUnicode on nicode on Windows 95/98/ME SystemsWindows 95/98/ME Systems file name: UnicoWS.dllfile name: UnicoWS.dll

Allows you to write a single Unicode Allows you to write a single Unicode application for all platformsapplication for all platforms


Design goals for MSLUDesign goals for MSLU Cannot slow down Unicode applications Cannot slow down Unicode applications

on WinNT/Win2K/WinXPon WinNT/Win2K/WinXP When possible, provide a consistent, When possible, provide a consistent,

identical Unicode subset of the Win32 identical Unicode subset of the Win32 API across all platformsAPI across all platforms

Based on the Millennium API setBased on the Millennium API set Supports Win95/98 alsoSupports Win95/98 also

Allow developers to take advantage of Allow developers to take advantage of the new languages on Windows 2000 the new languages on Windows 2000 and Windows XP without abandoning and Windows XP without abandoning older platformsolder platforms


Other design goalsOther design goals As small as possible:As small as possible:

Dll is ~160kbDll is ~160kb Has over 440 API wrappersHas over 440 API wrappers Has more than 40 "stub" wrappersHas more than 40 "stub" wrappers

Frugal with resourcesFrugal with resources No registration requiredNo registration required No dependenciesNo dependencies No special file location requirementNo special file location requirement

No inherent "DLL hell" issuesNo inherent "DLL hell" issues Cannot be put in the system directory!Cannot be put in the system directory!


PerformancePerformance

No slowdown at all on WinNT/2K/XPNo slowdown at all on WinNT/2K/XP No extra DLL loadNo extra DLL load Native APIs called directlyNative APIs called directly

Minimal slowdown on Win95/98/MEMinimal slowdown on Win95/98/ME Need time to convert strings to ANSINeed time to convert strings to ANSI For functions with out params, need to For functions with out params, need to

convert strings to Unicodeconvert strings to Unicode


MSLU's custom loaderMSLU's custom loader

No Virginia, you cannot do a true static No Virginia, you cannot do a true static linklink Your application statically links to the loaderYour application statically links to the loader

The loader works in a similar way to The loader works in a similar way to the /DELAYLOAD functionality in VC++the /DELAYLOAD functionality in VC++

Should work with other C++ compilers, Should work with other C++ compilers, tootoo

Supports the ability to override any API Supports the ability to override any API but get all the benefits of the loader!but get all the benefits of the loader!

No fear of failure:No fear of failure: DLL not presentDLL not present Low memory scenariosLow memory scenarios


AgendaAgenda

What is Unicode?What is Unicode? Why Unicode in Why Unicode in youryour application? application? Introducing the MS Layer for Introducing the MS Layer for

UnicodeUnicode Integration of the layerIntegration of the layer Covered APIs and functionalityCovered APIs and functionality What the layer is What the layer is notnot What about Visual Studio.Net?What about Visual Studio.Net? Where to get MSLUWhere to get MSLU


Integration of the layerIntegration of the layer

Just add UnicoWS.lib to the link list, and Just add UnicoWS.lib to the link list, and you are done!you are done! 1) Add it after static libraries like the CRT1) Add it after static libraries like the CRT 2) Add it before all of the libraries that it 2) Add it before all of the libraries that it

uses (such as kernel32.lib and shell32.lib)uses (such as kernel32.lib and shell32.lib)

Making sense of the rules:Making sense of the rules: The linker resolves references Left to RightThe linker resolves references Left to Right When it reaches the end, it wraps aroundWhen it reaches the end, it wraps around

uafxcw.lib libcmt.lib unicows.lib kernel32.lib gdi32.lib user32.lib oleaut32.lib oledlg.lib shell32.lib comdlg32.lib advapi32.lib

winspool.lib


How the DLL is loadedHow the DLL is loaded Setting an override:Setting an override:

Add a function that you use to load Add a function that you use to load the DLLthe DLL

"Set the hook" by adding this line:"Set the hook" by adding this line:

extern "C" HMODULE (__stdcall *_PfnLoadUnicows) (void) = &LoadUnicows;extern "C" HMODULE (__stdcall *_PfnLoadUnicows) (void) = &LoadUnicows;

Good place to handle failureGood place to handle failure

Default behavior: LoadLibrary!Default behavior: LoadLibrary!

Fallback to the shared location if Fallback to the shared location if you must (not preferred at all)you must (not preferred at all)

When all else fails, fail APIs When all else fails, fail APIs gracefully!gracefully!


Overriding individual APIsOverriding individual APIs

Add your function (using the same Add your function (using the same signature as the Win32 header files signature as the Win32 header files define)define)

"Set the hook" with this line of "Set the hook" with this line of code:code:

extern "C" FARPROC Unicows_<api>W = (FARPROC)&<your function>;extern "C" FARPROC Unicows_<api>W = (FARPROC)&<your function>;

Overriding "stubbed out" APIsOverriding "stubbed out" APIs The new PSDK will have notes for The new PSDK will have notes for

every API -- including any special every API -- including any special issues, when neededissues, when needed


Detecting when APIs are wrapped Detecting when APIs are wrapped properlyproperly

Current technologies do not Current technologies do not support MSLU:support MSLU: Dependency Walker (DEPENDS.EXE)Dependency Walker (DEPENDS.EXE) DUMPBIN.EXE /IMPORTSDUMPBIN.EXE /IMPORTS

Using these methods anyway to Using these methods anyway to determine what is linked and what determine what is linked and what is notis not


Loading your own Loading your own "unicows.dll""unicows.dll" How LoadLibrary works for loaded How LoadLibrary works for loaded

DLLsDLLs If DLL of the same name is already If DLL of the same name is already

loaded, LoadLibrary uses that DLLloaded, LoadLibrary uses that DLL

How to stay independent of othersHow to stay independent of others Rename unicows.dllRename unicows.dll Use the loader override to load the Use the loader override to load the

DLLDLL


Integration samplesIntegration samples

Taking a few samples from the PSDKTaking a few samples from the PSDK Showing off a customer integration or Showing off a customer integration or

twotwo


AgendaAgenda




Covered APIs and functionality Covered APIs and functionality - 1- 1 Wraps almost every "W" Win32 APIWraps almost every "W" Win32 API

The few that are not wrapped are ones that The few that are not wrapped are ones that already support Unicode, such as the IME already support Unicode, such as the IME APIsAPIs

Stubs you can override are present for APIs Stubs you can override are present for APIs not explicitly supportednot explicitly supported

Wraps several non "W" APIs when Wraps several non "W" APIs when needed for Unicode supportneeded for Unicode support Unicode support (e.g., IsWindowUnicode)Unicode support (e.g., IsWindowUnicode) Consistent user messaging (e.g., Consistent user messaging (e.g.,

CallWindowProcA)CallWindowProcA) Common, known issues (e.g., ExtTextOutW)Common, known issues (e.g., ExtTextOutW)


Covered APIs and functionality Covered APIs and functionality - 2- 2 User messaging supportUser messaging support

Similar to NT's user messaging Similar to NT's user messaging support for ANSI applicationssupport for ANSI applications

ThreadsafeThreadsafe Can handle multithreaded applications Can handle multithreaded applications

properlyproperly


Covered APIs and functionality Covered APIs and functionality - 3- 3(Playing well with the other children)(Playing well with the other children) Works properly with the Unicode Works properly with the Unicode

versions of MFC, ATL, and the CRT versions of MFC, ATL, and the CRT in both VS 6.0 and VS 7.0in both VS 6.0 and VS 7.0

Interoperability with other MSLU Interoperability with other MSLU clientsclients in processin process out of processout of process

Interoperability with non-Unicode Interoperability with non-Unicode applicationsapplications


AgendaAgenda




What the layer is notWhat the layer is not

Not a rewrite of the Win9x platformNot a rewrite of the Win9x platform No NT-specific functionality addedNo NT-specific functionality added

No "Unicode-only" language supportNo "Unicode-only" language support No supplementary character supportNo supplementary character support No new international supportNo new international support No new functionality added, at allNo new functionality added, at all

Not an "NT emulator" for Windows Not an "NT emulator" for Windows 95!95!

Not a layer over components that Not a layer over components that are still shipping new versionsare still shipping new versions


Other components, other Other components, other solutionssolutions RichEditRichEdit

2.0: Unicode support2.0: Unicode support 3.0: Bidirectional support3.0: Bidirectional support 4.0: Supplementary character support4.0: Supplementary character support

Windows Common Controls (comctl32.dll)Windows Common Controls (comctl32.dll) 5.80: full Unicode support5.80: full Unicode support

Uniscribe/GDI+Uniscribe/GDI+ MLangMLang Common Language Runtime (CLR)Common Language Runtime (CLR)

System.Globalization namespaceSystem.Globalization namespace System.Text namespace's encoding supportSystem.Text namespace's encoding support


AgendaAgenda




What about Visual What about Visual Studio.Net?Studio.Net? VS.NET should include the newest VS.NET should include the newest

Platform SDK; unicows.lib is in there!Platform SDK; unicows.lib is in there!

CLR languages like VB.NET, C#, and CLR languages like VB.NET, C#, and MC++ have their own delayload solutionMC++ have their own delayload solution PInvoke does not support calling different PInvoke does not support calling different

libraries on different platformslibraries on different platforms Do not call MSLU on NT platforms!Do not call MSLU on NT platforms!

You will have the same runtime issue You will have the same runtime issue with PInvoke in your MC++ codewith PInvoke in your MC++ code


using System;using System;

using System.Text;using System.Text;

using System.Runtime.InteropServices;using System.Runtime.InteropServices;

public class SystemAPI public class SystemAPI

{{

[DllImport("kernel32")][DllImport("kernel32")]

private static extern uint GetVersion();private static extern uint GetVersion();

[DllImport("kernel32", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)][DllImport("kernel32", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)]

private static extern uint OsGetSystemDirectoryW(StringBuilder lpBuf, uint uSize);private static extern uint OsGetSystemDirectoryW(StringBuilder lpBuf, uint uSize);

[DllImport("unicows.dll", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)][DllImport("unicows.dll", EntryPoint="GetSystemDirectoryW", CharSet=CharSet.Unicode)]

private static extern uint MsluGetSystemDirectoryW(StringBuilder lpBuf, uint uSize);private static extern uint MsluGetSystemDirectoryW(StringBuilder lpBuf, uint uSize);

public static uint GetSystemDirectory(StringBuilder lpBuf, uint uSize)public static uint GetSystemDirectory(StringBuilder lpBuf, uint uSize)

{{

if(GetVersion() < 0x80000000)if(GetVersion() < 0x80000000)

return(OsGetSystemDirectoryW(lpBuf, uSize));return(OsGetSystemDirectoryW(lpBuf, uSize));

elseelse

return(MsluGetSystemDirectoryW(lpBuf, uSize));return(MsluGetSystemDirectoryW(lpBuf, uSize));

}}

}}

class Class1class Class1

{{

static void Main(string[] args)static void Main(string[] args)

{{

StringBuilder pSysDir = new StringBuilder(256);StringBuilder pSysDir = new StringBuilder(256);

SystemAPI.GetSystemDirectory(pSysDir, 256);SystemAPI.GetSystemDirectory(pSysDir, 256);

Console.WriteLine(pSysDir);Console.WriteLine(pSysDir);

return;return;

}}

}}

C#.NET syntaxC#.NET syntax


Imports System

Imports System.Text

Imports System.Runtime.InteropServices

Public Class SystemAPI

Declare Function GetVersion Lib "kernel32" () As Integer

Declare Unicode Function OsGetSystemDirectoryW Lib "kernel32" Alias "GetSystemDirectoryW" _

(ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer

Declare Unicode Function MsluGetSystemDirectoryW Lib "unicows.dll" Alias "GetSystemDirectoryW" _

(ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer

Public Function GetSystemDirectory(ByVal lpBuf As StringBuilder, ByVal uSize As Integer) As Integer

If (GetVersion() > &H80000000) Then

GetSystemDirectory = OsGetSystemDirectoryW(lpBuf, uSize)

Else

GetSystemDirectory = MsluGetSystemDirectoryW(lpBuf, uSize)

End If

End Function

End Class

Module Module1

Sub main()

Dim stSysDir As StringBuilder = New StringBuilder(256)

Dim sapi As New SystemAPI()

sapi.GetSystemDirectory(stSysDir, 256)

Console.WriteLine(stSysDir)

End Sub

End Module

VB.NET syntaxVB.NET syntax


AgendaAgenda




Where to get MSLUWhere to get MSLU

the .LIB ships in the Platform SDK!the .LIB ships in the Platform SDK!http://www.microsoft.com/msdownload/platformsdk/http://www.microsoft.com/msdownload/platformsdk/

setuplauncher.aspsetuplauncher.asp

The DLL is a redistributable The DLL is a redistributable component:component:

http://www.microsoft.com/msdownload/platformsdk/http://www.microsoft.com/msdownload/platformsdk/setuplauncher.aspsetuplauncher.asp

(it will be somewhere up there, not sure where yet!)(it will be somewhere up there, not sure where yet!)


Questions?Questions?


Don't Forget Your Evals!Don't Forget Your Evals!

The MicrosoftThe Microsoft®® Layer Layer for Unicode™ on for Unicode™ on

WindowsWindows®® 95/98/ME Systems95/98/ME Systems

Unicode on Downlevel Windows (IUC 18) Unicode Across Windows Michael S. Kaplan Trigeminal Software, Inc. Cathy A. Wissink Microsoft.

Documents

unicode slide

unicode applications

downlevel windows iuc

windows xp

single unicode application

version of windows

windows michael

consistent unicode story