MinGW-w64 and the NT native APIs

So with the move to 64bits it was necessary to update my development environment moving from the now ancient MinGW tool set to the newer and 64bit supporting MinGW-w64. It was not entirely a smooth transition however, there were numerous differences between the two run-time libraries which had to be resolved.

There was one outstanding problem which could not be easily resolved. In MinGW-w64, the driver development kit headers (ddk\ntdkk.h, ddk\ntapi.h) , which I was previously using to access the NT native APIs, are not compatible with the standard win32 headers. This means, out of the box, there is no way to use the NT native APIs in MinGW-w64. (winternl.h is worthless)

For a while I was forced to make do with the lack of NT native API headers. Use of the Native API was done in the shit way where each project would have to include all of the definitions which they required. At some point I finally got sick of this and thought to myself, surely I am not the only one to have these issues. And that was a new beginning …

After some searching I finally discovered the ReactOS Native Development Kit. This set of headers was specifically designed for use in a win32 project and as such plays nicely with windows.h. A small number of changes were still required to build with MinGW-w64 as there were some definitions that the MinGW-w64 headers already had defined, but it did not take long to remove these from the NDK.

Advertisements
Posted in Uncategorized | Leave a comment

64 bits – sometimes going against the grain gets you bitten

When I installed 64bit windows I used custom paths for the “Program Files” directories.
x64: C:\ProgsNT
x86: C:\ProgsNT\x86

Windows setup was not at all willing to facilitate this arrangement, but after a small hack it became complacent. Anyway, the install succeeded and all was seemingly well. That is until a few days later when I finally discovered the unexpected consequences of this arrangement.

C:\ProgsNT\x86\x86\x86\x86\x86\x86\x86
This ridiculous set of nested directories was the unexpected consequence of this arrangement. Various files which had been installed were randomly scattered on the different nest levels of this mess of directories.

My non-standard directory layout has uncovered a strange bug in the msi installer system, and now I need to reinstall windows again to fix this mess. Serves me right for being so picky.

x86: C:\ProgsNT86
So I have now changed the 32bit “program files” directory to this, the msi installer is no longer shitting itself, but I need to redo all my setup again due to this stupid decision. But I have learned a valuable lesson, be really careful when going against the grain, or you might get bitten.

Posted in Uncategorized | Leave a comment

c++ bullshit – Character arrays and the null terminator

The C++ standard states

There shall not be more initializers than there are array elements. [Example:
char cv[4] = "asdf"; // error
is ill-formed since there is no space for the implied trailing ’\0’. —end example ]

Well that seems kind of reasonable at first glance, however on closer inspection this rule makes it impossible to define a non null terminated character array using a string literal as the initializer. The only “legal” way to accomplish this would be to use array syntax and specify each character individually.

char cv[4] = {'a','s','d','f'};

This is BULLSHIT, it makes the code ugly, harder to write, harder to read, and more difficult to maintain. This rule imposes unreasonable constraints on the programmer, he should not be forced to define a character array this way. The rule itself however is not in error, it is the fact that there is no way to define a non null-terminated string literal that is the problem.

Now most programmers would just submit to this tyranny, but I shall not yield. I shall hack the compiler to disable this rule until such time (probably never) that those retards who maintain the c++ standard correct this grievous mistake.

GCC compiler hacks

Posted in Uncategorized | Leave a comment

64 bits – the beginning

Finally I have done it, after almost a year of preparation (not continuously of course)  I have at last installed a working 64 bit install of windows 2003. So much work was needed just to get to this point and so much more will be needed just to get back to where I was with the 32bit install.

Tasks completed

  • Created 64bit version of Inexperience Patcher, no source code was available for the old one so I wrote my own code from scratch, though I did use the assets from the existing one.
  • Created 64bit WPA crack (I really could not find an existing one for x64). I used the 32bit crack Anti-WPA 2.3 as a guide, luckily the obfuscation and encryption was near identical so I did not loose too much hair.
  • Created install iso: Located suitable source iso. Used nlite to tune and remove bloat. Slip-streamed hacked AHCI driver. Fixed custom “Program Files” directory.
  • Added preliminary 64bit support to ExeModifier.
  • Patched ntfs.sys (using ExeModifier) to completely disable permissions (a totally worthless feature on a single-user machine which only gets in the way).

Tasks outstanding (off the top of my head)

  • Determine solution for the 32/64 conflict of DLL loading. Its retarded that the loader just stops searching after the first dll it finds regardless of whether it can load it or not.
  • Learn and understand x64 exception handling and add support to ExeModifier
  • Research the user-kernel-user exception swallowing issue and find a solution, seriously what the fuck were they thinking.
  • Port all remaining mods to 64bit
Posted in Uncategorized | Leave a comment

VirtualBox, 32bit host, 64bit guest, surpassing the 3584MB limit

VirtualBox limits the maximum amount of ram a VM can use on a 32bit host to 3584MB. There is no technical reason for this specific limit, whilst its true most 32bit hosts are limited to ~3.5GB of ram, servers with PAE enabled can far surpass this limit. My desktop runs win2k3 server, and can utilize all 16GB of installed ram. Wanting to run a 64bit guest with a decent amount of ram I set about to fix this pointless limitation.

After downloading the source for VirtualBox I was able to locate the problem code, when built for 32bit the constant MM_RAM_MAX_IN_MB is set to 3584. It was believed that to fix this it would be a simple matter of increasing MM_RAM_MAX_IN_MB and recompiling.

#if HC_ARCH_BITS == 64
# define MM_RAM_MAX_IN_MB           UINT32_C(2097152)
#else
# define MM_RAM_MAX_IN_MB           UINT32_C(3584)
#endif

A simple matter in theory; there was however no way I was going to actually try to build this from source, as with most large projects, it would take forever just to setup the build environment and dependencies, it would then likely fail to build in a numerous inexplicable ways.

Since only a single constant was required to be changed it was decided to modify the binary instead. A simple find and replace, 3584 -> 16384 was applied to VBoxSVC.exe, this was facilitated by a simple patch tool written for this purpose, the tool takes confirmation for each address to prevent altering unrelated code (in my case every instance of 3584 was replaced, this might not be the case for every VBoxSVC.exe version so checking the disassemble is a must). Testing found it to work fully, a 64bit guest was run and was able to utilize 12GB of memory on the 32bit host.  http://mjsstuf.x10host.com/files/vbox-mod.rar

32bit-host-64bit-guest-b

Posted in Uncategorized | Leave a comment

xp/2k3 on modern machines: the i7 and Hyper-threading

After some time running with win2k3 on the i7-4790k, I finally noticed that threads were not being correctly scheduled for hyper-threading. This caused a major performance hit when running 4 threads since the OS was making no effort to keep them on separate cores.

After a great deal of research and experimentation, I finally utilized the CPU enumeration API (which I had not previously known to exist) and discovered that the OS detected the CPU as a “single core with 8 HT units instead of 4 cores each with 2 HT units” (such insanity).  On finding this thread it became obvious that the problem was cpuid related. It seems that Intel in their infinite wisdom decided to change the meaning of one of the fields in the cpuid data.

The only solution that could be determined was to patch the Kernel to force a topology of two logical threads per core. ntkrnlpa.exe was disassembled and all occurrences of the cpuid instruction were located, thank god for IDA-pro without it I would be fucked. A single function was located which retrieved the cpuid data, that function was replaced and the relevant field was overridden with the desired value.

extern "C"
int __stdcall CPUID(int a1, int* a2, int* a3, int* a4, int* a5)
{
    int EAX, EBX, ECX, EDX;
    asm("cpuid" : "=a"(EAX), "=b"(EBX),
        "=c"(ECX), "=d"(EDX) : "a"(a1));
    if(a1 == 1) {
        EBX = (EBX & 0xFF00FFFF) | 0x00020000; }
    *a2 = EAX; *a3 = EBX;
    *a4 = ECX; *a5 = EDX;
}

This patch was applied using my exe modifier utility, and then the original Kernel was replaced with my patched version. On reboot the machine still worked and when running the CPU enumeration again it now correctly detected the 4 cores with two threads each. When testing with 4 threads the correct scheduling was observed, each core was now given a single thread consistently. ntoskrnl_HT_FIX.rar

Posted in Uncategorized | Leave a comment

xp/2k3 on modern machines

I use win2k3 32bit on my modern machine with i7-4790k CPU and 16GB ram. Using a 10+ year-old OS has problems and I would love to upgrade but win2k3 is the last real version of windows. This thing has suffered numerous freezes/blue-screens, some caused by the crappy ATI drivers for my aging x300. It might be the 16GB of ram which causes the problems, the ATI drivers are not said to work with win2k3 let alone more than 4GB of ram. I like to avoid OpenGl since crashes are very likely when running that.

win2k3

Posted in Uncategorized | Leave a comment