On running dos executables on x64

Well it became necessary to make updates to one of my old projects, the Vgm player. And that happens to be built with a dos application SNASM68K. At this point I have fully migrated over to 64bits so this presented a small issue. How to run this application which Microsoft in their infinite retardation decided to no longer support. Their excuse; that amd64 lacks support for V86; is utter bullshit, I know for a fact that the Ntvdm code base supports x86 emulation.

Anyway I needed to run this application, one simple way would have been to use a VM but that is kind of shitty and inconvenient. There is an existing solution,  NTVDMx64 by Leecher1337. This is an x64 build of the official Ntvdm built from the leaked windows NT4 source code. I did not use this for two reasons, 1: its does not support xp/2k3. 2: I dislike how invasive the patch is to integrate it into CreateProcess, also it would have been significant work to adapt the patch for xp/2k3.

Another option. Reactos Ntvdm , this recreation of Ntvdm works on XP/2K3 and can be built in a standalone mode so does not require the invasive integration into CreateProcess. The disadvantage however is that without the CreateProcess integration one cannot start the dos executables in the normal way, all callers would need to be modified to start the dos application by proxy of ntvdm.exe

The solution: I really do not need system wide support for dos applications, I really only have one or two dos apps which I cannot replace, so why not just build ntvdm into each dos application, so that is what I did.  I have the modified the Reactos Ntvdm to load the dos application appended onto the end of the Ntvdm binary. This creates standalone x64 compliant dos applications.

dosEx64.exe – dos executable patcher
SNASM68K.exe – 64bit compatible



Posted in Uncategorized | Leave a comment

Fixing the XP/2K3 calculator paste delay

The paste functionality in the classic windows calculator is implemented by generating button presses, as defined by the paste text. In windows 2000 and before these button presses can be seen. In XP this visual feedback was removed, the corresponding delay however, remains. This delay can be aggravating, especially when doing heavy calculations, after years of suffering this inconvenience I finally snapped and resolved to do something about it.

Fixing: To locate the relevant code I first located calls to GetClipboardData, of which there was one. The nearby loop which parsed the clipboard string contained no delays and simply forwarded the characters elsewhere by sending WM_COMMAND to some other window. With that avenue failing to bare fruit I instead located calls to the Sleep function, I found a single call which happened to be the offending code. The call to sleep was a fixed 20ms sandwiched in-between two calls to SendMessageW with BM_SETSTATE. I changed the delay to 0ms and the horrible delay was gone.

the code of interest

Patched calc.exe for windows 2003 x64: no-delay-paste-calc.zip

Posted in Uncategorized | Leave a comment

On Tool Assisted Speed Runs and the playback thereof

The standard way to watch a tool assisted speed-run is via a pre-recorded video. This sucks, not only are you having to download a very large video file, but the quality of said file, even when bit-rates are high will still be very substandard. The other option for watching is to download the specific game rom, emulator, and button press file. This is just inconvenient, as you have to trawl all over the internet to locate required pieces.

As the options discussed previously were unacceptable, a new solution was needed. The solution that was found was to bake the button press data into a modified version of the rom, the tool assisted speed run would then run like a built-in demo. This procedure is very game specific and may not even be possible for all games depending on how complex their lag behavior is, the game I have used is the MegaDrive game Sonic 1.

De-synchronization and lag frames

The main issue facing tool assisted speed-runs is desync due to the difference in timing between various emulators and the real hardware. The main cause of this desync (in the case of Gens-rerecording) is the fact that the emulator records controller input synchronous to the display frames, this means a single mismatched lag frame will cause all subsequent controller input to be desyned.

The other and much more insidious cause is the fact that (in the case of Sonic1) during the execution of the game engine the interrupts are left enabled, this is such that if a lag frame occurs, the music (sonic 1 uses main cpu for music) and palette (LZ water), will continue to update. There is another unfortunate side effect however, the frame counter is incremented in the vertical interrupt handler. This counter is the timing source for many in-game effects some of which effect the game-play such that desync can occur.

Solutions to desync

The main cause of desync is simple to correct, instead of recording the controller state on each display frame one shall instead record the actual controller state read by the game code. With this method of recording, lag shall no-longer cause controller desync.

The solution for the secondary cause is not so simple. The value of the frame timer is critical, even being off by 1 can significantly change the behavior of the game. There is no way to model the value of this variable as it is linked to the lag frames and thus the specific timing of the recording emulator. The only solution is to record its exact value for each frame.

Implementation – Controller Data

The controller data consists of two bytes for each event, the first being the timer increment and the second being the controller state. To perform the conversion of controller data it was necessary to modify the gens-rerecording emulator, specifically controller reads and frame timer reads were hooked, each controller read is considered the start/end of an event, reads to the frame timer are noted and used to determine the timer increment for each event.

Implementation – Sonic 1

The Sonic 1 game was modified to playback the converted controller data, the controller read function was modified to read from the controller data. The increment of the frame timer was removed from the vertical interrupt handler, instead the frame timer is updated explicitly in the controller read routine, its value increased by the amount specified in the timer increment byte of the controller event.

Sonic1 speed-runs exploit glitches in the game to achieve insane speeds and often skip entire levels all together. The in-game camera however is limited to just 16 pixels per frame scrolling. With this limitation much of the TAS video is that of just scrolling. To correct this I removed the scroll speed limit from the game, the camera shall now stay centered on sonic no matter how fast he moves. Specifically I increased the scrolling speed to 32px left and 64px right. If sonic moves faster than this such as wrapping to the end of the level then the entire screen is reloaded.

One would think such drastic changes to the game would cause desync, but having corrected the lag frame issues I can now make major changes without causing a desync. Any reader who understands the Sonic1 engine would wonder how removing the camera position limit did not result in desync as the game objects are effected by screen position. Well I simply duplicated the screen position variables, the original 16 pixel limit is still used internally to keep the engine happy.

One might also think that all these changes would cause massive slowdowns, what with the checking of two screen positions, particularly with the sprites, which have to be drawn with one  screen position but have their onscreen bit set using another. And the doubled, or quadrupled scroll speed. But actually the slowdown is about the same of maybe even a bit less! This is because some of the existing code in sonic1 is slow as fuck, I made many optimizations both major and minor, particularly the horizontal scroll, I can update 64 pixels about the same speed as the game used to mange 16.

Converted runs

Sonic The Hedgehog (W,) (REV 01) [!]Aglar.zip
Sonic The Hedgehog (W,) (REV 01) [!]no_zips_Aglar.zip
source code – github

I would really like to do other sonic games as well but this really took to long, and also the newer sonic games being much more complicated will likely have more serious issues that may or may not be solvable.

Posted in Uncategorized | Leave a comment

MinGW-w64 and the NT native APIs

So with the move to 64bits it was necessary to update my development environment moving from the now ancient MinGW tool set to the newer and 64bit supporting MinGW-w64. It was not entirely a smooth transition however, there were numerous differences between the two run-time libraries which had to be resolved.

There was one outstanding problem which could not be easily resolved. In MinGW-w64, the driver development kit headers (ddk\ntdkk.h, ddk\ntapi.h) , which I was previously using to access the NT native APIs, are not compatible with the standard win32 headers. This means, out of the box, there is no way to use the NT native APIs in MinGW-w64. (winternl.h is worthless)

For a while I was forced to make do with the lack of NT native API headers. Use of the Native API was done in the shit way where each project would have to include all of the definitions which they required. At some point I finally got sick of this and thought to myself, surely I am not the only one to have these issues. And that was a new beginning …

After some searching I finally discovered the ReactOS Native Development Kit. This set of headers was specifically designed for use in a win32 project and as such plays nicely with windows.h. A small number of changes were still required to build with MinGW-w64 as there were some definitions that the MinGW-w64 headers already had defined, but it did not take long to remove these from the NDK.

Posted in Uncategorized | Leave a comment

64 bits – sometimes going against the grain gets you bitten

When I installed 64bit windows I used custom paths for the “Program Files” directories.
x64: C:\ProgsNT
x86: C:\ProgsNT\x86

Windows setup was not at all willing to facilitate this arrangement, but after a small hack it became complacent. Anyway, the install succeeded and all was seemingly well. That is until a few days later when I finally discovered the unexpected consequences of this arrangement.

This ridiculous set of nested directories was the unexpected consequence of this arrangement. Various files which had been installed were randomly scattered on the different nest levels of this mess of directories.

My non-standard directory layout has uncovered a strange bug in the msi installer system, and now I need to reinstall windows again to fix this mess. Serves me right for being so picky.

x86: C:\ProgsNT86
So I have now changed the 32bit “program files” directory to this, the msi installer is no longer shitting itself, but I need to redo all my setup again due to this stupid decision. But I have learned a valuable lesson, be really careful when going against the grain, or you might get bitten.

Posted in Uncategorized | Leave a comment

c++ bullshit – Character arrays and the null terminator

The C++ standard states

There shall not be more initializers than there are array elements. [Example:
char cv[4] = "asdf"; // error
is ill-formed since there is no space for the implied trailing ’\0’. —end example ]

Well that seems kind of reasonable at first glance, however on closer inspection this rule makes it impossible to define a non null terminated character array using a string literal as the initializer. The only “legal” way to accomplish this would be to use array syntax and specify each character individually.

char cv[4] = {'a','s','d','f'};

This is BULLSHIT, it makes the code ugly, harder to write, harder to read, and more difficult to maintain. This rule imposes unreasonable constraints on the programmer, he should not be forced to define a character array this way. The rule itself however is not in error, it is the fact that there is no way to define a non null-terminated string literal that is the problem.

Now most programmers would just submit to this tyranny, but I shall not yield. I shall hack the compiler to disable this rule until such time (probably never) that those retards who maintain the c++ standard correct this grievous mistake.

GCC compiler hacks

Posted in Uncategorized | Leave a comment

64 bits – the beginning

Finally I have done it, after almost a year of preparation (not continuously of course)  I have at last installed a working 64 bit install of windows 2003. So much work was needed just to get to this point and so much more will be needed just to get back to where I was with the 32bit install.

Tasks completed

  • Created 64bit version of Inexperience Patcher, no source code was available for the old one so I wrote my own code from scratch, though I did use the assets from the existing one.
  • Created 64bit WPA crack (I really could not find an existing one for x64). I used the 32bit crack Anti-WPA 2.3 as a guide, luckily the obfuscation and encryption was near identical so I did not loose too much hair.
  • Created install iso: Located suitable source iso. Used nlite to tune and remove bloat. Slip-streamed hacked AHCI driver. Fixed custom “Program Files” directory.
  • Added preliminary 64bit support to ExeModifier.
  • Patched ntfs.sys (using ExeModifier) to completely disable permissions (a totally worthless feature on a single-user machine which only gets in the way).

Tasks outstanding (off the top of my head)

  • Determine solution for the 32/64 conflict of DLL loading. Its retarded that the loader just stops searching after the first dll it finds regardless of whether it can load it or not.
  • Learn and understand x64 exception handling and add support to ExeModifier
  • Research the user-kernel-user exception swallowing issue and find a solution, seriously what the fuck were they thinking.
  • Port all remaining mods to 64bit
Posted in Uncategorized | Leave a comment