nGlide and QEMU

[Zeus]

I'm starting this thread for discussions regarding Kjliew's 3Dfx Glide patch for QEMU and nGlide.

After some brief testing I can say things are looking quite good. Not spectacular, but not bad either. Performance wise I get 50fps average with most Windows Glide2x games I tested so far and around 30fps with Glide3x games that don't utilize lfb heavily.

Tests are done on Intel i7 6700K 4.0 GHz and Windows 98 SE as a guest operating system. QEMU is 4.1.0, Win32 version and nGlide latest dev version. Acceleration used is TCG, HAX crashes with Win98 and WHPX isn't available for 32-bit builds.

qemu_1.jpg (75.14 KiB) Viewed 11922 times

qemu_2.jpg (50.04 KiB) Viewed 11922 times

qemu_3.jpg (65.07 KiB) Viewed 11922 times

I will soon publish the first wave of the Glide implementation issues I found in the patch. In the meantime I have an important question for you, Kjliew.

Can you rewrite the patch so Glide API functions will be executed from the same thread that created QEMU window? Perhaps you are wondering why all Glide wrappers freeze with your patch and fullscreen exclusive mode.

That's because the patch calls Glide functions from a different thread that created QEMU main window. What's even worse QEMU (SDL2) message pump is executed only between Glide API invocations.

The requirement for D3D fullscreen mode is that 3D devices need to be created and released from the same thread that created a window. nGlide can cover such cases and can synchronize itself, but like any other software it needs windows message pump to be functioning during Glide API execution (which QEMU don't provide waiting for Glide API call to be complete).

Some games like Hitman forget to call grSstWinClose() at exit, directly calling grGlideShutdown(). This create a situation where nGlide doesn't have a chance to synchronize itself.

The sequence is as follows:
-grGlideShutdown() [from other thread]
-Message pump [activates synchronization]
-FreeLibrary(glidexx.dll)
-grGlideShutdown() [from window thread, crash to desktop due to the fact glidexx.dll is already unloaded]

In such scenario I have two choices. Not to offer fullscreen mode and Alt+Enter switch feature for QEMU (very bad) or to hook your FreeLibrary() call, making as a result glidexx.dll libraries loaded permanently within QEMU process.

Another workaround is to create a second window from the same thread that calls Glide API functions (this is exactly what you do with CreateWindow option in glide.cfg), but it's even worse, so I don't consider it as a reasonable solution.

Please let me know what are the options.

[kjliew]

Tests are done on Intel i7 6700K 4.0 GHz and Windows 98 SE as a guest operating system. QEMU is 4.1.0, Win32 version and nGlide latest dev version. Acceleration used is TCG, HAX crashes with Win98 and WHPX isn't available for 32-bit builds.

You really want QEMU 64-bit build with nGlide 64-bit build. That will give you free 10~15% performance boost for free just by recompiling. And, you really want a form of VM acceleration over TCG, that gives you another 2x~4x performance.

Can you rewrite the patch so Glide API functions will be executed from the same thread that created QEMU window? Perhaps you are wondering why all Glide wrappers freeze with your patch and fullscreen exclusive mode.

That's because the patch calls Glide functions from a different thread that created QEMU main window. What's even worse QEMU message pump is executed only between Glide API invocations.

Yes, I did come to realize that QEMU rendering thread is different from the main thread where the device models are instantiated. In fact, QEMU threaded its internal workloads very extensively. That's why I had to rely on posting SDL messages for window handling. I knew the problem quite well as I stumbled on resizing window during early development. Unfortunately, I am afraid this is something that we will have to live with and find ways around it. It is not a good idea to patch out QEMU extensively to change its original designs. The Glide pass-through is just another device model, and it has to live with what the architecture provides, so that it won't break easily as upstream QEMU continues to improve. I have done extensive testing with OpenGlide and psVoodoo, if the later is what more of a resemblance of nGlide for Direct3D9 backend. IIRC, psVoodoo full-screen worked when QEMU switched to full-screen with Ctrl-Alt-F as psVoodoo simply scaled its rendering context to QEMU window context, but I will double-check.

At the moment, the only infrastructure we have is, if the DLL exported a "_setConfig@4" symbol, then it will be called to inform the wrappers of running within QEMU. I hope that this will be something useful for wrappers to be informed about such special situation. My locally patched OpenGlide and psVoodoo take this approach to support special QEMU plumbing.

Code: Select all

    setConfig = (void *)(GetProcAddress(hDll, "_setConfig@4"));
    if (setConfig)
        (*setConfig)(WRAPPER_FLAG_QEMU);

If you need anything more than this, you are welcome to propose the ideas and we can see if they are feasible.

Some games like Hitman forget to call grSstWinClose() at exit, directly calling grGlideShutdown(). This create a situation where nGlide doesn't have a chance to synchronize itself.

I thought only DOS Glide games would have done this. I have workaround this in the OVL. I will double check on how this is handled in the DLLs. I believe the game you referred to is Hitman: Codename 47 (2000). Is it Glide2x or Glide3x?

Not to offer fullscreen mode and Alt+Enter switch feature for QEMU

Personally, that would be perfectly fine for now until we can find better solution.

[Zeus]

You really want QEMU 64-bit build with nGlide 64-bit build. That will give you free 10~15% performance boost for free just by recompiling.

Yes, that's the plan, for the future.

And, you really want a form of VM acceleration over TCG, that gives you another 2x~4x performance.

I would like to, but as far as I'm aware there is no way to run Win98SE with HAXM, no matter if the build is 32 or 64-bit. That's a problem assuming 85% of all Glide games are Windows. I'm not sure WHPX will be that fast.

I am afraid this is something that we will have to live with and find ways around it.

Alright. Not a problem. Perhaps faking grSstWinClose() for games that forget to call it will rectify the problem. What we need is another call before FreeLibrary(glidexx.dll). Once that's fixed, true fullscreen exclusive won't have to be sacrificed.

IIRC, psVoodoo full-screen worked when QEMU switched to full-screen with Ctrl-Alt-F as psVoodoo simply scaled its rendering context to QEMU window context, but I will double-check.

Ctrl-Alt-F is also a windowed mode, but a borderless one (and stretched, yes, not uprendered; nGlide will work the same with DX9 and VK backend).

At the moment, the only infrastructure we have is, if the DLL exported a "_setConfig@4" symbol, then it will be called to inform the wrappers of running within QEMU.

Interesting. Thanks.

I thought only DOS Glide games would have done this. I have workaround this in the OVL. I will double check on how this is handled in the DLLs. I believe the game you referred to is Hitman: Codename 47 (2000). Is it Glide2x or Glide3x?

Yes, that's the game. Glide3x.

Personally, that would be perfectly fine for now until we can find better solution.

This feature is important for some nice additions glide wrappers provide like high resolutions (real HD or 4K), Vsync, Integer scaling, aspect ratio correction, gamma correction (in case of dx9) etc. Windowed mode gives merely a stretched original resolution (640x480 in most cases).


BTW, I noticed some weird thing with QEMU. It's working 15% faster on my machine if host is Win7 x64 (53fps in Mortal Kombat 4 vs 39fps on Win10 x64).

There is also one annoying issue with SB16 emulation. It can freeze a playing sound until new sound is played (command 0xf9 unimplemented or something). I understand this is out of our scope and it's not something we would like to fix, but I have concerns whether anybody would be willing to play their favourite Glide games with such conditions.

Maybe WinXP as a guest with HAXM (assuming it works) and AC97 would be a better solution after all. The questions is, how fast would that be.

[kjliew]

I would like to, but as far as I'm aware there is no way to run Win98SE with HAXM, no matter if the build is 32 or 64-bit. That's a problem assuming 85% of all Glide games are Windows. I'm not sure WHPX will be that fast.

Oh, yeah though WHPX is not as fast as HAXM/KVM, it is still at least 2x faster than TCG. You should try out HAXM with Windows 2k/XP guests for games that work without compatibility layers (NFS3/4, NFS Porche 2000, Unreal etc.) It makes such a huge difference!

Perhaps faking grSstWinClose() for games that forget to call it will rectify the problem. What we need is another call before FreeLibrary(glidexx.dll). Once that's fixed, true fullscreen exclusive won't have to be sacrificed.

Yes, that's the game. Glide3x.

Ah, it's Glide3x! That is where I don't have the workaround because grSstWinClose() in Glide3x requires a window context that I don't have it handy when grGlideShutdown() is called. I would have thought all Glide3x games would have been more sane in respecting the API. I can definitely workaround it but I am not sure if there is any real usage of having multiple Glide window contexts in-flight (technically this is possible, not sure if anything really use this). I can do a simple single-window context to address this. If there is any multiple-window context situation, then I would expect the code would have called grSstWinClose() with the intended window context.

Ctrl-Alt-F is also a windowed mode, but a borderless one (and stretched, yes, not uprendered; nGlide will work the same with DX9 and VK backend).

Yes, you are right. However, I would have expected that this is the best practice nowadays as DE/WM are also 3D clients and doing true mode switching for full-screen is not going to play well in the future.

This feature is important for some nice additions glide wrappers provide like high resolutions (real HD or 4K), Vsync, Integer scaling, aspect ratio correction, gamma correction (in case of dx9) etc. Windowed mode gives merely a stretched original resolution (640x480 in most cases).

I was hoping that you would still be able to do some of these nice additions without an explicit mode switch for fullscreen.

BTW, I noticed some weird thing with QEMU. It's working 15% faster on my machine if host is Win7 x64 (53fps in Mortal Kombat 4 vs 39fps on Win10 x64).

There is also one annoying issue with SB16 emulation. It can freeze a playing sound until new sound is played (command 0xf9 unimplemented or something). I understand this is out of our scope and it's not something we would like to fix, but I have concerns whether anybody would be willing to play their favourite Glide games with such conditions.

Maybe WinXP as a guest with HAXM (assuming it works) and AC97 would be a better solution after all. The questions is, how fast would that be.

I am not too concerned about Win7 nowadays. Win10 and Linux are the future. Win7 will find a hard time booting in future when all new PCs are mandatory for UEFI class 3 after year 2020. You can have AC97 in Win98SE guest, too, that's how my Win98SE Glide gaming VM was setup. In fact, you can also have both SB16 and AC97 at the same time. Windows games will use AC97 and DOS games run in Win98 DOS box will use SB16 for games that work better in such setup (eg. BC3000, Pyl). My GLIDE2X.OVL pass-through works in Win98 DOS box as well as in real DOS.

[Zeus]

You should try out HAXM with Windows 2k/XP guests for games that work without compatibility layers (NFS3/4, NFS Porche 2000, Unreal etc.) It makes such a huge difference!

You bet I will! And nGlide compatibility list is full of XP patches we were creating for years with Gamecollector and some other authors, so running titles that require them won't be a problem.

I can do a simple single-window context to address this.

Never spotted any software with multiple context usage. You can safely treat it as single.

Yes, you are right. However, I would have expected that this is the best practice nowadays as DE/WM are also 3D clients and doing true mode switching for full-screen is not going to play well in the future.

I was hoping that you would still be able to do some of these nice additions without an explicit mode switch for fullscreen.

Sure, I can do that. For some reason I thought this would be unintuitive, but perhaps you're right. Still though, I'm planning to include fullscreen exclusive mode (Alt+Enter), not only for Vsync and Gamma correction unavailable with windowed DX9, but also for people (including myself) that *want* to have the ability to switch display mode.

I am not too concerned about Win7 nowadays. Win10 and Linux are the future. Win7 will find a hard time booting in future when all new PCs are mandatory for UEFI class 3 after year 2020.

Yeah, time flies like crazy. It's hard to think nGlide will be 10-years old in the next 2 weeks.

You can have AC97 in Win98SE guest, too, that's how my Win98SE Glide gaming VM was setup. In fact, you can also have both SB16 and AC97 at the same time. Windows games will use AC97 and DOS games run in Win98 DOS box will use SB16 for games that work better in such setup (eg. BC3000, Pyl). My GLIDE2X.OVL pass-through works in Win98 DOS box as well as in real DOS.

WOW! I didn't know that. Thank you!

[kjliew]

Attached Glide3x 9x DLL. Let me know if you need one for 2K/XP to check out HAXM. Glide3x games are much improved on models and details and would benefit tremendously with accelerated VM.

It seems that I did have the workaround in-placed for everything else including Glide3x DJGPP DXE during my adventure with MESAFX and QDOS/Q2DOS/H2DOS source ports. Everything had already been prepared, it just so happened that I did not see the same need for Windows While the QEMU Glide pass-through can potentially support multi-window Glide apps with simple modification, the wrappers are limited to single-window.

[Zeus]

Attached Glide3x 9x DLL.

Now it works like a charm.

Let me know if you need one for 2K/XP to check out HAXM.

That would be great, the problem is, I installed XP SP3 x86 today and unfortunately I don't have any good news:

-QEMU freezes with HAXM right after boot logo sequence (I used cirrus), TCG works
-AC97 freezes exactly the same as SB16 with W98SE
-Launching any glide software results with "The application failed to initialize properly - 0xc0000005" error

I also tried installing AC97 with W98SE. It won't detect it no matter what. Any ideas? Does it need to be a specific driver?

[kjliew]

-QEMU freezes with HAXM right after boot logo sequence (I used cirrus), TCG works
-AC97 freezes exactly the same as SB16 with W98SE
-Launching any glide software results with "The application failed to initialize properly - 0xc0000005" error

I also tried installing AC97 with W98SE. It won't detect it no matter what. Any ideas? Does it need to be a specific driver?

Attached the 2KXP pack.

I don't exactly recall if I had ever booted XP with HAXM, but 2K worked. Sorry I no longer have Intel system to check out HAXM, but you can try without the "-vga cirrus" and load VBEMP XP driver instead. Both cirrus and stdvga work fine with WHPX for XP guest.

For 2K/XP, the OS requires SCM to create and start the kernel driver, then only userspace DLL can access it. Place the FXPTL.SYS in %SystemRoot%\system32\drivers and run INSTDRV to create and start the driver.

I supposed you have enabled AC97 with "-soundhw ac97,sb16". Win98SE should have prompted for new PCI multimedia device. Win98SE does not have inbox AC97 driver, but you can find one. You need to modify the INF file to match QEMU AC97 PCI instance.

win98.png (43.44 KiB) Viewed 11795 times

[Zeus]

Sorry I no longer have Intel system to check out HAXM, but you can try without the "-vga cirrus" and load VBEMP XP driver instead. Both cirrus and stdvga work fine with WHPX for XP guest.

-vga std worked. Though installing VBEMP NT resulted with a blue screen of death.

Place the FXPTL.SYS in %SystemRoot%\system32\drivers and run INSTDRV to create and start the driver.

Can you rework it so it installs properly from any location? When I clicked on INSTDRV from my software folder it told me "Driver installed" when in fact it wasn't.

Clicking on INSTDRV from "drivers" finally worked, but only for glide.dll and glide2x.dll. Glide3x.dll crashes with "The application failed to initialize properly - 0xc0000142" if I don't enable HAXM.

I supposed you have enabled AC97 with "-soundhw ac97,sb16".

Yes.

Win98SE should have prompted for new PCI multimedia device.

And here's my problem, I don't see the device. Very odd. On a guest XP it's detected properly.

[Zeus]

Here's my #1 report regarding Glide implementation itself.

1) Broken data format in glide2x.grDrawPlanarPolygon()

Most likely format or stride issue. You can notice the problem in the demo of Summoner.
Once you recognize the problem you may want to check glide.grDrawPlanarPolygon() as well.

2) Broken CHROMARANGE glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grChromaRangeModeExt()
-grChromaRangeExt()

3) Broken TEXCHROMA glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grTexChromaModeExt()
-grTexChromaRangeExt()

4) Broken PIXEXT glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grSstWinOpenExt()
-grColorMaskExt()
-grStencilFuncExt()
-grStencilMaskExt()
-grStencilOpExt()
-grBufferClearExt()
-grLfbConstantStencilExt()
-grTBufferWriteMaskExt()

5) Broken COMBINE glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grColorCombineExt()
-grAlphaCombineExt()
-grTexColorCombineExt()
-grTexAlphaCombineExt()
-grConstantColorValueExt()

6) Broken glide3x.grAlphaBlendFunctionExt()

glide3x.grGetProcAddress() doesn't pass pointer for:

-grAlphaBlendFunctionExt()

7) Broken PALETTE6666 glide3x extension

glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly

8) Broken FOGCOORD glide3x extension

glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly

9) Broken TEXFMT glide3x extension

glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly

10) Broken TEXTUREBUFFER glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grTextureBufferExt()
-grTextureAuxBufferExt()
-grAuxBufferExt()

11) Broken GETGAMMA glide3x extension

Either glide3x.grGetString(GR_EXTENSION) doesn't pass extension string correctly or glide3x.grGetProcAddress() doesn't pass pointers for:

-grGetGammaTableExt()

12) Missing "Version Info" in guest DLLs resource

Total Annihilation Kingdoms (glide3x) detects Voodoo presence from Version Info.
You can use nGlide and official drivers for reference and game demo version for testing purposes.

13) Incorrect QEMU window size with multiple glide2x.grSstWinOpen() calls

Gex 2: Enter the Gecko calls grSstWinOpen() three times at launch. 512x384, 640x480 and 512x384. For some reason the last call is ignored. You can use demo or full version for testing purposes.

14) Incorrect depth state with Bio Freaks (glide2x)

Depth buffering should be enabled during simulation in Bio Freaks. I would check grDepthBufferMode(), grGlideGetState() and grGlideSetState(). The problem must be somewhere there.