Restoration of a few games' EXEs versions

Discuss coding, porting and creating games
NY00123
4-bit nibble
Posts: 30
Joined: July 4th, 2015, 11:05 am

Re: Restoration of a few games' EXEs versions

Post by NY00123 » April 9th, 2017, 3:29 pm

Hey there,

While I don't have a lot of new actual contents to report about, there's still what to write.

1. To begin with, anybody who's been messing with the modified Wolf3D codebase from my repository should probably read this point. Not long ago, I've found out that Borland C++ effectively truncates too long identifiers behind-the-scenes. There's an "Identifier Length" option which tells the max. identifier length, and has to fall in the range 1-32 (at least under Borland C++ 3.0).

So it's been a good chance for me to not just shorten identifier names, but also do some other changes. A significant one is the usage of a new GAMEVER_WOLFREV macro for covering different ranges of Wolf3D codebase revisions.

Also, the project file names have been renamed. In particular, note these renames: WL1APO12.PRJ -> WL1AP11.PRJ, SODFOR14.PRJ -> SODFG14B.PRJ. There is indeed a new "SODFG14A.PRJ" project, see below.

Finally, VERSION.H and VERSION.EQU now internally used "EXEDEF" macros that match the corresponding project files. These macros are *never* directly used in any other source file (on purpose). GAMEVER_WOLFREV should be used for this, combined with macros like UPLOAD and GOODTIMES, as well as the new GAMEVER_NOAH3D macro.

2. I've added a project file for another EXE, named SODFG14A.PRJ. This is the "v1.4" release mistakenly labelled as "V1.0" in-game, which was apparently released on 1992-11-12, according to Litude. To compare, SODFG14B.PRJ (previously named SODFOR14.PRJ) is assumed to be released on 1993-01-12.

In terms of actual code found in the C and ASM sources, SODFG14A.PRJ and SODFG14B.PRJ are exactly the same. Furthermore, both EXEs were originally built using Borland C++ 3.1. However, in other ways, SODFG10.PRJ is more similar to SODFG14A.PRJ. This includes differences like the order in which objects get linked, as well as the "V1.0" sign-on screen. In fact, ignoring my addition of the "EXEDEF" macros, the exact same project file can theoretically be used for SODFG10 and SODFG14A altogether, with a bit different VERSION.H/EQU definitions.

3. For some weird unknown reason, the instructions for building SODFG10.EXE (previously SODFOR10.EXE) have changed a bit. After building the EXE, you now have to remove WL_PLAY.OBJ, re-open the project with Borland C++ 3.0 and then again press on F9 to build the removed code. Only then, LZEXE91 may be used. I really don't know why this is the case, especially since it's not required in an earlier revision of the codebase (I've verified this). It might be related to source file(s) layouts and/or the way a table of definitions internally used by the compiler is structured.

4. I've also updated the notes regarding the compression of the Activision EXEs. To summarize, you can use LZEXE 0.91e, rather than 0.91, to faithfully recreate the EXEs from 1998. These two versions of LZEXE seem to differ, a little bit, by the contents of the decompressor stub as embedded in the packed EXEs (0.91e appears to add a "push ax" and a later "pop ax", if there's no other change). Since UNLZEXE8 checks the first 232 bytes of the decompressor stub, this should explain why it fails to unpack any of the Activision EXEs, while UNLZEXE7 can do so. Of course, if any of you attempts to hide that "LZ91" string (found in the first 32 bytes of the EXE), then UNLZEXE7 will fail, too.

I'll also add a few bonus points here:

5. So now, I don't exactly have the means to build this as-is, but let's continue anyway. The file http://www.freewebs.com/choksta/wolflist.txt lists a "WOLF3D_J.EXE" executable, which comes from a Japanese localization of the game made for DOS/V (not to be confused with the PC-98 port i.e., WOLF98.EXE). It is reportedly 254046 bytes long. Well, it turns out, that when you modify WL6GT14A so the JAPAN macro is defined instead of GOODTIMES, Borland C++ 3.1 outputs an EXE with the exact same size. Again, though, I cannot confirm it's the exact same EXE. In fact, I'm quite sure it isn't, given the presence of the (wrong) GT sign-on screen, heh.

6. One may be wondering if WOLF3D.EXE, as originally bundled on 1995 along with the Wolfenstein 3D sources, can be re-created? In fact, maybe it can simply be built from the sources as released back then? Well, it turns out the answer is "yes", under a few conditions. First, you should use Borland C++ 3.0. Secondly, make sure all files have their original modification dates, not older than 1995. Otherwise, expect to get a few minor differences between the 1995 EXE build and your build.

Can the same be done from my repository? Well, if we ignore differences in debugging symbols, then again the answer is "yes". Basically, you should begin from (a copy of) WL6AC14.PRJ, then replace the Activision SIGNON.OBJ with the GT one, select "On" for "Source Debugging" in Borland C++ 3.0 (under Debugger Options) and finally make sure that GAMEVER_WOLFREV is the same as for WL6GT14B. You may wish to modify the output directory to something other than OBJ\WL6AC14, and possibly also update the Include and Library directories (so files from BC30 are used instead of BC31). Watch out, though, in case you also want to replace the GAMEVER_EXEDEF_WL6AC14 macro. Reason is, not only you should edit it under "Compiler -> Code Generation", but it also has to be manually changed for a few ASM sources (under params passed to the assembler).

User avatar
MrFlibble
Forum Administrator
Posts: 1604
Joined: December 9th, 2010, 7:19 am

Re: Restoration of a few games' EXEs versions

Post by MrFlibble » April 19th, 2017, 8:29 am

Sorry for a bit of off-topic here. I just played The Catacomb Abyss shareware with your Reflection Keen port, and I must say you're doing an excellent job. What I particularly liked is:
  • correct aspect ratio (it shouldn't be surprising but not all ports of DOS games do that, sadly)
  • switching between windowed and fullscreen mode
  • optional bilinear filtering
Generally I like your idea of taking inspiration from Chocolate Doom. In my opinion, Wolf3D engine games lack their own "Chocolate/Crispy" ports pretty badly. Any chance you could do something similar for Wolfenstein 3-D if you have the time? Rise of the Triad also is pretty lacking in the source port department (apparently none of the available software mode ports properly deal with aspect correction), and also there's the code of In Pursuit of Greed no one has done anything with so far AFAIK.

NY00123
4-bit nibble
Posts: 30
Joined: July 4th, 2015, 11:05 am

Re: Restoration of a few games' EXEs versions

Post by NY00123 » April 28th, 2017, 2:19 am

Hey there,

Thanks for your comments on Reflection Keen / Reflection Catacomb Abyss, as well as trying it out!
Generally I like your idea of taking inspiration from Chocolate Doom.
As I said beforehand, there are still enough ways in which it isn't exactly like Chocolate Doom (e.g., some efforts are required in order to emulate certain bugs, but then again this can also be a difficulty in Chocolate Doom). Nevertheless, yeah, I try to not modify original behaviors unless there's a really good reason (e.g., a crash or a hang).
In my opinion, Wolf3D engine games lack their own "Chocolate/Crispy" ports pretty badly. Any chance you could do something similar for Wolfenstein 3-D if you have the time? Rise of the Triad also is pretty lacking in the source port department (apparently none of the available software mode ports properly deal with aspect correction), and also there's the code of In Pursuit of Greed no one has done anything with so far AFAIK.
This is surely a great idea, and I can understand the sentiment. I can't really say (i.e., promise) if I'll get something done with any of the above atm, though. Also, In Pursuit of Greed looks like a really obscure game. I did get to play it, and it may be (somewhat) OK for deathmatches, but unfortunately it looks like the single player part isn't really favored (across the few players who've recently tried it). I also recall encountering a few crashes, here and there (well, maybe in the very few deathmatches I weirdly initiated about 2 year ago).

Oh yeah, this may be a bit of a surprise for the readers, but yeah: Again 2 years ago, I did get to complete the whole IPoG campaign! Truly, these respawning creatures made it difficult, but luckily there was some glitch which generally made it possible to use some kind of a rocket inventory indefinitely (i.e., its supply never really ran out).

Also, about reproducing original bugs in games, it's important to get feedback from other users who know to spot differences. This might be doable with Wolfenstein 3D, and *maybe* also ROTT, but I don't really see it happening with IPoG or even the supported Catacombs games. The availability of demo playback and/or game mods (for the original DOS versions) also has a great impact on this.

User avatar
MrFlibble
Forum Administrator
Posts: 1604
Joined: December 9th, 2010, 7:19 am

Re: Restoration of a few games' EXEs versions

Post by MrFlibble » April 30th, 2017, 8:09 am

NY00123 wrote:
April 28th, 2017, 2:19 am
Also, about reproducing original bugs in games, it's important to get feedback from other users who know to spot differences. This might be doable with Wolfenstein 3D, and *maybe* also ROTT, but I don't really see it happening with IPoG or even the supported Catacombs games. The availability of demo playback and/or game mods (for the original DOS versions) also has a great impact on this.
Personally I don't feel that the recreation of bugs is a top priority (in fact I would prefer this to be optional, similar to compatibility modes and options in Boom derivative ports). A functional port of Wolf3D which properly corrects the aspect ratio similar to BStone (and maybe with an option to disable the wall thumping FM sound) would be nice. AFAIK there is no way to force the correct aspect ratio in Wolf4SDL (which appears to be otherwise an accurate port), although I've read somewhere that Blzut3 made a custom patch, and ECWolf isn't exactly a very conservative port either. Same goes for ROTT where there is simply no software mode port to properly address the aspect ratio issue.

I've only completed the first episode of In Pursuit of Greed, and played a bit the other two episodes. The problem is that the enemies are generally pretty weak, and the challenge comes more from the randomly spawned pickups. Also, there seems to be a lot of Hexen style switch hunt in level design, sometimes excessively so. I think I did like the early v2.0 demo better as it certainly is more challenging in terms of enemies.

NY00123
4-bit nibble
Posts: 30
Joined: July 4th, 2015, 11:05 am

Re: Restoration of a few games' EXEs versions

Post by NY00123 » May 7th, 2017, 12:26 pm

OK, I want to correct a few mistakes of mine. Afterwards, I'll ask about the new macros used in the sources recreation repository.

1. First, about Blake Stone: Aliens of Gold, I mistakenly mentioned in DHW and RGB that demo no. 3 is the first to be played in version 3.00. In fact, this is the case only in the registered version. The LastDemo variable is still initialized to 3 in shareware v3.0, but the actual demo number is calculated mod 3.
2. Secondly, about the projects SODFG14A.PRJ and SODFG14B.PRJ, I mistakenly hinted that there are differences between term, in the orders objects get linked. This is incorrect. In fact, these two projects are almost identical (and hence are both similar to SODFG10.PRJ, ignoring Borland C++ version originally used). Basically, the only real difference (other than SIGNON.OBJ) is that Source Debugging is toggled "On" in SODFG14A.PRJ and "Off" in SODFG14B.PRJ.

Now, I'm wondering if my choice of new macros (like GAMEVER_WOLFREV) is not favored by some of you? Maybe there's a better approach?

I'll say that I still want these points to apply:
- Macros shouldn't be as long as they used to be (or else the compiler may trim some of them).
- There should be one EXEDEF macro per EXE, mentioned *only* in VERSION.H and VERSION.EQU. I basically think it's (somewhat) better in terms of maintenance. It's also a good way to clarify that shareware v1.1 isn't of the the same revision as registered v1.1.

The thing is, maybe I should've used names like "GAMEVER_WOLF3D_PREREG" for shareware versions 1.0 and 1.1. But then, GAMEVER_WOLFREV can be used to (roughly) describe a range of code revisions, and not necessarily specific game versions.

There's also the special case of GAMEVER_WOLFREV being compared to 19921007L in ID_PM.C. (Yeah, I could use 19921008L, but I tried to be consistent, so I always used "<=" in comparisions, rather than ">"; Admittedly a bit of an imperfect situation.)

NY00123
4-bit nibble
Posts: 30
Joined: July 4th, 2015, 11:05 am

Re: Restoration of a few games' EXEs versions

Post by NY00123 » June 6th, 2017, 4:40 pm

Think it's time for another update. Let's begin with a few initial notes:

- It turned out the Blake Stone codebase had a silly typo specific to AOG. In the AOG versions of the enemy_t enum, en_goldmorphshot was mistakenly defined instead of en_arc_barrier. It was also used instead of en_arc_barrier in AOG-specific code pieces, but luckily it didn't occur that much.
Interestingly, the EXEs *still* built as expected! Thanks to Aryan_Wolf3D for reporting this.

- Following some feedback, I've changed some stuff with the Wolfenstein 3D game versions *yet again*. Basically, instead of comparing GAMEVER_WOLFREV to mere numbers, helper macros are used, prefixed with GV_WR (e.g., GV_WR_WL1AP10). Note that there's still an exception to this rule in ID_PM.C (with the AJR bug-fix).
Sorry if this has caused more mess for anybody, but hopefully it won't happen that much start.

Now, hopefully these points will be more interesting:

- I suppose at least a few users have wondered about this, especially after an earlier reference of mine in DHW. Thanks to some assistance, you can now recreate the DOS EXE from the Wolfenstein 3D 6-episodes Japanese version!
As expected, there wasn't really much code to add, since it's already found under the "JAPAN" ifdefs. I did used that old recreated FOREIGN\JAPAN\GFXV_WJ6.H file I previously referred to.
It was almost there, *really*. Only the value of CONTROLS_LUMP_END had to be corrected. Other than these points, and the different sign-on screen, WJ6IM14.PRJ is essentially the same as WL4GT14.PRJ, just with JAPAN defined instead of GOODTIMES.

- Next, there's basically a bonus. Usually, I wouldn't cover such game's versions, since they're generally not intended for public usage. However, he following version of Wolfenstein 3D apparently started to get widely accessible even in 1992, and may even be considered another "semi legitimate" shareware release (although I won't link to it here).
That's right, I'm talking about recreating the EXE from that Wolfenstein 3D March 1992 alpha/beta build! Recreated code is now in my repository. One surprise to mention is the various occasions in which S3DNA and the alpha are grouped together. There are also a few cases in which S3DNA and v1.0 (and possibly also the alpha) are grouped, but don't recall that many of these. Also, there's a somewhat(?) interesting name for some unused variable in WL_TEXT.C (present in the alpha only); Leaving for you too see it, heh.
In addition, there's this empty stub, which I originally added to ID_VH.C under the name of VW_NullStub, while recreating v1.0. Turns out, it was (almost surely) named VW_SetDefaultColors. Reason is that is called from MM_ShowMemory in Catacomb 3-D, and the code is almost the same in that Wolf3D March 92 build. Furthermore, except for VW_SetDefaultColors, this implementation of MM_ShowMemory is also present in Keen Dreams, albeit access to it from KD_MAIN.C was originally disabled in the sources (using FRILLS). There are also other cases in which some alpha code is similar to Keen Dreams and/or Catacomb 3-D code piece.

- Finally, I've added a new "HOWTO.md" file, partially describing some points about the process of recreating EXEs built with Borland C++ 2.0/3.0/3.1.

LATE POST EDIT (June 7 2017): Looks like I've forgotten to mention one more thing. Did you know that I originally started this codebase as a git repository hosted by GitHub? Well, this is what happened, basically hosting the Catacomb Abyss sources, modified to recreate CATABYSS.EXE from shareware version 1.13 (QA [0]). Just a week later, though, I moved to a Mercurial repository hosted by BitBucket, and added a few Wolf3D and SOD versions (Wolf3D shareware v1.4, SOD demo v1.0 and the Activision versions).
I did get rid of the original git repository as originally hosted on GitHub. Luckily, though, I've still had a local copy of this repository. It is now up on BitBucket in the following link (note it's still a git repository, *not* a Mercurial one): https://bitbucket.org/NY00123/game-srcc ... it-legacy/

User avatar
MrFlibble
Forum Administrator
Posts: 1604
Joined: December 9th, 2010, 7:19 am

Re: Restoration of a few games' EXEs versions

Post by MrFlibble » June 10th, 2017, 6:09 am

NY00123 wrote:
June 6th, 2017, 4:40 pm
- Next, there's basically a bonus. Usually, I wouldn't cover such game's versions, since they're generally not intended for public usage. However, he following version of Wolfenstein 3D apparently started to get widely accessible even in 1992, and may even be considered another "semi legitimate" shareware release (although I won't link to it here).
That's right, I'm talking about recreating the EXE from that Wolfenstein 3D March 1992 alpha/beta build! Recreated code is now in my repository.
Does this mean that you can produce a recreation of the original alpha executable without all that "keeper of the alpha" nonsense hacked into it? Or was the replaced text in the data files, not in the EXE?

NY00123
4-bit nibble
Posts: 30
Joined: July 4th, 2015, 11:05 am

Re: Restoration of a few games' EXEs versions

Post by NY00123 » June 10th, 2017, 2:57 pm

MrFlibble wrote:
June 10th, 2017, 6:09 am
Does this mean that you can produce a recreation of the original alpha executable without all that "keeper of the alpha" nonsense hacked into it? Or was the replaced text in the data files, not in the EXE?
I don't recall encountering anything to do with the BBS or other such non-original text, so I conclude it's only the data which was changed.

What is confusing, though, is there are currently (at least) two "distributions" of the data, which are (somewhat?) commonly scattered around:
1. In one of them, VSWAP.WL1 is dated March 11 (apparently ~1 hour before the beginning of March 12); The VGAGRAPH, MAPTEMP and AUDIOT related files are all dated March 12, and so is WOLF1V.EXE. There's an additional "TEXT" file from March 26 mentioning some BBS, and the rest of the files (new or edited) are all dated April 18.
2. There's a later distribution, with all files dated "April 17 2003", except for README.TXT which is dated "April 22". The included CONFIG.WL1 files differ, as there seem to be differences in the high scores entries. HELPART.WL1 has some additional pages of text, and a couple of lines at the end of it (with a mention of Gramcracker) are gone. The file "TEXT" is also gone, with its contents moving to somewhere within README.TXT. There are additional 9 saved games. VGAGRAPH.WL1 differs (details below). WOLF1V.EXE differs as well, having 5 extra bytes appended to its very end, with the text "MsDos".

As for VGAGRAPH.WL1: It turns out that in the alpha, this file has a few 16x16 tiles of the same kind used in Keen 4-6 (and Dreams), just VGAed.

There are also two variants of these again: Unmasked, and masked. The unmasked ones appear to be used in the overhead map (F10+O in the alpha or Tab+O in v1.0). There's also similar code in Catacomb 3-D.

Both sets of tiles were apparently intended for use in TED5. However, one of the masked tiles (probably the SS pointing west, spawning on every difficulty) was mistakenly relocated to offset 0, while there's corrupted data in the tile's correct/expected offset. So maybe the mappers still used EGA graphics with TED5, as hinted by the way TED5 was offered for ROTT map editing.

Comparing again the above two distributions of the alpha: It looks like minor edits were applied to a few 16x16 tiles, unmasked and masked, in the later distribution.

WDC can reportedly be used to view the contents of VGA*GRAPH.WL1, including the tiles. I've also modified ModId (an xGAGRAPH importer+exporter based on (l)modkeen), so it can be used to import and export such tiles.

Post Reply