Read on Twitter
So Doom (1993) has a neat bit of encryption in it. It's not very strong encryption, but it's still encryption.
And it's not used in any sort of way you'd normally expect: not copy protection, or multiplayer anti-cheat, or anti-tampering on saves... It's to slow down FAQs.
And it's not used in any sort of way you'd normally expect: not copy protection, or multiplayer anti-cheat, or anti-tampering on saves... It's to slow down FAQs.
So here's the code I'm talking about, the macro SCRAMBLE
It looks annoyingly complicated but it's not, really.
It's taking an 8-bit number and shifting around some of the bits.
It looks annoyingly complicated but it's not, really.
It's taking an 8-bit number and shifting around some of the bits.
If you diagram out what's happening, it makes slightly (BUT ONLY SLIGHTLY) more sense.
It kinda looks like they started with a a "reverse the order of these bits" function but then switched it so the 4 and 32 don't get switched, they just go straight through.
It kinda looks like they started with a a "reverse the order of these bits" function but then switched it so the 4 and 32 don't get switched, they just go straight through.
So, how is this code used?
Well, in m_cheat.c, it's used to build a lookup table which has all the values pre-encrypted, so it can quickly look them up later. Then, when you press a key, it translates it through this table:
Well, in m_cheat.c, it's used to build a lookup table which has all the values pre-encrypted, so it can quickly look them up later. Then, when you press a key, it translates it through this table:
This code mainly takes care of checking what you've typed while playing Doom, to see if it matches one of the cheat codes, like IDKFA or IDDQD. If what you typed matches one, it returns a true value to the calling code, and the game can turn on (or off) that cheat.
But the interesting part is how those cheat codes are defined in the source code (and therefore the executable).
Here's one of them, the classic IDKFA:
Here's one of them, the classic IDKFA:
the // idkfa part is just a comment to the programmers reading and writing the code, the only part the compiler cares about is the "0xb2,0x26,..." part.
But why is it written out as 0xb2, 0x26? is this scancodes or something? NOPE!
But why is it written out as 0xb2, 0x26? is this scancodes or something? NOPE!
when they wrote the cheats into the source code, they pre-encrypted them with this SCRAMBLE function.
Then, when the m_cheat.c code is checking your keypresses, it also SCRAMBLEs them and checks if they match.
Then, when the m_cheat.c code is checking your keypresses, it also SCRAMBLEs them and checks if they match.
since it's comparing scrambled values against scrambled values, it doesn't need a descrambler of any sort. it just checks that they both scramble the same way. But why do they do it this way?
it turns out the pre-scrambling is the entire point.
They didn't want "IDKFA" and the other cheats to be written out in plain text in the source code.
Now, the source code wasn't released until 6 years later, so why did they care?
They didn't want "IDKFA" and the other cheats to be written out in plain text in the source code.
Now, the source code wasn't released until 6 years later, so why did they care?
because things like strings are incorporated into the binary file of the program directly. They're not compiled like the C instructions are.
So anyone could have pulled out a hex editor and started scrolling through DOOM.EXE until they found some text, like "IDKFA"
So anyone could have pulled out a hex editor and started scrolling through DOOM.EXE until they found some text, like "IDKFA"
which, because it's in the source right next to all the other cheats, would immediately tell them that the other cheats are idmus, idchoppers, iddqd, idkfa, idfa, idspispopd, idclip, idbehold*, idclev, and idmypos.
which'd obviously immediately get put into every cheats ... I almost said "website" there but there weren't very many websites in 1993.
every cheats bulletin board file and usenet FAQ.
every cheats bulletin board file and usenet FAQ.
so by pre-encrypting them in the source they made it so you couldn't quickly find the cheats by searching the EXE for cheat-like phrases.
I dunno how much this actually slowed them down.
It'd be interesting to go look through the archives of the usenet groups and see how long it took for various cheats to get mentioned.
It'd be interesting to go look through the archives of the usenet groups and see how long it took for various cheats to get mentioned.
but hackers are crafty. You'd need a debugger (and for a game running in 32bit mode on DOS those were not going to be easy to come by) but you could set breakpoints on keypress and you'd probably find the cheat-checking code pretty quick, and then figure out how they're encoded
I checked usenet. And the first result in comp.sys .ibm.pc .games.action (where most Doom discussion was happening) for "idkfa", was at 8am on December 15th, 1993.
Elias 'CaveMan' Papavassilopoulos posted this list of cheats:
Elias 'CaveMan' Papavassilopoulos posted this list of cheats:
That thread, titled "idNews: sorry- not tonight" by David Taylor, seems to be the first time any Doom cheats were mentioned on usenet. https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/ArY43Es_1bw/uZilNsl8EwkJ
Doom's shareware episode was released at midnight on December 10, 1993.
So the answer for "how much did this slow down hackers?" seems to be "at most, 5 days"
So the answer for "how much did this slow down hackers?" seems to be "at most, 5 days"
it's possible they were mentioned elsewhere on usenet prior to that post, I wasn't able to really search.
That seems to have been the main usenet group for Doom stuff at the time, so it would have been quickly cross-posted had it shown up elsewhere first.
That seems to have been the main usenet group for Doom stuff at the time, so it would have been quickly cross-posted had it shown up elsewhere first.
There was later a whole set of Doom-related newsgroups for Doom, but they didn't start getting created until January, 1994.
https://doomwiki.org/wiki/Usenet_groups
https://doomwiki.org/wiki/Usenet_groups
BTW, it's amusing to see that the THE VERY SAME PLACE that excitedly talked about this new game, "Doom", back in 1993, is now talking about Outer Worlds. This post is like 30 minutes old.
Usenet never died, it just slowed down.
Usenet never died, it just slowed down.
BTW, 3 hours after Elias 'CaveMan' Papavassilopoulos posted their list of cheats, Jim Mcnicholas posted their own... in a suspiciously similar order and phrasing.
J'ACCUSE, JIM MCNICOLAS!
J'ACCUSE, JIM MCNICOLAS!
you totally ripped off Elias's list and then didn't credit them. ADMIT IT
David Taylor (of id Software) would confirm the cheats (and add on to, since hackers hadn't found iddt cheat yet) later that day, around 5pm
that was supposed to have a link: https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/QL8WehGZhps/7GRT3t0N514J
An interesting thing about finding the cheats:
As early as the 11th, someone (Eugen Wolwod) had spotted the activation messages for the cheats in the executable, but couldn't find how to activate them yet.
https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/hDUPzSnmT1Q/Waj4S9D1viwJ
As early as the 11th, someone (Eugen Wolwod) had spotted the activation messages for the cheats in the executable, but couldn't find how to activate them yet.
https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/hDUPzSnmT1Q/Waj4S9D1viwJ
and on the 12th, people were hacking their save game files to get all weapons, even though they didn't have any cheat codes yet:
https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/hDUPzSnmT1Q/0y3Qiv8bhHUJ
https://groups.google.com/d/msg/comp.sys.ibm.pc.games.action/hDUPzSnmT1Q/0y3Qiv8bhHUJ
The fact that they found longer ones like choppers & spispopd makes me think that the cheats were finally figured out by someone debugging the EXE (rather than just typing key combinations until they found something that worked), and they missed iddt because it's elsewhere
all the other cheats are in st_stuff.c, but iddt is in am_map.c
https://github.com/id-Software/DOOM/blob/77735c3ff0772609e9c8d29e3ce2ab42ff54d20b/linuxdoom-1.10/am_map.c#L287
https://github.com/id-Software/DOOM/blob/77735c3ff0772609e9c8d29e3ce2ab42ff54d20b/linuxdoom-1.10/am_map.c#L287
and compilers being what they are, that's likely going to end up somewhere else in the executable.
(They would have found it if they did a search on what code calls cht_CheckCheat, and checked each of the arguments)
(They would have found it if they did a search on what code calls cht_CheckCheat, and checked each of the arguments)
but they probably didn't think they needed to do that.
They knew how it was encoded from looking at cht_CheckCheat, and then followed it back to the group of cheats all in a row, from st_stuff.c
They knew how it was encoded from looking at cht_CheckCheat, and then followed it back to the group of cheats all in a row, from st_stuff.c
this is a serious problem in this kind of software archaeology:
it's often easy to prove something exists in the code, and to find more related things, but much harder to be sure you found them all, or prove they doesn't exist.
it's often easy to prove something exists in the code, and to find more related things, but much harder to be sure you found them all, or prove they doesn't exist.
it's one of those known-knowns/known-unknowns/unknown-unknowns sorts of thing: You can't find the thing you missed because you don't know that you need to look for it.
you find 5 things, does that mean you found all 5 things there are? or that you were only able to find 5 out of any number?
cross-group searching doesn't work terribly well in google groups so I thought I'd check usenet archives on internet archive and was shocked to see "alt.dolcett" on the front page. dolcett, if you don't know (DO NOT GOOGLE IT) is an obscure gore-porn artist from like the 80s.
and I was like "why is that one so popular?" and then I remembered: Oh yeah, one of the users who posted there ate someone.
That'd be Armin Meiwes, a German who ate another German back in early 2001.
It was, uh, "consensual" (if you can say that about CANNIBALISM), but he's very in jail now.
https://en.wikipedia.org/wiki/Armin_Meiwes
It was, uh, "consensual" (if you can say that about CANNIBALISM), but he's very in jail now.
https://en.wikipedia.org/wiki/Armin_Meiwes
anyway, how's THAT for a Foone-trademark ADHD tangent?
"Boy isn't this part of a DOS game's code interesting? and how did usenet figure out the cheats? Also, did you know a german guy ate someone for sexual reasons back in 2001?"
"Boy isn't this part of a DOS game's code interesting? and how did usenet figure out the cheats? Also, did you know a german guy ate someone for sexual reasons back in 2001?"
anyway, I didn't mean to end it there ,but it looks like there's not a good online interface to the other archives of usenet stuff that'd let me do a cross-group search based on time, so I'm gonna have to give up on this for now.
I don't have time or space to work on it
I don't have time or space to work on it
but it's entirely possible (if not likely) that the cheat codes in Doom were discovered in another usenet group first, or off-usenet in some other less-well-archived place, like IRC or a BBS.
but at the very least, we have an upper bound on how long this obfuscation slowed down hackers:
5 days!
5 days!
it's interesting that id left the obfuscation in the code even in later versions. The first doom update (1.1 on the 16th) was posted after fans had discovered all the cheats, and removing obfuscation would have lowered the memory requirements of the game by 260 bytes.
but I guess:
1. even in 1993, that's not THAT much
2. most people didn't have usenet access, and this kept them from figuring out the cheats on their own.
1. even in 1993, that's not THAT much
2. most people didn't have usenet access, and this kept them from figuring out the cheats on their own.
Plus regarding #1, the official memory requirements of Doom were "4 megs of ram".
Doom actually uses about 3.5mb of RAM, but you were unlikely to actually have that amount. So it had ~500k of spare memory, 1/4th of a kilobyte isn't worth worrying about.
Doom actually uses about 3.5mb of RAM, but you were unlikely to actually have that amount. So it had ~500k of spare memory, 1/4th of a kilobyte isn't worth worrying about.
and this was DOS: it's not multi-tasking. It's not like how where if Firefox uses 400mb less, that 400mb can now go to running Sublime Text.
With DOS, DOOM was all that was running, there's nothing else going that could use that free memory.
With DOS, DOOM was all that was running, there's nothing else going that could use that free memory.
(strictly speaking that's not EXACTLY true. There's TSRs, terminate-and-stay-resident programs, which were widely used for DOS device drivers and such)
BTW, Elias Papavassilopoulos would go on to make a Doom wad (with Steve McCrea and Simon Wall) called The Unholy Trinity in August of 1994.
It was a level based on Trinity College in Cambridge.
https://doomwiki.org/wiki/The_Unholy_Trinity
It was a level based on Trinity College in Cambridge.
https://doomwiki.org/wiki/The_Unholy_Trinity
Trinity College is part of the University of Cambridge, and Elias Papavassilopoulos was posting from a University of Cambridge email address.
and if you search their name, you find a linked in for a Elias Papavassilopoulos who got a Master of Arts at the University of Cambridge, studying from 1988 to 1997.
They're now a managing director at Barclays Investment Bank in London.
They're now a managing director at Barclays Investment Bank in London.
It's been pointed out that Cambridge always gives you a bachelor of arts, so that's not very useful info. The actual degree is in Theoretical Physics.
And they've now replied with some info on how they got the cheats out of Doom. (THEY WROTE THEIR OWN 32BIT X86 DISASSEMBLER!) https://twitter.com/blowupmeister/status/1189287214796161027
And thanks for reading. If you'd like to support me making future threads like this, feel free to throw me a dollar or two on ko-fi: https://ko-fi.com/fooneturing
or set up a monthly donation on patreon: https://www.patreon.com/foone
BTW, it was asked if Wolfenstein 3D worked the same way, since the original newsgroup post says they're similar.
Well, Wolfenstein 3D uses totally different cheats (they're all key combinations, not typed words) which are kinda implicitly hard to locate in a hex dump.
Well, Wolfenstein 3D uses totally different cheats (they're all key combinations, not typed words) which are kinda implicitly hard to locate in a hex dump.
But there's also a special codeword you can give the game when starting up to enable them, and THAT could be encoded. Is it, though?
Nope.
https://github.com/id-Software/wolf3d/blob/05167784ef009d0d0daefe8d012b027f39dc8541/WOLFSRC/WL_PLAY.C
Nope.
https://github.com/id-Software/wolf3d/blob/05167784ef009d0d0daefe8d012b027f39dc8541/WOLFSRC/WL_PLAY.C
BTW it's been asked "why did id Software even bother with this?" and Wolf3d might be the answer:
It's mentioned in the Wolfenstein 3D hint manual that people hacking Wolf3d caused the hidden contest in it to be called off (because it wouldn't be fair)
It's mentioned in the Wolfenstein 3D hint manual that people hacking Wolf3d caused the hidden contest in it to be called off (because it wouldn't be fair)
That contest was this weird object hidden in a maze deep in Episode 2 Level 8.
There was no equivalent contest in Doom but maybe they wanted to make Doom a little harder to hack because of how quickly people were taking apart Wolf3D and mapping it.
