IRC logs for #opendune on OFTC at 2010-02-07
⏴ go to previous day
00:05:53 <DorpsGek> SVN: truebrain (r871) -Fix (r861): grr @ self, don't commit src/main.c after running decompiler
00:06:16 <DorpsGek> SVN: truebrain (r872) -Add: named last 3 functions of ChunkFile handlers (Close, Seek, Read)
00:06:28 <TrueBrain> there, fully named B4B1, the ChunkFile handlers :)
00:06:39 <TrueBrain> 399 functions named
00:07:50 <DorpsGek> SVN: truebrain (r873) -Fix (r870): rename the files in src/ too
00:08:01 <TrueBrain> too many mistakes in too little code .. going to bed
00:08:31 <TrueBrain> at least I assume with my libtoc.so, the decompiler problems where fixed
00:09:09 <glx> but fixing my libtoc/libjit would be better :)
00:09:15 <TrueBrain> Nyerguds is one weird person ...
00:09:22 <TrueBrain> takes offense in everything and anything you say
00:13:26 <TrueBrain> I don't feel joining his ranting .. so I made a nice reply :)
00:14:31 <glx> hmm strange, if I enable voice only in dosbox I have the voices
00:14:52 <glx> if I do the same in JIT I have no voice and no new code
00:15:08 <TrueBrain> maybe because a IN or OUT gives the wrong signal back
00:15:25 <TrueBrain> and remember I did run the JIT on at least two sound drivers ones
00:15:30 <glx> but it should at least load a file somewher and call it
00:15:30 <TrueBrain> (and left out the AB00)
00:16:31 <TrueBrain> Xaroth: SVN request: sort like: first dir, then files
00:16:51 <TrueBrain> Xaroth: SVN request: get a svn log based on a dir
00:17:09 <glx> hmm right, voice has been treated already it seems :)
00:18:51 <TrueBrain> I believe r315 or something
00:19:34 <TrueBrain> it is in the logs :)
00:20:50 <TrueBrain> k, now really: night
00:41:00 <glx> oh voices are deactivated before the drivers init
00:41:13 <glx> there should be some memory check
02:15:07 *** proyvind has joined #openDune
04:20:12 *** TrueBrain has joined #openDune
04:20:12 *** resistance.oftc.net sets mode: +ov TrueBrain TrueBrain
04:25:30 *** TrueBrain has joined #openDune
04:25:30 *** resistance.oftc.net sets mode: +ov TrueBrain TrueBrain
11:29:54 *** DorpsGek has joined #openDune
11:29:54 *** ChanServ sets mode: +o DorpsGek
11:52:43 <tneo> clicked in end title screen: [EMU] INT 0x21 AL 0x4C application termination
11:55:32 <Xaroth> you completed the game?
12:07:44 <TrueBrain> and then it terminates, indeed :)
12:07:48 <TrueBrain> or at least, as far as I know :)
12:09:55 <tneo> true TrueBrain, but it should get me to the menu, not the prompt with a jump error :-P
12:10:09 <Xaroth> that EMU line isn't a jump error
12:10:15 <Xaroth> that's the emu telling you the app exited
12:10:16 <TrueBrain> jump error? The 0x4C is just a clean termination :)
12:11:47 <TrueBrain> but it should go to the main menu? Hmm ... then something is wrong indeed :p
12:16:37 <TrueBrain> tneo: no, the code explicitly terminates the game after reaching the last level
12:17:01 <TrueBrain> but yes, that could be done a bit more nice :)
13:20:12 <glx> after some more testing I think the easiest driver should be mt32mpu.adv :)
13:21:05 <glx> though C55 sounds better than XMI
13:22:18 <TrueBrain> is C55 or XMI not midi?
13:22:50 <glx> there's midi somewhere in them
13:23:31 <glx> but I need to read the source, and decompiler failed to show that
13:26:57 <TrueBrain> problem with multiple drivers :)
13:28:48 <glx> but I ran JIT in clean trunk and clean txt
13:30:10 <TrueBrain> I cant really read JIT output :p
13:30:13 <glx> anyway the first thing would be to have a working jit/toc without needing your libs
13:30:50 <glx> well it's just to show all found functions, but decompiler seemed to miss all calls to AB00
13:31:14 <TrueBrain> can happen if it can't find anything going into there
13:31:23 <TrueBrain> which is most likely due to multiple drivers already in the txt :)
13:32:05 <glx> or it doesn't understand dynamic calls
13:34:27 <TrueBrain> well, it does, but it also filters out functions which are seemly never called
13:34:32 <TrueBrain> clearly, something breaks there :)
14:10:52 *** blathijs has joined #openDune
14:27:57 <glx> finally have a working jit/toc
14:28:06 <glx> problem was how I built tcc ;)
14:29:41 <glx> now I use "./configure --cpu=x86 --cc="gcc -m32"
14:48:28 <glx> hmm using unpatched toc I get decompile AB00, but it is never called :)
14:49:08 <glx> probably because "called from" lines looks wrong
14:49:26 <TrueBrain> at least you now have the code :p
14:57:11 <glx> yes I have the code but I don't know how it's called ;)
14:57:25 <TrueBrain> well, you only need to know how it is decoded, not? :)
15:04:30 <glx> I know there's a function to tell the driver which ports to use
15:05:16 <glx> but I need to find where it's called ;)
15:05:29 <TrueBrain> remove 1DD7 and AB00, and run JIT again
15:13:49 <glx> ho I thinks it's because 2756:04F8 playing with cs:ip
15:14:00 <glx> you know fuzzy returns ;)
15:14:27 <TrueBrain> good possibility :)
15:23:56 <glx> if I understand correctly, fuzzy returns are indeed jumps
15:24:29 <TrueBrain> they are returns, but not to the place they came from
15:24:36 <TrueBrain> so, indirect jumps, with recovering the stack
15:25:01 <glx> yes original return is theorically done at the end of this "jump"
15:25:45 <glx> works well in JIT, harder in libemu ;)
15:26:42 <TrueBrain> well, the decompiler doesnt know how to handle those situations, no :)
15:27:00 <glx> it's impossible for it to handle them
15:27:39 <TrueBrain> just need special glue
15:27:44 <TrueBrain> overlays are no different
15:30:36 <glx> so I'll need to patch the function and it'll understand ?
15:31:11 <TrueBrain> well .. that is not so easy to patch up
15:32:49 <glx> this one is a push(dx);push(ax);pop(ip);pop(cs);return
15:33:22 <glx> it should be possible to add a switch there
15:33:30 <TrueBrain> but that would be manual
15:34:48 <glx> decompiler handles patched code ?
15:36:46 <glx> but I can run JIT, then decompiler, then patch code, then run decompiler again
15:37:32 <glx> with a log from JIT run I can find the fuzzy calls
15:37:42 <TrueBrain> that is, you can run decompiler again
15:37:47 <TrueBrain> but it won't magicly link the functions
15:37:53 <TrueBrain> it simply does not record fuzzy returns
15:38:01 <TrueBrain> is in my todo, but never got implemented ;)
15:38:36 <glx> well if I know what the call should be, I just add it and let decompiler find the function to decompile
15:39:05 <TrueBrain> the decompiler wont :)
15:39:28 <TrueBrain> well, it now became 'extern void f__'
15:39:30 <TrueBrain> but yes, that would work
16:23:08 <glx> lol sometimes decompiler does strange things :)
16:23:24 <glx> like adding unreachable code in 01F7
16:24:45 <TrueBrain> it makes 1 bug in 01F7
16:25:00 <TrueBrain> still not sure why .. and I am not going to fix it :p
16:25:22 <TrueBrain> that is the one, yes
16:25:45 <glx> as I said unreachable code
16:26:04 <TrueBrain> as I said yesterday, it has a bug there :)
16:27:30 <glx> first step done (JIT, decompile, patch)
16:32:53 <glx> I still have the 2B6C bug btw
16:34:20 <glx> I have cs__AB00.c now \o/
16:47:16 <glx> it runs, and all found AB00 are decompiled
16:53:22 <glx> of course for now I fake the port 0x331 in libemu so there's unresolved stuff in driver
20:59:12 <glx> yeah I have a fake MPU-401 :)
21:00:02 <glx> it does nothing except handling 0xFF on command port
21:00:51 <glx> but that's enough to get a bunch of output on data port
21:01:04 <glx> and that means it works :)
21:11:36 <tneo> glx, are you working on the sound area?
21:12:05 <glx> jit and toc are fun to use :)
21:12:52 <TrueBrain> when you know how to use it, yes :)
21:13:34 <glx> I even force the decompiler to do what I want ;)
21:13:53 <glx> with a little p__ function
21:19:53 <glx> like now, I have an undefined reference
21:21:16 <glx> an AB00 function calls an AB00 function and the decompiler did not see that
21:21:44 <TrueBrain> weird ... is recursion which is used
21:21:48 <TrueBrain> but okay ... it is broken, yes :)
21:21:51 <TrueBrain> it needs a rewrite, yes :)
21:21:56 <TrueBrain> maybe with my bachelor project
21:22:00 <TrueBrain> Thursday more on that ;)
21:26:07 <glx> let's add an extern void somewhere
21:36:15 <glx> then I run openttd, which shows a missing call in patched.c
21:36:39 <TrueBrain> try starting opendune :p :p
21:36:47 <glx> I fix it, rerun decompiler and magically the extra extern void is no longer needed
21:37:28 <TrueBrain> there is some bug .. but I already told you that :p
21:42:38 <glx> hmm the undefined reference is back
21:45:08 <glx> ho makes sense AB00:1B48 calls AB00:2336
21:45:35 <glx> decompiler probably don't look forward
21:50:12 <TrueBrain> decompiler traces all calls that are called, and handles them
21:50:14 <TrueBrain> (or at least, it should)
21:50:25 <TrueBrain> but it became so messy, code-wise ...
21:50:30 <TrueBrain> I hacked into stuff to do that analyses
21:53:33 <glx> lol it sees extern void even when commented
21:54:23 <TrueBrain> yup, it just scans for it :p
21:55:13 <glx> so a /* TODO decompiler bug: extern void f__XXX */ solves it ;)
22:00:08 <glx> hmm it seems I need to patch another place :)
22:00:59 <glx> it correctly writes the dynamic call, except it does it for AB00 when in real run it's 44AF
22:01:31 <glx> and if I fix that by hand it just reverts it
22:02:27 <TrueBrain> hmm ... weird, AB00 and 44AF ..
22:02:34 <TrueBrain> it might happen 44AF is still considered an overlay
22:02:40 <TrueBrain> but that shouldn't bump it to AB00 :p
22:03:20 <glx> or 44AF segment was not free with JIT
22:03:31 <TrueBrain> also not possible really
22:03:35 <TrueBrain> those runs should be identical
22:05:31 <glx> well it's a kind of overlay, just a permanent one ;)
22:07:22 <TrueBrain> 33E0 .. local to 1FB5 .. k .. should be easy enough
22:16:45 <TrueBrain> for a file to be opened from a PAK, these conditions have to be met:
22:16:46 <TrueBrain> if (loc06 != 0xFFFF && (g_fileInfo[loc06]->variable_12 & 0x10) != 0 && loc08 == 0 && (emu_get_memory16(emu_ss, emu_bp, 0xA) & 0x2) == 0) {
22:16:58 <TrueBrain> loc06 is the index of the fileinfo thingy
22:17:06 <TrueBrain> loc08 means: is not available via direct fopen
22:17:12 <TrueBrain> and the param is most likely the file map
22:17:38 <TrueBrain> so there is a lot of checking for absolutely no reason .. as the value for loc08 was found by opening the file, and closing it again .. why ... if you are going to use it anyway, if it was there
22:18:06 <TrueBrain> it is always checked if the file is in the pak file .. why .. if it can be opened directly
22:18:09 <TrueBrain> etc etc .. stupid function
22:20:31 <TrueBrain> glx: am I right to assume the pak files themselves are also in the array of FileInfo? (you created that, not?)
22:22:39 <TrueBrain> yeah .. the PAK files do not have 0x10 in the last byte
22:22:42 <TrueBrain> which means they can't be openend
22:23:05 <TrueBrain> 0x10 - If set, file is openable
22:26:08 <glx> .,.../* 42EB() */ PACK uint8 unknown_42EB[0x1D67]; /*!< list of all files. */ <-- file_list.txt is just that
22:26:26 <glx> I guess FileInfo array reflects it :)
22:26:30 <TrueBrain> but at two addresses?
22:26:39 <TrueBrain> yes, FileInfo has a pointer
22:28:25 <TrueBrain> it is very clear defined which file should be in which file
22:29:28 <TrueBrain> fun fact: you can have a PAK in a PAK
22:29:32 <TrueBrain> nothing in the code disallows that
22:29:48 <TrueBrain> the code is also build recursive
22:30:05 <TrueBrain> I would not have made it like that, when I would have wrote it
22:30:14 <TrueBrain> I would have made that when it was a file inside a PAK, it would open it raw
22:30:20 <TrueBrain> they open the PAK via the PAK opener
22:36:42 <TrueBrain> many inefficient routines here ...
22:36:58 <glx> I'm in a loop: add missing call in patched.c, toc, revert 2 files, fix 2 other files, compile, run
22:37:47 <TrueBrain> glx: are all the calls in AB00 you need resolved?
22:38:15 <glx> a lot has been found by JIT :)
22:38:34 <glx> just the decompiler doesn't know them
22:38:35 <TrueBrain> does the decompiled code of AB00 contain all you need?
22:39:55 <glx> at least I understand a part of it
22:39:55 <TrueBrain> then just copy it to src/, and leave the JIT for what it is :p
22:40:49 <glx> opendune still doesn't run ;)
22:41:01 <glx> I find a missing call to AB00 each time
22:41:12 <TrueBrain> okay, check this out: open file inside pak means: open pak, find filename, use FileInfo_FindIndex_ByName, see if it matches what we were looking for ...
22:41:15 <TrueBrain> why not a name match? :s
22:41:40 <TrueBrain> you should have enough power over the decompiler to make him aware of all functions
22:41:47 <TrueBrain> even if that means a bit of manual patching
22:41:54 <TrueBrain> but when you copy that to src/, it survives a rerun
22:42:14 <glx> rerun don't remove anything :)
22:42:37 <TrueBrain> you speak very cryptic, so I have no clue what you are trying or what is the problem :)
22:43:49 <glx> there's no problems, it's just the decompiler won't decompile a function if there's no reference to it, and there's not reference until opendune tells me there's an unresolved call
22:44:08 <glx> then I add it in the patched function and I restart the loop
22:44:30 <TrueBrain> ah, so it isnt an infinite loop, it just will take you a while :p
22:44:45 <TrueBrain> bit silly that the decompiler doesn't run it himself
22:44:55 <TrueBrain> more reason to make a new version :p
22:45:27 <TrueBrain> owh, running shows you it
22:47:24 <TrueBrain> I just moved 25 lines to this:
22:47:27 <TrueBrain> g_fileInfo[loc0E].variable_08 = fi->variable_08;
22:47:39 <TrueBrain> Well, more like: g_fileInfo[loc0E].variable_08 = g_fileInfo[g_fileInfo[fileInfoIndex]->parentIndex].variable_08;
22:51:12 <TrueBrain> the pak file is closed with a zero-length field I assume?
22:51:55 <TrueBrain> a lot of extra code to allow paks in paks
22:52:01 <TrueBrain> does it ever happen, and do we want to support it?
22:52:56 <glx> like tar in tar for openttd (dunno if it's supported)
22:53:23 <TrueBrain> openttd also only looks 1 level deep
22:54:07 <TrueBrain> 200 lines of code reduces to this:
22:54:49 <TrueBrain> now I need to understand the code ...
22:56:21 <TrueBrain> really ... who wrote this crap
22:56:28 <TrueBrain> they use all references mixed up
22:57:15 <TrueBrain> g_fileInfo[loc0E].parentIndex == emu_get_memory16(0x33E0, emu_di << 4, 0x8)
22:57:20 <TrueBrain> g_fileInfo[loc0E].variable_08 = g_fileInfo[g_fileInfo[fileInfoIndex]->parentIndex]->variable_08;
22:57:22 <TrueBrain> g_fileInfo[loc0E].variable_0C = g_fileInfo[emu_get_memory16(0x33E0, emu_di << 4, 0x8)].variable_0C + fileLength;
22:57:36 <TrueBrain> those things between the [] in the second and third line, are also identical
22:57:45 <TrueBrain> so I now have 3 variables here to name the same, of which one 2 are tested
22:57:52 <TrueBrain> how inconsistant can one be?
22:58:15 <glx> why not use parentIndex directly?
22:58:28 <glx> well it may be a compiler thing too
22:58:50 <TrueBrain> either way, as soon as it opens a PAK file, it reads all entries and maps them
23:01:07 <TrueBrain> ah .. now I start to understand ... unknown files in the PAK file are simply ignored
23:04:28 <TrueBrain> ah, and the other checks ensure it is not in the wrong PAK file
23:15:03 <TrueBrain> LOL! All code supports PAK in PAK, but not one line
23:15:09 <TrueBrain> it has a seek which ruins it completely, for all
23:15:28 <TrueBrain> stupid ... really ...
23:16:18 <TrueBrain> but yes, 1FB5 makes it feel like a normal file, fseek(0) does all what you expect, while the file is inside a pak
23:19:42 <TrueBrain> another thing doesnt feel right ... it seems that for every file inside a pak, it does the remapping of all files inside the pak file, instead of only once
23:21:11 <TrueBrain> or at least .. it makes one nasty assumption on it
23:27:59 <TrueBrain> when saving a file which is inside a PAK, there is some code to handle such instance
23:31:27 <TrueBrain> I fail to see how that can be useful for us, so I will disregard it
23:35:15 <TrueBrain> owh, it makes a copy of a file in a PAK file when writing, outside the PAK
23:36:01 <TrueBrain> or at least, I _think_ that is what it wants to do
23:36:16 <glx> hmm something weird happens, I have a wrong emu_cs somewhere
23:36:22 <TrueBrain> nah, strike that, as it does not check if the file already exist local
23:36:28 <TrueBrain> glx: that happens very easy, yes :)
23:36:34 <TrueBrain> for the overlay I had to do a lot of patching ...
23:36:40 <TrueBrain> lets hope AB00 does no longer segment access ;)
23:37:24 <glx> everything is ok, and suddenly emu_cs is wrong
23:37:39 <TrueBrain> you forgot an overlay() call?
23:37:50 <TrueBrain> it basically fixed emu_cs
23:40:10 <TrueBrain> lol, File_Open() already set NoInput, but, just to be 100% sure, it opens files with File_Open_NoInput :p
23:40:29 <glx> better safe than sorry ;)
23:42:54 <TrueBrain> somehow I think the dune2 developers were under the assumption that:
23:43:02 <TrueBrain> made a copy of a :p
23:59:21 <TrueBrain> ah, if a PAK entry would have been set with flag 0x2 or 0x4, it would have been read to the memory directly
23:59:28 <TrueBrain> but it never happens
continue to next day ⏵