IRC logs for #opendune on OFTC at 2009-09-13
⏴ go to previous day
02:31:19 <proyvind> btw. regarding legal issues
02:31:46 <proyvind> the dune trademark does at least belong to the dino de laurentiis company
02:32:24 <proyvind> I think you might possible run into issues on that end if you attract enough attention..
02:32:58 * proyvind changed the name from dune legacy to doon lunacy to be on the safe side himself..
02:35:50 <proyvind> or is it perhaps herbert properties, llc who holds the trademark? hm..
02:36:34 <proyvind> if you search through the dune2k forums you'll find others experiencing various legal issues etc...
06:49:48 <nsz> if you have enough attention then legal issues does not matter
06:50:00 <nsz> see google youtube itunes ...
08:44:38 <TrueBrain> proyvind: tnx for the info :)
08:49:34 <TrueBrain> reading a few posts on dune2k .. they really don't have a clue about overlays :p I read the wildest stories :) I guess that is an advantage of doing it what I did, looking at it in runtime, instead of looking in IDA :)
08:53:16 <TrueBrain> installing mingw32 ...
09:15:45 <TrueBrain> there, made MarkJ happy (I hope :p)
09:22:11 <TrueBrain> a good morning to you too :)
09:29:10 <Xaroth> TrueBrain: I'm still stunned about the amazing thing you did... i think I've ran the intromovie around.. 40 times already since waking up :P
09:29:22 <TrueBrain> you are a bit sick :p
09:29:29 <TrueBrain> I will prepare a version for you that runs the whole intro :)
09:30:11 <Xaroth> I'm crazy, yes, but not sick :P
09:30:18 <Xaroth> Overjoyed with happyness :)
09:30:30 <TrueBrain> I have to say, I am happy about it too :)
09:30:39 <Xaroth> I'm just a bit more.. OTT :P
09:31:24 <TrueBrain> okay, compiling fails ..
09:31:34 <Xaroth> hrnf, unubtu wants to update :o
09:32:34 <TrueBrain> hmm .. ncursus doesn't work on Windows
09:40:49 <TrueBrain> so ... porting to Windows is going to be more tricky :(
10:00:06 <TrueBrain> k, 'fixed' most of it, now I am only missing itimerval on windows ..
10:05:37 <TrueBrain> I typed something .. where did it go :s
10:05:52 <TrueBrain> I often type in the wrong window, and don't notice it :p
10:16:36 <TrueBrain> libemu.dll compiled
10:16:40 <TrueBrain> no idea if my timer works .. lets find out
10:18:19 <TrueBrain> For windows it is funny ... I create a thread which triggers a signal every N time :p
10:19:50 <TrueBrain> lol, linking takes 700MB of RAM, and 100% CPU .. dunno if it ever finishes ...
10:20:50 <TrueBrain> 2 minutes ... I don't think it will finish :p
10:20:55 <TrueBrain> nice bug, I guess ...
10:24:51 <nsz> for elf linking gold linker is much faster than gnu ld
10:25:16 <TrueBrain> this is COFF linking which seems to be screwed
10:25:30 <TrueBrain> i686-mingw32 target
10:33:25 <TrueBrain> whoho, after 5 minutes it finished :s
10:33:57 <TrueBrain> now it can't find the dlls ...
10:36:21 <TrueBrain> stack overflow .. hmm ..
10:39:20 <TrueBrain> there is also a TINY problem ... threads .. dual core ...
10:43:57 <TrueBrain> so .... I don't know how to solve this :s
10:47:11 <Xaroth> then.. we'll work from linux for now?
10:52:20 <TrueBrain> hmm ... biggest public is using the Windows OS :p
10:53:11 <TrueBrain> and I know my application demands a lot of stack .. but .. I didn't expect it to be such a big problem :p
11:00:54 <TrueBrain> even if I delay everything a lot the stack overflows ...
11:05:10 <TrueBrain> k, Xaroth, linux only for the time being :p When I did some optimizations, the stack-demand will decrease, so maybe later it magicly works :p
11:25:20 <TrueBrain> okay, I think I have the 'resume' part to work ...
11:30:03 <TrueBrain> just there seems to be a bug in the indication if overlaying is active or not :s
11:30:37 <TrueBrain> corrupting the data :p ghehe :)
11:30:45 <TrueBrain> oh well, I can always start from scratch again .. I think :p
11:33:15 <TrueBrain> hmm .. yes .. flagging overlay yes/no can be tricky :)
11:52:14 <TrueBrain> ha, finally found what went wrong :) I forgot to recover a few values on a hardware call :)
11:53:34 <TrueBrain> k .. sadly enough this means I need to start from scratch with creating the call-graph .. oh well :)
11:57:25 <TrueBrain> hmm, I thought I was using 1.00, but in the bottom right corner it says 1.07 :p
11:58:18 <TrueBrain> something is wrong ... I lost my mouse :'(
12:16:02 <TrueBrain> I am really suprised those overlays split so nicely :)
12:19:10 <TrueBrain> okay .. what is next ... either of the 3: 1) make it possible to name functions, 2) make compiling faster (either by storing the .o or by combining files), 3) make another teaser
12:19:18 <TrueBrain> takes 1 minutes to compile now :p
12:19:54 <Xaroth> at which part can I help :P
12:19:59 <TrueBrain> oeh, I also have to hunt down what the overlay manager does, and I need to check if I can automate adding usleeps at the right place :p
12:20:05 <TrueBrain> helping me decide what I should do next? :p
12:20:48 <Xaroth> yeh, i was more aiming at which of the 3 :P
12:21:06 <TrueBrain> that is not code you can step into :p
12:21:23 <TrueBrain> well, if you like, you can read through the source published, and try to figure out what function does what
12:21:33 <TrueBrain> but that might be very tricky
12:22:02 <TrueBrain> f__NNNN\tmore_nice_name <- is what I will make as file-format to put a bit readability in all of this
12:22:06 <Xaroth> I can always give it a go :P
12:22:25 <TrueBrain> the code currently lacks a good deal of comments which can guide you through that process
12:22:36 <TrueBrain> like it doesn't say it does an interrupt call, it just pushes a few things and does a call
12:22:52 <TrueBrain> I know it is an interrupt call, as I know how they look, but I guess I should add that as comment ;)
12:23:19 <Xaroth> is it possible for the decompiler to add such comments?
12:23:29 <TrueBrain> he knows it is an interrupt call
12:23:40 <Xaroth> that should help quite a bit then :)
12:24:05 <TrueBrain> I guess it could even tell what it is calling, like: int 0x21, DOS interrupt, AH 0x3D, Open File
12:24:19 <Xaroth> that would be even more kickass :P
12:24:27 <TrueBrain> but it takes time to write that :)
12:24:37 <TrueBrain> and currently it is on low priority, as I want this thing to work first :p Ghehe :)
12:34:49 <Xaroth> anyways, 3 would mean more interest from others, 2 and one are improvements for the project
12:37:54 <TrueBrain> for now I have some weird data ... jumps from places where is no code :p
12:59:04 <TrueBrain> okay, I understand the problem .. now a solution :p
13:37:02 <TrueBrain> okay, I need a way to tell the decompiler which functions we defined ourself
13:37:15 <TrueBrain> I mean, say I defined f__0070_0040 myself
13:37:22 <TrueBrain> I want it to use thatone, and not the one he generates
13:37:32 <Xaroth> double table with pointers to the replacement functins?
13:37:35 <TrueBrain> so either I can do that via a text-file, which might be hard to keep track
13:37:37 <Xaroth> if in table, call, else, call standard
13:37:43 <TrueBrain> or I can make a header ..
13:37:47 <TrueBrain> tables will be HUGE :p
13:37:56 <TrueBrain> but it can be staticly solved
13:38:26 <TrueBrain> so I was tihkning like: /**\n* Function blabla text here\n * Replaces: 0070:0040\n * Replaces: 0070:0042 ....
13:39:34 <TrueBrain> when creating the src/decompiler/* files, it first checks some other dir, say, src/converted or what ever
13:39:51 <TrueBrain> finds all the 'Replaces' entries, and doesn't do the decompiler step for them
13:40:17 <TrueBrain> but my quesiton in this: is that maintainable?
13:43:31 <Xaroth> you know more about it than me :P
13:44:50 <TrueBrain> something like that
13:45:07 <TrueBrain> the (ax) means that the function should be called with the ax value as first parameter
13:45:14 <TrueBrain> might be useful .. dunno if it will be :p
13:45:55 <TrueBrain> the ones not having () are functions that can no longer be called
13:48:45 <nsz> you are using comments for metadata?
13:48:53 <TrueBrain> yes; do you have any better suggestions?
13:49:22 <nsz> static counst char foo[] ? :)
13:49:43 <TrueBrain> hehe, I rather don't have it in C code, as it is only needed for the static parser
13:50:01 <TrueBrain> what I have now is a bunch of txt files, which contain the whole application mapped out, what jumps to what, what was in memory, etc etc
13:50:13 <TrueBrain> a static analyser runs over it, creating C files from it
13:50:22 <TrueBrain> of course slowly you want to replace those C files with your own work
13:50:37 <TrueBrain> but I also want to be able to rerun the analyser in case I find a missing function or what ever
13:50:54 <TrueBrain> so I need something that tells the analyser there is a replacement function and it should use thatone
13:54:10 <TrueBrain> so any suggestions? :)
13:55:05 <nsz> i don't know how much you need that info
13:55:32 <nsz> if it's only used in some code analyser then it can live in a separate file
13:55:44 <TrueBrain> but wouldn't that be terrible hard to maintain over time?
13:56:42 <nsz> if you merge functions, then can it be maintained at all?
13:56:59 <TrueBrain> well, in my above paste, I think it can
13:57:15 <TrueBrain> it is just a matter of keeping track I guess .. but at least then you have it in front of you .. brr .. I hate such design questions ;)
13:58:58 <nsz> i mean if there is a loop that zeros an array, but you replace it with memset, then memset won't have a seg:off address
13:59:20 <nsz> and eventually none of your code will have such address
13:59:26 <TrueBrain> but it does replace something that does have a seg:off ;)
13:59:40 <TrueBrain> which is the point ;) It replaces things, and the analyser needs to know about that
14:00:31 <TrueBrain> say, I replace f__0070_0040 with something else. What I want at that stage is that all calls to that function go to my new function. How to feed that info to the analyser
14:05:05 <nsz> maintain a void (*f__0070_0040)() = fnew; list?
14:06:02 <nsz> well if your analyser parses the code anyway then it is not needed to be in c
14:17:19 <nsz> then the compiler will tell you if there are namecollisions
14:17:34 <nsz> (ie the same area is assigned to multiple functions)
14:17:55 <nsz> but this looks like an ugly hack as well..
14:17:56 <TrueBrain> hmm, there is a very good idea :) But I can make the analyser also detect that based on the comment
14:21:46 <nsz> hm but then you should run the analyser often to make sure the comment syntax is correct
14:25:42 <nsz> anyway the two solutions are a search-replace distance away from each other
14:32:09 <TrueBrain> whoho, made everything MUCH faster by generating a lot less files :p
14:32:21 <TrueBrain> patching now is only a bit harder :p
14:32:35 <TrueBrain> biggest file is 200k :p
14:34:24 <TrueBrain> so, yeah, the policy will be: you copy out the function you want to implement, you rerun the analyser, and it removes all links to it
14:36:32 <Xaroth> and in the end we only have re-implemented code?
14:37:32 <TrueBrain> that is the idea ;)
14:45:52 <TrueBrain> * Decompiled function f__0070_0040_0005_1DBE()
14:45:55 <TrueBrain> * Implements: 0070:0040:0005:1DBE ()
14:46:00 <TrueBrain> ghehe, the reality is a bit less nice, with the length and crc added to it :p
14:46:15 <Xaroth> it's good enough I think
14:46:42 <Xaroth> I don't think nice is something affordable when decompiling :P
14:46:57 <TrueBrain> well, the length and crc fields are a bit nasty
14:47:01 <TrueBrain> they are RARELY needed
14:47:08 <TrueBrain> but because of those 1 or 2 instances, you need them everywhere
14:48:03 <Xaroth> it's good enough though, maybe a bit more fiddling before it works, but at least it'll work :)
14:52:31 <TrueBrain> okay, I need to patch up the overlay handlers a bit, hmm .. think think think ... IT HURTS! NO!!!!!
14:53:06 <TrueBrain> my code is becoming UGLY! :p
15:53:57 <Xaroth> playing a bit of ttd on one of them 'competative' servers.. good god some poeple don't know how to build tracks...
15:59:06 <TrueBrain> ==24643== All heap blocks were freed -- no leaks are possible.
15:59:09 <TrueBrain> I love my applications :)
16:02:19 <TrueBrain> k, rewrote the complete overlay handler
16:02:23 <TrueBrain> it is much nicer now :)
16:02:31 <TrueBrain> (well, the result, not the code generating it :p)
17:25:14 <TrueBrain> there! Done! GHA! I can now override any function :)
17:29:49 <TrueBrain> so now the hard work can start ;)
17:30:05 <Xaroth> read: Now Xaroth can do some work
17:30:07 <TrueBrain> while I was working on this, I started to wonder if it would really help to optimize the code any further
17:30:24 <TrueBrain> a human can do it 1000x better than any machine ever can :p
17:31:45 <TrueBrain> I now have a small snippet of the whole, and it is 34k
17:31:52 <TrueBrain> I guess we talk about 500k lines or something
17:32:06 <TrueBrain> (with the assumption 1 bytecode transforms into 1 line)
17:32:36 <TrueBrain> I never expected anything different :)
17:32:39 <TrueBrain> 1 to 2 years, I estimated
17:33:03 <nsz> that's about right assuming 1k line fix/day on avg
17:34:22 <TrueBrain> the executable is 371616 bytes long; lets assume 71616 bytes are data, leaving 300k of bytes. Say 2 bytes transforms to 1 line of C code, we talk about 150k lines. Then there will be tons of header-lines (which are included in the 34k above)
17:34:32 <TrueBrain> @calc 150000 / (365 * 2)
17:34:32 <DorpsGek> TrueBrain: 205.479452055
17:34:42 <TrueBrain> 205 lines per day :p Hehe :)
17:34:49 <TrueBrain> I think it will take a bit longer :p
17:34:53 <nsz> you said that the exe is compressed
17:35:07 <TrueBrain> I said my tool can handle compressed executables :)
17:35:27 <TrueBrain> when an application uses overlays, it is hard to also use compression
17:38:25 <TrueBrain> (the assembly before the line)
17:40:01 <nsz> not for me as i'm not used to asm and the c code has the same info
17:41:33 <TrueBrain> well, later I can try to decypher classic C structures, but for now I just want to work a bit with the code :)
17:41:38 <TrueBrain> like ... translating the overlay manager ;)
17:42:01 <nsz> before starting working on the c code please release the emu code
17:42:37 <TrueBrain> haha :) Why? (just wondering :p)
17:42:43 <TrueBrain> I need to find a project page .. hmm ...
17:43:04 <TrueBrain> somewhere to host it
17:43:10 <nsz> push it to some random place
17:43:46 <nsz> what's wrong with the opendune.org site?
17:43:49 <TrueBrain> Xaroth was also asking for a big slap in the face :p
17:43:54 <TrueBrain> I don't want to mangle the two projects
17:44:59 <TrueBrain> and you can only run one SVN instance per IP ...
17:45:11 <TrueBrain> oh, I guess I can upload a mercurial to my dev-space for now
17:45:57 <TrueBrain> oh, wtf, I just push the whole tool ..
17:46:05 <TrueBrain> I do need to seperate the libemu from the rest, but I will do that some other day
17:47:50 <TrueBrain> and the code is GPLv2
17:47:57 <TrueBrain> (there is no COPYING file yet)
17:48:16 <TrueBrain> any patches are welcome ;) :p
17:49:09 <TrueBrain> hg clone <url above>
17:49:21 <nsz> nah there is no hg server listening
17:49:53 <TrueBrain> lol, mercurial used my openttd.org email address ..
17:51:23 <TrueBrain> here it only gives one warning
17:51:47 <nsz> warning: deprecated conversion from string constant to ‘char*’
17:52:28 <nsz> src/libtoc/nodes/asm_jmp.cpp:111: error: ‘abs’ was not declared in this scope
17:52:47 <TrueBrain> let me guess: Debian?
17:54:06 <TrueBrain> it is known that debian flavours have another include-tree than Gentoo
17:54:15 <TrueBrain> in result you will receive such problems :) Just include which ever header is needed ;)
17:54:21 <TrueBrain> and let me know the final diff ;)
17:54:34 <nsz> i don't care about debian or gcc or..
17:55:12 <TrueBrain> standards or not, both windows mingw include-tree as gentoo include-tree compiles this code without warning :)
17:55:25 <TrueBrain> it is debian which .. well .. thinks he knows better :) Ghehe :)
17:55:56 <TrueBrain> oh, do compile in 32bit mode btw
17:56:00 <nsz> you haven't included the standard header where abs is defined
17:56:00 <TrueBrain> and you need TCC, in 32bit mode :)
17:56:06 <nsz> i have only 32 bit machines
17:56:39 <TrueBrain> I really wonder why it does compile here ;)
17:58:43 <TrueBrain> I hate it why some compilers don't warn about important things :(
17:58:53 <TrueBrain> like me using a 8bit value as if it was a 16bit :(
18:00:01 <TrueBrain> ah, gcc 4.3 complains .. gcc 4.1 works :p
18:00:15 <nsz> gcc -ansi -pedantic -Wall -Wextra
18:00:29 <nsz> you can't use gcc extensions then though ;)
18:00:48 <TrueBrain> I shouldn't be using them, or not much anyway :p
18:01:23 <nsz> like unnamed struct in union, or attribute(used)
18:05:32 <TrueBrain> this is weird ... netbeans shows no errors, and 'make' in console does ...
18:06:16 <TrueBrain> ah, wait .. I am still partly using gcc 4.1
18:08:37 <Xaroth> TrueBrain: so, what's the plan now then? :o
18:08:46 <TrueBrain> fixing those errors ;)
18:08:56 <Xaroth> what errors, in libemu?
18:09:28 <Xaroth> more idling time for me \o/
18:09:56 <nsz> im not sure if the zm thing is right
18:10:18 <TrueBrain> that warning can safely be ignored
18:10:35 <TrueBrain> are all of those includes needed?
18:10:42 <TrueBrain> stdarg is already included by node.hpp, so why in node.cpp?
18:12:59 <TrueBrain> let me add -Wextra (-Wno-unused-parameter)
18:13:42 <TrueBrain> the compiler of a friend of mine showed more errors ... namely in asm_mov.c, I used a 8bit value in a 16bit case-switch
18:13:55 <TrueBrain> I wonder why they don't show up here :s
18:14:31 <nsz> integers are casted upwards
18:14:47 <TrueBrain> this is like: uint8 v; if (v > 0x100)
18:15:20 <TrueBrain> check yourself ;) src/libtoc/nodes/asm_mov.cpp
18:23:01 <TrueBrain> nsz: let me know if the tool turns out to be anything useful for you :)
18:23:07 <TrueBrain> now I need to get Xaroth busy ... ;)
18:23:19 <TrueBrain> oh, and nsz, the tool is very .. error prune, so to say
18:23:24 <TrueBrain> it doesn't do 90% of the checks it should do :)
18:23:32 <TrueBrain> (like checking if files exist, dirs exist, ...)
18:23:49 <TrueBrain> but as said, the application needs a good cleanup :)
18:24:12 <TrueBrain> (it is a very typical application of things build on eachother, without trying to generalize anything :p)
18:24:27 <TrueBrain> at a certain point you only start to care about getting it to work :p
18:24:58 <TrueBrain> Xaroth: okay ... now an approach for Dune2 ..
18:25:14 <TrueBrain> I will run the JIT a few times and try to generate as much as C code as possible, and push that in the SVN
18:25:25 <TrueBrain> it won't work, as there are tons of minor places that should be touched
18:25:30 <TrueBrain> but most important: things need to be identified
18:25:38 <TrueBrain> I don't know if you are up for that task, but you can give that a spin?
18:26:28 <TrueBrain> oh, and nsz, as final thing: the idea of libjit.so is that you use it like: LD_PRELOAD="libjit.so" ./main ;)
18:27:48 <TrueBrain> it cleverly takes over a few fital functions :)
18:28:02 <Xaroth> but define.. 'identified' ?
18:31:42 <TrueBrain> euh .. well .. yeah .. I guess any content is as good as any other, to put in SVN now ...
18:32:24 <Xaroth> well we can create a temp svn repo that's unavailable to work from
18:32:33 <Xaroth> until we got 'something' that we're confident to be a start point
18:33:00 <Xaroth> well, 5000 useless commits that will keep showing looks a bit awkward
18:33:25 <TrueBrain> in worst case, we can trash the SVN :p
18:33:49 <TrueBrain> well, I can put it in a mercurial for now
18:40:02 <TrueBrain> with hg diff you get the patch
18:40:16 <TrueBrain> if you branch, you can always see the changes you made
18:40:30 <TrueBrain> remember ... problem with windows and stack? :)
18:44:48 <TrueBrain> so ... overlay manager :)
18:46:29 <Xaroth> added 1 changesets with 104 changes to 104 files
18:47:25 <Xaroth> :o see some "Jump does not resolve" bits
18:47:30 <Xaroth> something tells me they... don't work?
18:47:40 <TrueBrain> yes; first, if you ahve any questions, just ask
18:47:48 <TrueBrain> that comment means that the jump was never seen by the JIT
18:47:58 <Xaroth> I'm just looking around atm, see what it all looks like
18:47:59 <TrueBrain> it doesn't mean the code isn't available, but it means the JIT didn't see it
18:48:06 <TrueBrain> so it is not 100% sure it does the call (and to where)
18:48:23 <TrueBrain> normally this means the code never ever comes there
18:50:26 <TrueBrain> I think I should add a qsort :p
18:50:32 <TrueBrain> files are now not really in any order ;)
18:50:56 <Xaroth> so, the stuff in src/*.c are 'overrrides' ?
18:51:06 <TrueBrain> well, if they have the @implements, yes
18:51:14 <TrueBrain> that is where everything should go to
18:51:23 <TrueBrain> and slowly decompiled/* should get empty ;)
18:51:29 <Xaroth> so, mv decompiled/* src/
18:51:35 <Xaroth> ... only in the not-so-easy-way
18:52:20 <Xaroth> what's the difference between *stuff and **stuff?
18:52:31 <TrueBrain> the pointers, you mean?
18:52:44 <TrueBrain> pointer to a pointer ;)
18:53:05 <Xaroth> so it points to a pointer that points to data..
18:53:07 <TrueBrain> means that at 'argv' there is an array of pointers to chars
18:53:30 <TrueBrain> pointers are very simple things, but you need to get used to them :)
18:53:45 <Xaroth> I understand the concept of pointers
18:53:51 <Xaroth> just got confused at seeing double..
18:54:11 <TrueBrain> well, the same as a single pointer, only for the (*stuff) instead of the (stuff)
18:54:29 <TrueBrain> uint8 **map_2d or uint8 map_2d[][]
18:54:38 <TrueBrain> I only dislike the latter notation
18:55:05 <Xaroth> *argv[] makes sense to me :P
18:55:06 <TrueBrain> and there it gets fuzzy, as is it (*map_2d)[] or *(map_2d[])
18:55:14 <TrueBrain> so I never combine those
18:56:04 <TrueBrain> shit, auto-generated files miss a tag ...
19:01:50 <TrueBrain> oeh .. I guess I should also add in the headers where it jumped from and stuff :)
19:01:53 <TrueBrain> can be very useful ..
19:03:30 <TrueBrain> yes, lets add where calls come from .. else it will be a bit of a gamble :)
19:05:58 <TrueBrain> Xaroth: btw, all the NNNN:MMMM:XXXX:YYYY pairs you see: NNNN -> code segment; MMMM -> instruction pointer, XXXX -> length of function, YYYY -> CRC of function
19:06:05 <TrueBrain> For Dune2, the latter 2 can be ignored
19:08:14 <TrueBrain> and the pair: 0000:0000:0000:0000 means: hardware
19:10:31 <TrueBrain> hehe, the data is only sometimes a bit tricky .. :)
19:10:41 <TrueBrain> the 'Called From' properties are not always what you expect :)
19:14:30 <Xaroth> what does it mean then?
19:14:43 <TrueBrain> well, it shows where the call was coming from
19:14:53 <TrueBrain> normally this makes sense, but sometimes .. let me give you an example
19:15:32 <TrueBrain> the 'called from' is in fact set to the function which returns from the f__01F7_2A06_002A_FCED() call
19:15:41 <TrueBrain> and it is not the function you see the tail from
19:15:54 <TrueBrain> yet, it is called at that point, as the function call at the tail suggests
19:16:01 <TrueBrain> this is due to how the decompiler works
19:16:16 <TrueBrain> I can maybe correct for it ... but that is something for another week :p
19:16:32 <TrueBrain> (and you are insanely smart if you understand what I just said, as I completely failed to explain it :p)
19:16:57 <Xaroth> you lost me at the tail part :P
19:17:24 <TrueBrain> in reality what happens on in a 16bit CPU, is that at the end of the function of f__01F7_2A06_002A_FCED() it returns. It does this by setting cs and ip to the value after the point that called him
19:17:27 <TrueBrain> in this case, line 2
19:17:50 <TrueBrain> to keep track of various of important values, I designed the system to make that like a jump (not really, but similair)
19:18:07 <TrueBrain> therefor the system says the jumps is from 01F7:2B74 to 261F:004A
19:18:15 <TrueBrain> this is what the 16bit CPU does
19:18:20 <TrueBrain> in reality of course it goes via line 2
19:18:31 <Xaroth> so it can 'skip' something
19:18:49 <TrueBrain> well ... it has to do that I need to validate if the value it wants to return to, is really the position you expect it to be
19:18:57 <TrueBrain> and if not, it needs to tell you where that 'error' came from
19:19:07 <TrueBrain> I reused that validation algorithm, but it is not 100% correct :p
19:19:19 <TrueBrain> euhm ... I really should start to make sense :p
19:19:48 <TrueBrain> line 2 is 'fake'. I inserted it in the analyser. A 16bit CPU doesn't really execute that command
19:20:03 <TrueBrain> instead, at 'return' of the 01F7 thingy, it goes directly to 261F:004A
19:20:44 <TrueBrain> this is how C structures are transformed to assembly. It is very logic, and very useful, but not useful when making C out of it again
19:21:08 <Xaroth> it's logical, but not logical to understand :P
19:21:09 <TrueBrain> so in this snippet it appears that 261F:004A is called directly from 01F7:2B74, but in reality it comes via line 2
19:21:27 <TrueBrain> (but from the CPU point of view, it was called directly from 01F7 :p)
19:22:48 <Xaroth> what's that stuff in main()
19:22:58 <Xaroth> those extern calls being linked
19:23:12 <TrueBrain> euh, when the hardware wants to call something
19:23:16 <TrueBrain> it needs to know the pointer of the function
19:23:39 <TrueBrain> for example: when you move your mouse, the 2something is called
19:23:41 <Xaroth> hardware return pointers
19:23:47 <TrueBrain> but it needs to know the name of the function ;)
19:23:55 <TrueBrain> hardware _call_ pointers ;)
19:24:06 <Xaroth> returning calls from the hardware
19:24:13 <Xaroth> rather than returns from the hardware
19:25:19 <Xaroth> now to figure out the entry point where it goes from main to insanityland
19:25:31 <TrueBrain> hehe, that is not explicitly marked ;)
19:25:38 <TrueBrain> but it is the 01F7:0100 one
19:27:58 <TrueBrain> sorry, COM files, EXE (MZ) files .. so confusing :p
19:28:16 <TrueBrain> the reason that part looks so idiotic, is that it still allows the JIT to be attached
19:28:21 <TrueBrain> but it needs a bit of tricking to allow it ;)
19:28:29 <TrueBrain> else I can't resume a crash.bin :)
19:29:27 <Xaroth> now I just need a crash course in assembly/emu calls
19:29:39 <TrueBrain> it does what it say :)
19:29:43 <TrueBrain> but ask anything you don't understand
19:29:51 <TrueBrain> (nobody said this would be easy ;)
19:30:11 <Xaroth> the calling thing I get
19:31:07 <TrueBrain> emu_decb(&emu_get_memory8(emu_ds, 0x00, -0x6794)) <- that is guessable
19:31:12 <TrueBrain> and your first guess will be correct
19:31:23 <TrueBrain> the 'b' and 'w' means 'byte' (1 byte) or 'word' (2 bytes)
19:33:33 <Xaroth> emu_get_memory8 is retrieving memory at a certain location
19:33:59 <TrueBrain> 100 points for Xaroth
19:34:02 <TrueBrain> told you it was easy ;)
19:34:24 <TrueBrain> where that memory location really is, is not important :)
19:39:07 <TrueBrain> part one of overlay rewrite, done :)
19:39:16 <TrueBrain> 146 sweet lines of C :)
19:39:29 <TrueBrain> it shows a few things that are just WEIRD! Assembly can be so inefficient
19:40:12 <TrueBrain> example: ax contains the fileno. It is put on the stack for later reference, then this happens:
19:40:18 <TrueBrain> emu_dx.x = emu_get_memory16(emu_ss, emu_bp, -0x4); // Fileno of 'dune2.exe'
19:40:38 <TrueBrain> the expensive memory request is totally unneded, if it would have used 'ax', and if it would have set 'ax' just 1 line later
19:42:11 <TrueBrain> next set of code is 3224 lines long ... LOL! :)
19:43:24 <TrueBrain> I really should fix that 'Called From', it is annoying :p
19:43:24 <Xaroth> I should have spent more time learning useful stuff like C or ASM...
19:43:35 <TrueBrain> assembly is 'stupid'
19:43:38 <TrueBrain> it is really primitive
19:43:41 <TrueBrain> it does what it say
19:43:50 <TrueBrain> making sense of it, is the problem
19:44:04 <TrueBrain> emu_orw(&emu_di, emu_di);
19:44:22 <TrueBrain> it is very simple what it does there, but the implications are a bit higher. In fact what it does is check if 'emu_di' is 0
19:45:18 <Xaroth> I think at this point it's figuring out what all the variables are that's being used.
19:45:45 <TrueBrain> when you know what a function does, the rest is simple :)
19:45:55 <TrueBrain> but as you notice, things are a bit spread out
19:46:00 <TrueBrain> and the generated code isn't always that clear :)
19:46:08 <Xaroth> movX sets a value to something, right?
19:46:25 <TrueBrain> you can rewrite them always by removing the 'mov', the first &, and change the ',' in a = ;)
19:46:32 <TrueBrain> f__01F7_2A06_002A_FCED <- that opens a file. Maybe something to start with :)
19:46:50 <Xaroth> Actually I was looking at 01F7_0000 :P
19:46:56 <TrueBrain> at 0070 are interrupts, one of them is the DOS interrupt (the one with a lot of FROMs :p)
19:48:30 <Xaroth> now, a confusing one.. emu_movw(&emu_get_memory16(emu_cs, 0x00, 0x291), emu_dx.x); .. that's setting a word value to a retrieved value?
19:48:34 <Xaroth> is that like *bla = val
19:49:05 <TrueBrain> emu_get_memory16(emu_cs, 0x00, 0x291) = emu_dx.x
19:49:09 <Xaroth> it gets a memorypiece from directions, and sets it.. that sounds like a pointer being set
19:49:25 <TrueBrain> the & in this case is just a way to allow emu_movw to do that :)
19:49:43 <TrueBrain> void emu_movw(uint16 *dest, uint16 src) { *dest = src; }
19:50:11 <Xaroth> it's setting a pointer
19:51:24 <Xaroth> emu_get_memory16 returns a pointer to a certain memoryblock, right?
19:51:37 <TrueBrain> emu_get_memory16 gets the 16bit value from the memory
19:51:42 <TrueBrain> &emu_get_memory16 gets the pointer to it
19:51:48 <TrueBrain> (a lot of magic is going on there, don't ask ;))
19:57:28 <TrueBrain> sometimes things are in the details
19:57:39 <TrueBrain> 'es', 'ss', 'cs', 'ds' .. all so closely related :)
19:57:47 <TrueBrain> you might want to read about 'general' use of those values
19:57:59 <TrueBrain> people can always abuse it, but in general they are only used for certain things :)
20:01:31 <Xaroth> still a bit confused by all that, but it's slowly getting there :P
20:02:01 <TrueBrain> remember I have been looking at such code for the past 3 months :p
20:02:05 <TrueBrain> before that I knew almost nothing about it :)
20:02:25 <Xaroth> Somehow I think my brain will melt slightly faster than yours :P
20:02:33 <TrueBrain> believe me, I am now toasted
20:02:38 <TrueBrain> that is how I feel :p
20:03:01 <Xaroth> but technically right, this code compiles into something that runs?
20:03:14 <TrueBrain> it currently crashes
20:03:20 <TrueBrain> because I didn't add the required usleeps()
20:03:43 <TrueBrain> install gdb, and you can fix those yourself :p
20:03:49 <TrueBrain> wait tillit crashes
20:04:03 <TrueBrain> shows you tons and tons of the same function(s) called over and over
20:04:21 <TrueBrain> you will notice a memory compare with 0x76BN or something
20:04:24 <Xaroth> actually have gdb already o_O
20:04:35 <TrueBrain> then it needs an elegant way of having an usleep there :p
20:04:45 <TrueBrain> not sure yet what the best method is, because there are several of those places :)
20:04:50 <TrueBrain> they are all timers, counting down
20:05:39 <TrueBrain> assuming you have gcc installed
20:05:58 <TrueBrain> oh, you need btw libemu.so from the linux package
20:06:04 <TrueBrain> if it isn't changed ..
20:06:07 <TrueBrain> let me upload a recent version :p
20:06:22 <TrueBrain> and you need the data dir
20:07:11 <Xaroth> warning: incompatible implicit declaration of built-in function exit
20:07:23 <Xaroth> but it also moans about libemu, yeh
20:08:16 <Xaroth> cannot find, -lncursesw
20:09:23 <Xaroth> libncurses5 is already the newest version
20:09:46 <Xaroth> well, it wants to install something
20:09:59 <TrueBrain> you don't really need the 'w' variant
20:11:25 <Xaroth> loooootsa stuff it wants to install
20:14:41 <TrueBrain> the main problem while fiddling with this stuff: how to test you didn't damage anything :p
20:17:28 <Xaroth> Program Termination: jumped to 217E:01BA, which is not decompiled.
20:17:47 <TrueBrain> try not moving your mouse inside the application, or press any key :p
20:18:01 <Xaroth> I don't even get a new window
20:19:10 <TrueBrain> when I execute ./opendune, I get a Window ...
20:19:22 <TrueBrain> do you have the data dir there?
20:19:29 <TrueBrain> I did not compile those checks in :p
20:19:57 <Xaroth> hm, lemme check if everything's there
20:20:50 <Xaroth> oo, some file not found error now
20:21:38 <TrueBrain> you should have a Makefile, data, decompiled, include, libemu.so, memory, opendune and src file/dir
20:23:40 <TrueBrain> what does it say exactly?
20:25:24 <TrueBrain> you don't have the source of libemu there, so it can't show you :)
20:25:51 <TrueBrain> and all the same, right? :)
20:26:18 <TrueBrain> open the files, and you will see it is waiting for the clock :)
20:26:27 <TrueBrain> I believe 39 in this case
20:28:15 <Xaroth> don't see a cs__2533.c tho :o
20:28:40 <Xaroth> might be because file browser isn't sorting
20:30:12 <Xaroth> 30 calls 24, 24 calls 29
20:30:30 <TrueBrain> and one of them does a compare against a 72Bsomething
20:31:14 <Xaroth> 30 does some emu_get_memory16s
20:31:51 <TrueBrain> put an usleep(0) above the function, recompile, and you will see it doesn't fail there anymore :p
20:31:54 <TrueBrain> just that isn't a real solution :)
20:32:07 <Xaroth> above as in, start of it
20:32:31 <TrueBrain> my english is failing :p
20:32:52 <Xaroth> hah, it's warning about my usleep
20:33:13 <Xaroth> and another nice error
20:33:31 <Xaroth> so this is what you've been doing to get the intro movie to work, i take it
20:33:44 <TrueBrain> but now we can make more permanent solutions
20:33:54 <TrueBrain> by understanding what those 3 functions do, and make a good C variant of it :)
20:34:14 <TrueBrain> that is another problem I am trying to fix :p
20:34:23 <TrueBrain> that is the overlay manager inserting calls to itself
20:35:12 <TrueBrain> slowly I am unrolling what it does, and why
20:35:33 <TrueBrain> f__01F7_02C9_0001_0196 <- I need to know what thatone does :p
20:36:43 <TrueBrain> I haven't opened it :p
20:36:52 <TrueBrain> it immediatly returns
20:37:15 <TrueBrain> yes, that is part of a 'retf' in assembly :)
20:37:21 <TrueBrain> one of those default constructs :)
20:37:40 <TrueBrain> then I completely and totally miss why it does that ..
20:37:46 <TrueBrain> I guess it is a method to add a hook to it ...
20:38:35 <TrueBrain> * Called From: 217E:0551:0020:0368 ()
20:38:37 <TrueBrain> * Called From: 217E:0551:0017:FBD1 ()
20:38:47 <TrueBrain> same address, but different length/crc
20:39:05 <TrueBrain> no, it is the overlay manager (217E)
20:39:20 <Xaroth> overlay 2 modifying overlay 1?
20:39:47 <TrueBrain> oh, lol, nevermind, I get it :)
20:39:53 <TrueBrain> the values only are not :)
20:40:21 <TrueBrain> k, removed that function .. I love doing that
20:44:18 <TrueBrain> sometimes things are really magic to me :)
20:44:34 <TrueBrain> Xaroth: this will take a long long time :p
20:44:41 <TrueBrain> I hope you have the time :)
20:45:13 <TrueBrain> emu_ax.l = emu_get_memory8(emu_es, 0x00, 0x1A);
20:45:16 <TrueBrain> emu_ax.x = (int8)emu_ax.l;
20:45:17 <TrueBrain> emu_andb(&emu_get_memory8(emu_es, 0x00, 0x1A), 0xF7);
20:45:22 <TrueBrain> stuff like that ... it makes my head spin :)
20:45:31 <TrueBrain> 'es' is the overlay table header
20:45:53 <TrueBrain> the weirdest part most likely is that it doesn't do anything with the value :p
20:53:45 <nsz> i just realized that if i hide the window the simulation is much faster
20:54:08 <TrueBrain> the video-driver is not one of the fastest ;)
20:54:20 <TrueBrain> every N msecs it redraws the complete screen :p
20:54:48 <nsz> btw i had to tweak stuff a bit
20:55:15 <nsz> i used gcc flags ./libfoo.so flags
20:55:24 <nsz> gcc flags libfoo.so flags
20:55:33 <TrueBrain> it does ./libemu.so
20:55:49 <nsz> the makefile does not have the ./
20:55:57 <TrueBrain> yes, you ened to launch it with LD_LIBRARY_PATH="." :p
20:56:01 <nsz> also the working directory structure was not clear
20:56:22 <nsz> now dune2 is up and running
20:56:31 <TrueBrain> what is not clear about it? (for me everything is clear, as I wrote it :p Gheheh :))
20:56:39 <TrueBrain> with the JIT, you can play it :p
20:56:45 <nsz> well the jit it segfaulted
20:56:58 <nsz> because there was no /txt
20:56:58 <TrueBrain> this isn't the cracked version, so after lvl1 you need to give a valid phrase :p
20:57:19 <TrueBrain> I have a script which takes care of that :p
20:57:30 <TrueBrain> like it also needs a file in decompiled/ :p
21:02:46 <TrueBrain> this is like: follow the white rabbit .. pff :p
21:07:58 <TrueBrain> whoho, my first while(1) loop :p
21:09:28 <TrueBrain> I guess that f__217E_078A_001C_43A7 reads the required amount of memory needed to load the overlay
21:12:33 <TrueBrain> it should be fully playable :)
21:13:11 <TrueBrain> the mouse acts weird
21:13:23 <TrueBrain> you just have a slow computer :p :p :p
21:13:43 <TrueBrain> but yes, it tends to hang from time to time .. the JIT needs to compile :p
21:13:58 <Xaroth> gets less as you go, probably :P
21:14:06 <TrueBrain> that is the general idea :)
21:14:11 <TrueBrain> nsz: check the memory usage :)
21:14:17 <TrueBrain> for what it does, it is very low :)
21:14:58 <TrueBrain> that is not that hard :)
21:15:18 <TrueBrain> but I am pretty proud on the fact it runs with low memory, despite the fact it has both the memory as the compiled functions in its memory
21:17:09 <nsz> hey my mouse disapppeared
21:17:17 <TrueBrain> that happens from time to time yes, and I have no idea why
21:17:27 <TrueBrain> it also randomly presses buttons and shit
21:17:31 <nsz> i just had to hide the win
21:23:03 <TrueBrain> slowly I am filling in the blanks what a value does in the overlay table headers :p
21:23:05 <TrueBrain> still guessing a lot :)
21:26:11 <nsz> have you played the first level through?
21:29:32 <TrueBrain> structures getting more and more compelx :p
21:29:40 <TrueBrain> 323 lines big is my file already :p
21:30:05 <TrueBrain> given I am busy for 2 hours now
21:30:10 <TrueBrain> I can do 150 lines an hour :p
21:30:38 <TrueBrain> I noticed there are a few simple things the analyser can do, like functions that only go one way, including that already in the code
21:31:23 <TrueBrain> if you do the math, you see I have all bytes from 5A9 till 61F :)
21:35:37 <nsz> could not find file options.cfg, insert disk...
21:35:54 <TrueBrain> it might be I removed that file yes :p
21:36:00 <TrueBrain> I thought it wasn't used :$
21:37:57 <TrueBrain> uploaded new opendune-data.tar.bz2
21:38:47 <nsz> i already have 7575 files under txt/ ;)
21:39:06 <TrueBrain> and now the thing I love most about my emulator: I can launch up DOSBox and compare stuff :)
21:39:15 <TrueBrain> nsz: don't forget to pack the shit and send it to me :)
21:39:30 <TrueBrain> I need to make a tool which merges those .txt files :)
21:40:19 <nsz> i'll play a bit more with it
21:40:29 <TrueBrain> play as long and as often as you like ;)
21:40:35 <TrueBrain> the more you do, the more is mapped :)
21:40:40 <TrueBrain> (including missing files ;))
21:41:44 <Xaroth> that means there's work to do for TrueBrain :P
21:41:54 <TrueBrain> normally when you do a call, it returns on the instruction after the call
21:41:58 <TrueBrain> but in some cases that doesn't happy
21:42:05 <TrueBrain> then it puts a fuzzy return on the stderr
21:42:09 <TrueBrain> as I don't have a real method to handle those
21:42:32 <TrueBrain> as you can see it is always a 3[45]NN:0000 and 3[45]NN:0002
21:42:38 <TrueBrain> (besides the first one at 01F7)
21:42:44 <TrueBrain> this is the overlay manager sneaking in
21:44:06 <nsz> it does not end with 0000/0002 here
21:44:16 <TrueBrain> it is a cross-pair every time
21:44:54 <TrueBrain> Fuzzy return at depth 0. Expected E4F5:20C0, but got 34B8:0000 from 0642:075C
21:44:55 <TrueBrain> Fuzzy return at depth 0. Expected 34B8:0002, but got E4F5:20C0 from 261F:01B9
21:45:02 <TrueBrain> or so I hope anyway, as else there is more work to do :)
21:45:19 <TrueBrain> 261F here is the overlay manager, and 34B8 is the overlay header for the E4F5 overlay :p
21:52:33 <TrueBrain> I almost have the whole overlay shit mapped out :)
21:52:42 <TrueBrain> it works nicely, with DOSBox for the in-memory view, on runtime
21:52:45 <TrueBrain> and the code for a static view :)
21:52:51 <TrueBrain> but .. it is slow process this way :)
22:05:16 <TrueBrain> I have a piece of code here, I can't figure out what it does ...
22:05:27 <TrueBrain> it checks something of the memroy size, but I can't put my finger on what it does exactly ...
22:14:56 <TrueBrain> hmm, the missing options.cfg is 'ignored' at startup :p
22:15:03 <TrueBrain> can have resulted in random behavoir :p
22:22:58 <TrueBrain> dune2 keeps stats about its overlay uses :)
22:23:04 <TrueBrain> which is mostly: cache hits and cache misses
22:23:14 <TrueBrain> looks very poor, the performance :p
22:24:52 <TrueBrain> lol, Nyer so misunderstood something :)
22:33:20 <TrueBrain> I am really trying to understand this overlay stuff, but it is a bit weird
22:33:28 <TrueBrain> when you call an overlay, it loads it in the memory
22:33:52 <TrueBrain> then when another overlay is loaded, the overlays in the first 0x12B0 bytes are unloaded, but kept on disk
22:34:03 <TrueBrain> when they call again while still on disk, they are directed to there directly
22:34:19 <TrueBrain> when there is need for memory, overlays are removed from memory too
22:34:30 <TrueBrain> I don't get where this 0x12B0 values comes from, and what it has to do with all this ...
22:39:12 <TrueBrain> @calc 0x448b - 0x34f1
22:39:43 <TrueBrain> there is just 0xF9A bytes of memory for overlays .. the biggest is 0x24D
22:45:37 <TrueBrain> okay, MORE than enough for one day :)
22:45:47 <TrueBrain> I should be in bed already ... I have classes early in the morning
22:45:50 <TrueBrain> I wish you all a good night :)
23:09:21 <nsz> Could not find file 'SAVEFAME.DAT'. Insert disk 0 containing 'SAVEFAME.DAT' into any drive
continue to next day ⏵
