IRC logs for #opendune on OFTC at 2010-02-02
⏴ go to previous day
09:05:01 <Xaroth> and, got blown up again yet? :P
09:07:53 <TrueBrain> sigh, why did glx not just commit :p Now I have to wait for him to get online again :)
09:11:13 <Xaroth> can't connect to warod's box, so can't get into the big irc chan :/
09:12:06 <TrueBrain> why do you bounce for that?
09:12:44 <Xaroth> Because there's a lot of rivalty in eve, and you can have people try to hack into your systems for just visiting a site
09:13:02 <Xaroth> hence, using a cond^h^h^hbounce for irc
09:13:18 <TrueBrain> then you really take a game one step too far
09:13:28 <TrueBrain> but okay .. I won't tell you it is easier to hack a bouncer ;)
09:13:39 <Xaroth> it's not a bouncer, it's a ssh box :P
09:13:58 <Xaroth> so in order to get to me, he'll have to hack -that- box, then find some way to hack back to me
09:13:59 <TrueBrain> use client certificates + SSL IRC connection, and you are safe too
09:14:25 <TrueBrain> owh, trace your home IP
09:15:23 <Xaroth> I don't mind people getting to the eve stuff, I just don't like to have to call my boss for certain reasons :P
09:15:32 <TrueBrain> yeah, that would work indeed :)
09:15:59 <TrueBrain> well, you have to be pretty good to hack an average linux box
09:16:36 <Xaroth> yeh, but still, safety first :P
09:16:44 <TrueBrain> no, you say it wrong
10:38:16 <DorpsGek> SVN: truebrain (r840) -Add: C-ified Save_Main() and Save_Chunk()
10:40:32 <DorpsGek> SVN: truebrain (r841) -Update (r840): make the decompiler happy again
10:54:48 <DorpsGek> SVN: truebrain (r842) -Fix (r840): forgot one extern declaration
11:01:30 <TrueBrain> I am going to avoid a buffer for saving stuff like houses etc
11:01:38 <TrueBrain> and just store the data direct in the file
11:01:52 <Xaroth> still there, waiting for him to get back
11:02:24 <TrueBrain> ghehe :) Lucky enough you have a phone :p
11:02:40 <Xaroth> you reckon that autosave is doable in the cirrent situation?
11:02:54 <TrueBrain> the problem about autosave is, that the naming of savegames is very strict
11:03:07 <TrueBrain> _save%03d.dat, but it has to be incremental
11:03:40 <TrueBrain> so I have to teckle loading to fix that
11:03:43 <Xaroth> yeh, but we can at some point change that restriction
11:11:34 <TrueBrain> that looks almost readable
11:12:00 <TrueBrain> dunno if it is the best solution ... hmm ..
11:12:22 <TrueBrain> we will need something like OpenTTD if we want this to advance .. but okay .. for now this works just fine
11:12:40 <TrueBrain> and much better readable then the original code .. and avoids stupid copying back and forward of data
12:29:10 <TrueBrain> wow ... the 'store map' stuff gets highly inefficient the more is on the map, and the more you uncover the map
12:40:58 <TrueBrain> okay, one chunk left ... 'INFO' chunk
12:41:01 <TrueBrain> thatone seems complicated :)
12:45:51 <TrueBrain> weird ... words are stored LE, where dwords are stored BE
12:53:47 <TrueBrain> no .. chunks are BE, data inside it are LE
12:57:26 <TrueBrain> playerCreditsNoSilo is stored twice :s
13:04:26 <TrueBrain> any patch to fix the endian stuff is fine by me; couldn't review it anymore as you were offline :p
13:05:23 <TrueBrain> please add tabs between #ifdefs
13:05:27 <TrueBrain> so it is clear what is between them and what not :p
13:05:37 <TrueBrain> and please make sure to test save/load
13:05:55 <TrueBrain> look at os/strings.h for how to tab :)
13:06:21 <TrueBrain> and please makt 'int x' 'uint32 x'
13:06:27 <glx> save seems to work, load doesn't even for mingw, but maybe it's because it's not the latest version
13:06:42 <TrueBrain> well, if load doesn't work, it means save didn't store it in BE :)
13:06:54 <TrueBrain> #if __BYTE_ORDER == __LITTLE_ENDIAN <- I don't think they have __BYTE_ORDER__ defined ;)
13:07:02 <TrueBrain> for WIN32 it is suffecient to define it as LE
13:07:22 <glx> I used breakpoints in msvc and the value is swaped
13:07:39 <TrueBrain> maybe hexedit the savegame?
13:17:05 <TrueBrain> glx: either way, the savegame format is NOT BE/LE safe .. so there is need for a lot of work anyway :)
13:19:03 <glx> Program Termination: jumped to 4367:006F, which is not decompiled.
13:19:03 <glx> The jump was triggered at decompiled/cs__B4B1.c:69
13:19:03 <glx> The jump appears to originate from B4B1:0050.
13:19:03 <glx> Please send the file 'memory/crash.bin' to upstream developers.
13:19:05 <glx> Crash-dump in 'memory/crash.bin'
13:19:15 <glx> one of the savegame was just empty ;)
13:19:23 <TrueBrain> he can't handle that :)
13:23:42 <glx> of course it works only for x86 family ;) (but same for windows)
13:23:57 <TrueBrain> Windows doesn't work on any other CPU :)
13:24:12 <TrueBrain> even WinCE requires x86 :p
13:25:50 <TrueBrain> savegames look identical, so I think I did a good job, content wise .. now I wonder coding-style wise ;)
13:26:20 <TrueBrain> except Fog stuff, I removed all emu calls for saving :)
13:27:21 <DorpsGek> SVN: glx (r843) -Fix (r839): make os/endian.h MSVC and MINGW compatible
13:32:23 <glx> asm looks more complicated than C :)
13:32:54 <TrueBrain> well, it is not really C-ification, more a rewrite :p
13:33:36 <TrueBrain> as I said, I have no real doubt about the content .. more about the coding style :)
13:40:40 <TrueBrain> the NNN_Save can be rewritten to just walk the array, instead of using the (much slower) Find variant
13:40:48 <TrueBrain> but I guess that is something for later, when making that into normal pools :)
13:44:21 <glx> it saves only non generated data
13:44:37 <TrueBrain> yes, but it is also very stupid: the more you uncover, the less efficient it becomes
13:44:49 <TrueBrain> when you see the whole map, it is twice the size then the map is big
13:45:07 <TrueBrain> so on average, I think it takes more data in this method
13:45:44 <TrueBrain> no, not twice .. 150%
13:46:06 <TrueBrain> but yes, it does tell us what g_map is / does :)
13:46:40 <TrueBrain> well, if 0x8000 is empty, yes
13:46:59 <TrueBrain> the tiles without 0x8000 indicate the generated SpriteID
13:47:07 <TrueBrain> the ones with 0x8000 are 'overrides'
13:47:45 <TrueBrain> 38FA is weird .. it is only written once ... I can't even find if it is loaded back from the savegame
13:52:03 <glx> ,...emu_push(emu_cs); emu_push(0x058D); emu_cs = 0x1A34; emu_Unit_Create();
13:52:03 <glx> ,...emu_addw(&emu_sp, 0xC);
13:52:03 <glx> ,...emu_get_memory16(emu_ds, 0x00, 0x38FC) = emu_dx;
13:52:03 <glx> ,...emu_get_memory16(emu_ds, 0x00, 0x38FA) = emu_ax;
13:52:12 <TrueBrain> already made it a Unit ;)
13:52:19 <TrueBrain> but it is stored in the savegame
13:52:23 <TrueBrain> yet I cant find where it is read :)
13:52:53 <glx> indeed that's the only place I find it set to something different than 0
13:54:20 <glx> ok loading is in B4C4 if my search skills are right
13:54:29 <TrueBrain> see src/function_names.txt ;)
13:54:47 <TrueBrain> but okay, no comments on my patch?
13:55:36 <glx> oh it reads the unit index on load (as only the index was saved)
13:56:28 <glx> ,...emu_push(0x020C); f__B4C4_054C_0031_FE26();
13:56:28 <glx> ,...emu_addw(&emu_sp, 0x8);
13:56:28 <glx> ,...emu_cmpw(&emu_si, 0xFFFF);
13:56:29 <glx> ,...if (emu_si == 0xFFFF) goto l__0224;
13:56:31 <glx> ,...emu_push(emu_cs); emu_push(0x021C); emu_cs = 0x0FE4; emu_Unit_Get_ByIndex();
13:56:31 <glx> ,.../* Check if this overlay should be reloaded */
13:56:33 <glx> ,...if (emu_cs == 0x34C4) { overlay(0x34C4, 1); }
13:56:33 <glx> ,.../* Unresolved jump */ emu_ip = 0x021C; emu_last_cs = 0xB4C4; emu_last_ip = 0x021C; emu_last_length = 0x0010; emu_last_crc = 0xFF92; emu_call();
13:56:56 <glx> that's the loading of 38FA
13:57:35 <DorpsGek> SVN: truebrain (r844) -Add: C-ified all Save routines and avoid using the emulator layer for it
13:57:42 <TrueBrain> feel free to do that now ;)
13:59:43 <TrueBrain> my decompiler keeps fucking up ... oh well
14:00:46 <DorpsGek> SVN: truebrain (r845) -Update (r844): fix up decompiler
14:08:04 <TrueBrain> now up to the Loading part ....
14:08:09 <glx> ok 38FA is related to palace weapon, saboteur and fremen
14:08:11 <TrueBrain> if I can figure that out, in detail, I can fix autosave :)
14:08:19 <TrueBrain> Only to the Harkonnen
14:08:24 <TrueBrain> it only does the Missle
14:08:31 <TrueBrain> but I wonder why it is stored seperatly
14:08:54 <glx> f__1423_04F2_0016_CD6B <-- set there
14:09:10 <TrueBrain> Unit Type 0x12 = 18 = UNIT_MISSLE_HOUSE
14:09:29 <glx> and called from GUI_Widget_SpriteTextButton_Click()
14:09:42 <TrueBrain> so: Launch Missle! :p
14:10:09 <glx> ,...,...case 0x2A: /* "Launch" */
14:10:09 <glx> ,...,...case 0x2B: /* "Fremen" */
14:10:09 <glx> ,...,...case 0x2C: /* "Saboteur" */
14:10:16 <TrueBrain> it is not only called from the GUI :)
14:10:28 <TrueBrain> that is something that function should handle then
14:10:34 <TrueBrain> Fremen are not created under unit type 18
14:10:35 <glx> yes auto fire in structure loop
14:12:09 <glx> so we just need an advanced enough save to resolve unresolved stuff in load ;)
14:13:10 <TrueBrain> nah, that will be easier to fix :)
14:25:34 <Xaroth> I know, but yer missing an I in Missile :P
14:26:20 <Xaroth> UNIT_MISSLE_HOUSE << Missle :P
14:27:19 <Xaroth> btw, I'm back home (as you guessed seeing I'm once again able to copy/paste)
14:27:36 <TrueBrain> but then I expect you to make a patch to fix it
14:27:37 <Xaroth> dad's downstairs talking with some people.. I finally figured out how they call what he had
14:27:52 <Xaroth> they call it Dissociative fugue
14:27:59 <Xaroth> worst part is, fugue is a french word :P
14:28:16 <Xaroth> that -is- how 'we' call it :/
14:29:05 <Xaroth> first paragraph of the description bit is exactly what happened
14:37:17 <TrueBrain> I am suprised they call it 'rare'
14:37:20 <TrueBrain> it is not that rare?
14:38:13 <Xaroth> this is an extreme form
14:38:18 <TrueBrain> btw, missle is how Dune2 writes it
14:38:27 <TrueBrain> well .. it sucks either way :)
14:39:24 <DorpsGek> SVN: truebrain (r846) -Fix: Xaroth has issues with how I write the word 'missile', so this should make him happy
14:41:19 <TrueBrain> Mwhahahahahahaha :)
14:41:25 <Xaroth> Missle is incorrect spelling :P
14:41:34 <TrueBrain> I just had to do it :)
14:45:04 <TrueBrain> k, emu_Load is not the true loader :p
14:45:24 <TrueBrain> emu_Load_Info would be his name :)
14:50:02 <DorpsGek> SVN: truebrain (r847) -Add: named most of the important load functions (correctly)
14:54:13 <TrueBrain> now to decypher all weird things it does while loading .. tralala ...
14:55:02 <TrueBrain> glx / Xaroth: should I make a saveload.c, or (save.c and load.c) ?
14:55:35 <glx> are there any common parts ?
14:55:57 <glx> can be split for now I think
14:57:08 <TrueBrain> hmm .. Save_Main .. maybe that should be named Save
15:02:41 <DorpsGek> SVN: truebrain (r848) -Fix: gave a few functions a bit more sane names
15:04:17 <TrueBrain> seriously ... it checks if 'filename' is NULL for Load ...
15:04:20 <TrueBrain> that is just stupid
15:05:44 <glx> hmm maybe I'll try to convert GameLoop_LevelEnd
15:07:24 <TrueBrain> the game now terminates on failure to load
15:07:31 <TrueBrain> or: failure to open the file to load
15:08:11 <TrueBrain> something for the todo list :)
15:11:00 <TrueBrain> loading is weird ... open file, open file, close file, open file, read file
15:11:16 <TrueBrain> close file, read rest of file
15:12:20 <glx> ok GameLoop_LevelEnd is called only from GameLoop_Main, so I think it can go in opendune.c
15:12:50 <TrueBrain> I am pretty sure we will be moving things around a lot, so yup :)
15:20:08 <TrueBrain> when you save a game, the Scenario data is set to 0x0290 (first 2 bytes)
15:20:21 <TrueBrain> when loading, something special happens when that isn't the case
15:20:45 <TrueBrain> no, dune.log has nothing that looks like it
15:21:42 <TrueBrain> yes, they are 0x0257
15:21:59 <TrueBrain> nobody knows where those numbers come from, but that is not important here ;)
15:22:19 <TrueBrain> nope, makes no sense :p
15:22:31 <TrueBrain> but okay ... there is a version number in the savegame, that is _very_ useful :)
15:25:54 <TrueBrain> okay ... it constantly starts from the begin of the savegame, tries to find a chunk
15:26:03 <TrueBrain> I say, lets change that: read the next chunk, and try to handle it
15:26:18 <TrueBrain> or would the order be important? Hmm ..
15:26:19 <Xaroth> might be useful to know to the version numbers of 1.00 and 1.07
15:26:26 <Xaroth> maybe introduce some backward compatibilty? :o
15:26:56 <Xaroth> uint16 version number?
15:27:22 <TrueBrain> by the looks they made tons of internal releases
15:27:56 <Xaroth> at least we'll have enough space to.. increase :P
15:28:07 <TrueBrain> rarely a true issue :)
15:28:16 <TrueBrain> one can consider to use the 0x02 as '0.2'
15:28:26 <TrueBrain> so bump that every release
15:28:33 <TrueBrain> but in OpenTTD we stepped down from that approach
15:28:38 <TrueBrain> as it turned out to be VERY annoying :p
15:28:55 <Xaroth> well, at some point we need to re-do the format anyhow
15:29:02 <TrueBrain> well, chunks are good
15:29:07 <TrueBrain> so we can keep this, that is okay
15:29:33 <Xaroth> chunks as in |info||map data||unit data||whatever data||more whatever data| etc right?
15:29:35 <TrueBrain> because of the magic header ('FORM'), we can also add compression and stuff
15:29:59 <TrueBrain> 'INFO'<uint32 length field><byte data of length size>
15:30:02 <TrueBrain> and it starts with:
15:30:08 <TrueBrain> 'FORM'<uint32 total length field>
15:30:17 <TrueBrain> one exception: 'SCEN', has no length field
15:31:39 <SmatZ> [16:28:17] <TrueBrain> one can consider to use the 0x02 as '0.2' <== having only 253 versions left?
15:31:52 <TrueBrain> SmatZ: 253 releases
15:32:02 <TrueBrain> where each can have 256 versions
15:32:11 <TrueBrain> but okay .. OpenTTD has shown that works annoying, so it was a stupid idea of mine :)
15:32:42 <TrueBrain> although it allows you to branch a release, and still bump the savegame version
15:32:51 <TrueBrain> but .. there things broke for OpenTTD, as loading that in the current trunk is a BITCH
15:36:09 <TrueBrain> hmm .. yes .. you first need to load in the INFO chunk, as it contains the version
15:36:52 <Xaroth> how about versioned chunks?
15:37:10 <TrueBrain> is not in current savegames
15:37:35 <planetmaker> btw, would it make sense to load savegames and ignore chunks you don't understand?
15:37:46 <TrueBrain> planetmaker: yes, in general it does
15:37:47 <Xaroth> planetmaker: could work, could fail miserably
15:37:49 <TrueBrain> OpenTTD for example does
15:37:56 <planetmaker> it fails on those
15:37:59 <TrueBrain> (to use it as reference :p)
15:38:13 <TrueBrain> well, for reading TTDp savegames ;)
15:38:36 <planetmaker> hm. But not for reading savegames from patched OpenTTD versions? That would make no sense...
15:38:38 <TrueBrain> but in general you want all chunks for the game, yes
15:38:54 <Xaroth> [@TrueBrain]: is not in current savegames << and can TTD open OTTD saves? :P
15:38:57 <TrueBrain> but tools that for example only want to show the map, can ignore all other chunks :)
15:39:03 <TrueBrain> Xaroth: I believe it cannot
15:39:39 <TrueBrain> but that is because we use a slightly different format, if my memory is not mistaken
15:40:12 <TrueBrain> as for the oldloader.c I wrote, I could not use the OpenTTD framework, but had to manually load most parts :)
15:40:24 <glx> png format is a good example for unknown chunk skip :)
15:40:37 <Xaroth> well, we can do somewhat similar for opendune
15:40:50 <Xaroth> having it able to load old saves for as long as it remains feasable
15:41:10 <Xaroth> (seriously, at around 0.6-0.7 who would want to load retro saves with all the features we built in?)
15:41:18 <TrueBrain> I even think I can make it possible to load Dune 1.00 savegames
15:41:28 <TrueBrain> Xaroth: you would be suprised ...
15:41:39 <TrueBrain> in OpenTTD, people still load old old old savegames
15:41:44 <TrueBrain> (and mostly, they still do load :))
15:41:58 <TrueBrain> 1 thing: we need a storage per savegame version, and a validation to see if they are still valid ;)
15:42:04 <planetmaker> Xaroth, having the ability to load OLD savegames should never be forfeit IMO
15:42:14 <Xaroth> planetmaker: at some point you might have no choice
15:42:28 <planetmaker> The other way around... well. Not necessary, but nice with patches
15:42:34 <TrueBrain> you have always a choice :) Just a matter of effort ;)
15:42:37 <Xaroth> my point is, that we can still deviate from the original save format, without breaking backwards compatibility, while maintaining some form of branch-related compatibility
15:43:02 <planetmaker> Then save-load would need a compatibility mode ;-)
15:43:09 <Xaroth> will be a bitch to implement tho, but it might be worth it
15:43:21 <TrueBrain> for patches you can insert a PATC chunk, which indicates it is a patch savegame
15:43:35 <TrueBrain> which is ignored by normal releases ;)
15:44:37 <DorpsGek> SVN: truebrain (r849) -Add: identified a variable in Scenario
15:45:14 <TrueBrain> the total length field is ignored, lol
15:48:53 <Xaroth> but is the data valid? :P
15:51:18 <TrueBrain> I only fail to see how it skips the SCEN chunk
15:52:20 <TrueBrain> it does protect against all kind of overflows while reading
15:53:24 <TrueBrain> ah, special code, skipping the first chunk
17:05:55 <glx> you don't know this function ?
18:49:58 <TrueBrain> someone should fix that (/me points to glx for making that function :p)
18:51:55 <TrueBrain> glx: quick scan shows it looks fine :) I am gone for the rest of the night, so feel free to commit :)
18:52:13 <TrueBrain> Xaroth: maybe you can ask him what he defines as 'unstable' as it shouldn't crash very often .. once in the 30 minutes, no more
18:52:20 <TrueBrain> so maybe he is getting crash.bins we want to know about :)
20:08:54 <DorpsGek> SVN: glx (r850) -Add: C-ified GameLoop_LevelEnd()
21:44:34 <TrueBrain> we have a new roommate
22:21:10 <TrueBrain> fun fact: if savegame not 0x0290, then it runs an 'old' loader
22:21:22 <TrueBrain> so I think the INFO chunk never changed
22:25:58 <TrueBrain> emu_addw(&emu_get_memory16(emu_ss, emu_bp, -0x8), 0xE6);
22:25:59 <TrueBrain> emu_addw(&emu_get_memory16(emu_ss, emu_bp, -0x8), 0x16);
22:26:03 <TrueBrain> I still have those WTF moments when reading the code
22:27:13 <SmatZ> it's next to each other...
22:27:20 <TrueBrain> even a -O0 should fix that :p
22:27:34 <SmatZ> the only reason I can think of is that 0xE6 could overflow
22:27:36 <TrueBrain> just _no_ optimization :) I am suprised there are no comments :p
22:27:44 <SmatZ> but the code needs flags set after adding 0x16
22:27:57 <TrueBrain> nope, flags are ignored afterwards
22:28:04 <TrueBrain> but yes, that would have been an option :)
22:29:23 <SmatZ> maybe the compiler was miscompiling with optimisations enabled :-p
22:29:33 <SmatZ> and this was fast enough anyway ;)
22:33:05 <TrueBrain> lol, the current player is not in the INFO block
22:33:15 <TrueBrain> so the 'compatible' loader has to find the PLYR block to find which is the human player
22:33:46 <TrueBrain> so many possibilities for improvements ....
22:41:39 <TrueBrain> do I start to use 'PLYR' as integer .. or do I make 'strcmp's ....
22:41:54 <TrueBrain> as integer gives warnings ... as strings is slow
22:42:59 <TrueBrain> or hexvalues, but then nobody understands it :p
22:45:49 <TrueBrain> it is better, I guess :)
22:48:54 <TrueBrain> hmmm ... 'exit()' is not C89 ...
22:56:55 <TrueBrain> whoho, my own written reader does what I expect him to do ... good news! :)
22:57:00 <TrueBrain> more tomorrow .. night all :)
continue to next day ⏵