IRC logs for #openttd on OFTC at 2025-12-27
⏴ go to previous day
00:00:07 <xarick> my suspicion is PriorityQueue
00:00:20 <xarick> something over there just feels wrong
00:07:44 <xarick> queue.Insert([j + 1, j % interval + 1 + GSBase.RandRange(interval)], GSBase.Rand());
00:07:44 <xarick> what I'm actually inserting in the priority queue
00:07:49 *** WormnestAndroid has quit IRC (Read error: Connection reset by peer)
00:08:05 *** WormnestAndroid has joined #openttd
00:08:33 <xarick> multiplied by 2 million, so queue got 2 million arrays
00:10:12 <xarick> garbage collector possibly steps overheads here, i dunno
00:10:29 <xarick> will investigate tomorrow
00:21:32 <_glx_> garbage collection of huge ScriptList is known to be slow (set and map are cleared one item at a time)
00:23:31 <_glx_> PriorityQueue uses a vector, so should be quite fast to clear
00:24:44 <_glx_> but queue destructor has to release the objects
00:59:52 *** Wolf01 has quit IRC (Quit: Once again the world is quick to bury me.)
01:02:07 *** tokai|noir has joined #openttd
01:02:07 *** ChanServ sets mode: +v tokai|noir
01:09:05 *** tokai has quit IRC (Ping timeout: 480 seconds)
02:48:52 *** WormnestAndroid has quit IRC (Ping timeout: 480 seconds)
02:49:14 *** WormnestAndroid has joined #openttd
03:09:39 *** WormnestAndroid has quit IRC (Read error: Connection reset by peer)
03:09:54 *** WormnestAndroid has joined #openttd
03:11:32 *** MinchinWeb[m] has quit IRC (Ping timeout: 480 seconds)
03:12:50 *** MinchinWeb[m] has joined #openttd
04:43:01 <DorpsGek> - Update: Translations from eints (by translators)
05:03:19 *** limyx826 has joined #openttd
05:03:19 <limyx826> I had checked the pull request #14220 and #14233.
05:04:22 <limyx826> While I get the why the need to do so but the way it implement kind of lose the charm of the TTD.
05:07:51 <limyx826> Just stating my opinion.
06:35:39 *** Artea has quit IRC (Read error: Connection reset by peer)
07:08:19 *** WormnestAndroid has quit IRC (Remote host closed the connection)
07:08:22 *** WormnestAndroid has joined #openttd
09:11:48 <andythenorth> limyx826: your image on reddit is weird, new menu looks like this
09:13:57 <limyx826> andythenorth: I was using OpenGFX
09:23:46 <peter1138> Hmm, the version string is pretty invisible when you font with fine lines.
09:35:46 <_zephyris> Same is true for town names, transparent signs, etc.
09:36:22 <_zephyris> User problem IMO, unless there's a way we can softly promote bold (eg. Pre-populate the cfg with it)
09:39:25 <peter1138> Well, we switched to preferring non-bold in autodetection too, for consistency (only the Linux code preferred bold, and that preferred the super super heavy stuff, not regular bold)
09:40:10 <peter1138> The default config is to use OpenTTD Sans, so users have set up Arial or whatever manually.
09:40:35 <peter1138> Some of that choice might be because it was set up before the font existed.
09:42:15 <peter1138> It's also possible that the font loading code doesn't work for some people, but those people don't realise there's meant to be a custom font...
09:42:34 <peter1138> (The Windows API we use to load a non-installed font is sort of hidden.)
09:47:13 <peter1138> I need to test with the toyland intro game :)
09:53:44 <peter1138> Using something closer to the original image that andythenorth refers to.
09:55:52 <peter1138> Padding went from too much to too little.
10:00:26 <_zephyris> Meh, stylistically semi-transparent doesn't match
10:03:40 <_zephyris> Nuke user settings occasionally...
10:18:02 <peter1138> Nuke those old pathfinder settings :)
10:18:38 *** Superuser has joined #openttd
10:19:20 <Superuser> I have a question about the nightly ga900f5756c. That reference doesn't seem to correspond to a git commit. How do I know which commits were included?
10:22:42 <xarick> a900f5656c4ca627ef45d91c3f30a7f405352f6b ?
10:23:10 <Superuser> Looks like it, just as you typed it I cross-referenced with a translation commit and it seems to be in there
10:23:20 <Superuser> strange that they have two different references
10:23:48 <Superuser> but I suppose it's master-g{commit}. I do not know why the "g" is there though
10:24:17 <xarick> g probably for genuine, dunno
10:25:31 <_zephyris> Rocks always looking the same makes my life easier!
10:33:24 <xarick> funny that it activates itself during an update
10:33:58 <xarick> i thought it was a feature
10:37:39 <xarick> investigating garbage collector spikes
10:38:39 <xarick> I inserted 2 million items, each item being [ item, value ] arrays
10:38:54 <xarick> priority queue spikes like that
10:39:27 <xarick> every 2.5 secs or so i get those spikes even paused
10:41:24 <xarick> oh, maybe it's every 7 seconds, can't really tell
10:41:42 <LordAro> Superuser: g is for git
10:42:05 <Superuser> it might be more useful as -git-{commit} then
10:42:55 <Superuser> though it is kind of implied when it says "master" and OpenTTD hasn't used Subversion for perhaps a decade
10:43:14 <LordAro> commit hashes are hex, g can't be part of it
10:48:11 <xarick> now i am pushing the items from the queue into a main array. the queue becomes empty, and the main array will contain those [ item, value] arrays in it.
10:48:54 <xarick> spikes become more spikier
10:49:56 <xarick> what can be done to allieviate this?
10:50:50 *** SigHunter has joined #openttd
10:51:02 <peter1138> Something about not putting 2 million items into a queue or an array.
10:53:02 <xarick> while (!queue.IsEmpty()) {
10:53:02 <xarick> array.push(queue.Pop());
10:58:18 <xarick> let me try a 4k rectangle directly into a ScriptList
11:03:26 <xarick> and it has 16 million items
11:03:33 <xarick> much more than 2 million
11:07:43 *** Superuser has quit IRC (Quit: WeeChat 4.8.1 - Please send me a message through MemoServ if I'm offline as I don't have a bouncer)
11:07:44 *** Superuser has joined #openttd
11:12:32 *** Superuser has joined #openttd
11:42:57 *** toktik is now known as Guest34537
11:48:28 *** Guest34537 has quit IRC (Ping timeout: 480 seconds)
12:02:00 <rito12_51026> Rubidium: Figured it out for some reason the _POSIX_C_SOURCE is not defined while _POSIX_SYNCHRONIZED_IO is, so the header files are not included but functions from them are used. If I remove the check for _POSIX_C_SOURCE and include those headers directly it compiles, so the headers do exist.
12:10:40 <xarick> make garbage collector smooth again
12:29:05 <xarick> should I go into the rabbit hole?
12:57:15 *** toktik has quit IRC (Remote host closed the connection)
13:12:11 <_glx_> well, one option to make GC less visible could be to run it more often
13:14:33 <_jgr_> I doubt that would help much. You'll incur the performance cost even if nothing ends up being actually deleted.
13:14:52 <peter1138> Make it suspendable :D
13:18:05 <Borg> actually.. this might be sane
13:18:25 <Borg> I remember I had problems in DOOM w/ FPS shuttering.. on large maps w/ complex-doom
13:18:42 <Borg> it was GC.. I adjusted threshold and timeout.. resulting calling it more often and for smaller values
13:18:49 <Borg> and it was huge difference
13:19:06 <_glx_> there's no timeout for GC
13:21:43 <xarick> wow, an unexpected crash!
13:22:46 <xarick> i remember a few years ago openttd seemingly crashing out of nowhere after resuming from sleep.
13:23:26 <xarick> well, this is not "resuming from sleep" but similar, i had breakpoints in the garbage collector, then advanced to next instruction, and suddenly... i get this crash
13:24:00 <xarick> and yes, the framerate graph window is open
13:24:50 <_glx_> yeah it's not a "real" crash then
13:25:59 <xarick> it's a division by zero :|
13:27:44 <xarick> I was waiting to reproduce this crash for years
13:28:25 <_glx_> yes but it happens because you made the tick extra long
13:28:29 <vldb1245> I discovered that chatGPT doesn't know C++ well at all
13:28:42 <vldb1245> Java or Javascript understands the best
13:30:10 <_jgr_> Even if it's not a "real" crash, drawing nothing seems more reasonable than doing a division by 0
13:30:46 <_glx_> y values look way off too
13:37:39 <_glx_> ho there's a while loop dividing by 2
13:47:18 <xarick> margin is 8, starts at 1
13:47:41 <xarick> the loop was run >2>4>8, 3 times
13:49:14 <xarick> 3 / 2 = 1 > 1 / 2 = 0, 0 / 2 = 0
13:55:39 <xarick> not sure what I'm screenshoting but
13:56:16 <xarick> point 511 is related to what chrono measured at the point it resumed?
13:57:50 <xarick> 3'168'381'550 but it's forced to be stored as int, that makes it negative
13:59:28 *** emperorjake has joined #openttd
13:59:28 <emperorjake> I found a potential bug in RC4. I'm loading older vanilla saves from before wallclock is a thing, and the game behaves as if it's in wallclock mode even though it's in calendar mode. It's fixed after a save and reload.
14:03:04 <emperorjake> The game date progresses very slowly as if it's in wallclock mode, I have it set to 10080 minutes per year
14:03:44 <emperorjake> See if you can reproduce it. Set minutes per year to 10080, and then load an older pre-wallclock save
14:06:13 <emperorjake> Yeah that's a bug. The finances are out of sync with the date
14:08:22 *** hexdog has quit IRC (Remote host closed the connection)
14:09:01 <talltyler> Ah, `minutes_per_calendar_year` doesn't seem to be set in afterload
14:09:30 <talltyler> So it's not set when loading an old savegame.
14:11:04 <talltyler> Anyone have an old savegame handy to test with?
14:11:26 <xarick> the rabbit hole led me to OverflowSafeInt32
14:11:44 <xarick> should it be used for framerate graph?
14:11:49 <emperorjake> This is the 1.12 title game I tested with
14:12:51 <xarick> that looks like an easy fix
14:13:18 <xarick> i was running around trying to figure out why y was negative
14:15:04 <_glx_> yes `y` values are completely off it seems, but if the division by 0 is prevented it will just return because nothing to draw I think
14:16:48 <xarick> Point2D<int>, could it accept Point2D<OverflowSafeInt32> instead or is this nonsense?
14:20:23 <xarick> framerate uses Point to store measured times
14:20:23 <xarick> ```/** Coordinates of a point in 2D */
14:20:23 <xarick> using Point = Coord2D<int>;```
14:20:47 <xarick> i thought about Coord2D<OverflowSafeInt32>
14:20:50 <_glx_> no need to overcomplicate 🙂
14:23:46 <_glx_> talltyler: intro game from master is also a valid super old game I think
14:32:01 *** Superuser has quit IRC (Quit: WeeChat 4.8.1 - Please send me a message through MemoServ if I'm offline as I don't have a bouncer)
14:43:48 <xarick> testing garbage collector running every frame:
14:44:23 <Rubidium> regarding the garbage collector: I doubt there's much that can be done about it. As I doubt the problem is in the 'collecting of garbage', but rather in the step before that... determining what is garbage, i.e. what objects are not reachable anymore. For this it needs to traverse the heap and all its objects. That's not going to be fast due to the random access nature. Also suspending is going to add a
14:44:29 <Rubidium> lot of complexity, as you need to revisit elements you've seen before if you know they are changed. After all... you could be swapping the contents of an object during the suspend of the GC run, but you need to update the appropriate structures for that.
14:45:28 <Rubidium> having said that... there is an easy solution: just limit each script to say 50k objects. And all your long GC runs are gone
14:45:43 <_glx_> first step would be to time each part of SQSharedState::CollectGarbage
14:50:11 <Rubidium> time with I'd say: first while loop, last while loop and finally middle while loop. The middle loop is the thing that actually releases, but that will only do something when you've got circular references and release that. For xarick's performance checks I doubt there will be a lot of actual 'lost' circular references
14:57:16 <_glx_> yeah it's just a lot of objects at a single moment
14:59:33 <xarick> i assume [ item, value ] objects?
15:01:01 <xarick> the idea for the priority queue was to create a list of items to be inserted in random order in an empty list, to exploit lower_bound and emplace routines without the sorter bias
15:02:45 <xarick> i also needed reproduceability
15:04:51 <_jgr_> All collectable Squirrel objects are stored in a single linked list, so taking a fair amount of time to iterate through it is not unexpected
15:06:08 <_jgr_> There are plenty of fancier strategies for GC but these are probably overkill and a lot of hassle to retrofit into Squirrel
15:07:48 <_glx_> in normal use cases GC is mostly fine
15:12:10 <_jgr_> That said you could probably keep purely leaf objects out of the GC chain and just the ref count for those, but it may not be worth the bother
15:13:28 <xarick> nop, can't think of an alternative for my purposes
15:15:19 <xarick> i also have the peculiar requirement of "item" to be counted from 1 to interval (2 million)
15:17:08 <xarick> yeah, I really need the PriorityQueue for what I aim to test
15:18:32 <xarick> wait, I think I had an idea!
15:19:02 <xarick> make 2 priorityqueues, one for "item", one for "value"
15:27:34 <ode_> _jgr_: Oh yeah when is the next pack/update coming out
15:30:55 <Borg> linked list? then you can do lazy pickup..
15:31:13 <Borg> if first pickup wont mark object as free.. next pass will
15:41:14 <Rubidium> Borg: how can you be lazy? You can only free things when you know it can't be reached from the elements on the stack/globals. Or do you mean lazy in the context that you 'only' need to check fromo the stack/globals, and once that's done you only free a bit (and start from the beginning the next run)
15:49:52 <xarick> I found an alternative solution on my end
15:51:32 <xarick> instead of 1 priorityqueue with 2 million [ item, value ] arrays, it's 2 priorityqueues, one with 2 million item, other with 2 million value
15:52:53 <xarick> then i have 2 other arrays where I push
15:55:53 <xarick> ```local itemqueue = GSPriorityQueue();
15:55:53 <xarick> local valuequeue = GSPriorityQueue();
15:55:53 <xarick> local itemarray = array(0);
15:55:53 <xarick> local valuearray = array(0);
15:55:55 <xarick> local priority = GSBase.Rand();
15:55:55 <xarick> itemqueue.Insert(j + 1, priority);
15:55:57 <xarick> valuequeue.Insert(j % interval + 1 + GSBase.RandRange(interval), priority);
15:55:57 <xarick> } while (++j % interval != 0);
15:55:59 <xarick> while (!itemqueue.IsEmpty() && !valuequeue.IsEmpty()) {
15:55:59 <xarick> itemarray.push(itemqueue.Pop());
15:56:01 <xarick> valuearray.push(valuequeue.Pop());
15:56:03 <xarick> for (local l = 0; l < itemarray.len(); l++) {
15:56:03 <xarick> list[itemarray[l]] = valuearray[l];
15:59:43 <xarick> squirrel needs something similar to a tuple
16:00:03 <xarick> the closest thing is array :/
16:00:11 <Borg> Rubidium: you just check as much as you can and free what can be freed and give up.. resume from last ptr
16:00:29 <Borg> its linked list.. I assume new allocs go to tail
16:00:56 <Borg> if object will dangle for a while due to a temporary preasure.. its not problem right? they will be pick up later?
16:01:07 <Borg> if you try to optimize for stupid case like some alloc 2M objects.. then dont.
16:01:32 <Borg> either start aggrsively pickup (you dont care about lag) or.. just go alone.. if out of mem happens. crash
16:02:28 <Borg> thats why I asked how GC is trigger.. never looked at that part of the code in OpenTTD
16:07:19 <Rubidium> Borg: in Squirrel there are two methods of GC. The first is reference counting. If the object isn't referenced it gets freed. This is easy and simple. Until someone does object A -> object B -> object C -> object B. Now if object A does not point to B anymore, B is still referenced by C. The GC run is there to free those 'islands' of self-referencing stuff. That means that you need to look from the
16:07:25 <Rubidium> global allocations (stack/globals) and basically scan all elements to see whether object B or C are referenced. Doing that for each object is very expensive, so we just mark all elements that can be reached every once in a while and then free anything that can't be reached. The marking is the step that costs a lot of time (also when there's nothing to free).
16:09:34 <xarick> garbage collector only finds 4 now
16:10:58 <Rubidium> you can't easily suspend the marking. Given the previous example, add an object D which refers to nothing and A and D are on the stack. Now start the marking of reachable objects. First we'll do D, which has no 'children'. Now we suspend the marking, and the script makes D refer to B and A refer to nothing. Next the marking continues, sees object A refering to nothing and it's done. Now B and C are
16:11:04 <Rubidium> released erroneously because they could not be reached. So... you need to account for any changes you do while doing the 'mark' run, which gets even more expensive quickly.
16:20:16 <Borg> okey.. a bit more complicated indeed...
16:20:57 <Borg> anyway.. OpenTTD time ;)
16:26:03 <xarick> added a few counters. the queue started with a size of 15
16:26:38 <xarick> it only became emtpy when count1 = 1883
16:26:46 <xarick> what the heck is happening in there
17:26:46 <_glx_> count2 keep 0 if there's nothing to GC
17:27:36 <_glx_> so count3 matches count1
17:28:08 <_glx_> in all cases you should have count2+count3 == count1
17:29:12 <_glx_> hmm no, that assumption is wrong
17:30:48 <_glx_> and yes queue is always 15 long when entering the first loop
17:31:36 <_glx_> because the first thing done by the function is adding 15 objects
17:33:38 <_glx_> then direct children of these objects are added, then their children...
18:02:00 *** Wormnest has joined #openttd
18:05:29 *** Flygon has quit IRC (Remote host closed the connection)
20:24:51 *** Borg has quit IRC (Quit: leaving)
21:04:23 <xarick> master ScriptList is finally competitive at something
21:04:49 <xarick> oh wait, those are wrong versions of openttd
21:05:28 <xarick> they're all 5c96a2c19a, but i forgot to rename in excel
21:06:52 <peter1138> talltyler, oh teh joeks
21:07:13 <peter1138> But yeah, my initial attempt was way over the top and complicated.
21:08:41 <xarick> that node_handle commit is one of the rare situations where std::map/set has an advantage over btrees
21:11:18 <xarick> master could still be faster
21:11:27 <xarick> the teal graph proves it
21:15:29 <xarick> remember when ValuateTests in master was 800 seconds? look at it now
21:37:32 <thevbw> nothing to contribute yet but I offer a massive THX. for supporting OpenBSD and Linux systems so well!
21:55:47 *** MinchinWeb[m] has quit IRC (Ping timeout: 480 seconds)
21:59:42 *** MinchinWeb[m] has joined #openttd
23:43:14 *** WormnestAndroid has quit IRC (Read error: Connection reset by peer)
23:43:15 *** WormnestAndroid has joined #openttd
23:56:47 *** MinchinWeb[m] has quit IRC (Ping timeout: 480 seconds)
continue to next day ⏵
