IRC logs for #openttd on OFTC at 2024-04-12
⏴ go to previous day
00:18:16 <yiffgirl> huh, i only saw closed prs with backport requested tags -
00:18:16 <yiffgirl> ohhhh right because they've been merged into trunk and not release, gotcha
00:36:17 <yiffgirl> how do you do the backports? is it cherry-pick or more involved?
02:46:13 *** gnu_jj_ has quit IRC (Ping timeout: 480 seconds)
02:51:50 *** D-HUND has quit IRC (Ping timeout: 480 seconds)
04:41:36 <DorpsGek> - Update: Translations from eints (by translators)
05:18:38 *** quantom2 has joined #openttd
05:18:38 <quantom2> talltyler: Wow... So why it isn't added to the game? Seems like a cool feature...
05:46:00 <truebrain> slowly gathering more data. Top CPU, bottom memory. The white line is the baseline, at the 14th. Green is better than the 14th, Red is worse than the 14th.
05:46:43 <truebrain> You can see that some games aren't all that stable in terms of performance, and can deviate with a few percent. But trends are also very visible 🙂
05:49:42 <truebrain> or another way of looking at the same data .. now the colours mean: difference with the day before. So you can see hotspots even better 🙂
05:51:42 <truebrain> but this last one only shows when it changes, not how long that persists. The first is a bit more clear in that 🙂 Same data, different ways of looking at it 😛
05:54:00 <truebrain> And this graph shows the difference between the pre-build 12.0 version and the custom build 12.0 version. It should all be this yellow colour, basically. But you can see that with some games, the CPU is not behaving uniformly. Yet othergames are. In other words, these performance measurements do need to be taken with a pinch of salt, and only when several games turn colour at once, it is relevant.
05:54:00 <truebrain> A single game just sometimes changes colour for no good reason other than: measuring performance sucks balls 😛
06:18:22 *** keikoz has quit IRC (Remote host closed the connection)
06:47:29 <kuhnovic> I love these diagrams, this can be a really powerful tool to keep openttd in check!
07:06:02 <truebrain> kuhnovic: or to proof the senseless "OpenTTD has gotten 200% slower since NNN" 😛 Anyway, glad you like them 🙂
07:11:54 <kuhnovic> We got Moore's law to fix that problem 😛
07:12:17 <truebrain> it also, as it turns out, is not true 😛
07:20:30 *** Ox7C5 has quit IRC (Ping timeout: 480 seconds)
07:26:22 <kuhnovic> Shhhht! Don't tell anyone!
07:39:29 <truebrain> doubt I would consider that a bug, but for sure a bug not worth fixing
08:01:12 <truebrain> ugh, I sometimes hate the Python stdout buffering .. that moment you run something in a container, and you don't see any logs until N log-lines have been produced ... it is annoying
08:01:21 <truebrain> (and complaining about it helps!)
08:03:39 <LordAro> does python do any special amount of stdout buffering?
08:03:53 <truebrain> it has this PYTHONBUFFER<something> thing
08:04:00 <truebrain> but I never actually looked into what it actually does etc
08:04:14 <LordAro> there's PYTHONUNBUFFERED
08:04:20 <LordAro> stdout is buffered by default though
08:04:29 <truebrain> I know, I was complaining about that 😛
08:13:37 <truebrain> luckily enough, there are other ways to see my container is doing stuff .. but the logs aint one of them! 😛 (it is my own fault, I should use `log.info`, and not `print`)
08:22:40 <peter1138> truebrain: Well, benchmark builds from 2004 to really see that 😉
08:23:08 <truebrain> most people use 12.2 as NNN, for some reason. So I don't have to (yet!) 😛
08:28:50 <locosage> last good version (tm)
08:29:01 <truebrain> I used NNN as a placeholder for <something> a bit earlier today
08:29:55 <truebrain> so much easier if you can see what people reply to
08:31:13 <truebrain> GitHub is funny .. on an organisation I can see what the runners are doing. On my personal user, I cannot
08:32:11 <truebrain> takes ~3 hours to get through 20 days of performance testing .. that is not bad
08:32:15 <truebrain> could be worse, at least
08:32:49 <peter1138> How long until GitHub start charging you...
08:33:10 <truebrain> did you know there are projects that run fuzzing on GitHub runners every night, for a few hours?
08:33:26 <truebrain> once I learnt that, I realised that we are being polite in our CI usage
08:34:31 <truebrain> (ironically, Google is promoting that behaviour 😛 )
08:34:46 <reldred> woo, pay went in, let's try and not blow a months pay in one night buying 2nd hand hifi equipment and bits for my old volvo
08:35:22 <truebrain> reldred: sounds like an excellent plan
08:35:29 <truebrain> may it take two nights
08:36:20 <truebrain> is that your excuse?
08:37:31 <reldred> like I need any excuses to hurtle myself into financial ruin with niche hobbies
08:37:53 <truebrain> at least it is better than having an average hobby
08:37:56 <truebrain> now that is a waste of money 😄
08:38:07 <reldred> what would be an average hobby?
08:38:18 <reldred> I'd say just about most hobbies are usually niche
08:39:31 <truebrain> walking, gym, driving motorbike
08:39:52 <reldred> yeah but those, apart from the motorbike, typically aren't too expensive
08:40:05 <reldred> oh that reminds me, I should buy that cycle trainer
08:40:19 <truebrain> peter1138: I know less people that cycle than do any of the other 3 I mentioned
08:40:27 <truebrain> so not sure where we should put the line 😛
08:40:46 <truebrain> collecting stamps .. also such a non-niche thing to do 😛
08:40:54 <_glx_> truebrain: Vcpkg CI run times are huge
08:40:55 <truebrain> (how many people can I insult in 5 minutes, I like this challenge)
08:41:40 <truebrain> _glx_: yeah, but that is MS. So one can argue that they pay for it indirectly themselves 😛
08:41:55 <truebrain> but yeah, plenty of projects where their CI takes HOURS to finish
08:42:30 <truebrain> but the amount of projects that run that fuzzing-lite stuff on the CI, that just blew my mind. I always considered that rude, to consume so much CPU time for such a long amount of time, every single day
08:42:40 <truebrain> so running a few performance tests for a few days, that is nothing compared to that 🙂
08:43:33 <truebrain> then again .. remember that time someone rebased 15+ PRs, for no good reason other than "rebase is good"? 😄 That also took a few hours of CI time 😛
08:43:57 <LordAro> you could argue that's on us for not closing them sooner :p
08:44:20 <truebrain> I am not allowed anymore ... so I remove "me" from that set of "us" 😛
08:45:19 <peter1138> eBay listings for "untested" "job lot" actually mean "all this shit is broken", right...
08:45:41 <truebrain> they stopped testing it after it broke, yes
08:55:02 <reldred> nice, ordered the rollers. I can set that up in my bedroom away from my housemate where nobody can see me and embarrass myself in the comfort of my own home
09:13:47 <peter1138> reldred: Saves damaging the bike too.
09:13:56 <peter1138> Someone crashed their new bike last night 😦
09:16:22 <locosage> LordAro: space-twitter.com 🤣
09:17:35 <reldred> peter1138: Oh dear! Fortunately my bike is hardly new or nice
09:33:39 <kuhnovic> I hope your hand wasn't between the handle and whatever it was you crashed into
09:35:51 <truebrain> LordAro: lolz @ article .. now that is some fine incompetence 😄
09:36:23 <truebrain> "I don't understand regex!" - "No worries, just use a string replacement"
09:52:21 <_glx_> Oh a broken regex can do the same
10:53:03 <LordAro> "Can we please do all of our testing on the builders etc. using at least one space in the path. It should be easy to catch these errors"
10:56:42 <peter1138> Today my task is to work around for all the people who don't understand dates formatted as yyyy-mm-dd.
10:59:32 <reldred> the work around is to take them out front of the building and club them
11:00:52 <reldred> as an example to the rest if they do not enter the data correctly
11:02:40 <peter1138> It's not even data entry, just display.
11:03:22 <LordAro> reldred: you remind me of Clarkson telling the One Show (fluffy evening talk show) that teachers should be shot in front of their families
11:19:34 <peter1138> "branches are cheap"
11:53:19 <peter1138> I guess it's not actually wrong. It's not a false positive though, it's... intended behaviour.
11:58:49 <_glx_> Not the first report for these
12:01:36 <peter1138> Won't fix seemed most appropriate.
12:06:42 *** Flygon has quit IRC (Quit: A toaster's basically a soldering iron designed to toast bread)
12:20:38 <peter1138> And me wondering too 🙂
12:25:13 <peter1138> The strgen version has "// Just ignore the error" but...
12:38:35 *** keikoz has quit IRC (Ping timeout: 480 seconds)
12:50:42 <truebrain> Moar data! This time March and April of this year. 2024-03-01 is the 100% in this image. Most seems to be noise of the measuring more than anything. One notable is the first of this month. Some games improved, but the game with 1 single big town did not. Now to find a way to visualise that to be a bit more clear 😄
12:52:44 <truebrain> Tweaking the colours and making 2024-04-01 the 100% makes it a bit more clear that something changed 🙂
12:53:29 <truebrain> and now I know what I want on the visualizer 🙂 A way to tweak the colouring, and a way to change what is the 100% 😄 Just ... creating that will be "fun" 😛
12:56:28 <truebrain> and started the last 15 runs, which completes all of 2024 so far. w00p
13:00:03 <truebrain> 1st of April had several of those pesky Codechanges 😛 Guess one of them is bad for when you have a HUGE town, but good for all other scenarios 😛
13:01:31 <truebrain> biggest-town-ever, the name says
13:01:37 <truebrain> I believe LordAro to tell the truth
13:01:53 <LordAro> i have no memory of where it came from, think i found it on the forum
13:02:17 <locosage> pfft forum can easily call 100k that xD
13:03:08 <LordAro> it covers the entire map, i know that
13:03:51 <truebrain> owh, wait, I can ofc just request the profile difference ... let's see what that tells me
13:04:25 <locosage> towns that cover the whole map tend to not be a single town...
13:04:35 <locosage> unless the map is pretty small
13:06:49 <truebrain> meh. CPU profiling did not pick up on the change. Most likely as it runs the map a lot shorter than the benchmarking does
13:07:59 <truebrain> well, no, I should word that better: it picks up on a smaller change, just 4% (instead of 20%+)
13:08:07 <truebrain> it blames it on ClearTownHouse
13:09:16 <truebrain> (and in result, on RunTileLoop)
13:09:29 <truebrain> that kinda would make sense why a savegame with a big town is impacted like that 😛
13:10:32 <locosage> ok, so it's 50mil town but it's not natural, they don't grow like that
13:12:20 <locosage> with a house count of 2.3mil it should be over 200mil
13:12:51 <locosage> though I guess it wouldn't get there as it would start dying them moment it's allowed to grow
13:13:15 <truebrain> hmm, the only commits in that range modify cargo production/acceptance of industries
13:13:20 <truebrain> so not really sure what happened here
13:14:54 <truebrain> lol, that savegame is funny either way .. a lot of time is spend in RunTileLoop
13:15:02 <truebrain> and according to the profiler, in CmdRemoveLongRoad 😄
13:15:52 <truebrain> as in, the profiler (which takes measurements every 10ms or so) sees RunTileLoop in 92% of the time it looks in the stacktrace 😄
13:17:26 <locosage> wonder where CmdRemoveLongRoad comes from, I don't remember towns removing any roads
13:18:46 <peter1138> New rule, only one commit a day so that we know which one causes a performance regression...
13:18:58 <truebrain> no please no no no 😛
13:19:11 <peter1138> Or, abuse the CI, run it for every merge to master.
13:19:35 <truebrain> just takes an hour or so per commit
13:19:57 <peter1138> Hmm, well, maybe not the CI itself.
13:20:38 <truebrain> I am not really curious what change caused it, as nothing obvious stands out 🙂
13:20:46 <peter1138> That makes me more curious 🙂
13:20:47 <truebrain> not because it matters, but because I am curious 😄
13:21:14 <truebrain> as yes, industries changed, but .. how the fuck do towns care?
13:21:26 <kuhnovic> Everything is connected!
13:22:20 <truebrain> or is every house secretly an industry? 😛
13:24:19 <peter1138> I should remember to rebase with --ignore-date when I push a PR :/
13:24:35 <peter1138> My original commit dates are confusing in the log and in blame.
13:24:59 <truebrain> `git log --graph --pretty=format:'%C(auto)%h%d (%cd) %cn <%ce> %s' upstream/master` (partially from stackoverflow, hence the bloat)
13:25:15 <truebrain> the `%cd` is the important part 🙂
13:25:49 <peter1138> Ah, hopefully this.
13:25:57 <truebrain> I have been wondering if I should make the capability that you can run a single PR through the benchmark .. which indirectly means running a single commit
13:26:25 <truebrain> so that indeed would help out in these cases, to find the exact commit causing the issue 😛 But honestly, I should just use kcachegrind or something, and do that locally 🙂
13:27:38 <peter1138> Fortunately I can do whatever regressions I like in the GUI code as you only test the game loop 😉
14:03:36 <peter1138> CmdRemoveLongRoad is not called for me with that save. Hmm.
14:04:55 <truebrain> (in performance-config/save)
14:05:49 <peter1138> Okay, it is this one.
14:06:09 <peter1138> Wonder why your profiler blames that function which is never called :S
14:06:29 <truebrain> that would be troubling, yes
14:06:49 <truebrain> when it looks what the game is doing, it finds the nearest function to that
14:06:56 <truebrain> so it might just be optimizer playing tricks
14:07:57 <truebrain> (this is a profile against a release build, mind you)
14:09:14 <locosage> it's definitely that town but somehow it has way more houses in the save than on screenshot
14:11:00 <locosage> ha, I guess someone fixed u16 overflow since xD
14:11:16 <locosage> 2347849 % 2**16 = 54089
14:13:18 <peter1138> Seems right. It's a full 2048x2048 map with a 3x3 grid, so 9 squares out of 16 are houses, `2048 * 2048 * 9 / 16 = 2359296`. Right ballpark.
14:13:53 <truebrain> running it with a Debug build takes FOR EVAH 😛
14:15:10 <truebrain> a release build is 150 times faster
14:15:34 <peter1138> Hm, no industries at all.
14:15:57 <truebrain> it is one of the reasons I like this map in the performance suite 🙂
14:16:04 <truebrain> it tests something no other savegame tests 😛
14:16:46 <peter1138> Have you got one of Xarick's 70k ships saves then? :p
14:17:28 <peter1138> (Saves are too new for historical testing)
14:17:36 <truebrain> one of the problems 😉
14:17:42 <truebrain> I also tend to avoid 4k x 4k maps in the test suite
14:17:46 <truebrain> they are just horrible for testing 🙂
14:18:20 <truebrain> okay, I can reproduce the slowdown locally too .. 8.8s -> 9.4s
14:18:30 <kuhnovic> There are more things than just map size that make that save horrific 😛
14:19:53 <truebrain> the other thing I noticed, is that the save needs to run at least 1000 ticks to be somewhat stable (ideally even longer) .. and ideally I want that done in 10s
14:19:57 <truebrain> so yeah, that would be the other issue 😛
14:20:06 <truebrain> wentbourne is the only save in the set that doesn't meet that criteria
14:20:12 <truebrain> but not using wentbourne is not done, ofc 😛
14:20:37 <locosage> 1000 ticks is like nothing
14:20:45 <locosage> that's not likely to catch even a single monthly tick
14:21:28 <locosage> though I guess monthly ticks are more of a lag issue than average performance
14:22:25 <truebrain> bbbbiiiisecting ... let's find the actual commit 🙂
14:25:18 <locosage> wentbourne is a good example of an actual game that ran into performance issues
14:25:25 <locosage> rather than just some synthetic test
14:25:49 <peter1138> .6 seconds? Is that my backdoor?
14:26:48 <peter1138> Is it gameloop time or saveload time, I wonder.
14:26:52 <truebrain> but so all games stay the same and some even improve... but this one does not
14:27:03 <truebrain> time to make two profiles 🙂
14:27:46 <peter1138> So weird because there are no industries.
14:27:59 <peter1138> DoCreateNewIndustry is not called, so it's not that.
14:28:00 <truebrain> that is why I am curious about this one 🙂
14:28:07 <truebrain> it seems totally unrelated
14:28:24 <truebrain> but it can be clearly measured
14:28:51 <truebrain> poeh, long time ago since I made a profile out of anything .. `-p` should be enough, right?
14:29:24 <peter1138> clang said it was ignoring it when I specified it, but then I didn't bother to switch to Debug.
14:29:28 <truebrain> let's hope the profiling is not very very slow 😛
14:29:45 <locosage> those vectors can probably use some pre-sizing...
14:30:50 <peter1138> Do vectors that don't exist cause a performance regression?
14:31:23 <truebrain> hmm .. when I compiled with `-p`, and run the game, I should get a profile file, not?
14:31:29 <truebrain> ah, yes, `gmon.out`
14:32:28 <LordAro> peter1138: that's my only thought
14:32:30 <truebrain> took 10s without profiling .. let's see how long it takes with 🙂
14:32:38 <LordAro> it'll be initialising the vectors regardless
14:33:05 <peter1138> Okay, DoCreateNewIndustry is called, but not often, maybe not in the first 1000 ticks.
14:33:24 <peter1138> LordAro, that would be vectors that exist.
14:34:11 <truebrain> hmm .. how to open a gprof in kcachegrind again
14:35:23 <LordAro> truebrain: double click the file on the desktop
14:36:05 <truebrain> `gprof` has issues with my 466MB executable
14:37:05 <truebrain> and cachegrind takes for-ever to execute
14:37:10 <truebrain> so either way, it takes a long time 😛
14:37:43 <truebrain> `--72939-- warning: L3 cache found, using its data for the LL simulation.`
14:37:43 <truebrain> What a pointless "warning". I have no more information than I had before reading it, and I have no clue why it is "warning" me about it 😛
14:42:31 <truebrain> so yeah, seems I have no sensible way of creating a profile for this savegame .. sad 😦
14:45:46 <truebrain> lot of these tools struggle with our 500MB executable size .. but if you strip it a bit, they fail to make sense of the debug symbols 😛
14:46:58 <truebrain> okay, pprof does make a more sensible profile now; at least, it no longer blames functions that are clearly not to blame .. so let's compare it with pprof ..
14:48:31 <truebrain> guess I should use these binaries in the performance suite too, to get more decent profiles .. they are just so gigantic!
14:54:12 <truebrain> okay, pprof is telling me there is no difference between the two versions. lol
14:54:33 <truebrain> it is a sneaky little bastard! Well, first food, more looking into this after 😄 Too curious now to let go 😛
15:32:31 <xarick> guess i'm gonna do some PR's now 🙂
16:07:12 <Eddi|zuHause> that's why you never play HC in that kind of game... crazy damage can come out of nowhere
16:31:00 <peter1138> What is GitHub trying to show me here?
16:41:34 <_glx_> you have too many branches maybe
16:47:19 <_glx_> ah that's why `-s win32:bufsize=2048` didn't have any effect on the weird sound with my mingw build
16:49:14 <_glx_> (with `samples` it works)
17:11:36 <peter1138> _glx_: hmm, can you add some debug for the error_code.message()?
17:12:59 <peter1138> Hmm, maybe it's not working for me too.
17:14:13 <_glx_> it doesn't enter in the error_code part of the if
17:20:59 <peter1138> It's because mtime is uint64_t, and uint64_t - uint64_t doesn't really work for sorting...
17:21:33 <peter1138> But it used to work because the result of substraction is stored in an int.
17:22:35 <peter1138> Hmm, maybe the old mtime is seconds not milliseconds.
17:24:02 <_glx_> ```[2024-04-12 19:21:52] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Ninfingley Transport, 16 Avr 1960.sav 2021-05-01 23:18:45.5326558 13264384725532
17:24:02 <_glx_> [2024-04-12 19:21:52] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Test GS spin.sav 2023-08-23 17:34:27.2432838 13337285667243
17:24:02 <_glx_> [2024-04-12 19:21:52] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 13 Sep 1980.sav 2024-01-19 14:42:44.3826001 13350148964382
17:24:02 <_glx_> [2024-04-12 19:21:52] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 1980-07-31.sav 2024-01-19 14:28:24.1066834 13350148104106```
17:25:26 <_glx_> for windows it was `// Convert from hectonanoseconds since 01/01/1601 to seconds since 01/01/1970`
17:28:36 *** dwfreed has quit IRC (Ping timeout: 600 seconds)
17:28:57 *** dwfreed has joined #openttd
17:29:43 <peter1138> (*this).mtime != other.mtime
17:29:47 <peter1138> Who does (*this).? 😄
17:30:08 <_glx_> should do the same as this->
17:30:26 <peter1138> Yeah, I know what it does 🙂
17:31:53 <_glx_> ```[2024-04-12 19:29:17] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Ninfingley Transport, 16 Avr 1960.sav 1619911125
17:31:53 <_glx_> [2024-04-12 19:29:17] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Test GS spin.sav 1692812067
17:31:53 <_glx_> [2024-04-12 19:29:17] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 13 Sep 1980.sav 1705675364
17:31:53 <_glx_> [2024-04-12 19:29:17] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 1980-07-31.sav 1705674504```
17:32:24 <_glx_> using master (last field is fios->mtime)
17:34:36 <peter1138> cast to seconds works
17:36:27 <peter1138> Really the comparison is wrong, but whatever 🙂
17:37:29 <_glx_> ```[2024-04-12 19:36:09] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Ninfingley Transport, 16 Avr 1960.sav 13264384725
17:37:29 <_glx_> [2024-04-12 19:36:09] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Test GS spin.sav 13337285667
17:37:29 <_glx_> [2024-04-12 19:36:09] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 13 Sep 1980.sav 13350148964
17:37:29 <_glx_> [2024-04-12 19:36:09] dbg: [misc:0] D:\Mes documents\OpenTTD\save\Wartston Transport, 1980-07-31.sav 13350148104``` casting to seconds
17:37:43 <peter1138> Different values but same magnitude.
17:38:13 <_glx_> yeah different date offset it seems
17:39:32 <_glx_> but order is correct for me now
17:42:07 <Eddi|zuHause> probably every day?
17:43:44 <andythenorth> Daily lunch? Monthly lunch? Bi-daily lunch?
17:48:57 <peter1138> It's Timberwolf dropped a video day.
18:03:05 *** Wormnest has joined #openttd
18:10:56 <andythenorth> NotStations time?
18:12:00 *** matusguy has joined #openttd
18:24:28 <truebrain> right, I was trying to profile what the hell that one commit has for an impact on an game without industries
18:29:56 *** gelignite has joined #openttd
18:31:46 <truebrain> now doing 50k ticks, in the hopes that shows a bit better what is going on 😛
18:32:45 <truebrain> that should show up on any profile, not?
18:33:28 <truebrain> also means it is not saveload routines btw
18:37:15 <truebrain> lolz ... okay, this method of profiling is useless to me 😛 The older version is slower than the newer, it claims
18:39:15 <peter1138> Running it for longer affects the relative timings?
18:40:33 <peter1138> I don't know. You said you increased the number of ticks, and the older version is slower now.
18:40:48 <peter1138> But I don't know what you meant, which is why it was a question.
18:40:50 <truebrain> no, not what I said 🙂 I increase the ticks, and the difference is 85s -> 89s
18:41:03 <truebrain> but when I run a profile, the "good" version becomes slower than the "bad" version
18:41:09 <truebrain> because timings in profiling sucks balls
18:41:21 <truebrain> pretty sure if I run the profile 5 more times, I get 5 totally different numbers
18:41:34 <truebrain> but when I run the game, it is always ~85s for the "good", and ~89s for the "bad"
18:41:53 <truebrain> it is just I fail to profile the game; just in general, I cannot get a sane profile out of the game 🙂
18:42:12 <truebrain> callgrind keeps saying: `==168107== brk segment overflow in thread #1: can't grow to 0x489e000`
18:43:43 <truebrain> and also, kcachegrind shows nothing in relation to each other ... this is all not what I expect to find 😛
18:43:49 <truebrain> last time I did this, years ago, it all made more sense 😄
18:44:54 <truebrain> and now I managed to close the side-bar in kcachegrind, and I see no way to reopen it
18:47:24 <truebrain> it tells me that `main` used 0s, including all its children. It says `main` has no children
18:49:14 <peter1138> 111 packages to install kcachegrind. Can't be arsed.
18:52:27 <peter1138> I keep thinking that maybe it's some combination of changes and that is just the one that triggers it, rather than the cause.
18:52:35 <peter1138> But then... there's no industries...
18:52:48 <truebrain> I expect it to be something weird, honestly
18:53:19 <truebrain> but it is just funny how much difficulty I have creating a profile locally
18:54:02 <truebrain> but, pprof does something, and what is fully, it blames 60% of the CPU on `GetTileType`
19:02:26 <truebrain> okay, let's give up on profiling. Let's approach this a different way. First, let's check if this happens with other compilers too
19:04:33 <truebrain> GCC: 7.7s. Clang: 6.2s
19:04:36 <truebrain> wth .. that is insane 😛
19:10:11 <truebrain> okay, both compilers show a jump in runtime
19:11:08 <truebrain> but never knew that clang and gcc could have that much performance difference between each other
19:14:44 <truebrain> the game tries to build 1 (!) industry
19:14:50 <truebrain> and that seems to cause this 😛
19:15:14 <truebrain> if I disable that one industry, it returns to where one would expect
19:15:35 <_glx_> vector + change from iterator to indexed loop ?
19:22:26 <truebrain> okay, it is actually a behavioural change
19:22:34 <truebrain> `TryBuildNewIndustry` is called a different amount of times
19:22:56 <truebrain> with 5000 ticks, I see `TryBuildNewIndustry` 4 times before the change, and 8 times after
19:23:19 <truebrain> which suggests that commit isn't an actual code-change 😦
19:25:49 <truebrain> so if I would be to guess, the random generator creates different numbers
19:25:53 <truebrain> which ofc can cause all kind of effects
19:26:11 <truebrain> which suggests in one case it does something and in the other case it doesn't, causing another Random() to be called
19:26:35 <peter1138> 1 industry shouldn't cause that?
19:26:45 <truebrain> no, it is not the one industry
19:27:45 <truebrain> what I can tell now, `TryBuildNewIndustry` is called a different amount of times
19:27:50 <truebrain> which can have a lot of reasons
19:28:13 <truebrain> but as it is a chance-based roll
19:28:22 <truebrain> I am going to guess it is because the randomizer for some reason is different
19:30:24 <_glx_> but this commit doesn't touch that
19:30:49 <truebrain> the result disagrees with you
19:31:39 <peter1138> In DoCreateNewIndustry:1809.
19:31:56 <peter1138> That would do a different number of RandomRange(256) with this change.
19:32:12 <truebrain> well, there you have it 🙂
19:32:20 <truebrain> a code-change that did just a tiny bit more 😛
19:32:28 <truebrain> no, the random is different
19:32:31 <truebrain> so anything can happen
19:32:39 <truebrain> I mean, it could try to build a totally different road
19:32:51 <truebrain> like, it is a totally different game from that point on
19:32:56 <truebrain> you can't even compare them anymore 😄
19:33:08 <truebrain> not saying the change it bad btw; like ... so what, that the random is different
19:33:23 <truebrain> but it does explain the difference 🙂
19:33:26 <peter1138> Should I call that RandomRange() enough times to counteract it?
19:33:39 <truebrain> for me personally? No. I mean, any change can impact the random generation
19:33:53 <truebrain> it just sneaked up on us there 😉
19:33:53 <_glx_> I can see i->produced and i->accepted can be smaller
19:34:01 <peter1138> That Random only has an effect if the industry is actually built.
19:34:07 <peter1138> And no industries can be built, afaik.
19:34:39 <truebrain> well, I can validate by running some addition randoms ofc
19:35:06 *** HerzogDeXtEr has joined #openttd
19:35:39 <peter1138> Yeah, a bank is created eventually.
19:35:40 <truebrain> yup, back to the original timing
19:36:14 <peter1138> So your profiler also needs to output random seed information...
19:36:24 <truebrain> you see how subtle performance metrics can be 🙂
19:36:53 <peter1138> Although probably it changes enough accidentally that it wouldn't be a useful metric.
19:38:09 <truebrain> full vision of this year so far 🙂
19:38:17 <truebrain> (14th of january as 100%)
19:39:11 <truebrain> so on the memory front (below) we did awesome
19:39:16 <truebrain> on the CPU front it is so-so 😛
19:40:40 <andythenorth> nice charts are nice
19:40:58 <michi_cc> truebrain: You mean we might have to update the CPU specs on Steam after all? 😛
19:40:59 <locosage> saves that actually matter seem to be ok on cpu
19:41:18 <truebrain> michi_cc: it still isn't a lie 😄
19:41:44 <truebrain> but okay, I learnt one thing from all this performance bla, is that after-the-fact analysis are fucking shit
19:42:25 <_glx_> I can see some ` uint maxcargoes = (indspec->behaviour & INDUSTRYBEH_CARGOTYPES_UNLIMITED) ? static_cast<uint>(i->produced.size()) : 2;`
19:43:19 <_glx_> /* Clear all output cargo types */
19:43:19 <_glx_> /* Query actual types */
19:43:19 <_glx_> uint maxcargoes = (indspec->behaviour & INDUSTRYBEH_CARGOTYPES_UNLIMITED) ? static_cast<uint>(i->produced.size()) : 2;
19:43:32 <_glx_> that might change some stuff
19:43:47 <peter1138> We already found the cause...
19:44:55 <peter1138> I don't think it's useful to try and match old versions seed-for-seed, just knowing that the performance is only due to randomness is enough.
19:45:48 <_glx_> oh indeed, different i->produced length
19:46:52 <truebrain> and I would hate if we start to change the code to make a performance benchmark happy
19:46:55 <truebrain> that is the wrong way around 😄
19:47:07 <_glx_> but the callback handling seems broken by just looking at the code
19:48:03 <_glx_> if INDUSTRYBEH_CARGOTYPES_UNLIMITED it will loop from 0 to 0 to get the cargo via the callback
19:51:02 <peter1138> This isn't in the 14.0 branch so that's something at least 🙂
19:53:00 <_glx_> relying on array size for something that should be the constant used to define array size
19:56:29 <_glx_> ok only 2 locations it seems
19:57:19 <peter1138> Still compiling it, tbh.
19:58:37 <peter1138> Any industry sets using this?
19:58:42 <_glx_> that's the only locations I found too
19:58:43 <LordAro> i'm suspicious about why the commit hash doesn't link
19:59:02 <truebrain> he tried to change the random generator to create a binary to infect milions of machines
19:59:05 <truebrain> so he is just a bit pissed atm
19:59:12 <andythenorth> peter1138: FIRS probably
19:59:28 <peter1138> LordAro: Maybe a collision.
20:00:12 <truebrain> haha, indeed, collission
20:00:13 <andythenorth> is it CB 14B, 14C?
20:00:16 <truebrain> so you need to add the `e` 😛
20:00:41 <truebrain> well, no, we have a collision of 7 characters
20:00:44 <truebrain> so all shorts need 8
20:00:48 <truebrain> something like that is how GitHub works 😛
20:02:05 <_glx_> lol and github shows only 7 🙂
20:02:07 <peter1138> Now it links, but still only shows 7...
20:02:19 <truebrain> that is just weird 😛
20:02:36 <peter1138> There's no collision in my master, so I guess it's a branch?
20:02:54 <truebrain> no, I believe if there is a N char collision between ANY commit, it requires N+1
20:04:42 <truebrain> they once wrote about this ... but I have a hard time finding that back
20:04:42 <truebrain> must have dreamed it
20:05:00 <peter1138> It's fine. I'll start filling in the full hash 😉
20:05:23 <peter1138> andythenorth: I'd like more than a "probably" to test it.
20:06:17 <andythenorth> is CB 14B / 14C?
20:07:26 <andythenorth> FIRS does not appear to use it any more
20:08:33 <_glx_> `cargo_input` and `cargo_output` in nml
20:08:57 <andythenorth> not used in FIRS, nor nml example industry
20:10:45 <truebrain> _glx_: will you also be pressing buttons for 14.0? 🙂
20:11:01 <_glx_> needs changelog update too
20:11:38 <peter1138> None of the industry sets I have available trigger a breakpoint...
20:12:03 <_glx_> so we found a bug with users complaining
20:12:31 <truebrain> as with, we have plenty of those 😛
20:12:39 <_glx_> yeah, I missed some letters
20:13:48 <_glx_> and it was an accidental find, very hard to spot in the PR diff itself
20:14:04 <truebrain> xz backdoor was also an accidental find
20:14:08 <truebrain> (will this ever get old?)
20:15:13 <_glx_> hehe I found it because you were looking for the performance change 😉
20:16:32 <LordAro> today i improved the runtime of a tool by 90%
20:16:55 <andythenorth> was it the test runner?
20:17:04 <peter1138> The 600ms performance change...
20:17:18 <LordAro> wait, that's not precise enough - i cut 90% off the runtime of a tool
20:17:54 <LordAro> turns out there was a file reader class that was using exceptions to know whether it was at the end of a line or not
20:18:05 <LordAro> turns out, exceptions, not fast
20:19:37 <andythenorth> can you make Iron Horse compile faster?
20:20:01 <_glx_> didn't try the script, I was just reading 🙂
20:20:04 <LordAro> this was done by actually profiling the runtime and looking at it
20:20:12 <LordAro> "oh, exception handling is 90% of the runtime"
20:25:51 <truebrain> Okay, so maybe running the performance set on PR isn't such a bad idea. And it wouldn't even take that much time if I don't do all the stuff around it. About 20min CI time. Might be worth testing
20:28:34 <truebrain> (Mostly as this after-the-fact checking of how/what/why is just annoying :P)
20:38:38 *** nielsm has quit IRC (Ping timeout: 480 seconds)
20:38:49 <_glx_> trying to understand how changelog script works, seems to be a pain when wanting to update only release branch
20:44:19 <_glx_> hmm yeah, seems it only lists non backported stuff on master, that's not what I'd want
20:49:55 <truebrain> I wrote it to automate the massive changelog. So yeah 😛
20:52:03 <truebrain> But a 'git log' mostly works for these few changes? Dunno, didn't check how many there are 🙂
20:52:04 <_glx_> it's perfect for beta 🙂
20:54:24 <truebrain> The backport PRs help ofc for after betas
21:11:58 *** HerzogDeXtEr has quit IRC (Read error: Connection reset by peer)
21:12:42 <_glx_> I hate doing changelogs
21:15:11 <truebrain> I do too. Hence the automation 😛
21:23:32 <locosage> _glx_: citymania event uses 0x14b
21:30:46 <peter1138> Does not seem to be a useful NewGRF.
21:36:35 *** keikoz has quit IRC (Ping timeout: 480 seconds)
21:37:47 <peter1138> Anyway it doesn't use the unlimited flag so isn't broken.
21:38:38 <locosage> peter1138: it is useful for the event it was made for :p
21:38:51 <locosage> 14b is used to randomize input cargoes
21:41:18 <locosage> peter1138: which flag is that?
21:43:29 <locosage> ah, flag18 I guess, yeah, didn't need that one as only 3 inputs
21:43:52 *** matusguy has quit IRC (Remote host closed the connection)
21:46:15 <locosage> well, it's not that hard to set it...
21:52:39 *** gelignite has quit IRC (Quit: Stay safe!)
21:58:17 *** amal[m] has quit IRC (Quit: Client limit exceeded: 20000)
22:39:55 *** Wolf01 has quit IRC (Quit: Once again the world is quick to bury me.)
continue to next day ⏵
