IRC logs for #openttd on OFTC at 2023-08-27
⏴ go to previous day
00:29:59 *** Eddi|zuHause has quit IRC ()
00:44:42 *** Eddi|zuHause has joined #openttd
00:46:10 *** Eddi|zuHause has quit IRC ()
00:49:13 *** Eddi|zuHause has joined #openttd
01:35:41 *** Wormnest has quit IRC (Quit: Leaving)
02:12:56 *** debdog has quit IRC (Ping timeout: 480 seconds)
02:14:15 *** D-HUND is now known as debdog
03:58:25 *** esselfe has joined #openttd
06:19:15 *** HerzogDeXtEr has joined #openttd
07:26:00 *** HerzogDeXtEr has quit IRC (Read error: Connection reset by peer)
07:34:27 <_zephyris> frosch123: Super useful to know. I've been trying to get the sprites organised in sensible standard-sized templates depending on where they pop up - this kinda info is vital.
08:19:32 <truebrain> really puzzled by that comment over autoclean in that PR 😄
08:20:21 <truebrain> it sounds like a nice addition too, but I don't see it 😛
08:23:25 *** gelignite has joined #openttd
08:29:20 <andythenorth> so if my GS builds *lots* of industries, what do we think `IndustryBuildData` is going to do in the monthly / random industry construction loop?
08:30:04 <andythenorth> it's likely the number of industries on the map far exceeds the trigger for placing a new one?
10:00:11 <truebrain> hmm .. I am reworking the crashlog stuff .. I can now really easily add the settings in the crash.log (well, crash.json, but who is counting) .. is that useful, I wonder
10:00:35 <truebrain> it does double the size of those log files 😄
10:05:54 <locosage> is crash log size even a concern?
10:06:39 <locosage> crash sav is probably much bigger anyway
10:14:29 <truebrain> the main difference between linux and windows is when a crash savegame is made .. linux always does it, windows only on demand .. I guess that explains why often people have trouble attaching the savegame as "it is not there"
10:23:46 <truebrain> hmm ... only Linux doesn't have a popup on crash; I guess that is really difficult, as you have to assume the SDL library didn't crash 😛 The other OSes hook into known-good OS functions .. meh 😛
10:30:36 <_jgr_> On Linux it'd probably be more pragmatic to fork/execve another process for a dialog box
10:31:05 <truebrain> _jgr_: that works if the SDL driver functions; so any crash before would still be problematic, but yeah, that is not a terrible idea 🙂
10:31:25 <frosch123> i thought crashpad already does a fork
10:31:34 <truebrain> crashpad, yes, but we are not using crashpad
10:31:35 <_jgr_> In any case, on Linux STDOUT is not a black hole like it is on Windows
10:31:38 <truebrain> so not sure how that is related 😄
10:32:04 <frosch123> but if you go for writing you own second process, it may be easier to have a bootstrap process, which runs the real ottd as subprocess
10:32:24 <truebrain> I really want to stay as far away as possible from that as I can 🙂
10:32:39 <frosch123> we are not using crashpad? why does cmake shout at me for not having it installed then?
10:32:52 <truebrain> read carefully frosch123 , it doesn't say crashpad at all 🙂
10:33:02 <truebrain> just because it ends with "pad" doesn't mean it is crashpad 🙂
10:33:04 <_jgr_> Even with a multi-process setup "bootstrap" and "main" don't have to be different executables
10:33:27 <frosch123> "unofficial-breakpad" <- is that some kind of truecrash?
10:33:34 <truebrain> no, that is vcpkg 😦
10:33:40 <truebrain> they "plan to fix it" ...... for a few years now
10:34:53 <truebrain> anyway, breakpad is in-proc; for crashpad you need to ship a second executable
10:34:59 <truebrain> which is a world of hurt you don't want to step into
10:35:38 <truebrain> _jgr_: yeah, I was thinking about another parameter like `-crash` that has a fast path to only show a dialog .. still .. lot of work, for very few people 😄
10:39:19 <frosch123> is anyone already working on adding baseset parameters?
10:39:39 <truebrain> I saw _zephyris talking about it; you would have to check with him 🙂
10:40:31 <frosch123> adding them to openttd.cfg is kind of weird. i tried `graphicsset = "OpenGFX2 Classic = 0 1 0 0"`. older versions would happily ignore that, show no error, and fallback to some default baseset of their choice
10:40:54 <frosch123> but if adding the parameters there, it would have to remain a single string enclosed by " "
10:41:08 <frosch123> so there could also be some other separator than "="
10:41:15 <frosch123> i just copied the syntax from the newgrf config
10:41:41 <frosch123> not sure whether i should also add the uniqueid or md5sum
10:42:55 <locosage> make a separate section like static newgrfs
10:43:43 <frosch123> what section title and what content?
10:44:09 <locosage> idk, [graphicsset] with same content as any other static grf, no?
10:44:12 <frosch123> if some other version of ottd edits "graphicsset", i do not want the parameters to carry over
10:45:21 <truebrain> You could do that by explicit removing the section when an older config version is loaded
10:45:21 <frosch123> alternative is to deprecate "graphicsset" entirely, and add a separate new thing
10:52:16 <andythenorth> can we do the multi-ship-cargo-holds thing from JGRPP?
10:53:41 <_zephyris> frosch123: I haven't started anything, I was just getting ideas... I think it's probably beyond my ability!
11:03:50 <truebrain> So ... I have 3 options, and it is hard to pick. On crash, I can either: (1) [Windows/Mac only] show a dialog that asks to transmit the crash to us. It most likely works, as most of our crashes are not in the HTTP-stack. (2) restart the process with a special parameter that shows a dialog asking exactly that question. We are now in a known-good-state, so that will work fine. (3) on next startup,
11:03:50 <truebrain> show a popup: the game crashed in the last run, want to transmit the crash report?
11:03:58 <truebrain> (3) seems to be the easiest cross-platform
11:04:08 <truebrain> (2) could be an extensions on (3)
11:04:21 <truebrain> (1) is not going to work on Linux
11:07:23 <locosage> (3) won't work if game doesn't even start
11:38:13 <truebrain> the downside of (3) is that it requires the user to start OpenTTD again; but I guess that is not unlikely to happen in most cases. Which would be sufficient. (2) would require a lot of extra and special code .. (1) would be the cheap solution. Meh, tricky 🙂
11:38:31 <frosch123> (4) do it like the opt-in-survey. ask in advance whether to submit crash reports
11:39:03 <truebrain> wouldn't that be weird? On startup, ask: he, dude, say, just in any case, when it happens so, unlikely, I know, the game crashes .. should we send it to the developers?
11:39:11 <truebrain> feels weird 😄 But yeah, that is for sure an option too 🙂
11:39:35 <frosch123> isn't that also how other software does?
11:39:47 <truebrain> `As backtrace() is only implemented in 10.5 or later` .. guess I am rewriting this function for MacOS, as 10.5 .. is old 😛
11:39:50 <frosch123> on first start show the privacy stuff popup with all the options
11:39:59 <truebrain> frosch123: most software ask on crash whether to send the crash report 🙂
11:40:01 <frosch123> survey, crash report, ads to support ootd
11:40:19 <truebrain> but okay, we could rework the survey window to ask for consent, that is true
11:40:27 <truebrain> "Send crashreport yes/no"
11:40:55 <truebrain> hmm, thinking about it, as we have one nag screen already, why not make full use of it ..
11:41:17 <truebrain> there is still a chance that sending the crash report won't work on crash, as we are in an unstable state .. but JGRPP has some nice code to catch a crash in a crash 😛
11:43:45 <truebrain> yeah, I was initially thinking you meant two dialogs
11:43:52 <truebrain> but if we fiddle it into a single dialog, that should be fine
11:44:09 <truebrain> just .... means .... I need to touch GUI code ...... 😦 😦
11:44:41 <frosch123> but you only need to add a setting to an existing gui, instead of writing a whole process with a new window
11:44:54 <truebrain> except that the window will need some refactoring to look pretty
11:44:57 <truebrain> creating a window is easy
11:45:03 <truebrain> but the constant compile->check circle
11:45:07 <truebrain> is just ... fucking annoying 😛
11:45:26 <frosch123> oof, be careful, don't go down the andy route
11:45:27 <truebrain> and then ofc, the infinite amount of "feedback" from people
11:45:31 <truebrain> "try this, try that"
11:45:44 <truebrain> I wish I could write GUIs in YAML or something 😛
11:46:30 <_jgr_> For development, it's worth turning LTO off so that compiling isn't so slow
11:46:49 <truebrain> the debug builds do LTO? oof 😄
11:47:04 <truebrain> but even so, the cycle is just annoying to deal with; always puts me off making any GUI changes, I have to admit
11:47:16 <truebrain> especially if the window doesn't popup on startup 😄
11:47:18 <_jgr_> I never use debug builds TBH
11:47:32 <truebrain> ah; in that case, Debug builds don't do LTO 😛
11:47:35 <truebrain> the linking is very fast 🙂
11:47:43 <_jgr_> Without any optimisation, codegen is just abysmal
11:47:59 <truebrain> I reach 300 frames per second; I don't have that experience 🙂
11:48:10 <frosch123> i always do debug build, compilation is fast, but grfscan is slow :p
11:48:19 <truebrain> clean up your grfs! 😄
11:49:16 <frosch123> i wanted to optimise the grfscan, but then learned it is fast in release build :p
11:49:27 <truebrain> we should still cache the results 🙂
11:49:31 <truebrain> it is "fast", but not -that- fast 🙂
11:51:14 <truebrain> the annoying momemt you can't generalize some print statements, as Windows needs wchar .. and converting from char to wchar is annoyingly difficult 😛
11:51:31 <truebrain> (and not something I want to do during a crash, runtime)
11:51:35 <locosage> idk what happened to lto but master started to build fast in release too for me recently
11:52:16 <truebrain> time to see if the crash log on MacOS "just works" .. would be nice
11:53:05 <truebrain> the width is annoyingly small, but that is not my problem 😛
11:53:58 <truebrain> and after that, MacOS says: this process crashed, want to send the report to Apple? 😄
11:55:13 <frosch123> there is now a crash.json instead of crash.log?
11:56:13 <frosch123> last week there was a discussion whether to make json lib mandatory, now there is a breakpad lib and json is used in crash handler :p
11:56:35 <truebrain> the discussion about the json lib was for the crash handler, or did you forget? 😄
11:56:50 <frosch123> i do not keep up with the chat log, but sometimes things take surprising turns
11:57:10 <truebrain> the exact context we had the conversation in, was the crashlog 🙂
11:57:14 <frosch123> oh, i did not know what the json was for last week
11:57:25 <truebrain> it was like in a line above before you replied 😛
11:57:36 <frosch123> i only voiced in on whether to have a dependency discussion on a header-only lib
11:58:32 <truebrain> so to catch you up to speed: I want to transmit crashes to us when they happen, instead of depending on users to make a report
11:58:50 <truebrain> for that, I added minidump to Linux / MacOS, as only Windows had that (in the form of breakpad)
11:59:09 <truebrain> and I am now generating a JSON file with metadata, so we can analyze both and create decent crash reports in Sentry
11:59:29 <truebrain> "catch you up to speed", wow, that is just terrible English, holy crap
12:00:01 <truebrain> and that can't be surprising, as I am talking about it for 2 years now 😛 😄
12:10:17 <truebrain> Draft, as I am looking for opinions 🙂
12:12:25 <truebrain> you are not allowed to attach a `.json` file to a ticket on GitHub
12:49:09 <truebrain> I wrote down the opinions I remember; curious about other peoples input on this
12:55:58 <Eddi|zuHause> sounds reasonable.
12:56:38 <frosch123> i have no idea about the dependencies of libpng. but when i think a about optional dependencies, i think about all those people who run an ottd server on a raspberry
12:56:53 <frosch123> loading heightmaps is no essential part of a server
12:57:05 <truebrain> and that is where the slippery slope is
12:57:11 <truebrain> is it really not essential?
12:57:18 <truebrain> a dedicated server can start from a heightmap
12:57:30 <truebrain> so what is "essential"?
12:57:34 <frosch123> if you say 🙂 i would not be sure about that really working :p
12:57:47 <truebrain> they can download it via the CLI even
12:57:50 <truebrain> but then not load it ..
12:58:09 <truebrain> always felt weird .. but people were really: "dependencies only required if you REALLY need them"
12:58:20 <truebrain> also, dedicated servers can make screenshots .. many do, in fact
12:58:29 <truebrain> they would be BMP format when libpng is not found 😛
12:58:36 <truebrain> so it is not "essential" ... but .. I consider it essential
12:58:44 <truebrain> similar for zlib, btw
12:58:50 <frosch123> dedicated servers can make screenshots?
12:58:58 <frosch123> that uses to be something people need to patch explicitly
12:59:04 <frosch123> is it now possible by default?
12:59:12 <truebrain> you need to set a resolution, as far as I remember
12:59:20 <truebrain> but there have been several patches for it
12:59:27 <truebrain> as it used to .. not work 😛
12:59:38 <frosch123> yes, that's my state of knowledge :p
12:59:51 <truebrain> anyway, change libpng for zlib or lzma, and we have a similar discussion
13:00:18 <truebrain> lzma I can understand, as it isn't always that easy to install that dependency
13:00:44 <truebrain> anyway, I just wanted to point out that there is a constant argument to be made to not make such dependencies "required"
13:00:50 <truebrain> but one can always argue: you don't actually need it 🙂
13:01:09 <truebrain> hell, even libcurl is just encouraged ...
13:01:33 <truebrain> so maybe I should make my PR make nlohmann, png, zlib and libcurl all required?
13:02:42 <frosch123> zlib is only for old savegames?
13:02:43 <Eddi|zuHause> i'd view it from a different angle: how many hoops do you have to go through to avoid the code paths where the lib is used in case it is not found
13:02:48 <frosch123> though iirc png depends on zlib
13:03:18 <truebrain> so if no lzma, it is better savegames are stored with zlib than l77 😛
13:03:24 <truebrain> or even worse: no compression!
13:04:16 <truebrain> anyway, I will keep it with nlohmann, and see how that goes 🙂 Our binaries have all libraries anyway, which 90% of the users use 🙂
13:05:12 <Eddi|zuHause> i don't think i've ever had all the libraries that configure told me...
13:06:16 <frosch123> in other news, ogfx2 readme says on building "it will take a long time...". it's now running for an hour and not done :p
13:06:55 <Eddi|zuHause> is it using more than one core?
13:07:27 <Eddi|zuHause> at one point i aliased "make" to "make -j6"
13:07:47 <frosch123> it does not use make
13:07:48 <truebrain> don't run `make -j$(nproc)` on MacOS .. it will execute `make -j` 😄
13:08:25 <Eddi|zuHause> run all the jobs. will that include steve jobs? :p
13:14:41 <truebrain> create 1 draft PR, make 4 other PRs out of it as they have nothing to do with the actual draft PR 😄
13:27:59 <truebrain> _jgr_: just as a heads-up, as I am converting from a buffer to a JSON object, I can't reuse your `TryCrashLogFaultSection`. But I will do something similar.
13:39:12 <truebrain> _jgr_: you auto-detect `sigaction`; was there any platform you know that doesn't support `sigaction`?
13:40:52 <_jgr_> truebrain: No, it's probably unnecessary now
13:41:57 <_jgr_> I originally added that back in 2016 and I was fairly cautious about that sort of thing back then
13:42:09 <truebrain> I can understand 🙂 k, tnx 🙂
13:42:13 <_jgr_> "I can't compile it" bug reports are a pain to deal with
13:42:30 <truebrain> with our switch to C++17 we removed a lot of those reports 😛
13:42:53 <truebrain> I see you moved a lot of stuff to some mmap'd area
13:42:58 <truebrain> you had that much issues with memory?
13:43:52 <_jgr_> It's mainly because I'm making a (possibly futile) effort to try to log crashes resulting form stack overflows and heap corruption issues
13:44:26 <truebrain> I like the `SA_ONSTACK`; that at least should help debugging things 🙂
13:44:28 * andythenorth busy appreciating features
13:56:37 *** HerzogDeXtEr has joined #openttd
14:12:54 <truebrain> ```Writing crash savegame...
14:12:54 <truebrain> Something went seriously wrong when attempting to fill savegame section of the crash log.
14:12:54 <truebrain> Writing crash savegame failed. Please attach the last (auto)save to any bug reports.```
14:12:57 <truebrain> it actually works 😄
14:13:59 <truebrain> what longjmps and clever signal handlers can accomplish ... 😄
14:20:30 <truebrain> _jgr_: just to confirm I am reading it right: you never implemented the `TryCrashLogFaultSection` for MacOS, right?
14:21:10 <_jgr_> Getting access to a Mac to actually test things is an issue for me
14:21:19 <truebrain> I now have a VM running \o/
14:21:23 <truebrain> so I will look into it 🙂
14:21:26 <truebrain> but more checking I wasn't going mental 😛
14:21:47 <truebrain> first, Windows .. that is also mostly a copy/paste from your work, so should be easy 🙂
14:35:13 <truebrain> `Something went seriously and unrecoverable wrong when creating the crash log. Aborting.` 😄
15:07:39 <truebrain> awh, you cannot use a `std::function` with a `__try` block 😦
15:09:56 <truebrain> well, I guess I just add a few `&` to fix the problem 😛
15:13:04 <truebrain> so after dinner, time to do MacOS! \o/ 🙂
15:15:55 *** Kitrana1 has quit IRC (Quit: Leaving.)
15:40:36 <truebrain> hmm, can't test MinGW .. I guess I just believe that the work in JGRPP is solid 😄
15:45:11 <truebrain> _jgr_: for your information, MacOS is a direct copy of Unix; it even works as expected 🙂
15:52:13 *** Wormnest has joined #openttd
16:01:53 <truebrain> hmm .. how do I backup a `back_insert_iterator` and restore it to a previous value when I want to .. I guess I can't ..
16:07:07 <truebrain> right, that should prevent a dialog from showing on Windows, even if the savegame could not be created \o/ 😄
16:07:09 <alfagamma_0007> Is that even possible?
16:07:29 <alfagamma_0007> I mean , will the crash report be logged as well?
16:08:02 <truebrain> what a silly question; no, it isn't possible, but I did it anyway? 😄
16:08:21 <truebrain> so yes, clearly this is possible 🙂
16:11:05 <truebrain> and no, the crash inside the crash will not be reported with a crashlog; it will only tell you it failed, nothing else (as there is nothing else to report on, ofc)
16:11:56 <alfagamma_0007> A crashing recursion
16:12:13 <truebrain> it is exactly not that
16:12:45 <alfagamma_0007> Nvm me dum dum
16:27:35 *** Flygon has quit IRC (Quit: A toaster's basically a soldering iron designed to toast bread)
16:33:45 <truebrain> right, when these 5 PRs are reviewed, I will rebase 11232 again 🙂 Otherwise I keep on rebasing 😛
16:48:46 <andythenorth> object limit is 64000 now eh?
16:48:57 * andythenorth as though we don't have docs or src to read :P
16:50:29 <locosage> oh, can do 5 state rivers now
16:51:59 <locosage> even 7 state, 2401 object
16:59:33 <frosch123> approaching 5 hours... so many gentoo memories 🙂
16:59:49 <truebrain> I hope you are doing other things in the meantime 😄
17:00:05 <truebrain> like reviewing my PRs! (lolz, sorry, couldn't resist 😄 )
17:00:16 <truebrain> how is eints on sqlite going? Any progress lately?
17:00:56 <frosch123> sqlite eints was mostly finished, it just did not work well
17:01:13 <frosch123> now i think the problem is acutally not in eints, but in the sync script
17:01:43 <frosch123> ah, i wanted to run the sync script locally with the real eints instance, to actually check
17:01:50 <frosch123> did not do that yet
17:02:10 <frosch123> but eints+script locally needs like 20 seconds to sync, so if it needs 40 minutes in the cloud, there must be some throttling
17:02:21 <truebrain> _glx_: owh, so that was the * 2 doing ... my mistake was to think it had to do with the amount of files, but it has to do with their length (wide strings) .. lol, oops 😄
17:02:52 <truebrain> throttling? If any, it is inside eints .. nothing else does any form of throttling ..
17:03:36 <frosch123> well, as said, i did not confirm my theory yet. but my guess right now is that github somehow things eints-sync is a coin miner and restricts it :p
17:04:09 <truebrain> frosch123: I doubt that, really 🙂
17:04:41 <andythenorth> broke my object IDs 😄
17:05:21 <truebrain> I split up the eints job, giving a bit more visibility what is going on .. total runtime of 4 minutes, 90s of it is in a git clone, lol
17:05:27 <truebrain> guess it should do a shallow clone of depth=1
17:05:40 <frosch123> anyway, while sqlite would probably scale better in theory for more projects, right now it is a lot slower than the json files
17:05:51 <truebrain> and it takes a minute between `Done` and `Check updates from git` .. not sure it does between those two moments, as it doesn't tell 😛
17:06:03 <truebrain> frosch123: meh; that is too bad 🙂
17:06:25 <frosch123> well, it's mostly that the json stuff runs in 20 seconds locally :p
17:06:45 <frosch123> which is very different to the production version
17:06:58 <truebrain> so 20s locally already sounds like a long time
17:07:07 <truebrain> ```Sat, 26 Aug 2023 18:40:33 GMT HEAD is now at a3cd07e Change: [CI] switch to reusing workflows (#85)
17:07:07 <truebrain> Sat, 26 Aug 2023 18:41:01 GMT [2023-08-26 18:40:33] Check updates from git```
17:07:15 <truebrain> for OpenGFX, for example
17:07:26 <truebrain> 28 seconds between those two lines .. script doesn't say what it does there 😄
17:11:57 <truebrain> `Sat, 26 Aug 2023 18:40:32 GMT [2023-08-26 18:39:36] Update from git` ... those timestamps don't add up 😄
17:12:15 <truebrain> okay .. so the problem is somewhere else:
17:12:26 <truebrain> ``` [2023-08-26 18:39:38] Upload/Merge translations
17:12:26 <truebrain> [2023-08-26 18:40:19] Download translations```
17:12:32 <truebrain> this is for OpenGFX 🙂
17:15:25 <truebrain> fastest is appstore btw
17:15:51 <truebrain> fewer languages, which helps; but still 🙂
17:16:08 <truebrain> still takes ~20 seconds to upload the files 😄
17:16:42 <truebrain> frosch123: want a copy of the JSON files from production?
17:17:16 <_glx_> though opengfx is only 3 strings
17:17:36 <truebrain> `122M ./openttd-master`
17:20:09 <frosch123> if you multiply that with 150 (50 languages, upload+download+extra), you do not get to 40 minutes
17:20:19 <truebrain> I DM'd you a tarball; before we point a finger to GitHub, see what happens locally when you restore a backup
17:20:32 <truebrain> frosch123: not sure where you get 40 minutes from; the current runtime is ~4 minutes for openttd-master
17:22:54 <frosch123> lol whut? when i started with the problem i assumed it starts at 20:00 and finishes at 20:40
17:23:09 <truebrain> a total eints run was like 20-ish minutes, was the complaint
17:24:01 <truebrain> I now split it up in a chunk per project, to give a bit more visibility, and a full run now takes ~10 minutes
17:24:12 <truebrain> the problem is, the number is going up
17:24:20 <_glx_> I think github does better now on that side
17:24:31 <truebrain> 2 months ago it was on average 9m23s
17:24:39 <truebrain> it currently is 10m30s
17:24:49 <truebrain> there is a clear trend 🙂
17:25:10 <truebrain> my guess was that it has to do with the history
17:25:35 <_glx_> our services are too stable 🙂
17:25:50 <frosch123> ok, 10 minutes is a completely different figure that i assumed
17:27:13 <frosch123> btw. locally the most time was spent on recomputing the language statistics for all languages after uploading the baselng, and for every single one again after uploading the translation
17:27:25 <truebrain> how does it make those backup files?
17:27:43 <truebrain> I assumed it was a move
17:27:47 <frosch123> we can disable those backup files, they are a weird debugging feature, unused otherwise
17:28:14 <truebrain> I assumed it could be useful if something got corrupted?
17:29:41 <truebrain> ah, yes, it is a rename; so that should be fine
17:29:59 <truebrain> anyway, see if you can reproduce the slowness when you use the json files I just gave you; might be interesting to see if locally it is still 20s
17:30:13 <frosch123> oh, we already reduced the backup files to 1 :p instead of the default 5
17:30:33 <truebrain> it already consumes more diskspace than I find reasonable 🙂
17:32:11 <truebrain> maybe switching to simplejson or ujson might already give some speedup .. much faster JSON libraries; normall you don't notice that, but with files these sizes, it might
17:32:36 <frosch123> i found the json time neglibe locally
17:33:34 <truebrain> let me know if that still holds true with production-worthy JSON files 😄
17:34:51 <truebrain> every time when I try to read eints source code, I just ... blank
17:35:01 <truebrain> so many abstractions and redirects
17:35:14 <frosch123> both my test data and the production data are 61MB
17:35:23 <frosch123> (not counting the bup)
17:35:41 <frosch123> truebrain: it's not different for me :p
17:35:59 <frosch123> right now i am trying to run lang_sync download on production
17:36:06 <truebrain> I am still so tempted to make a new version of eints, that lives on Cloudflare (and doesn't need a backend) .. it is so tempting 😛
17:36:13 <frosch123> and i have to debug it, because the errors are not helpful
17:36:37 <frosch123> the string validaiton is kind of isolated in eints
17:36:47 <frosch123> so is is not out of question to rewrite the rest
17:36:51 <frosch123> but also incredible boring :p
17:36:58 <truebrain> and not the first time 😄
17:37:08 <truebrain> but I am a bit annoyed we can't offer translation access for NewGRFs
17:37:30 <frosch123> that did not fail for technical reasons
17:37:57 <truebrain> well, I know at least a few technical reasons; but what other reasons are there?
17:39:02 <frosch123> 3/4 years ago, whenever eints was moved to github, andy was the only potential user, and he did not want to give anything push rights, nor liked eints making PRs
17:39:33 <frosch123> and with eints on coop there was already the problem that 90% of projects, which were offered for translations, were dead
17:39:38 <truebrain> GitHub matured a bit; you now just have to install a GitHub Apps to the repo; so there is that at least 🙂
17:39:58 <truebrain> we also still have the open request for the OpenTTD on Android fork
17:40:11 <truebrain> guess that is not all that important atm, but we also never managed to do that .. lot of manual labour to add any project
17:40:17 *** Kitrana has joined #openttd
17:41:20 <truebrain> but anyway, Cloudflare Workers can query a DB (implemented on (distributed?) SQLite)
17:41:48 <truebrain> so when talking about eints, I keep wondering if it can't just live on Cloudflare .. eints is such a memory hungry, single-instance, "nobody knows how it works" kinda thing
17:43:41 <frosch123> anyhow, on the plus side: it is possible to make eints 2x or 3x times faster, by reducing the number of times eints rechecks the string validity
17:44:28 <truebrain> A Cloudflare database (called D1) can be at most 500MB
17:44:31 <truebrain> guess that should be fine 😛
17:45:21 <frosch123> sqlite did not turn out particulary fast, because for the steps we are talking about (upload/download) it has to essentially fetch the entire table
17:45:44 <frosch123> at which point just loading a json file is faster
17:45:53 <truebrain> that I understood 🙂
17:47:37 <truebrain> but in case it wasn't clear: I would not port eints to Cloudflare Workers in a 1-on-1 fashion; you can't even, as it doesn't support Python 😛
17:47:54 <truebrain> it would be more an actual rewrite 🙂 Knowing you run on SQLite 🙂
17:48:48 <truebrain> speed also would be less of a problem on such platform .. as it autoscales like a mofo 😛
17:49:31 <truebrain> anyway, just something that keeps happening when I look at the eints code .. I get lost .. 😄
17:49:46 <truebrain> but first on my agenda, automated crash reports!
17:50:48 <_glx_> and automatic duplicate detection ?
17:51:02 <truebrain> sentry does that for us, yes
17:51:26 <truebrain> it basically looks at the backtrace and goes: this is the same issue!
17:51:31 <truebrain> and just record the fact it has seen the issue N times
17:51:39 <_glx_> even from different OS ?
17:51:49 <truebrain> dunno, that I kinda doubt
17:51:58 <truebrain> but something to find out 🙂
17:52:28 <truebrain> part of the issue is a bit that our crashlog is in the backtrace .. and they differ per OS 🙂
17:52:51 <truebrain> I might just strip them when sending a report to Sentry; just to keep things a bit readable for us
17:53:49 <truebrain> when all my PRs I have now are done, next job is to make the CI send the breakpad symbols to a symbol server every night .. then we need to find a crash we can produce on every OS, then we can see how to send the data to Sentry 🙂
17:53:58 <truebrain> sadly, I am not aware of any crash you can trigger easily 😛
17:54:55 <_glx_> we could support alt+0 in release just for the tests
17:54:55 <andythenorth> reverse a recent fix PR 😛
17:55:16 <_glx_> or add a console command
17:55:39 <truebrain> ALT+0 is so static .. I rather have a real crash, so we can see if the backtraces etc are useful
17:56:06 <frosch123> how about "kill -6" ?
17:56:20 <truebrain> try doing that on Windows, or at the same time on different OSes 😄
17:56:54 <frosch123> ok, then load a corrupt savegame
17:57:13 <truebrain> that might be the way to do it, yes
17:57:16 <peter1139> I could merge some new feature...
17:57:30 <truebrain> it is funny, we have so many bugs, but so little crashes (that are reproduceable)
17:57:53 <frosch123> make an uncompressed save, and then alter some bytes in it :p
17:58:09 <_glx_> when they are we fix them quite fast
17:58:13 <truebrain> randomly smashing on the keyboard 😛
18:00:30 <truebrain> lalala, nothing to see, lalala
18:01:50 <andythenorth> I can spin the game 😛
18:01:54 <andythenorth> that isn't a crash though
18:02:13 <truebrain> would be odd, if the game is rotating 😄
18:03:08 <frosch123> pff, lang_sync cannot even access the language-list on production
18:03:15 <frosch123> it works fine with curl, no idea what's wrong
18:03:33 <truebrain> did you update? lang_sync had a bug where it didn't send content-type
18:03:41 <truebrain> and Cloudflare, rightfully, doesn't let that through
18:06:45 <frosch123> "download" takes 20s from production to local disk
18:07:24 <truebrain> check the workflow eints logs to see what the workflow does, time-wise 🙂
18:08:56 <frosch123> so, that's not too bad
18:09:06 <truebrain> still a long time btw; but sure
18:09:16 <truebrain> so that accounts for 30s out of the 4 minutes 🙂
18:10:13 <frosch123> hmm, does it still do a complete git clone of the whole history?
18:10:18 <frosch123> i thought that was already removed
18:10:28 <alfagamma_0007> 14.0 release when
18:10:55 <truebrain> frosch123: the checkout is done before the scripts are called; so it should be there, and ready to be used
18:11:39 <truebrain> (earlier tonight it felt like the script was doing a clone; but that is not true, I already prepared a clone before the script is called)
18:11:50 <truebrain> either way, I do see it does something git-related
18:12:07 <truebrain> ```[2023-08-26 18:35:21] Update from git
18:12:07 <truebrain> [2023-08-26 18:36:32] Upload/Merge translations```
18:12:17 <truebrain> so for sure 70s is wasted on doing something git-related
18:12:28 <frosch123> yes, that step should not be there any more :p
18:12:56 <frosch123> upload/merge + download are the steps, where it talks to eints
18:13:03 <frosch123> the rest is only in the gh action
18:13:15 <truebrain> so you have some cleaning up to do in that script of yours 😄 🙂
18:13:37 <truebrain> well, it is not onloy "download" btw .. it should also make a commit 🙂
18:13:56 <frosch123> that's 5s, you see that in the log as well
18:14:13 <truebrain> not talking about time; you are talking about responsibilities
18:14:16 <truebrain> the script should make the commit 🙂
18:14:26 <andythenorth> will `make -j 19` compile FIRS faster? 😛
18:14:27 <truebrain> that is not part of anything in the gh action 🙂
18:14:48 <truebrain> the `git push` we can talk about .. currently it is part of the script; I am so-so about where that lives 🙂
18:15:00 <frosch123> ok, correction, with "gh action" i meant the script
18:15:17 <truebrain> then I am somewhat confused what you meant 😄
18:15:18 <frosch123> with "gh action" i mean every process that runs on github, and not in the eints process
18:15:36 <truebrain> I am now even more confused 🙂
18:15:45 <frosch123> optimising eints and changing json/db backend won't affect 2.5 of the 4 minutes
18:16:53 <truebrain> okay, so let's tune down what I noticed and what worries me: for OpenGFX, a very small amount of strings, I see this:
18:16:56 <truebrain> ```[2023-08-26 18:39:38] Upload/Merge translations
18:16:56 <truebrain> [2023-08-26 18:40:19] Download translations```
18:16:58 <frosch123> well, i am kind of lost as well :p everyone was complaining that json is too slow, and i thought it would take 40 minutes to run, and kept wondering where it would take so long. and now it is actually only 10, and only 30% of that are in eints
18:17:08 <truebrain> what we also have seen, that over the months, that number has been increasing
18:17:24 <truebrain> the rest is your own doing, I am afraid 🙂
18:17:45 <truebrain> as we run eints commands 6 times, those 40s on OpenGFX, translate into a hefty amount of time 🙂
18:18:01 <truebrain> similar btw, for OpenGFX:
18:18:03 <truebrain> ```[2023-08-26 18:40:33] Upload translations
18:18:03 <truebrain> [2023-08-26 18:41:01] Done```
18:18:23 <frosch123> i would assume it akes the same time for all projects
18:18:26 <truebrain> I was wondering if it had to do with cache issues, where it purges one project or something
18:18:43 <frosch123> yes, eints only has one project in memory
18:18:58 <truebrain> I thought my configuration option modified that
18:19:16 <frosch123> if you download ogfx, and anything/anyone accesses openttd-master inbetween, it will take N seconds to reload openttd-master
18:19:37 <truebrain> it runs on an LRU, not?
18:20:05 <truebrain> OpenMSX shows this, for example:
18:20:08 <truebrain> ``` [2023-08-26 18:42:47] Upload/Merge translations
18:20:08 <truebrain> [2023-08-26 18:43:26] Download translations```
18:20:11 <truebrain> which is 10s more than OpenGFX
18:20:14 <truebrain> while I expected VERY similar results
18:20:28 <truebrain> and again, the actual time is not what worries me .. but this was a fraction of the time when we launched eints
18:22:08 <frosch123> ok, sounds like i need to split-off the statistics part from the sqlite experiment then
18:22:57 <truebrain> if this is what it is, that is fine; but if this turns out to be a quadratic increase, we might get an unexpected surprise in a few months, where it takes a day to run, or something 😛
18:23:07 <truebrain> also realise that during this time, eints is totally unresponsive
18:24:06 <frosch123> only during 30% of the time :p
18:24:19 <truebrain> well, if it reloads projects as you say ... 😄
18:24:41 <truebrain> owh, I looked it up: we went from 20m per run to now 10m per run, because it runs in the new infra now
18:24:43 <truebrain> which has a better CPU
18:25:17 <truebrain> so the % was even worse when we started out with this problem 🙂
18:25:19 <frosch123> and probably a newer python?
18:25:30 <truebrain> no, I pinned the Python version to .. 3.8? still
18:25:34 <truebrain> I am scared of updating that
18:26:26 <truebrain> ```[2023-06-19 18:37:25] Upload/Merge translations
18:26:26 <truebrain> [2023-06-19 18:39:39] Download translations```
18:26:38 <truebrain> ``` [2023-08-26 18:36:32] Upload/Merge translations
18:26:38 <truebrain> [2023-08-26 18:37:47] Download translations```
18:26:41 <truebrain> both openttd-master
18:27:10 <truebrain> so part of the problem has been solved by just throwing more CPU against it 😛
18:28:22 <truebrain> the new workflow also seems to have helped, where eints has a bit more time between being export/import
18:28:27 <truebrain> of different projects
18:29:00 <truebrain> so part of this problem has gotten smaller .. so we have a few more months till it is back to 20min runs 😄
18:29:50 <truebrain> `"--project-cache", "10",` is set btw
18:30:05 <truebrain> `"--language-file-size", "10000000",` <- do I want to know what that does? Not really ...
18:32:17 <frosch123> it's some parameter in bottle to reject requests with larger payload
18:35:39 <truebrain> so long story short: SQLite is not giving any performance benefits; we still see a slow increase in time it takes to sync, but with new hardware / workflow, the runtimes have halved, and are 30% of the total runtime. The rest is git-related activities.
18:36:15 <truebrain> did I sum that up correctly? 🙂
18:38:23 <frosch123> and half of to 30% are string-checking, which is done 3 times instead of once
18:38:36 <truebrain> so that is something you can fix? 😄
18:38:49 <truebrain> and the 30% is only true for openttd-master, as it does a silly `git fetch` 🙂
18:38:54 <frosch123> the "git" thing should be the first step
18:39:01 <frosch123> likely easiest, and most effect
18:39:04 <truebrain> I just don't understand what it actually does
18:39:08 <DorpsGek> - Update: Translations from eints (by translators)
18:39:11 <truebrain> it keeps track of changes ...
18:39:16 <truebrain> but does a `reset --hard`
18:39:19 <truebrain> I am a bit confused 😄
18:39:35 <frosch123> it was written assuming the checkout is persistent
18:39:59 <frosch123> so it updates an assumingly old checkout from last run, and discards any stale commits from earlier runs
18:40:18 <frosch123> but the gh action always starts from scratch, so all those pulls and resets are useless
18:40:33 <truebrain> I am just a bit confused what that `changes` part is doing
18:40:48 <truebrain> owh, to see if anything has actually changed?
18:41:04 <truebrain> ah, okay .. so it tries to avoid uploading everything if nothing has changed
18:41:08 <truebrain> but it is an all or nothing scenario
18:41:37 <truebrain> we run a `--force`, so yeah 😛
18:42:12 <truebrain> ah, and now I see why the logs make no sense ... they are not in order
18:42:19 <truebrain> one is on stderr, the other on stdout, I think
18:42:21 <truebrain> and one is buffered
18:42:42 <truebrain> owh, no, the Python commands are send to stdout/stderr instantly, and the Python prints are not
18:42:55 <truebrain> it is just confusing, in the logs 😛
18:43:33 <truebrain> workflow first does `commit-to-git`, and `update-from-git` after, btw
18:44:54 <frosch123> yes, that part is unfortunate, but intentional 🙂
18:45:34 <frosch123> the "update-from-git" is essentially feedback to eints that confirms the commit
19:00:31 <frosch123> oh dear, i looked at the ogfx2 build script, to figure out how far it got
19:00:35 <frosch123> it's not even half done :p
19:01:33 <frosch123> i only wanted to make a testgrf :p
19:06:41 <truebrain> plenty of time to review PRs! 😛 😄
19:06:50 <truebrain> I am suckering you in, notice it? 😛
19:27:16 *** gelignite has quit IRC (Quit: Stay safe!)
19:33:38 <_zephyris> frosch123: Sorry.... It does an aweful lot of custom 32bpp->8bpp conversion in Python.
19:35:40 <frosch123> if i had known it advance that it would take 14 hours, i would have planned it differently :p
19:35:51 <frosch123> i am just used to andy complaining that 10s is very long
19:35:55 <andythenorth> "shorter letter, if I had more time"
19:36:17 <frosch123> yeti took some hours at it's worst time, then we optimised nml
19:36:25 <andythenorth> no but seriously what is ogfx2 doing, that can't be made faster with a multiprocessing pool?
19:36:49 <truebrain> 14 hours on 14 cores is still a long time 😛
19:36:53 <frosch123> it sure can be parallelized, but it does not do so yet
19:37:08 <_zephyris> Yup, currently single threaded...
19:37:29 <frosch123> it also generated stuff like snow transitions for tropic, but i guess those are just insignifciant jokes in the whole picture
19:39:22 <frosch123> i am kind of happy that "it takes a long time to build" is actually true for once
19:39:41 <truebrain> you don't have to take it with an andy grain of salt, you mean? 😄
19:40:12 <frosch123> it's currently at "infrastructure"
19:40:22 <frosch123> which may be the largest part, but no idea
19:41:02 <frosch123> inside the scripts it does most things for 1x, 2x and 4x zoom, which can probably be parallel as well
19:41:14 <andythenorth> feels like 'with' might be useful in some of these scripts
19:41:20 <andythenorth> or for i in [names]
19:42:46 <andythenorth> hmm....graphics processing looks about as intense as mine
19:42:53 <andythenorth> I always think it should be simpler, but it never is
19:44:25 <locosage> write in grf-py, processing is quite simple and can be parallelized in like half an hour 😜
19:44:43 <_zephyris> frosch123: The only script that really needs speedup is `custom_dither.py`, and that is enormously parallelisable.
19:44:46 <frosch123> i don't think grf-py does anything of this
19:45:37 <locosage> it may not do custom processing but it's quite simple to add
19:45:56 <locosage> and integrates with caching
19:46:49 <frosch123> ah, it's at "bridges" now
19:46:50 <locosage> _zephyris: why custom though? is pil dither not good enough?
19:49:20 <_zephyris> Yeab, definitely not. Eg. Mine can restrict dithering (per pixel) to specific hue groups in the ttd palette.
19:51:07 <_zephyris> Making sprites with recolouring and animated colours in a 32bpp first pipeline is quite easy with it.
19:55:20 <_glx_> I guess the fmt::print could be switched to ShowOSErrorBox at some point
19:55:35 <truebrain> no, you want it to continue
19:55:38 <truebrain> so that would be annoying, I guess
19:55:58 <truebrain> those that can, already should
19:56:02 <truebrain> well, MessageBox, but tomato tomato
19:56:19 <locosage> _zephyris: have you tried writing it with numpy? should be much faster than get/putpixel
19:56:44 <locosage> especially if can vectorize some stuff
19:57:05 <_glx_> or maybe open a console
19:57:20 <truebrain> always the best option 😄
19:58:13 <_glx_> but will need a mecanism to wait before exiting
19:58:31 <_glx_> else the output will most likely flash and disappear
19:59:04 <_glx_> hmm we have the crash window
19:59:07 <truebrain> everything important to the user should already show up and be showed pretty
19:59:14 <truebrain> just when to show filenames yes/no need a good cleanup
19:59:35 <truebrain> I didn't want to bother the user too much that creating a crash.dmp or crash.png failed, honestly
20:00:06 <truebrain> I could just do: `(failed to create crash.dmp)` as filename
20:00:15 <truebrain> informative enough 🙂
20:00:37 <_glx_> yeah perfectly fine solution
20:01:29 <andythenorth> locosage: is numpy notably faster?
20:01:54 <locosage> andythenorth: I don't remember how fast it is for processing every pixel
20:02:00 <_glx_> the fmt::print writing to void on windows are just the result of unification of crash handling
20:02:01 <locosage> but it's much faster in vector operations
20:02:17 <_zephyris> locosage: I suspect its not easily optimised in python, dithering propagates errors through the image which will always need big python loops. Ie. Not whole image/array operations.
20:02:24 <truebrain> _glx_: yup! And if you so happen to have a console, like I do, it is actually pretty useful, and feels more like the other targets 🙂
20:04:09 <locosage> lambdas are python, python is slow xD
20:04:37 <locosage> though may be faster than get/putpixel
20:05:07 <_glx_> we can add a call to CreateConsole() but for end user the logging of events is not very useful
20:05:51 <andythenorth> Horse graphics processing takes 5 or 6 seconds, but I haven't really explored optimising it
20:05:53 <locosage> oh, btw, `if i in dice_recolour_map.keys()` may be slower than just `i in dice_recolour_map`
20:05:54 <andythenorth> that's very long though
20:05:56 <truebrain> _glx_: if I manage to automate sending crash reports, nobody will care anymore 🙂
20:06:20 <andythenorth> it's not easy to instrument things inside a multiprocessing loop 😛
20:06:29 <andythenorth> guess I could test it in single-thread
20:07:26 <locosage> locosage: fastest is probably just `dice_recolour_map.get(i, i)`
20:08:36 <andythenorth> maybe I should log different pipeline actions
20:10:06 <locosage> locosage: fastest with .point I mean, numpy recolour should still beat that by a lot
20:11:15 <locosage> oh, wait, you recolour 8bpp?
20:11:19 <locosage> then definitely numpy
20:13:00 <locosage> as you can put them all in one array you can replace all colours at once
20:16:15 *** Kitrana has quit IRC (Quit: Leaving.)
20:16:57 <truebrain> Windows crashlog allocated 128k + 64k of memory ON THE STACK 😛
20:17:05 <truebrain> JGRPP already addresses that, I am going to copy it 😄
20:23:18 <truebrain> wchar_t crash_msgW[65536]; and co 🙂
20:26:33 <locosage> locosage: hm, I did this test though when writing 32bpp recolour and it's much faster to replace colours one-by-one
20:27:07 <locosage> when there aren't many colors to replace
20:28:47 <andythenorth> how fast is Horse in grf-py now?
20:28:58 <andythenorth> it's really slow with nml etc here
20:30:03 <locosage> it's not really comparable as it's crappy decomplied code
20:31:12 <andythenorth> total compile is 27 seconds for me
20:31:15 <andythenorth> that's pretty poor
20:31:24 <andythenorth> it was ~20 seconds but I added stuff
20:31:34 *** HerzogDeXtEr has quit IRC (Read error: Connection reset by peer)
20:31:45 <peter1139> truebrain, does it assume that malloc etc can't be used...
20:32:16 <truebrain> JGRPP uses VirtualAlloc, which seems like a fine one to use 🙂
20:34:22 <truebrain> `crash_msgW_length = ((crashlog_length + 16) * 3) / 2;` .. that kinda confuses me, what is the * 3 / 2 .. time to find out 😄
20:36:43 <_jgr_> Probably n ot the clearest way for me to have put it, but it's basically that the char based buffer is half the size of the wchar_t based buffer
20:38:00 <locosage> andythenorth: rn decompiled horse builds in 175 sec clean and 167 sec cached but 162 sec of those is OOM protection algorithm that is completely redundant for horse. So about 5s re-bulid time seems doable.
20:38:07 <locosage> and that I didn't really start optimizing it yet...
20:38:26 <_glx_> then bytes len to wchar_t len conversion with / 2 I think
20:38:33 <truebrain> _jgr_: `crashlog_length` is in char_t, so .. I don't get it 😄
20:39:04 <truebrain> _glx_: I wouldn't know what that part has to do with filenames, tbh .. seems unrelated
20:39:24 <truebrain> this is the buffer to convert the crash.log from char_t to wchar_t, if I get it right
20:39:32 <truebrain> so that is just `* 2`, isn't it?
20:39:59 <Eddi|zuHause> is that true for utf-8 strings?
20:40:35 <_glx_> there's most likely only ASCII here 🙂
20:41:16 <_jgr_> Clearly I should have commented this, as I can't remember why I added that now
20:41:30 <truebrain> so we are now both puzzled! \o/
20:41:48 <truebrain> maybe you now have cut-off crash.log text in the dialog
20:41:54 <truebrain> it is not like anyone actually reads it 😛
20:42:24 <_jgr_> It wouldn't be cut off, as the value is in sizeof(wchar_t)
20:42:30 <_jgr_> It's really a 50% safety margin
20:43:00 <_jgr_> Probably I was being too defnesive
20:43:10 <truebrain> 🙂 That I can appreciate 🙂
20:43:47 <_glx_> better get a too big buffer than the opposite 🙂
20:46:58 <_glx_> and I guess in the end it will still be shorter than the static buffer we used
20:47:44 <_glx_> as we already removed a lot of stuff from the crashlog
20:48:44 <_jgr_> I added a whole bunch of other stuff, so I needed to bump the length limit
20:50:26 <_jgr_> I was having the dialog fail to show up because of running out of stack
20:50:46 <truebrain> I am surprised it fits on the stack for vanilla 😛
20:51:00 <truebrain> one buffer is `static`, and the rest isn't .. which is also weird
21:01:41 *** keikoz has quit IRC (Ping timeout: 480 seconds)
21:03:22 *** Wolf01 has quit IRC (Quit: Once again the world is quick to bury me.)
21:04:17 <truebrain> _jgr_: crashlog_win.cpp, line 1068, should be `screenshot_filename` 🙂
21:04:27 <truebrain> (sorry, to lazy to make an issue / PR atm 😛 )
21:06:06 <_jgr_> Thanks, I'll fix that shortly 🙂
21:33:13 <truebrain> also the old code was really scared of overflowing buffers 🙂 But I guess if we just calculate it correctly, we shouldn't be scared ... I hope 😄
21:34:52 <truebrain> fits exactly, nice and snuggly 🙂
21:38:28 <truebrain> _glx_: something like this?
21:40:54 <truebrain> `src\crashlog.h(11): fatal error C1014: too many include files: depth = 1024`
21:46:03 <_glx_> that's a lot of includes
21:47:11 <truebrain> I now need to increase the size of that dialog, so the next line fits
21:47:19 <truebrain> but ... the resource files are ... troubling to read, for me 😄
21:51:30 <truebrain> ha, figured it out 😄 Tnx to a commit glx did to fix the size a few years back 😛
21:57:02 <truebrain> looking back, it should have been part of my previous PR _glx_ , sorry about that. Didn't know it would be this simple 😄
21:57:53 <_glx_> truebrain: yeah not the first time 🙂
22:02:06 <truebrain> MSVC IDE tells me `_snwprintf` might cause a buffer overrun .... huh?
22:02:16 <truebrain> the second parameter is for exactly that?!
22:03:42 <_glx_> it can't know you used it properly
22:03:46 <_jgr_> truebrain: Some of the snprintf variants don't null-terminate the output buffer in some cases
22:04:01 <truebrain> I looked thisone up, MSVC docs say it always adds a \0
22:04:05 <truebrain> and it never exceeds the length
22:04:23 <truebrain> _glx_: it is not a warning, it is a statement it makes. And as far as I can tell, it is wrong 😛
22:04:40 <truebrain> it also says the pointer could be null, which is also a weird claim 🙂
22:05:03 <truebrain> `char *crashlog_dos_nl = reinterpret_cast<char *>(filename_buf + filename_buf_length * filename_count);`
22:05:06 <truebrain> `convert_to_fs(crashlog_dos_nl, ...`
22:05:13 <truebrain> and it claims `crashlog_dos_nl` can be 0
22:05:28 <truebrain> I dunno ... that would be REALLY odd ..
22:05:33 <truebrain> and it can know this just fine
22:05:48 <_glx_> I only listen to compiler, intellisense is not always correct
22:06:14 <truebrain> it doesn't warn with static buffers
22:06:42 <_glx_> it's just a static analysis
22:06:55 <truebrain> okay, so a bit pointless in that sense ..
22:07:41 <_glx_> it's like some of the codeQL reports we discarded
22:08:23 <_glx_> they are right, but we do it properly and are not affected by the potential issue
22:08:43 <truebrain> double-checked with asserts that I am correct it does not overflow 🙂
22:10:44 <_glx_> hmm it's annoying the CodeQL run on master can be cancelled
22:12:18 <truebrain> I am a bit cheeky in how I did 11240 .. I allocate a bit more memory than JGRPP, but I also simplified the code by using a single snprintf .. it is a balance 🙂
22:16:02 <truebrain> I think I did everything correctly .. VirtualAlloc is a nice addition here 🙂
22:39:17 <_glx_> that's a funny syntax 😉
22:44:11 <truebrain> Will fix tomorrow, now some Zzz
23:03:39 *** Kitrana has joined #openttd
23:13:32 *** Kitrana has quit IRC (Quit: Leaving.)
23:46:48 *** agneli has quit IRC (Remote host closed the connection)
continue to next day ⏵
