IRC logs for #openttd on OFTC at 2022-07-09
⏴ go to previous day
00:09:42 *** WormnestAndroid has quit IRC (Read error: Connection reset by peer)
00:18:47 *** WormnestAndroid has joined #openttd
00:58:43 *** Smedles has joined #openttd
02:06:46 *** Wormnest has quit IRC (Quit: Leaving)
02:16:02 *** debdog has quit IRC (Ping timeout: 480 seconds)
03:16:48 *** D-HUND is now known as debdog
05:14:41 *** OsteHovel_ has joined #openttd
05:21:42 *** OsteHovel has quit IRC (Ping timeout: 480 seconds)
06:03:08 *** HerzogDeXtEr has joined #openttd
06:03:56 *** andythenorth has joined #openttd
06:09:49 *** andythenorth_ has joined #openttd
06:16:09 *** andythenorth has quit IRC (Ping timeout: 480 seconds)
07:30:39 <andythenorth_> so grf version numbers
07:30:57 <andythenorth_> date, or just an incrementing number?
07:32:25 <andythenorth_> semver doesn't work when releasing multiple grfs from a monorepo :P
07:33:52 * andythenorth_ might do name-megaversion-incremement
07:34:04 <andythenorth_> megaversion implies a grfid changed
07:34:28 *** sla_ro|master has joined #openttd
08:29:04 *** andythenorth has joined #openttd
08:34:02 *** andythenorth_ has quit IRC (Ping timeout: 480 seconds)
08:34:22 <TrueBrain> hihi, 3 commands never actually check if you are commanding the first vehicle in a chain
08:34:29 <TrueBrain> in result, one could reverse a wagon in a train
08:34:32 <TrueBrain> which ..... crashes
08:54:21 *** andythenorth_ has joined #openttd
09:01:34 *** andythenorth has quit IRC (Ping timeout: 480 seconds)
09:03:27 <TrueBrain> hmm .. autoreplace on hangars should always crash, but we have no reports about that
09:03:33 <TrueBrain> guess I don't understand the code well enough :P
09:04:13 *** Smedles has joined #openttd
09:23:48 <TrueBrain> funny how the fuzzer created a crash I can reproduce by injecting that specific command, but I don't see how it actually breaks ..
09:23:59 <TrueBrain> it suggests there is an aircraft that is not a primary vehicle but still is in a depot
09:24:06 <TrueBrain> (shadows are never on an actual tile)
09:24:21 *** gelignite has joined #openttd
09:24:27 <TrueBrain> fixing it is easy, but .. I rather understand it first :D
09:30:57 <TrueBrain> ah .. now the monkey comes out of the sleeve ..
09:31:32 <TrueBrain> that should be an invalid index not? Hmm
09:31:59 <TrueBrain> lol .. you can do auto-replace on tile 0 :D
09:35:29 <TrueBrain> and all the aircraft shadows etc are on tile 0 .. so yeah.. this makes sense now :P
09:43:15 <TrueBrain> you could mass start/stop vehicles of others (which failed to execute, but not the point :P)
09:44:31 <TrueBrain> okay, that was the last crash the fuzzer found yesterday .. let's see if it can find any others now :)
09:50:14 <TrueBrain> OpenTTD doesn't compile with MSAN .. lol? :D
09:54:30 <TrueBrain> well, it does compile, just strgen fails, so you cannot compile :D
09:55:12 <TrueBrain> StrMakeValid() seems to be the issue, but the error isn't very specific :P
10:25:33 <TrueBrain> the way I am fuzzing, runs 10k commands and then cycles the application
10:25:39 <TrueBrain> when one of those commands fails, it stores which command
10:25:48 <TrueBrain> now that is mostly stable, but sometimes it crashes because of a sequence of commands
10:25:53 <TrueBrain> sadly, that information is lost
10:26:03 <TrueBrain> our own crash.log indicates it is related to GroupStatistics
10:27:26 <TrueBrain> triggered by CmdSetAutoReplace
10:27:34 <TrueBrain> I wonder what the sequence of events is ... hmm
10:30:55 <TrueBrain> I wonder if I make this change in OpenTTD a bit pretty, if I can get OpenTTD accepted in oss-fuzz
10:31:14 <TrueBrain> then Google will run the fuzzer on many more machines .. so many more ways to find things :D
10:54:37 <TrueBrain> it is funny how the fuzzer manages to do things .. start new AIs, yup, he figured out how he can do that :P
11:17:41 *** sla_ro|master has quit IRC ()
11:18:34 <Samu> estimate is already == 0
11:18:42 <Samu> meaning this is the goal
11:19:05 <Samu> but then it detects that those directions have already been tried, but how...
11:19:28 <Samu> this is the first time estimate is == 0, so, I don't understand
11:21:38 <TrueBrain> "coverage 7.47%, no crashes yet"
11:22:23 <TrueBrain> seems you did a good job michi_cc , with the command conversion :)
11:23:36 <michi_cc> That depends very much on who you ask about this topic :)
11:24:36 <TrueBrain> haha, as with any change ;)
11:24:58 <TrueBrain> I am still shocked that you managed to convert to/from network with templates like this :P
11:25:55 <TrueBrain> I wonder if/how can I find out what is crashing the game with CmdSetAutoReplace ... tricky tricky ...
11:29:22 *** virtualrandomnumber has joined #openttd
11:30:38 *** virtualrandomnumber has quit IRC ()
11:36:21 <michi_cc> The templated (de-)serialization is actually not anything I would consider especially clever. I am a lot more surprised that I managed to wrangle the proper templates for untangling the command callbacks (for example around command_func.h:320).
11:37:14 <michi_cc> Especially with commands that return nothing extra, an extra cost and/or some other data.
12:14:35 *** andythenorth_ has quit IRC (Ping timeout: 480 seconds)
12:30:55 *** andythenorth has joined #openttd
12:45:23 *** frosch123 has joined #openttd
12:58:44 <Samu> okay i found why this was failing
12:59:04 <Samu> i also need to pass the lane
12:59:11 <Samu> not just the segment dir
12:59:36 <Samu> now i wonder if i also need to pass anything more
13:00:50 <Samu> j is the lane, lane 0 or lane 1
13:01:04 <Samu> 1 << 39 is still within 64
13:02:45 <Samu> something tells me I also need to pass the node #
13:03:43 <Samu> each segment can have 1 to 3 tiles, i call them node #0-#2
13:09:02 <Samu> segment_dir A, lane j (#0 and #1 per segment_dir - always have both), node i (#0 up to #2 per lane - can have different amounts for each lane)
13:10:14 <Samu> and each node have all the details about it
13:11:11 <Samu> so yeah, in theory, i also need to pass the node#
13:17:51 *** glx is now known as Guest4590
13:21:04 <TrueBrain> coverage 7.84%, no crashes yet
13:21:08 <TrueBrain> Total execs : 549 millions
13:23:00 <frosch123> how is coverage computed? just the executable (incl. squirrel and other barely used parts)? or also shared objects with infinitely more unused parts?
13:23:15 <TrueBrain> everything I added instrumentation to
13:23:23 <TrueBrain> which in this above percentage is all of OpenTTD's source
13:23:45 <TrueBrain> if I would be smart, I would whitelist the files that are interesting to fuzzing
13:23:45 <frosch123> ok, so including squirrel, excluding sdl :)
13:24:16 <TrueBrain> so this percentage is pretty high, given I only run commands
13:24:17 *** Guest4590 has quit IRC (Ping timeout: 480 seconds)
13:24:23 <TrueBrain> (I don't even run a gameloop)
13:25:02 <frosch123> wow, i jsut wanted to ask how much is run without commands :p but without gameloop it's surprising
13:25:21 <TrueBrain> yeah, and it also ignores all setup etc
13:25:29 <TrueBrain> this is just all traces from the start of a command
13:26:31 <TrueBrain> so far it found 8 bugs .. of which I now fixed 7 :P Well, I fixed 6 .. 1 was fixed by mister m :)
13:26:36 *** andythenorth_ has joined #openttd
13:26:50 <frosch123> how much slower would it be, if ottd would validate infrastructure-counts and other differentially-udpated caches after each command?
13:27:13 <frosch123> we had countless cases of desynced vehicle counts in groups, infrastructure counts and more :)
13:27:24 <TrueBrain> dunno :P How to check :D
13:27:32 <TrueBrain> I did consider also running a game-tick
13:27:38 <TrueBrain> to flush out any invalid state ..
13:27:40 <frosch123> i think -d desync=1 enables some checks
13:27:48 <TrueBrain> but for now I just focused on validating the command decoder :)
13:27:59 <TrueBrain> desync checks assume server/client
13:28:07 <TrueBrain> I currently only have server, and I inject commands :P
13:28:15 <frosch123> no, some checks just enable cache validation
13:28:53 <TrueBrain> I think we need to borrow some code from JGRPP to have something sane in those regards
13:29:00 <TrueBrain> if my memory serves me well, he added a bunch of extra validation
13:29:30 *** sla_ro|master has joined #openttd
13:29:30 <frosch123> so -d desync=1 enables that stuff
13:29:46 <frosch123> desync=2 enabled savegames every N ticks, i guess you do not want that :p
13:30:01 <TrueBrain> it also needs to crash, instead of log ..
13:30:35 <frosch123> true, it only dos Debug(...) prints, no asserts
13:30:50 <TrueBrain> but that is fixable ofc
13:31:09 *** andythenorth has quit IRC (Ping timeout: 480 seconds)
13:32:07 <frosch123> haha, actualyl it's a random mixture of debug-print and assert :)
13:32:10 <TrueBrain> and we should fix that MSAN works with OpenTTD, might also be interesting
13:32:33 <TrueBrain> I just really do not udnerstand what goes wrong with MASN :(
13:34:32 *** gelignite has quit IRC (Quit: Stay safe!)
13:35:54 <TrueBrain> okay, enabled CheckCaches() after every command ... let's see what it does
13:36:14 <TrueBrain> starting the fuzzer takes a while .. not because it is slow, but because it prints out a tons of things to stdout, which WSL doesn't really appreciate :P
13:37:04 <frosch123> yeah windows console is funny. on high cpu load it prints individual characters like a tele-typer
13:37:24 <TrueBrain> in good news: it doesn't insta-crash
13:37:40 <TrueBrain> but, we went from 20,000 executions per second
13:38:12 <frosch123> so the caches are worth it :)
13:38:29 <TrueBrain> well, CheckCaches is not the most efficient function we have :P
13:38:44 <TrueBrain> (I btw still don't do a game-tick, I just call CheckCaches with some modifications :P)
13:39:29 <frosch123> what kind of savegame do you use?
13:41:00 <TrueBrain> if we can get this accepted by oss-fuzz, the CheckCaches is worth it .. from what I read, oss-fuzz runs with 50k VMs or something silly .. so an hour for them, is ... a long time for me :P
13:41:23 <frosch123> nightly title screen? that means no newer features than 0.3 :p
13:41:39 <frosch123> no vehicle groups, no newgrf :)
13:42:02 <TrueBrain> but I think I have a bit modern title game .. let me check which it picked
13:42:28 <TrueBrain> ah, yes, the 0.3 :P
13:42:31 <TrueBrain> so, give me a better savegame :D
13:42:37 <TrueBrain> ideally a 64x64 or a 128x128 one
13:42:40 <frosch123> title game compeition at least asks to use every available thing :) no idea how true that is though
13:43:07 <frosch123> without newgrf, the release titlegame is probably best
13:43:11 <TrueBrain> best thing about this fuzzing is, it likes it if you load multiple with different configuration :)
13:43:22 <TrueBrain> I wouldn't mind NewGRFs being loaded honestly
13:46:40 <andythenorth_> lol I need something bigger than super_heavy_freight
13:46:46 <andythenorth_> ultra_heavy_freight
13:46:50 <andythenorth_> super_super_heavy_freight
13:47:16 <andythenorth_> hyper_heavy_freight?
13:47:17 <frosch123> ridiculous_heavy_freight and ludicrous_heavy_freight still available?
13:47:21 <TrueBrain> okay, loading the 12 opntitle.dat .. let's see how that goes
13:47:23 <andythenorth_> yup still available
13:48:16 <TrueBrain> did disable CheckCaches for now, as the slowdown hurts more than the exploring atm :D
13:48:34 <TrueBrain> I should compile 1 fuzzer with it enabled, and let it run to the ones without
13:48:40 <TrueBrain> so the ones without explore, and the one with validates
13:50:35 <TrueBrain> crash in linkgraph, nice :)
13:51:12 <TrueBrain> guess I should recompile before expecting code-changes to work, right?
13:53:08 <TrueBrain> that savegame is a lot slower to work with, haha :)
13:57:01 <TrueBrain> maybe one of you can understand this MSAN problem :)
14:00:37 <TrueBrain> okay, so this new map shows another problem that only happens with multiple commands ... which are very hard to debug .. hmmmmmm
14:02:49 <TrueBrain> how can I smartly make a fuzzer execute multiple commands from a single payload
14:03:01 <TrueBrain> use the first byte as length indicator? Seems like a poor choice ..
14:25:37 <frosch123> "last" is meant to be a valid char
14:25:41 <frosch123> but here it is "end"
14:25:57 <TrueBrain> I hope Rubidium can give you more insight on this :)
14:26:23 <frosch123> apprently since c++11 it is guaranteed to end in \0
14:27:19 <TrueBrain> oof, I made sure the fuzzer is now stable, which means it no longer runs 10000 commands before resetting the map .. the fuzzer slowed down from 20k per second to 100 per second ..
14:27:31 <TrueBrain> well, enabling CheckCaches and running a GameLoop now no longer actually hurts that much anymore :P
15:04:07 *** gelignite has joined #openttd
15:23:32 <Samu> i guess collision checking is still failing
15:24:49 <Samu> getting the parent node is also where the pf wastes most of the time :(
15:32:45 *** jpw has quit IRC (Ping timeout: 480 seconds)
16:25:49 *** andythenorth_ has quit IRC (Quit: andythenorth_)
16:38:10 *** Smedles has joined #openttd
17:04:03 <TrueBrain> yippie, I can reproduce the crash with CmdSetAutoReplace
17:04:08 *** lobster has joined #openttd
17:05:00 <TrueBrain> I just wish I understood this GroupStatistics blabla :P
17:05:05 <TrueBrain> code is really odd to read
17:09:16 <TrueBrain> you can do an autoreplace on a group of another vehicle type or something weird :P
17:09:58 <TrueBrain> which only fails because GroupStatistics validates this, but nothing else does .. lol? :D
17:10:23 *** Flygon has quit IRC (Quit: A toaster's basically a soldering iron designed to toast bread)
17:11:28 <glx> well yoy have autoreplace rules per group, and also a general autoreplace
17:12:17 <glx> and the GUI doesn't really help in there IRIC
17:13:58 <TrueBrain> yeah, so, nowhere the DoCommand seems to validate if the group is actually of the right type for the engine you are replacing
17:14:01 <frosch123> looks like "uninitialised" warnings are as useless as ever
17:14:13 <TrueBrain> does this means I can setup a replacement from one engine type to the other? :P
17:14:18 <TrueBrain> frosch123: it is? Awh :(
17:14:46 <frosch123> for me msan triggers on strgen, not sure whether it got to settingsgen
17:14:56 <TrueBrain> it could also trigger on strgen, yes
17:15:11 <TrueBrain> I run with "make -j9", so it is a toss-up which one crashes first :P
17:15:19 <frosch123> and it claims std::vector<byte>::data() is uninitialised
17:15:27 <frosch123> which makes no sense
17:16:07 <frosch123> acutally, it's similar to what you posted on gh about settingsgen
17:17:52 <TrueBrain> too bad MSAN doesn't work .. fuzzers indicate they can help weed out issues
17:24:58 <TrueBrain> I hope this one is correct ... it looks correct .. but hard to test auto replace in all the cases
17:26:32 <frosch123> MSAN fails when strgen tries to write "(undefined string)" into the lng.file
17:26:40 <frosch123> which is the first non-empty string in english.txt
17:27:05 <TrueBrain> clang page suggests adding some -f to get better traces
17:27:07 <TrueBrain> I did not try that yet
17:27:25 <frosch123> i added printf just before it fails
17:28:18 <frosch123> ok fair, it could be a dangling pointer, which still points to the right data
17:28:37 <frosch123> but then asan would detect it, and doesn't
17:29:41 <TrueBrain> yeah, ASAN is really happy with OpenTTD
17:29:51 <TrueBrain> found no issues so far with DoCommands
17:33:16 <frosch123> "__attribute__((no_sanitize("undefined")))" <- though it's funny that src/3rdparty/fmt/format.h contains that :)
17:33:53 <frosch123> TrueBrain: i changed my cmake variables so it only compiles openttd with msan, not strgen or settingsgen
17:34:10 <frosch123> when starting, it immedially reports icu :)
17:34:14 <TrueBrain> given the function it failed for settingsgen, I had little hope :P
17:34:44 <frosch123> so let's say, there is a reason MSAN is in clang only, not in gcc :p
17:34:54 <TrueBrain> okay, so close the ticket :P
17:36:43 <TrueBrain> next one fuzzers advise is the undefined behaviour SAN
17:37:57 <frosch123> TrueBrain: i tried fsanitize=undefined earlier with gcc build
17:38:13 <TrueBrain> the next is CFISAN, TSAN and LSAN
17:38:16 <TrueBrain> the latter two should be fine
17:38:24 <TrueBrain> no clue what the first actually does .. description is vague
17:38:29 <frosch123> i assumed from the name it may have been gcc's MSAN :p but it wasn't
17:40:53 <frosch123> i still like valgrind, it does not have insane requirements like "compile libc with instrumentation"
17:41:11 <TrueBrain> fuzzers support qemu, to avoid recompiling
17:41:24 <TrueBrain> they do instrumentation based on qemu
17:41:30 <TrueBrain> which is also insane, if you think about it :D
17:41:48 <TrueBrain> but ... it makes fuzzing much quicker, if it can explore if a trace is new or not
17:41:51 <TrueBrain> so worth the slowdown, I guess
17:42:03 <frosch123> as long as the tool does the insane stuff, it is fine. i have the problem with tools requesting insane things from me :)
17:44:03 <TrueBrain> well, for OpenTTD I am compiling the instrumentation in the binary, as that is a lot quicker than qemu :P But yeah .. annoying :D
17:50:24 <TrueBrain> with UBSAN I get Illegal Instructions
17:52:46 <frosch123> i wondered whether all the SANs were invented to mock the japanase, Truebrain-san
17:53:54 <TrueBrain> and I am going to give up on the CFISAN too ... linking takes a really really REALLY long time
17:58:07 *** MaverickFischer has quit IRC (Quit: Due to budget cuts, the light at the end of the tunnel has been turned off.)
17:59:37 *** MaverickFischer has joined #openttd
17:59:49 <TrueBrain> lol @ msys build times .. that hasn't improved :P
17:59:57 <TrueBrain> in fact .. it got worse :P
18:00:37 <TrueBrain> anyway, the fuzzer has found ~2300 unique command+arguments by now .. I am always amazed how it manages to do that .. :)
18:03:02 <frosch123> did the fuzzer try to autoreplace a train with a plane?
18:03:23 <TrueBrain> next it tried to replace a train with a train in a plane group
18:03:30 <frosch123> i do not see a check though
18:03:53 <TrueBrain> "CheckAutoreplaceValidity(old_engine_type, new_engine_type, _current_company)"
18:03:58 <frosch123> both CmdSetAutoreplace and CheckAutoreplaceValidity only use the source type
18:04:39 <frosch123> ah, IsEngineBuildable does it
18:04:46 <frosch123> so, three levels deep :p
18:04:49 <TrueBrain> was about to say, I am sure I found a path that did that
18:04:53 <TrueBrain> yeah .. it is silly
18:05:28 <frosch123> well, i hope you never consider to fuzz the saveload code :)
18:05:40 <TrueBrain> I gave up REAL quick
18:05:42 <TrueBrain> that is a total shitshow
18:05:52 <TrueBrain> I did consider fuzzing newgrfs after this
18:05:58 <TrueBrain> those, in theory, should not cause any crashes
18:06:48 <TrueBrain> no, savegames were really bad ... and even if it doesn't crash on load, it crashes seconds after load :P
18:07:03 <TrueBrain> we do really little sanity checks .. guess it makes sense .. would make loading insanely slow :P
18:20:53 <TrueBrain> I also considered fuzzing the network protocol itself, but that is a bit annoying, as it strongly depends on internal state ...
18:21:33 <frosch123> and commands don't?
18:21:43 <TrueBrain> I can always execute a command
18:21:51 <TrueBrain> but most network packets can only be done in a certain sequence
18:22:21 <TrueBrain> which makes it a lot more difficult to do something useful
18:25:07 <TrueBrain> basically I would have to make a meta-language, where it can create multiple packets to execute, so sequences can be tested
18:25:11 <TrueBrain> doable .. just annoying :P
18:25:32 <TrueBrain> for commands I just execute one at random, and see what happens :P
18:27:26 <TrueBrain> it would also helps commands to do it like that .. but meh .. I think this is enough for now :D
18:28:53 <frosch123> ok, so you could only fuzz the chat-message package and crash the unicode processing :)
18:29:16 <TrueBrain> well, it is not a bad idea to fuzz some specific functions that do tasks like that
18:29:24 <TrueBrain> but in that case, fuzz the unicode decoder ;)
18:29:31 <TrueBrain> we can just skip the rest :D
18:29:47 <TrueBrain> if oss-fuzz would accept OpenTTD, you could earn up to 20k dollar for doing stuff like that :P
18:30:21 <frosch123> unicode magic is my favorite magic. nothing compares to being able to send a cell broadcast message to brick all mobile phones in some area
18:30:41 <TrueBrain> every time I am like: this is the last one, right?
18:31:23 <frosch123> how do you earn 20k by oss-fuzz accepting ottd?
18:31:35 <TrueBrain> that is what oss-fuzz offers
18:32:07 <TrueBrain> it is one of their selling points
18:32:13 <TrueBrain> I just want access to their cluster, honestly ..
18:32:22 <TrueBrain> would go a lot quicker than on my local machine :D
18:33:07 <TrueBrain> the page is funny btw
18:33:23 <frosch123> i saw a 5 in the middle
18:34:17 <frosch123> i think 5 is the max per project, but you can apply for multiple projects
18:34:45 <TrueBrain> it is a nice gesture to get big OSS projects fuzz-tested
18:34:54 <TrueBrain> openssl etc are all in there these days
18:35:02 <TrueBrain> most popular JSON serializers
18:35:05 <TrueBrain> which is really good
18:35:14 *** Wormnest has joined #openttd
18:35:33 <frosch123> oh dear, "json serializer" is such a @work trigger word :p
18:36:02 <TrueBrain> lol, I had to deal enough with that @work too lately :P
18:36:47 <frosch123> someone asked in an internal community, which json parser to use
18:37:05 <frosch123> someone recommended som internal package "which wraps an external json parser"
18:37:48 <frosch123> turns out, it was like 10k homegrown-wrapper-code around a 200 LOC minimal external parser, which looked unmaintained
18:38:43 <dwfreed> I hear jansson is great if you need C
18:38:49 <TrueBrain> I have been toying with tag_invoke and boost::json .. I had to learn how to have a template function work on enums and another on non-enums .. which was .. interesting
18:39:09 <TrueBrain> learnt a lot .. also how "typeof" and "decltype" differ .. which was unexpected :P
18:39:10 <frosch123> there seem to be a common pattern in big corporation: the worst programmers are assigned to jobs far away from customers
18:39:32 <dwfreed> well, yeah, can't have their shitty code ruining a customer's day
18:39:36 <dwfreed> (it still will, though)
18:39:50 <frosch123> so the most fundamental stuff is done by people who learned fortran, and then were tasked to write a json parser in c++
18:39:50 <TrueBrain> yeah, but now 10 other people have a chance to patch up the shitty code :P
18:40:43 <frosch123> or just ignore it and copy nlohmann into their namespace
18:41:06 <TrueBrain> nlohmann is annoying
18:41:09 <TrueBrain> strict JSON sucks balls
18:41:31 <TrueBrain> "owh no, you had a comma at the end of a list, NOW I AM REFUSING TO READ YOUR FILE"
18:43:01 <TrueBrain> anyway, I like that you can have a default parameter with a std::enable_if to make sure a function is not considered when its validation fails .. which is such a weird thing
18:43:23 <TrueBrain> within the <template> tag, I expected those things
18:43:26 <frosch123> i like that std::enable_if is gone with c++20 :p
18:43:28 <TrueBrain> but that it also works for parameters ........
18:43:38 <TrueBrain> what is it replaced with?
18:44:48 <TrueBrain> means you can have templates to actually only work on enums? :P
18:44:59 <TrueBrain> so template <Enum ..> ? :D
18:45:45 <frosch123> template parameter restrictions become declarative, instead of enable_if-assertive
18:46:00 <frosch123> concepts themself are defined using duck-typing
18:46:11 <TrueBrain> so less SFINAE, more sanity?
18:46:39 <frosch123> though the assumption is that almost noone has to write custom concepts
18:47:13 <TrueBrain> it is basically ... "sure, I can understand what it does .. but fuck no way I can write that myself" shit :P
18:47:25 <frosch123> lambdas killed std::bind, concepts kill SFINAE, happy pancakes
18:47:51 <TrueBrain> I need to read up on the pros and cons of std::functions
18:48:02 <TrueBrain> bit of an issue with people binding "this" to a lambda to assign to a std::function
18:48:02 <frosch123> TrueBrain: usually the "i can understand what it does" only applies as long as it compiles
18:48:08 <frosch123> once it fails to compile, you are lost :)
18:48:22 <TrueBrain> frosch123: yeah, I cannot validate it does what it says it does .. I can just loosely understand what it might be doing :P
18:48:39 <frosch123> std::function are essentially std::unique_ptr<lambda>
18:48:49 <frosch123> a lambda moved to the heap
18:49:01 <TrueBrain> lol, that is a nice way of putting that; I am going to remember that :)
18:49:02 <frosch123> not inline-friendly, but universal
18:49:13 <TrueBrain> just the binding of "this" is a bit weird .. not really memory safe :P
18:49:28 <TrueBrain> what I see libraries do, is ask for a std::function as callback
18:49:37 <TrueBrain> and I see application implement that with [&this]{}
18:49:47 <frosch123> oh yes, [=, this] [&, =this] is a shitshow :)
18:49:51 <TrueBrain> so you are giving another library your pointer ... without tracking .....
18:50:06 <TrueBrain> so that sounds like a disaster waiting to happen
18:50:39 <TrueBrain> binding .. capture .. what-ever, you get what I mean I guess :P
18:51:10 <TrueBrain> anyway, I do wonder if there isn't a better way of doing these lambdas, but I don't think there is ...
18:51:13 <TrueBrain> so it is just meh :P
18:51:25 <frosch123> yeah, it assumed that the lbirary does not store the std::function, but only uses it during the call
18:51:45 <TrueBrain> well, these libraries store it to call the callback when something happens :P
18:52:09 <TrueBrain> but I guess they assume the library is instantiated by the object who gave the callback in the first place
18:52:13 <TrueBrain> so that "should be fine" :P
18:52:14 <frosch123> std::function is dynamic and thus linkable. you can replace all std::function with lambdas, by turning all users into header-templates
18:52:45 <frosch123> everyone can call a std::function, lambdas can onyl be called by templates
18:53:57 <TrueBrain> I just miss the time of (void)(*my_t)(int)
18:53:59 <TrueBrain> that I could understand :P
18:54:54 <TrueBrain> lol, TIL, you can also template a lambda (was reading cppreference.com) .. that is funny :D
18:55:04 <frosch123> fine, std::function is like "void (*functor)(void* cargo)"
18:55:40 <frosch123> or rather std::pair<void (*functor)(void* cargo), void*>
18:55:58 <frosch123> templated lambdas are new
18:56:20 <frosch123> i think c++17 added "auto" to lambada parameters
18:56:37 <frosch123> c++20 added the explicit <> param list
18:56:44 <TrueBrain> I really miss "explicit" for lambdas
18:56:48 <TrueBrain> std::visit is very nice
18:56:54 <TrueBrain> but it can do conversion without you realising
18:56:57 <TrueBrain> which is ... euh ... interesting
18:57:40 <frosch123> with concepts you often don't need the "template<>" anymore
18:58:00 <TrueBrain> sadly it will take for ever for compilers to properly support C++20 AND be available on the targets I work with :(
18:58:01 <frosch123> you can just use "auto" in the parameter list
18:58:11 <frosch123> you only need template<> for dependencies between multiple parameters
18:58:48 <frosch123> implementation is mostly done, but yeah enterprise distros will take ages
18:59:25 <TrueBrain> yeah, and shit like Yocto ... you need to upgrade to a whole new version to get the better gcc
18:59:28 <TrueBrain> just takes a long time
18:59:37 <frosch123> i am mostly surprised by clang becoming out-of-fashion again, and gcc getting better support
19:00:10 <TrueBrain> owh my, [*this] is allowed as capture? That ... is surprising
19:00:44 <frosch123> i think it was added for completeness :)
19:01:18 <frosch123> also, you can now have storage inside lambdas
19:01:31 <TrueBrain> how ... does that work
19:01:45 <TrueBrain> don't lambdas start off on the stack?
19:01:53 <frosch123> [foo=1] mutable (int a) { foo += a; return foo; }
19:02:10 <frosch123> (did not try to compile)
19:03:11 <frosch123> it allows the lambda to modify the variables it captured-by-copy :)
19:03:37 <TrueBrain> okay, that somewhat makes sense ...
19:03:44 <TrueBrain> but I think I am happy I never had to use that :P
19:03:49 <frosch123> you know how in modern languages const is the default and mutable needs declaration, while in c++ mutable is default and const needs declaration? for lambdas it's the other way around
19:04:58 <TrueBrain> I wonder how clever compilers are with lambdas ..
19:05:07 <TrueBrain> but then again .. who gives a shit :P
19:06:38 <TrueBrain> I also did not know you could do "foo=0" in a lambda
19:06:48 <TrueBrain> I really only used it in their most simple syntax :D
19:07:05 <frosch123> don't worry, i also never used it in real code :)
19:07:33 <TrueBrain> basically with these additions to lambdas, you only need int main() :P
19:08:20 <TrueBrain> funny, that godbolt without -O, actually puts the lambdas like a normal function in the assembly
19:10:05 <TrueBrain> okay, that [foo=0] stuff is scary as fuck
19:10:24 <TrueBrain> like a static piece of memory all of a sudden
19:10:58 <TrueBrain> I cannot imagine that not giving issues :P
19:10:59 <frosch123> it's not that magic
19:11:08 <TrueBrain> no, just hard to read what actually happens ;)
19:11:23 <frosch123> "int bar = 0; auto l = [foo=bar](...)" is just the same
19:11:26 <TrueBrain> it creates a variable in a weird place
19:11:34 <frosch123> it copies some value into the lambda state
19:11:58 <frosch123> all captures create variables (without -O)
19:12:22 <frosch123> what's the difference between copying "this", "bar", or "foo" into the lambda data?
19:12:47 <TrueBrain> It is now implicit that foo is within the scope of square
19:12:58 <TrueBrain> Just a readability thing
19:13:39 <frosch123> looks like the assignment is actually c++14 already
19:14:24 <frosch123> "auto"-parameters are also c++14
19:15:15 <frosch123> oh, there is some lambda syntax new in c++20 and removed in c++23 again
19:16:22 <frosch123> hmm, no, it's just some documentaiton detail
19:16:37 <frosch123> essentially a bugfix
19:16:59 <TrueBrain> oops, remove "mutable" :P
19:17:11 <frosch123> but there the storage is outside of the lambda
19:17:20 <frosch123> so if you move that lambda into a std::function it will be broke :)
19:17:40 *** lobstarooo has joined #openttd
19:18:53 <TrueBrain> I guess, but oof ..
19:18:56 <TrueBrain> lambdas surprise me :)
19:19:24 <TrueBrain> where does "foo" actually live in your godbolt I wonder ..
19:20:37 <TrueBrain> seemly on the stack of the parent (square)
19:20:43 <TrueBrain> so also outside the lambda :P
19:22:06 <frosch123> a lambda with capture is a class with operator()
19:22:19 <frosch123> you can move that class using the default-move operator
19:23:03 <TrueBrain> I don't really understand your std::function remark from earlier .. let's experiment :D
19:25:19 *** lobster has quit IRC (Ping timeout: 480 seconds)
19:25:21 *** lobstarooo is now known as lobster
19:29:13 <TrueBrain> ah, yeah, ofc, it copies the whole object
19:29:25 <TrueBrain> lol, this can really give surprisingly results :D
19:29:59 <TrueBrain> my code was a lot simpler :P
19:30:03 <TrueBrain> let's see what you did here ..
19:30:29 <TrueBrain> "auto" as function return type .. lol .. that is just evil :P
19:31:06 <frosch123> too lazy to type :)
19:31:21 <frosch123> it won't work in header prototypes :p
19:31:35 *** tokai|noir has joined #openttd
19:31:35 *** ChanServ sets mode: +v tokai|noir
19:32:01 <TrueBrain> I guess with your "boom" you mean it does something different then you intended? :P
19:32:06 <TrueBrain> as it works just fine ;)
19:32:35 <frosch123> example3ab should crash
19:32:51 <TrueBrain> well, if not crash, it does something
19:32:54 <frosch123> it writes _foo from an expired stackframe
19:32:56 <TrueBrain> but yeah, I understand :)
19:34:13 <TrueBrain> hmm .. so basically a std::function copies the lambda, if my experiments are any decent ..
19:34:18 <frosch123> now explain lambdas to your team next week, and get a raise :)
19:34:34 <frosch123> TrueBrain: technically it "moves" the lambda
19:34:51 <TrueBrain> the "foo" is not shared between the two calls
19:34:55 *** andythenorth has joined #openttd
19:35:00 <TrueBrain> so it did a copy somewhere :)
19:35:17 <TrueBrain> ah, the compiler did
19:35:31 <frosch123> hmm, oh, maybe you are right. it copies it...
19:35:48 <frosch123> yeah, it copies it :)
19:35:59 <TrueBrain> yeah, I can put "constexpr" before the lambda
19:36:00 <frosch123> std::function is move-only, but lambdas are copyable
19:36:09 <TrueBrain> so the compiler does that shit
19:36:31 <TrueBrain> but my godbolt shows a bit why I think that "foo=" is dirty
19:37:09 <frosch123> yeah, mutable lambdas are probably dirty :)
19:37:47 <frosch123> "foo=" itself is fine, the "mutable" is the problem
19:37:47 <TrueBrain> but I like your explanation, just imagine a lambda as a small class with the captured variables ..
19:37:58 <TrueBrain> yeah, fair, mutable makes it a bit iffy :)
19:38:20 *** tokai has quit IRC (Ping timeout: 480 seconds)
19:38:46 <frosch123> TrueBrain: compilers implement lambdas as classes
19:38:52 <TrueBrain> wow, my example, the compiler completely fails to optimize that :D
19:39:06 <frosch123> if you set a breakpoint inside a lambda, you can look at the instance variables
19:39:07 <TrueBrain> frosch123: funny :) So auto-generated classes ... like Python's "namedtuple" :D
19:39:43 <TrueBrain> ah, it is the std::function that kinda breaks optimization :P
19:40:30 <TrueBrain> that is fine, I guess :)
19:40:36 <glx> std::function breaks a lot of things no ?
19:40:43 <frosch123> yeah, heap allocations are not constexpr :)
19:41:00 <TrueBrain> yeah, but I was hoping the inspect would see it is a const after all .. it doesn't :P
19:41:20 <TrueBrain> glx: bit too vague of a statement ;)
19:41:24 <TrueBrain> like this blank statement of nothingness :D
19:41:34 <frosch123> anyway, the main point of std::function is that you can put your square() and main() into different modules, and link
19:41:57 <TrueBrain> yeah, I mostly have seen it lately in libraries for callbacks
19:42:07 <frosch123> std::function can hide lambdas as incomplete types, and yuo can still move and call them
19:42:23 <TrueBrain> instead of an interface you have to define a class for and give that as instance to the library
19:42:44 <TrueBrain> usage-wise, it is pretty nice, as you can define your callbacks much more local
19:42:57 <TrueBrain> instead of this generic place, where you have to distribute it back to the local place again
19:43:51 *** lobstarooo has joined #openttd
19:45:13 <TrueBrain> but I am used to the latter solution, so that made me wonder if std::function is actually a good solution, or that it craps all over the place :P
19:45:55 <frosch123> just replace your callbacks with coroutines :p
19:46:11 <frosch123> (it won't get easier)
19:46:19 <TrueBrain> coroutines in C++ are not there yet
19:46:23 <TrueBrain> which is a bit sad :)
19:46:42 <frosch123> c++20 has coroutines without standard library
19:46:56 <frosch123> not sure whether it was finished for c++23
19:47:22 <frosch123> i have no idea how much they are linkable though. they may be all-header-templates
19:47:27 <TrueBrain> can you use a class member as std::function btw?
19:47:47 <TrueBrain> euh, class function
19:48:10 <TrueBrain> I would guess that requires a small lambda wrapper of sorts ..
19:48:27 <frosch123> you can do it by storing a lambda [this] in the std::function
19:48:55 *** lobster has quit IRC (Ping timeout: 480 seconds)
19:49:01 <frosch123> std::function([this](){this->bar();})
19:49:01 *** lobstarooo is now known as lobster
19:50:56 <TrueBrain> cool .. tnx a lot frosch123 :) Nice to toy a bit with std::function/lambda :)
19:51:11 <frosch123> things that do signal/slot stuff, which track the validity of sender/receiver need the owner explicitly
19:51:39 <michi_cc> If you want to hide the lambda, std::bind is an alternative.
19:51:47 <frosch123> so in qt: connect(..., this, &member) is very different to connect(..., [this](){})
19:51:49 <TrueBrain> I have been told to not touch std::bind :P
19:52:05 <frosch123> in the former case the connnection will be freed if this is destroyed, in the latter it won'T
19:52:27 <frosch123> yeah, std::bind is crap from 90's, use lambda :)
19:52:36 <TrueBrain> how does it know this is being destroyed?
19:53:02 <frosch123> well, this must inherit from QObject
19:53:14 <TrueBrain> ah, so they have internal signalling for that
19:53:14 <frosch123> which can signal on destruction
19:53:39 <TrueBrain> Qt has a lot of practical things
19:53:45 <TrueBrain> sadly, their license is a bit ....... annoying :D
19:54:44 <TrueBrain> few months ago I had a colleague that tried to check if an object was destroyed via weak_ptrs
19:54:48 <TrueBrain> it was .. somewhat working
19:54:50 <frosch123> Rubidium: no, :290 assumes a 0-terminated string. :301 is valid with c++11, but wrong with c++98. data() is not guaranteed to be 0-terminated in c++98
19:55:44 <Rubidium> yeah, though we require C++11, right?
19:55:46 <frosch123> TrueBrain: i use shared_ptr and weak_ptr happily, works fine as long as you do not build circles
19:55:54 <TrueBrain> frosch123: exactly ;)
19:56:11 <TrueBrain> well, guess I could capture "this", put it in a weak_ptr, and check if it is still alive, for these callbacks
19:56:18 <TrueBrain> no, that doesn't work
19:56:31 <TrueBrain> I keep forgetting you don't have access to the shared_ptr from inside the class
19:56:35 <frosch123> Rubidium: yes, but i had to look that data() think up first, i did not remember the c++11 change
19:56:52 <frosch123> TrueBrain: std::shared_from_this
19:57:10 <frosch123> you have to derive your class from that, so it is intrusive
19:58:49 <Samu> I'm such a terrible coder
19:58:53 <frosch123> it's basically what you planned to implement yourself :p
19:59:10 <frosch123> std::weak_ptr member to this. assigned by std::shared_ptr constructor
19:59:21 <TrueBrain> most of the objects I have define a Create that returns an unique_ptr, which you can upgrade to shared_ptr if you so like .. but this forces the object to already be a shared_ptr from the start
19:59:25 <TrueBrain> guess there are worse things in life
19:59:40 <frosch123> no, i think it doesn't enforce that
20:00:08 <TrueBrain> funny to see I am not the first to run into this :P
20:00:10 <frosch123> it only needs to be owned by a shared_ptr, when you want to create additional shared_ptr
20:01:00 <frosch123> so it makes sense to do it during constrution, but you can make it more complicated :p
20:02:52 <TrueBrain> I wonder how that enable_shared_from_this works under the hood ... lets godbolt this :D
20:04:25 <TrueBrain> I like their example "Best", that forces you to use Create to avoid a non-shared_ptr
20:04:42 <frosch123> the shared_ptr constructor has an overload for types derived from std::shared_from_this
20:05:24 <frosch123> well, not an overload, but some kind of type-check
20:06:39 <TrueBrain> what I wonder, is how does "this" know where the shared pointer is :P
20:07:13 <frosch123> it adds a weak_ptr as member
20:07:41 <frosch123> [21:56] <TrueBrain> well, guess I could capture "this", put it in a weak_ptr, and check if it is still alive, for these callbacks <- you already knew that :)
20:08:20 <frosch123> you knew the solution before you knew someone else did it before :p
20:08:30 <TrueBrain> yeah, but I wonder is how they solved it :P
20:09:35 <TrueBrain> but yeah, that last page explains it
20:10:16 <TrueBrain> as that would be a lot safer way of dealing with callbacks
20:10:54 <frosch123> you should read about boost::asio
20:11:00 <frosch123> it does that kind of stuff all over the place
20:12:33 <frosch123> uses enable_shared_from_this for those aio callbacks
20:14:58 <frosch123> well, experimental-coroutines, so don't use it in code which you have to maintain :p
20:16:08 <TrueBrain> I even used an assignment in the capture! :P
20:16:19 <frosch123> i like how you immediately used [t=] :p
20:16:27 <frosch123> oh lol, you just said that
20:17:42 <TrueBrain> this is pretty cool :D I did not expect that someone already figured this out in the standard ... I should have known better :P
20:18:31 <frosch123> well, technically boost figured it out 10 years before the standard
20:18:39 <frosch123> but boost looks like #coop these days
20:18:39 <TrueBrain> yeah, ofc they did :P
20:18:50 <TrueBrain> but boost also makes sure your error messages are 2000 pages long
20:18:57 <TrueBrain> boost::hana can be the worst in that
20:19:09 <TrueBrain> as a string is templated, as in 'h', 'e', 'l', 'l', 'o'
20:19:16 <TrueBrain> now have a class with 20 members
20:19:21 <TrueBrain> and any error it throws is a page full
20:20:18 <TrueBrain> okay, this is cool .. I have something to annoy my colleagues with monday :P
20:20:46 <frosch123> that's fine, i also annoy my coworkers with things from ottd :p
20:21:16 <frosch123> some weeks ago i showed them openapi, using bananas as example
20:21:51 <Samu> this function has been so buggy
20:22:07 <TrueBrain> next week I am going to show the department how you can learn Rust with GitHub CoPilot, by using TrueGRF as example :D
20:22:08 <Samu> i am so bad at iterating backwards
20:22:48 <TrueBrain> Total run time : 12 hours, 29 minutes
20:22:48 <TrueBrain> Total execs : 2 millions, 4 thousands
20:22:56 <TrueBrain> coverage 7.39%, no crashes yet
20:22:56 <frosch123> usually i have to tell them "i really don't know this shit", so they understand i man the cyclops between the blind ones
20:23:06 <TrueBrain> lost a bit of coverage because I made some damaging changes to the corpus format :P
20:23:44 <TrueBrain> but it seems, commands no longer crash the game on the 12 opntitle.dat savegame :)
20:23:53 <frosch123> i have a running gag about dutch people with names starting with P
20:24:08 <frosch123> i have a dutch teammate named Peter, who also does lots of magic :p
20:24:59 <TrueBrain> right, enough C++ toying around .. time for some telly
20:25:05 <TrueBrain> tnx again frosch123 , this was very useful to me :D
21:07:43 *** andythenorth_ has joined #openttd
21:13:14 *** nielsm has quit IRC (Ping timeout: 480 seconds)
21:15:07 *** andythenorth has quit IRC (Ping timeout: 480 seconds)
21:19:02 *** andythenorth_ has quit IRC (Quit: andythenorth_)
21:46:41 *** Samu has quit IRC (Quit: Leaving)
22:08:19 *** sla_ro|master has quit IRC ()
22:24:48 *** frosch123 has quit IRC (Quit: be yourself, except: if you have the opportunity to be a unicorn, then be a unicorn)
23:16:37 *** WormnestAndroid has quit IRC (Ping timeout: 480 seconds)
23:17:02 *** WormnestAndroid has joined #openttd
23:17:07 *** WormnestAndroid has quit IRC (Read error: Connection reset by peer)
23:17:19 *** WormnestAndroid has joined #openttd
continue to next day ⏵
