IRC logs for #openttd on OFTC at 2020-09-24
            
00:28:54 *** Progman has quit IRC
00:43:37 *** HerzogDeXtEr has quit IRC
00:50:12 *** colde has quit IRC
00:50:37 *** innocenat_ has quit IRC
00:50:47 *** mindlesstux has quit IRC
00:50:48 *** dvim has quit IRC
00:52:29 *** mindlesstux has joined #openttd
00:54:50 *** mindlesstux has joined #openttd
00:56:19 *** innocenat_ has joined #openttd
00:56:21 *** colde has joined #openttd
00:57:22 *** dvim has joined #openttd
02:29:34 <Eddi|zuHause> why is it always night? (also ingame)
03:07:35 *** Wormnest has quit IRC
03:35:43 *** arikover has quit IRC
03:45:28 *** Eddi|zuHause has quit IRC
04:05:17 *** Eddi|zuHause has joined #openttd
04:42:18 *** urdh has quit IRC
04:50:12 *** Flygon has joined #openttd
04:53:21 *** D-HUND has joined #openttd
04:56:43 *** debdog has quit IRC
06:36:01 *** debdog has joined #openttd
06:39:23 *** D-HUND has quit IRC
07:17:50 *** snail_UES_ has quit IRC
07:42:08 *** andythenorth has joined #openttd
08:06:36 *** sla_ro|master has joined #openttd
08:15:26 *** andythenorth has quit IRC
08:21:01 *** andythenorth has joined #openttd
08:26:04 *** Gustavo6046 is now known as Guest1673
08:26:08 *** Gustavo6046 has joined #openttd
08:26:13 *** Guest1673 has quit IRC
08:57:20 *** namad7 has joined #openttd
09:00:52 <TrueBrain> that moment you find out eints can only handle a single connection per time ..
09:00:59 <TrueBrain> and if one stalls .. the server becomes unreachable
09:01:00 <TrueBrain> oops :)
09:02:57 <LordAro> heh
09:03:05 <LordAro> rewrite in async!
09:05:50 <TrueBrain> it is more: how can a backlog be set to one? :)
09:19:16 <TrueBrain> meh; frosch rewrote eints login system, and as result, the API bypass we used no longer work .. so the API cannot be controlled by the "translators" user :D
09:19:30 <TrueBrain> a good thing we failed to bring this to production on the VM .. it would never have committed anything :)
09:19:34 <TrueBrain> and it fails kinda silently ..
09:40:49 *** urdh has joined #openttd
09:41:53 <TrueBrain> okay ... atm I am battling how eints works more than anything else :D
09:43:58 <TrueBrain> I upload the base language, but it doesn't do anything ..
09:44:15 <TrueBrain> and bottle is leaking memory
09:47:35 *** andythenorth has left #openttd
09:50:05 <TrueBrain> okay, there are errors uploading the base language, which returns a 200, confusing the script thinking: it all went okay! :D
09:53:48 <TrueBrain> finally, found all the quirks and odd things :)
09:54:44 *** Ragavendra has joined #openttd
10:00:49 <LordAro> all of them? :p
10:02:56 <TrueBrain> to get the base language uploaded :P
10:03:05 <TrueBrain> sorry for not being overly specific, context is hard, I know :)
10:05:10 <LordAro> :p
10:05:21 <LordAro> hmm, still getting FORTIFY_SOURCE warnings when compiling OTTD
10:05:30 <LordAro> thought i fixed that..
10:06:24 <TrueBrain> holy crap, bottle uses temporary files for file uploads .. that explains why it is this darn slow :P
10:06:29 <TrueBrain> but okay, these files are also not closed
10:06:32 <TrueBrain> dangling for weeks :)
10:06:59 <TrueBrain> the reference to the file is, however, removed ..
10:07:10 <LordAro> oh dear.
10:08:28 *** Ragavendra has quit IRC
10:13:54 <TrueBrain> okay .. found a cheesy solution for now
10:14:03 <TrueBrain> seems bottle is also not really developed anymore
10:14:11 <TrueBrain> but we do run behind some versions .. it just doesn't fix this issue :D
10:23:02 <TrueBrain> next up: why the fuck is it using 500MB of RAM :)
10:25:36 <TrueBrain> size=73.5 MiB, count=1481885, average=52 B
10:25:38 <TrueBrain> how many?!
10:26:34 <TrueBrain> death by a thousand needles
10:28:07 <LordAro> nice
10:29:12 *** Ragavendra has joined #openttd
10:29:43 *** Ragavendra has joined #openttd
10:35:49 <LordAro> TrueBrain: do you know where in the CI the cmake BUILD_TYPE is set?
10:36:34 <TrueBrain> what does build-type do? No clue :P
10:36:48 <LordAro> debug vs release
10:36:52 <TrueBrain> for MSVC?
10:36:56 <LordAro> for cmake
10:37:12 <LordAro> it's presumably set somewhere...
10:37:16 <TrueBrain> well, cmake for Linux is only executed with:
10:37:17 <TrueBrain> cmake ..
10:37:18 <TrueBrain> for CI
10:37:21 <TrueBrain> and "deb"-shit
10:37:22 <TrueBrain> for releases
10:38:18 <LordAro> https://github.com/OpenTTD/CompileFarm/blob/master/release-linux-deb-gcc/files/run.sh#L26
10:38:21 <LordAro> hmm
10:38:28 <LordAro> i don't think we set the release config type anywhere in the CI
10:38:34 <LordAro> which means no optimisation
10:38:40 <TrueBrain> owh, we don't use deb-shit anymore for releases :P
10:38:56 <LordAro> you're sending me mixed signals :p
10:39:00 <TrueBrain> for Windows it is done I believe, something like https://github.com/OpenTTD/OpenTTD/blob/master/azure-pipelines/templates/release.yml#L92
10:39:09 <TrueBrain> you are asking me something I worked on ... in Jan 2019
10:39:31 <LordAro> and you should have a perfect memory!
10:39:41 <TrueBrain> haha :D
10:40:47 <TrueBrain> hmm .. from 505MB used RAM to 340MB used RAM with a few simple Python lines
10:40:51 <TrueBrain> at least that is a nice start
10:41:09 <LordAro> :)
10:41:15 <TrueBrain> (you got to love slots)
10:41:24 <TrueBrain> stupid client .. _ _ slots _ _
10:42:38 <LordAro> you must surely be able to turn that off :p
10:42:54 <TrueBrain> anyway, regarding to debug vs release, I know CMake has infrastructure for that
10:43:00 <TrueBrain> so possibly it is handled inside cmake somewhere
10:43:16 <TrueBrain> and possibly it is forgotten and nobody tested it
10:43:27 <LordAro> yeah, i'm fiddling with it a bit - the "default" results in a lot of incorrect-use-of-FORTIFY_SOURCE warnings
10:43:30 <TrueBrain> should be easy enough to validate with the nightlies, I guess :)
10:43:39 <LordAro> there appears to be "","debug" & "release"
10:48:15 <TrueBrain> just the amount of information eints keeps in memory, is the reason for most of the other memory usage
10:48:23 <TrueBrain> not really a lot I can do about that, it seems ..
10:55:16 <TrueBrain> cool, GitHub Actions has changed their UI a bit; much more pleasant on the eye now
11:15:37 *** andythenorth has joined #openttd
11:41:50 *** SpComb has quit IRC
11:44:40 *** tokai has joined #openttd
11:44:40 *** ChanServ sets mode: +v tokai
11:48:42 *** SpComb has joined #openttd
11:51:33 *** tokai|noir has quit IRC
12:12:13 *** matt21347 has joined #openttd
12:32:29 <TrueBrain> frosch123: if you are reading your backlog, eints does something weird that it keeps telling me: "Login successful!", twice, on pages where you can edit strings .. not sure what happened :P
12:32:44 <TrueBrain> and it doesn't go away :D
12:35:09 <TrueBrain> https://translator.staging.openttd.org/ <- it contains https://github.com/TrueBrain/OpenTTD-eints-test :)
12:35:58 <TrueBrain> and changing strings work :)
12:37:09 <TrueBrain> ah, I found how I can duplicate "Login successful!" message :)
12:37:40 <TrueBrain> it only works on a editing a string
12:37:44 <TrueBrain> and just add "?message=Login%20successful%21" behind the url
12:37:46 <TrueBrain> hit F5
12:37:49 <TrueBrain> SO MANY :D
12:38:10 <TrueBrain> "issues" is disabled on the eints repository?
12:38:15 <TrueBrain> one way of not getting bug-reports :D
12:40:42 *** D-HUND has joined #openttd
12:44:08 *** debdog has quit IRC
12:57:20 *** matt21347 has quit IRC
13:25:46 *** snail_UES_ has joined #openttd
13:43:46 *** Ragavendra has quit IRC
14:47:52 <DorpsGek_III> [OpenTTD/eints] TrueBrain opened pull request #10: Various of changes and fixes https://git.io/JU2EN
14:50:11 <DorpsGek_III> [OpenTTD/eints] TrueBrain opened pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2Ej
14:51:03 <TrueBrain> there we go .. with these 2 PRs, eints seems to work on staging :)
15:12:11 *** tokai|noir has joined #openttd
15:12:11 *** ChanServ sets mode: +v tokai|noir
15:16:12 *** snail_UES_ has quit IRC
15:19:12 *** tokai has quit IRC
15:30:37 *** snail_UES_ has joined #openttd
16:02:21 *** nielsm has joined #openttd
16:07:18 *** tokai has joined #openttd
16:07:18 *** ChanServ sets mode: +v tokai
16:14:03 *** tokai|noir has quit IRC
16:34:40 *** Wormnest has joined #openttd
17:28:46 *** frosch123 has joined #openttd
17:56:53 *** snail_UES_ has quit IRC
17:59:40 *** Flygon has quit IRC
18:16:00 *** Progman has joined #openttd
18:18:36 *** snail_UES_ has joined #openttd
18:39:16 <andythenorth> yo
18:39:25 <LordAro> o/
18:42:26 *** HerzogDeXtEr has joined #openttd
18:48:10 *** tokai|noir has joined #openttd
18:48:10 *** ChanServ sets mode: +v tokai|noir
18:54:56 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro opened pull request #8315: Codechange: Set CMAKE_BUILD_TYPE to default to debug if not otherwise set https://git.io/JU2X4
18:55:08 *** tokai has quit IRC
18:55:13 <LordAro> hmm, i suppose i should look at some of these 72 open PRs
18:55:26 <LordAro> though 4 of them are mine
18:56:52 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro approved pull request #8312: Fix #8311, b98c7763de4: Industry probability at map generation was sc… https://git.io/JU2XE
18:57:12 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro merged pull request #8305: Fix: Only check houses for cargo when generating subsidies with towns https://git.io/JUGId
19:02:49 <LordAro> @seen nielsm
19:02:49 <DorpsGek> LordAro: nielsm was last seen in #openttd 1 week, 2 days, 23 hours, 39 minutes, and 17 seconds ago: <nielsm> probably some regulations with delivery and inspection before they are allowed to run under own power
19:05:02 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro approved pull request #7028: Feature: Option to group vehicle list by shared orders https://git.io/JU2XF
19:07:49 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro approved pull request #7328: Feature: Improve restart command https://git.io/JU21T
19:08:01 *** jottyfan has joined #openttd
19:08:37 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro commented on pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/JU21m
19:09:29 *** jottyfan has quit IRC
19:10:36 <DorpsGek_III> [OpenTTD/eints] frosch123 approved pull request #10: Various of changes and fixes https://git.io/JU213
19:11:37 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro commented on pull request #7376: More than max_no_competitors could be created in network games https://git.io/JU21C
19:11:38 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro closed pull request #7376: More than max_no_competitors could be created in network games https://git.io/fjvY8
19:14:41 <DorpsGek_III> [OpenTTD/eints] LordAro merged pull request #10: Various of changes and fixes https://git.io/JU2EN
19:17:21 <DorpsGek_III> [OpenTTD/OpenTTD] michicc merged pull request #7328: Feature: Improve restart command https://git.io/fhxPf
19:22:46 <DorpsGek_III> [OpenTTD/OpenTTD] michicc approved pull request #8266: Feature: Make news and errors close hotkeys configurable https://git.io/JU2Mf
19:23:16 <DorpsGek_III> [OpenTTD/OpenTTD] michicc merged pull request #8266: Feature: Make news and errors close hotkeys configurable https://git.io/JJIWa
19:24:39 <DorpsGek_III> [OpenTTD/eints] frosch123 commented on pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2ML
19:25:05 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro commented on pull request #7575: Feature: Industry production graph https://git.io/JU2MZ
19:26:28 <DorpsGek_III> [OpenTTD/OpenTTD] michicc commented on pull request #8286: Change: Service depot also reset breakdown chance. https://git.io/JU2MR
19:29:21 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro commented on pull request #7708: Draft Change: Use Squirrel version 3.0.7 https://git.io/JU2MX
19:29:21 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro closed pull request #7708: Draft Change: Use Squirrel version 3.0.7 https://git.io/fjxWf
19:29:55 <DorpsGek_III> [OpenTTD/OpenTTD] michicc merged pull request #8312: Fix #8311, b98c7763de4: Industry probability at map generation was sc… https://git.io/JUuOi
19:29:56 <DorpsGek_III> [OpenTTD/OpenTTD] michicc closed issue #8311: Industry probability differs when set by property versus callback https://git.io/JUuOB
19:34:25 <LordAro> TrueBrain: do you feel like a break from the python? feel like taking a look at https://github.com/OpenTTD/OpenTTD/issues/7679 ? it seems like it could be the right solution, but...
19:36:20 <DorpsGek_III> [OpenTTD/OpenTTD] LordAro approved pull request #7883: Fix: Ensure a minimum gap of water upon generating terrain with original generator https://git.io/JU2DQ
19:46:25 *** Xaroth has quit IRC
19:48:31 <frosch123> LordAro: hmm, ai/gs always require a specific version of a library, right? should ottd just never select ai/gs libraries as upgrades?
19:48:37 <TrueBrain> LordAro: I am reading the issue and the PR, and it seems I think it is gibberish .. not sure if I am just tired, or that I cannot follow :P
19:49:14 <LordAro> ¯\_(ツ)_/¯
19:49:25 <TrueBrain> I will try this again Saturday :)
19:49:48 <frosch123> TrueBrain: i read the backlog. originally we disabled issues on all projects that were ported from devzone, since we did not know whether to migrate issues or not. other than that i do not know which bugs you already fixed, or what i shall look at :)
19:50:12 <TrueBrain> the ?message= I have no clue about
19:50:23 <TrueBrain> it seems to be stored in the session too, but never purged
19:50:29 <TrueBrain> that one is open, and I have not looked into it :)
19:51:32 <DorpsGek_III> [OpenTTD/eints] TrueBrain commented on pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2yy
19:52:44 <DorpsGek_III> [OpenTTD/eints] TrueBrain commented on pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2yQ
19:54:26 <TrueBrain> hmm .. guess if we run it UTC, we should either make the nightly UTC too
19:54:30 <TrueBrain> or put this an hour earlier
19:55:09 <TrueBrain> given the activity of translations, not sure it really matters
19:55:23 <TrueBrain> btw, frosch123 , as you might have spotted, I removed the "hourly" check if OpenTTD had a new commit to update translations
19:55:30 <TrueBrain> pretty sure that currently it is fine to check it every night
19:55:32 <TrueBrain> you agree?
19:56:27 <DorpsGek_III> [OpenTTD/eints] TrueBrain updated pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2Ej
19:56:41 <frosch123> sure, that was always a waste of resources. i just did not know better :)
19:57:10 <TrueBrain> I was thinking of running an update on every commit
19:57:15 <TrueBrain> but I considered even that a waste of resource :P
19:57:20 <TrueBrain> we can always add that if we like :)
19:57:30 <TrueBrain> guess that means the only question remains: in which repository is this best suited
19:57:47 <TrueBrain> given you can manually trigger it, eints sounds most natural to me, as people might look there first
19:57:58 <TrueBrain> but I agree that the workflow repo can be useful too, as it interconnects repositories of OpenTTD
19:58:03 <TrueBrain> up to you!
19:58:43 <TrueBrain> eints btw still links to openttdcoop on every page; possibly change that to GitHub and open up issues there?
19:59:12 <frosch123> yes, but the manual is only hosted on coop
19:59:31 <frosch123> it can stay like that until it breaks :)
20:00:46 <frosch123> anyway, either we keep eints master as "pure eints free of openttd stuff". or we abandon the idea of the "openttd" branch.
20:01:28 <TrueBrain> I also considered renaming master to something else, dunno, "upstream", and have master only tell about the branches and these few workflows
20:01:52 <frosch123> so, i am "no" to "workflows in master, when openttd branch exists". i am fine with "workflows in workflows repo" or "abandon the openttd branch, do everythnig in master"
20:02:10 <TrueBrain> so for now, workflow repository it is :)
20:02:12 <frosch123> in theory there is also the "openttdcoop" organisation
20:02:14 <LordAro> how feasible is adding a --coop flag to it to do the (minimal) amount of differing behaviour?
20:02:21 <frosch123> which is some kind of zombie graveyard
20:02:25 <TrueBrain> it is not minimal LordAro :(
20:02:28 <LordAro> :(
20:02:31 <frosch123> but we can transfer eints back there, and then fork it
20:02:51 <TrueBrain> I rather have eints in OpenTTD, where it has some "maintainers"
20:02:58 <TrueBrain> moving it to openttdcoop feels like moving it to a graveyard
20:03:41 <LordAro> i'd suggest a coop branch over an ottd branch
20:03:52 <LordAro> (i.e. swap existing stuff)
20:04:01 <frosch123> LordAro: there is no coop branch
20:04:36 <LordAro> i mean create one
20:04:38 <frosch123> or rather, the coop branch is still hosted on coop as hg queue
20:05:30 <frosch123> the difference between "master" and "openttd" is, that "openttd" says "welcome to openttd webtranslator, visit our homepage, sign up here", while "master" says something like "welccome, you probably want to customize this"
20:05:48 <TrueBrain> master really is upstream of openttd
20:05:51 <TrueBrain> basically
20:05:58 <TrueBrain> and nothing we can really do about that; as that is how software should work :D
20:06:09 <TrueBrain> "openttd" is the implementation
20:06:37 <TrueBrain> ideally, you make "eints" into a pip package
20:06:40 <TrueBrain> and allow overriding
20:06:43 <TrueBrain> but ... now I am pushing it :D
20:07:16 <frosch123> ok, big branch renaming? "master" -> upstream", "openttd" -> "openttd-ldap", "openttd-github" -> "master" ?
20:07:48 <TrueBrain> it will confuse the heck out of people who want to develop for it, just to fix our itch :P
20:07:53 <TrueBrain> I rather put the workflow in another repo
20:08:29 <LordAro> redo the whole eints package so that ottd/ottd-gh/coop stuff can be implemented as a plugin :)
20:08:41 <TrueBrain> or: redo eints :)
20:08:47 <LordAro> WT5 when
20:08:53 <TrueBrain> like bottle sounded like a good idea at the time
20:08:58 <TrueBrain> but it turned out, it was the wrong horse to bet on
20:09:00 <TrueBrain> which happens
20:11:52 <DorpsGek_III> [OpenTTD/workflows] TrueBrain opened pull request #6: Add: [Actions] export eints to git every night or upon request https://git.io/JU2SB
20:12:02 <DorpsGek_III> [OpenTTD/eints] TrueBrain commented on pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2SR
20:12:03 <DorpsGek_III> [OpenTTD/eints] TrueBrain closed pull request #11: Add: [Actions] export eints to git every night or upon request https://git.io/JU2Ej
20:13:26 <DorpsGek_III> [OpenTTD/eints] TrueBrain opened pull request #12: Fix: [Actions] label a step the same as the action it performs https://git.io/JU2S0
20:14:38 <DorpsGek_III> [OpenTTD/workflows] TrueBrain opened pull request #7: Codechange: update to latest black coding style https://git.io/JU2Su
20:15:24 <frosch123> TrueBrain: ah, i made python newbie mistake
20:15:49 <TrueBrain> you did?
20:16:01 <TrueBrain> I make so many .. my editor tends to fix most :P
20:16:06 <frosch123> why does flake not warn about setting mutable objects as default values for parameters? :p
20:16:15 <TrueBrain> oef .. it should
20:16:20 <frosch123> "def foo(bar = []):" is almost always wrong
20:16:23 <TrueBrain> yup
20:16:36 <TrueBrain> it is ONLY valid if 'bar' is set (completely) again .. even then ..
20:16:45 <TrueBrain> weird that flake8 doesn't say NO
20:17:58 <TrueBrain> renamed "openttd" branch to "openttd-ldap"
20:18:05 <TrueBrain> removed "openttd-oauth"
20:18:18 <TrueBrain> can't as it is protected .. lol
20:18:46 <TrueBrain> gone now :P
20:19:42 <TrueBrain> okay, if I made no mistakes, this should "just work" now after merging :)
20:23:36 <TrueBrain> https://github.com/pycqa/flake8-bugbear detects those things frosch123 :)
20:25:02 <TrueBrain> it does indeed find your issue :)
20:25:39 <TrueBrain> finds 1 warning in the 5 other repos (bananas / masterserver)
20:25:51 <TrueBrain> and none in DorpsGek
20:25:53 <TrueBrain> so that is not too bad :)
20:30:04 <DorpsGek_III> [OpenTTD/eints] frosch123 opened pull request #13: Fix: Edit-string page accumulated messages from all users for all users. https://git.io/JU293
20:30:37 <DorpsGek_III> [OpenTTD/eints] TrueBrain approved pull request #13: Fix: Edit-string page accumulated messages from all users for all users. https://git.io/JU29G
20:30:43 <TrueBrain> it reads as correct :P
20:31:25 <frosch123> hmm now that i rebased... should i install sentry_sdk, or also start eints only in containers...
20:31:41 <TrueBrain> sentry-sdk is a very small pip package
20:31:52 <TrueBrain> and it is only in openttd-github
20:32:23 <frosch123> no...
20:32:36 <frosch123> i am on master
20:32:57 <TrueBrain> owh, I deployed it on master too
20:32:59 <TrueBrain> hmm, that was unneeded
20:33:01 <TrueBrain> owh well
20:33:17 <TrueBrain> but okay, it is similar to requests_oauthlib
20:33:29 <TrueBrain> so if you installed requests_oauthlib in your venv, you should do sentry-sdk too
20:33:38 <TrueBrain> if you did requests_oauthlib in docker, well, you already using docker :)
20:33:43 <DorpsGek_III> [OpenTTD/eints] frosch123 merged pull request #13: Fix: Edit-string page accumulated messages from all users for all users. https://git.io/JU293
20:34:38 <DorpsGek_III> [OpenTTD/eints] frosch123 approved pull request #12: Fix: [Actions] label a step the same as the action it performs https://git.io/JU29W
20:35:17 <DorpsGek_III> [OpenTTD/eints] TrueBrain merged pull request #12: Fix: [Actions] label a step the same as the action it performs https://git.io/JU2S0
20:35:19 <TrueBrain> fixed the commit message :P
20:36:01 <TrueBrain> not via a PR, but any comments on https://github.com/OpenTTD/eints/commits/openttd-github are welcome
20:39:41 <TrueBrain> pretty sure https://github.com/OpenTTD/eints/commit/958d839cbfa47f51a7f9b1ba15048cbae984955d should go to master, but okay :)
20:40:55 *** blathijs has quit IRC
20:41:13 <TrueBrain> hmm .. I am now getting "Access denied" when I want to access my language .. what is this about ..
20:41:44 <frosch123> not team member? wrong api token?
20:42:09 <frosch123> what does your profile page say?
20:42:17 <TrueBrain> that I do not have access to anything
20:42:36 <TrueBrain> owh, I throw away some of my personal tokens today
20:42:43 <TrueBrain> I am guessing that I used that "temporary" :D
20:45:44 <TrueBrain> frosch123: yippie, the messages bug is fixed \o/ :D Nice :)
20:46:01 <TrueBrain> okay, so now the two workflows PRs, and we can test this :D
20:46:10 <frosch123> nice that you found it before we had to make a CVE entry :)
20:46:34 <TrueBrain> :D
20:46:34 *** HerzogDeXtEr1 has joined #openttd
20:46:44 <frosch123> sharing messages between all sessions is the special kind of funny :)
20:46:46 <TrueBrain> "history" of eints is weird .. if I change a string I already changed, it doesn't show in the history
20:47:00 <TrueBrain> it only shows if someone else changed it
20:47:14 <TrueBrain> I assumed the changed-time or what is it called would help in that
20:47:16 <TrueBrain> but it doesn't seem to
20:47:17 <frosch123> it may have to do with that "stable-time" paraemter
20:47:30 <frosch123> but yes, that part is also pretty weird to me
20:47:35 <TrueBrain> it is set to 600 .. it is not 6 hours later, and it still didn't add it to the history :P
20:48:36 <LordAro> coincidentally, i am currently watching Primer
20:49:07 <TrueBrain> not even reviewing my black-fix PRs? Pfft :P :P
20:50:17 *** blathijs has joined #openttd
20:51:15 *** HerzogDeXtEr has quit IRC
20:51:34 <TrueBrain> frosch123: I see the wiki dude replied ... not really in a way I can understand or process the response
20:51:36 <TrueBrain> but it is a response?
20:52:15 <frosch123> it is a "yes i got your mail" response
20:53:27 <frosch123> i did a ROTATE on the tasks
20:53:54 <TrueBrain> que?
20:54:00 <frosch123> do everything else wiki-related first, before thinking again what to do with the pl stuff
20:54:06 <TrueBrain> ah :D
20:54:08 <TrueBrain> smart ;)
20:54:11 *** HerzogDeXtEr has joined #openttd
20:54:14 <TrueBrain> now review my PRs! I want to test this :D
20:56:07 <frosch123> if i do 'return ""' or 'return "yolo"' in health, it creates a 200 response
20:56:08 *** HerzogDeXtEr1 has quit IRC
20:56:40 <TrueBrain> what I have now also makes the content-type a nice text/plain :)
20:56:53 <TrueBrain> by default bottle makes it a test/html
20:56:54 <frosch123> ok, the "" is text/html :)
20:56:59 <TrueBrain> no clue why ..
20:57:18 <TrueBrain> but okay, bottle has more oddities :)
20:59:38 <DorpsGek_III> [OpenTTD/workflows] frosch123 approved pull request #7: Codechange: update to latest black coding style https://git.io/JU2Hr
20:59:51 <DorpsGek_III> [OpenTTD/workflows] TrueBrain merged pull request #7: Codechange: update to latest black coding style https://git.io/JU2Su
21:00:05 <DorpsGek_III> [OpenTTD/workflows] TrueBrain updated pull request #6: Add: [Actions] export eints to git every night or upon request https://git.io/JU2SB
21:01:00 <DorpsGek_III> [OpenTTD/workflows] frosch123 approved pull request #6: Add: [Actions] export eints to git every night or upon request https://git.io/JU2HK
21:01:20 <DorpsGek_III> [OpenTTD/workflows] TrueBrain merged pull request #6: Add: [Actions] export eints to git every night or upon request https://git.io/JU2SB
21:01:22 <TrueBrain> PAM PAM PAMMMMMM
21:02:16 <TrueBrain> https://github.com/OpenTTD/eints-sandbox/commits/master
21:02:17 <TrueBrain> IT WORKS :D
21:03:04 <frosch123> i see, your test translation changes are just as silly as mine :)
21:03:06 <TrueBrain> okay, this makes me really happy :)
21:03:10 <TrueBrain> yes, they are :D
21:03:29 <TrueBrain> as far as I can tell, this means we are ready to move this to production
21:03:33 <TrueBrain> but please do give it a spin yourself :)
21:03:46 <frosch123> it is on testing?
21:03:55 <TrueBrain> https://github.com/OpenTTD/workflows/actions?query=workflow%3A%22Eints+to+Git%22 <- "Run workflow" button to force an eints->git
21:04:01 <TrueBrain> https://translator.staging.openttd.org/
21:04:17 <frosch123> hmm. yeah "staging", not "testing" :)
21:04:21 <TrueBrain> staging is linked to eints-sandbox, as you might have guessed
21:04:38 <TrueBrain> the nuance between testing and staging is very thin in the OpenTTD infra ;)
21:06:26 <frosch123> what non-member can we force to test the whole thing from sign-up to edit?
21:06:58 <TrueBrain> Xaroth, but he has too many numbers .. Xaroth65 Xaroth0 .. not sure how many we need
21:07:00 <frosch123> Eddi|zuHause: you look like you want to become a translator
21:07:05 <TrueBrain> that works too :)
21:07:27 <Eddi|zuHause> are you volunteering me? :p
21:07:37 <frosch123> yes :)
21:07:44 *** Xaroth has joined #openttd
21:07:53 <frosch123> TrueBrain: how did you do that?
21:08:01 <TrueBrain> magic :D
21:08:11 <Eddi|zuHause> do i need to know what you're volunteering me for?
21:08:58 <frosch123> Eddi|zuHause: you go to https://translator.staging.openttd.org/ click "request access" and continue until you can replace german texts with sächsich
21:09:24 <TrueBrain> addition: request access to de_DE please ;)
21:09:34 <Eddi|zuHause> does it matter if i'm not a native sächsisch speaker? :p
21:10:01 <frosch123> no, but staging only supports the important languages like dutch, german and ido
21:10:19 <TrueBrain> ido made me giggle .. no clue what language that is :P
21:10:23 <TrueBrain> I just needed an unstable one :D
21:10:32 <frosch123> TrueBrain: it's a competitor to esperanto
21:10:39 <TrueBrain> frisian was too close to home :P
21:10:45 <TrueBrain> owh ... one more silly language
21:10:48 <DorpsGek_III> [OpenTTD/team] Eddi-z opened issue #6: [de_DE] Translator access request https://git.io/JU2QZ
21:10:49 <TrueBrain> no wonder it is still unfinished
21:11:22 <DorpsGek_III> [OpenTTD/team] DorpsGek commented on issue #6: [de_DE] Translator access request https://git.io/JU2QZ
21:11:43 <DorpsGek_III> [OpenTTD/team] frosch123 commented on issue #6: [de_DE] Translator access request https://git.io/JU2QZ
21:11:46 <TrueBrain> I am considering muting DorpsGek from announcing DorpsGek shit :P
21:12:01 <DorpsGek_III> [OpenTTD/team] DorpsGek commented on issue #6: [de_DE] Translator access request https://git.io/JU2QZ
21:12:02 <DorpsGek_III> [OpenTTD/team] DorpsGek closed issue #6: [de_DE] Translator access request https://git.io/JU2QZ
21:13:44 <TrueBrain> but pretty sure the spam is only for a little while, so possibly not worth the effort :P
21:14:40 <Eddi|zuHause> so i have to manually go to translator.stagting.openttd.org i presume
21:14:46 <Eddi|zuHause> *staging
21:14:53 <frosch123> yeah :)
21:15:06 <Eddi|zuHause> as the bot said the non-staging url
21:15:33 <frosch123> the teams are shared with the real thing
21:16:17 <TrueBrain> yeah, I wasn't going to roll out a staging for that :D
21:16:20 <Eddi|zuHause> well, i successfully logged in
21:16:33 <TrueBrain> change something! Anything!
21:17:19 <TrueBrain> frosch123: I am not able to import the current eints history btw .. it seems I have to reset it from zero
21:17:31 <TrueBrain> as in, I have not found a way to upload files myself to the persistent volume
21:17:37 <Eddi|zuHause> did that do something?
21:18:23 <TrueBrain> up to frosch to start an export to git ;) :D
21:19:23 <TrueBrain> it might be good to reset the history btw; as the usernames will change, it might be confusing otherwise
21:19:43 <TrueBrain> possibly we can add a link to the new discussion page
21:19:48 <frosch123> TrueBrain: that's fine. as long as the history is not reset everytime the container restarts :)
21:19:57 <TrueBrain> it is not :)
21:20:08 <TrueBrain> https://github.com/OpenTTD/eints-sandbox/commit/82156b5544c407edb98a1841e0e3c63b673d4022
21:20:32 <frosch123> Eddi|zuHause: thanks :)
21:20:41 <TrueBrain> seems the full-chain test succeeded :D
21:20:43 <TrueBrain> \o/
21:20:58 <TrueBrain> the migrating to AWS went quicker than we both expected, I guess :)
21:21:32 <frosch123> now someone needs to write a bot that can post "/approve" :p
21:21:49 <frosch123> TrueBrain: yes, i did you know you could have persistent files like that
21:21:52 <Eddi|zuHause> TrueBrain: you'll never be able to migrate away from AWS, though :p
21:21:54 <frosch123> *not
21:22:15 <TrueBrain> frosch123: I know it existed, but the stories varied wildly how easy it would be
21:22:18 <TrueBrain> it turns out ... sick easy
21:22:32 <TrueBrain> Eddi|zuHause: no, it is containerized .. pretty sure I can deploy this in Azure within a week or so :)
21:22:44 <TrueBrain> just ... I do not plan to ever found out :)
21:23:48 <TrueBrain> https://gist.github.com/TrueBrain/466739f35e8ee54b7d8171873ad84f2f <- to give you an idea how simple it turned out to be
21:24:22 <TrueBrain> and it will cost us, an estimation, 0.03 dollar per month
21:24:47 <Eddi|zuHause> TrueBrain: the problem with showing a small solution like this is, it doesn't tell you how difficult it was to arrive at that solution :)
21:25:02 <TrueBrain> well, I planned 40 hours for this; I finished in 6 :P
21:25:30 <TrueBrain> but no, there was no real documentation .. just snippets of information
21:25:33 <Eddi|zuHause> TrueBrain: definitely charge the full 40 hours :p
21:25:33 <TrueBrain> as with most cloud-related stuff
21:25:36 <frosch123> Eddi|zuHause: the last time tb got any cake-payment was in 2014
21:25:39 <TrueBrain> I can send bills for this?!
21:26:09 <frosch123> https://github.com/pycqa/flake8-bugbear <- anything we need to do with that?
21:26:18 <frosch123> you mentioned you ran it on other repos?
21:26:45 <TrueBrain> pip install flake8-bugbear, and run flake8
21:27:08 <TrueBrain> but integrating it in the CI is difficult
21:27:17 <TrueBrain> it was just fun to run it, see if it showed any new things
21:27:26 <TrueBrain> attaching LGTM to eints might be more useful btw
21:27:30 <TrueBrain> LGTM for sure would pick up these things
21:27:32 <Eddi|zuHause> frosch123: i feel like it's a bad time to have a cake party
21:28:43 <TrueBrain> LGTM looks at many more of these kind of errors; it is really impressive tbh :)
21:29:32 <TrueBrain> https://lgtm.com/projects/g/OpenTTD/master-server/?mode=tree
21:29:37 <TrueBrain> still some minor warnings I need to fix there, for example
21:29:50 <TrueBrain> flake8 is oblivious for them :D
21:31:41 <frosch123> so, do you want to stick with saturday? or move to friday?
21:32:10 <TrueBrain> I am not here tomorrow, so that doesn't sound like a good idea :D
21:32:18 <TrueBrain> Saturday sounds fine by me
21:32:30 <TrueBrain> I have some things to prepare, and we need to run a pre-live test
21:32:34 <frosch123> yay, saturday :)
21:33:40 <frosch123> ah, i can also revoke my pat now...
21:35:56 <TrueBrain> :D Always a good thing to be able to do :)
21:36:33 <frosch123> hmm... so 184 people logged in to bananas once, but only 70 people uploaded something
21:36:52 <TrueBrain> lol
21:36:58 <TrueBrain> we migrated 184 people?
21:37:18 <frosch123> no, we migrated < 70
21:37:29 <TrueBrain> pfew :P
21:37:31 <frosch123> 70 = number of people migrated + new sign ups
21:37:43 <frosch123> 184 people authorized bananas to access their profile
21:38:15 <frosch123> so 114 just pressed login, clicked "yes, allow access", and logged out again
21:38:47 <TrueBrain> or just looked around, curious people etc
21:38:58 <TrueBrain> or got heavily confused by the "upload" form :D
21:39:07 <frosch123> :)
21:39:36 <TrueBrain> but still, I consider BaNaNaS v1.5 successful
21:39:43 <TrueBrain> weekly we have several updates
21:40:00 <TrueBrain> pretty sure Gadg8eer made most :P
21:40:03 <TrueBrain> I see that name a lot :D
21:42:18 <frosch123> 57 versions in 14 newgrf. not yet V level
21:43:12 <TrueBrain> okay, provisioning eints to production already, just so that is already running
21:43:24 <TrueBrain> let's see if it can run in 512MB RAM .. would be nice
21:43:33 <TrueBrain> more is problematic ..
21:44:09 <frosch123> it needs 400 here
21:44:25 <TrueBrain> 350MB after my patches in my last tests
21:44:30 <TrueBrain> (up from 550 .. )
21:44:34 <TrueBrain> hopefully it doesn't spike too much :)
21:50:00 <TrueBrain> right, provisioning is working .. now I have to wait till AWS internally propagate a DNS entry for the EFS .. which oddly enough takes a bit of time :)
21:50:11 <TrueBrain> creating a new EFS volume is near instant (scary fast)
21:50:16 <TrueBrain> but the DNS to the EFS .. poeh
21:53:21 <TrueBrain> importing OpenTTD ..
21:53:24 <TrueBrain> pam pam pammmmm
21:54:31 <TrueBrain> well, it still runs
21:55:40 <TrueBrain> 35.158.0.106 translator.openttd.org <- for the curious
21:57:39 *** tokai has joined #openttd
21:57:39 *** ChanServ sets mode: +v tokai
21:57:58 <DorpsGek_III> [OpenTTD/eints] TrueBrain created new tag: openttd-github-1.0.0 https://git.io/JU25V
21:58:01 <TrueBrain> right, lets try this ..
22:01:32 *** snail_UES_ has quit IRC
22:04:38 *** tokai|noir has quit IRC
22:05:13 <TrueBrain> after import eints uses less memory than after a restart
22:05:16 <TrueBrain> that is funny :)
22:05:35 <TrueBrain> the memory profile did show JSON decoder was holding on to 100MB of RAM
22:05:41 <TrueBrain> I was assuming it was strings that were given to other objects
22:05:52 <TrueBrain> but now I start to doubt it, and possibly the JSON loaded objects are kept alive
22:05:58 <TrueBrain> something to check out saturday :P
22:09:33 <TrueBrain> so because eints runs as a single instance, a deployment means a slight downtime
22:09:36 <TrueBrain> pretty sure we survive
22:11:14 <frosch123> the commit scripts also cause downtime
22:11:43 <frosch123> downloading all those langfiles blocks everything :)
22:11:51 <TrueBrain> and, and because you switched to cookies, you are also logged out
22:12:17 <TrueBrain> but because of GitHub, anything that requires access will auto-login
22:12:46 <TrueBrain> it is a pretty good user experience tbh
22:12:49 <frosch123> you still have to reenter the translation you were working on (i think)
22:13:09 <frosch123> anyway, we do not have 100s of concurrent translators :)
22:13:16 <TrueBrain> https://pasteboard.co/JsBXpOj.png <- memory usage :)
22:13:33 <TrueBrain> 100% = 512MB
22:13:38 <frosch123> what are the colors?
22:13:53 <TrueBrain> max, avg, min
22:14:12 <TrueBrain> the blue line is important ;)
22:14:23 <TrueBrain> the first peak is after import, the second after a restart
22:15:42 <DorpsGek_III> [OpenTTD/workflows] TrueBrain opened pull request #8: Add: [Actions] enable eints-to-git for production https://git.io/JU2dn
22:15:57 <TrueBrain> okay, I think everything is set for a launch on Saturday :)
22:17:06 <frosch123> \o/
22:18:14 *** snail_UES_ has joined #openttd
22:18:49 <TrueBrain> we let it run till then to spot any running issue
22:18:54 <TrueBrain> but I am pretty happy :)
22:19:16 <TrueBrain> now the question .. can we disable accounts.openttd.org saturday? It would mean nobody can signup for a new account for the wiki :P
22:20:47 <frosch123> or reset their passwords :)
22:21:53 <TrueBrain> looking at it from a risk perspective, it is worth blocking the very very unlikely event someone would want to do that :)
22:24:01 <TrueBrain> hmm, I cannot spot 1-2-3 why a reload uses so much more memory
22:24:35 <frosch123> anyway, eints does not need accounts either. so you could already have stopped it a week ago
22:24:48 <TrueBrain> valid point
22:24:56 <TrueBrain> well, I will have a very happy weekend in that case :P
22:25:08 <TrueBrain> and I will have to download a snapshot of a 600GB VM :P
22:25:17 <TrueBrain> does that really mean I can shut down Debian Etch?
22:25:20 <TrueBrain> :D \o/ :D
22:25:38 <frosch123> ldap and mail runs in a different vm?
22:25:53 <TrueBrain> yes
22:25:59 <TrueBrain> MySQL and LDAP have both their own VM
22:26:07 <TrueBrain> mail too, one for storing and one for spam filtering
22:26:16 <TrueBrain> dev-spaces is on yet another
22:26:32 <frosch123> etch is only old^6stable
22:26:35 <TrueBrain> there are a few things still served by that very old VM, mostly things like openttd.com redirects
22:26:44 <TrueBrain> well, it is also Django 1.2 :P
22:27:01 <TrueBrain> what is mode "t" in open() ?
22:27:12 <frosch123> line end conversion?
22:27:17 <TrueBrain> owh, text .. lol
22:27:26 <TrueBrain> "r" === "rt"
22:27:38 <frosch123> can't remember whether python3 ignores it
22:28:01 <TrueBrain> ignoring is a big word, it just has no additional effect :)
22:28:22 <frosch123> it's important in C on dos and windows
22:28:56 <frosch123> always funny how much dos is still in windows :)
22:29:10 <TrueBrain> in Python, you have to be explicit to use binary mode
22:29:13 <TrueBrain> text mode is otherwise implied
22:29:18 <TrueBrain> which .. I never like, those implicit shit :P
22:29:29 <frosch123> isn't it the other way around?
22:29:41 <frosch123> you have to set an encoding, or you will get Bytes ?
22:29:52 <TrueBrain> https://docs.python.org/3/library/functions.html#open
22:30:00 <TrueBrain> 't' - text mode (default)
22:30:21 <TrueBrain> The default mode is 'r' (open for reading text, synonym of 'rt')
22:31:04 <TrueBrain> okay, it is the "changes" that consume so much memory
22:31:36 <TrueBrain> ah, it contains all the current strings too
22:31:39 <TrueBrain> hihi, "changes" .. :D
22:32:47 <TrueBrain> lol .. the base language is in memory for all the languages too
22:33:55 <frosch123> don't ask me whether they can differ :)
22:34:11 <TrueBrain> I am still a bit puzzled why a restart consumes so much more memory
22:35:05 <TrueBrain> but at least Python is smart in that if you move strings from one object to another, it doesn't copy it
22:35:15 <TrueBrain> so it tells me the JSON decoder is using most memory
22:35:18 <TrueBrain> which is not really the case
22:35:35 <frosch123> TrueBrain: didn't you just answer that?
22:35:53 <frosch123> when you upload a language, it matches it against the baselanguage, so the same string is referenced
22:36:02 <TrueBrain> hmmmmmm
22:36:04 <frosch123> when loading from json, all translations get their own basetexts
22:36:13 <TrueBrain> that is a very good point, didn't realise that :)
22:36:19 <TrueBrain> that would indeed explain
22:36:40 <TrueBrain> you smart!
22:36:59 <frosch123> i just was your loopback device
22:37:30 <TrueBrain> I honestly did not consider that case yet :)
22:37:40 <TrueBrain> but it is a logic deduction :) Now lets verify if it is true
22:38:10 <TrueBrain> removes ~30MB
22:38:48 <TrueBrain> still missing ~60MB of RAM
22:39:12 <frosch123> what about the string id? STR_xyz
22:40:44 <TrueBrain> that seems to consume the most memory
22:43:03 <TrueBrain> removed 100MB RAM with some silly code
22:43:20 <frosch123> look up all loaded strings in a set?
22:43:31 <frosch123> "unknown" is also loaded often
22:43:35 <TrueBrain> https://gist.github.com/TrueBrain/77cf77f1dd7aa6e40d733dddc69c982d
22:44:18 <frosch123> i think you can do that for all items in node[]
22:44:20 <TrueBrain> that saves 60MB of RAM .. :)
22:44:45 <TrueBrain> it is really knowing how Python allocates memory, and HEAVILY abusing that :P
22:46:30 <TrueBrain> okay, testing it with strname, case, user and base_text
22:46:38 <frosch123> a lot of script languages do that for all strings by default
22:46:39 <TrueBrain> integers and new_text won't really help, I hope :)
22:46:50 <TrueBrain> down to 250MB RAM that way
22:46:57 <frosch123> :p
22:47:03 <TrueBrain> it is really worth it, tbfh
22:47:16 <TrueBrain> just not sure what the consequences are for base_text
22:47:22 <TrueBrain> as there all language get the same OBJECT
22:47:26 <TrueBrain> which is slightly different
22:47:51 <frosch123> strings are immutable, what should happen?
22:47:58 <TrueBrain> objects are not
22:47:59 <frosch123> i doubt there is any IS vs ==
22:48:04 <TrueBrain> base_text is an object
22:48:16 <TrueBrain> and I do not know if it is immutable :)
22:50:14 <TrueBrain> trouble of not knowing the code very well :)
22:50:26 <TrueBrain> if I only dedup strings, and only strings, I am at 290MB
22:50:30 <TrueBrain> which is already an improvement
22:50:58 <TrueBrain> (from 342)
22:51:12 <TrueBrain> so I need to know how base_text is used :)
22:51:56 <frosch123> it has custom __eq__ and __neq__ for a start
22:52:12 <TrueBrain> the only question is: is the object mutateble :)
22:52:53 <TrueBrain> it appears to be write-once
22:53:01 <TrueBrain> but ... how this code base is setup, it is difficult to trace
22:55:10 <TrueBrain> how languages are loaded, it appears to be shared too
22:55:19 <TrueBrain> so it should be fine? :P
22:57:50 <frosch123> yeah, looks good
22:58:02 <frosch123> looks like Text and Change are never modified after creation
22:58:33 <frosch123> can you declare classes as immutable?
22:58:39 <TrueBrain> you can, sort-of
22:58:43 <TrueBrain> and I was thinking the same :)
22:58:46 <TrueBrain> will try that this weekend
22:58:57 <TrueBrain> as that means we can cache the objects too
22:59:00 <TrueBrain> which saves even more memory
22:59:17 <TrueBrain> well, not new_text, as that is very likely to be unique :P
22:59:43 <frosch123> it's not as likely as you make it sound
23:00:14 <TrueBrain> every new_text should be somewhat unique, not?
23:00:29 <frosch123> in english.txt we have 15 texts for "None"
23:00:57 <TrueBrain> is it worth keeping all these Text in a dict?
23:01:00 <TrueBrain> well, lets find out!
23:01:35 <TrueBrain> we can ofc debate that, or just test it :D
23:01:40 <TrueBrain> 254 without
23:01:59 <TrueBrain> 262 with
23:02:01 <andythenorth> I am observing a 30GB python problem $somewhere else
23:02:05 <andythenorth> synchronicity :P
23:02:10 <TrueBrain> keeping it in a dict consumes slightly more than it saves :(
23:02:10 <andythenorth> must be the moon phase or something
23:02:49 <TrueBrain> let me try it in a different way ..
23:03:23 <TrueBrain> same result
23:03:49 <TrueBrain> so it appears to be unique-enough to not offset deduplication :)
23:04:53 <TrueBrain> but I am now using a stupid dict; possibly there is a better way
23:05:35 <frosch123> the dict is temporary? or do you keep it all the time?
23:06:05 <frosch123> in the latter case i would be worried abuot it accumulating size
23:06:22 <TrueBrain> it is only an issue if the container runs for long long periods of time
23:06:25 <TrueBrain> which is unlikely ;)
23:06:38 <TrueBrain> but yeah, ideally you remove any item with refcount == 1
23:07:09 <TrueBrain> but possibly even better, only on load
23:07:13 <TrueBrain> and throw away the dict after
23:07:23 <frosch123> hmm.. i can't remember whether languages we loaded on demand
23:07:24 <TrueBrain> but .. that is a bit difficult in this code-base
23:07:43 <frosch123> but possibly projects are always loaded as a whole
23:07:48 <TrueBrain> it appears they are
23:07:59 *** Lejving has quit IRC
23:08:09 <frosch123> ah, i think the optimisation was to only write languages to disk that changed
23:08:16 <TrueBrain> but okay, at least this proofs the issue is with how languages are stored on disk :)
23:08:18 <frosch123> instead of writing all languages when one language changed
23:08:42 <TrueBrain> finding a proper fix, is the next step :D
23:10:18 <TrueBrain> and ~250MB RAM is a nice place to be
23:10:23 <TrueBrain> I am totally fine with that
23:10:38 <TrueBrain> means I can provision it with 384 or something
23:10:42 <TrueBrain> enough wiggleroom
23:11:17 *** HerzogDeXtEr has quit IRC
23:11:24 <TrueBrain> so: proof Change and Text are immutable by making them immutable (after initalization)
23:11:39 <TrueBrain> and de-duplicate them in a smart way upon load
23:12:01 <TrueBrain> that means over time it will increase in memory, but containers restart often enough for that to not be a real issue
23:13:49 <TrueBrain> I am looking at the "upload_language" code, specific the function dealing with uploading non-base languages ..
23:13:56 <TrueBrain> I simply do not understand it sufficiently to get what it does
23:15:41 <frosch123> TrueBrain: what if you use system.intern() instead of the custom dict?
23:15:42 <TrueBrain> it overwrites the latest change (if override is set), instead of adding one, if it is different?
23:16:11 <TrueBrain> system.intern?
23:16:18 <frosch123> sys.intern()
23:16:23 <frosch123> https://docs.python.org/3/library/sys.html
23:16:50 <TrueBrain> I cannot free that
23:16:59 <TrueBrain> so that would mean memory slowly increases
23:17:11 <frosch123> it does that itself (it claims)
23:17:52 <TrueBrain> 1MB difference (with a gain for sys.intern)
23:18:25 <TrueBrain> owh, they are destroyed when refcount hits zero
23:18:31 <TrueBrain> sweet, yeah, that is the better one to use
23:18:34 <TrueBrain> but it only works for strings
23:19:31 <TrueBrain> owh, I was already deduplicating new_text.text btw, in my earlier tests, to be clear :)
23:21:36 <frosch123> TrueBrain: hmm, yes, looks like "override" violates the immutable
23:22:45 <DorpsGek_III> [OpenTTD/eints] TrueBrain opened pull request #14: Add: use sys.intern() on strings for classes using __slots__ https://git.io/JU2Nl
23:22:48 <TrueBrain> at least this is now safe to do ^^ :)
23:22:56 <frosch123> oh, "override" only updates the stamp and the user
23:23:10 <TrueBrain> yeah, it really confuses me tbh
23:23:15 <frosch123> well, and updates the "last_upload"
23:23:47 <TrueBrain> override is always off in our use-case btw
23:23:57 <TrueBrain> at least, lang_sync sets it to nothing
23:24:13 <frosch123> TrueBrain: why not sys.intern the base_text ?
23:24:13 <TrueBrain> a user possibly could upload a language file, but I assume that is blocked by roles
23:24:39 <TrueBrain> it is a Text() object
23:24:39 <frosch123> TrueBrain: i thought the --force option sets it
23:25:03 <TrueBrain> https://github.com/OpenTTD/eints/blob/master/scripts/lang_sync#L670
23:25:05 <frosch123> TrueBrain: ah, again :p
23:25:56 <DorpsGek_III> [OpenTTD/eints] frosch123 approved pull request #14: Add: use sys.intern() on strings for classes using __slots__ https://git.io/JU2NV
23:25:56 <TrueBrain> --force only exists in your eintsgit.py, which ignores the outcome of pull if there was anything new to do
23:26:13 <frosch123> ok, so more weird stuff :)
23:26:37 <frosch123> also, yes, uploads are disabled for everyone but "translators"
23:26:41 <TrueBrain> anyway, Change is not immutable; last_upload is indeed changed
23:27:15 <TrueBrain> I think the best way for base_text is to just get it from the base_language, in the same way the upload function does
23:27:23 <TrueBrain> there should be a match, in theory
23:27:29 <frosch123> ok, does sharing of Text() still save anything after the members are internalised?
23:27:31 <TrueBrain> I am just not sure that is true in reality :D
23:27:42 <TrueBrain> for base_text, another 40MB or so
23:27:52 <DorpsGek_III> [OpenTTD/eints] TrueBrain merged pull request #14: Add: use sys.intern() on strings for classes using __slots__ https://git.io/JU2Nl
23:27:53 <frosch123> TrueBrain: i think it is not true
23:28:12 <frosch123> i think it defines "outdated" strings
23:28:40 <frosch123> "translations that were not validated after gthe base language changed"
23:28:56 <TrueBrain> hmm, so that complicates it a bit :)
23:29:14 <frosch123> oh, and btw. that's the important information that is lost when clearing the efs
23:29:25 <frosch123> new eints will report all strings as up-to-date
23:29:33 <TrueBrain> ah!
23:29:38 <TrueBrain> it doesn't know the base language changed
23:29:39 <TrueBrain> makes sense
23:30:10 <frosch123> "confirm translation" is a eints-internal action
23:30:11 <TrueBrain> and there are more than a few outdated
23:30:20 <frosch123> that action does not affect the git repo
23:31:06 <frosch123> yeah, sorry. i knew that something got lost when deleting eints project. but i forgot what it was :)
23:31:06 <TrueBrain> how much of a problem is it, that we lose outdated?
23:31:38 <frosch123> last time it happend was when we moved from wt3 to eints
23:31:58 <frosch123> so, "every few years" is probably fine
23:32:24 <TrueBrain> otherwise what I can think off, is to replay every commit that changes the language
23:32:27 <TrueBrain> and import from "zero"
23:32:35 <frosch123> that's a waste :p
23:32:52 <frosch123> i can't even imagine how long that would take
23:32:52 <TrueBrain> scripting-wise, not difficult to do
23:32:55 <frosch123> a month?
23:33:17 <TrueBrain> only 4049 commits
23:33:25 <frosch123> TrueBrain: anyway, it does not help at all
23:33:33 <TrueBrain> takes ~30 seconds per commit .. so 2000 minutes
23:33:35 <TrueBrain> @calc 2000 / 60
23:33:35 <DorpsGek> TrueBrain: 33.3333333333
23:33:36 <frosch123> it would just mark all translations as outdated
23:33:39 <TrueBrain> over a day :P
23:33:47 <TrueBrain> ah, yes, ofc
23:33:48 <TrueBrain> okay
23:34:07 <TrueBrain> the other thing I can think off, is to start a special prepared container with a copy of the current JSON
23:34:11 <TrueBrain> which puts them on the EFS
23:34:15 <TrueBrain> as a "one-off"
23:34:35 <TrueBrain> but I would like to change all usernames to "translators", to remove any chance people confuse OpenTTD users with GitHub users :P
23:34:44 <frosch123> considering that outdated strings only affect unmaintained languages. it is better to not show any outdated strings to new translators, than with many strings that do not need touching
23:34:47 <TrueBrain> but, honestly, is it worth it?
23:35:06 <frosch123> no :)
23:35:11 <TrueBrain> German has 7 outdated strings! :P
23:36:06 <TrueBrain> it will just be what it is
23:36:27 <TrueBrain> owh, I see we can intern() a few more places, where Change is mutated; but that most likely won't really make a difference
23:36:29 <TrueBrain> this was the biggest blob
23:37:30 <frosch123> lol, STR_CONFIG_SETTING_PLANE_CRASHES_NONE gained an asterisk
23:37:41 <TrueBrain> OMG! :P
23:37:46 <frosch123> i guess i missed/forgot some drama
23:37:57 <TrueBrain> git log git log git log! :P
23:38:25 <frosch123> so, ottd has fine-print now :)
23:38:41 <andythenorth> pls, don't remind me
23:38:46 <TrueBrain> ok
23:38:59 * andythenorth happy place today
23:39:06 <andythenorth> apart from the 30GB python thing
23:39:18 <TrueBrain> give me the Python thing! I now know how to trace memory issues! :P
23:39:23 <TrueBrain> add _ _ slots _ _ everywhere! :P
23:39:24 <TrueBrain> :D
23:39:32 <andythenorth> we'd have to get you background checked
23:39:38 <andythenorth> dunno how that would go :P
23:40:06 <andythenorth> I did a talk to 50 people and finished a job description, so not a bad day
23:40:06 <TrueBrain> too bad you are in the wrong country; I already have sufficient of those in mine :P
23:40:10 <TrueBrain> pretty sure it counts for something :D
23:40:15 <frosch123> can you check the "dutch" box at the top?
23:40:26 <andythenorth> if it's EU, that's now negative points for our political culture I think
23:40:32 <andythenorth> probably automatic traitor or something
23:40:40 <TrueBrain> sounds right
23:40:41 <andythenorth> we're even walling off Kent because it's a bit close to France
23:40:51 <andythenorth> New Freedoms!
23:40:58 <andythenorth> but more paperwork, more borders
23:41:06 <andythenorth> such lol
23:41:20 <DorpsGek_III> [OpenTTD/eints] TrueBrain created new tag: openttd-github-1.0.1 https://git.io/JU2Aq
23:41:21 <michi_cc> But you have the control now!
23:41:34 <TrueBrain> YOU HAVE GOT THE POWERRRRRRR
23:41:35 <frosch123> andythenorth: oh, i was worried. i first read you had 50 job interviews in one day
23:41:50 <andythenorth> I read 10 CVs today, but not 50 job interviews :D
23:42:20 <TrueBrain> let's see what that mini-patch makes for a difference in production :D
23:42:37 <frosch123> andythenorth: what's the ratio of cv with photo vs cv without photo?
23:42:51 <andythenorth> we would remove them anyway
23:42:55 <andythenorth> so I have NFI :)
23:43:04 <TrueBrain> do you email CVs to other people? :D
23:43:08 <frosch123> ah, UK is that kind of country :)
23:43:11 <andythenorth> we try to remove bias-inducing information
23:43:15 <TrueBrain> do you have a policy on how to remove those CVs from their mailboxes?
23:43:23 <andythenorth> oddly yes
23:43:26 <andythenorth> it's GDPR stuff
23:43:28 <TrueBrain> it is not odd :)
23:43:31 <TrueBrain> it is mandatory :P
23:43:52 <andythenorth> we have a retention period for job applicant data
23:43:53 <TrueBrain> simple solution btw .. don't email CVs to other people
23:43:55 <TrueBrain> just print it :P
23:44:01 <TrueBrain> as you should :)
23:44:06 <TrueBrain> I am shocked how many companies still don't
23:44:22 <frosch123> TrueBrain: you can buy specialized software to manage CVs and their deletion
23:44:34 <TrueBrain> I am not surprised :)
23:44:39 <frosch123> i don't know how easily they leak though
23:44:51 <andythenorth> I am surprised that 'photo' would ever be useful in hiring
23:44:59 <andythenorth> unless it's for actors
23:45:04 <andythenorth> or impersonators
23:45:10 <TrueBrain> I once had someone that wrote League of Legends on his CV as Hobbies
23:45:11 <andythenorth> or ninjas
23:45:19 <TrueBrain> no clue how it was relevant, but I of course did ask about it
23:45:21 <TrueBrain> :P
23:45:36 <andythenorth> would a ninja photo be blank?
23:45:50 <TrueBrain> still one of the strangest things I saw on a CV .. and we hired the dude, so it does work :P
23:45:56 <TrueBrain> one of the best people we hired, tbh ..
23:46:02 <TrueBrain> not sure there is a correlation :D
23:46:05 <andythenorth> I don't mind weird CVs
23:46:19 <frosch123> andythenorth: here there are 3 types of cv: extroverts who have their face on the frontpage, covering half the page. introverts to put a tiny picture into some corner. and confident people who do not add a photo
23:46:36 <TrueBrain> I cannot believe people consider adding a photo ..
23:46:38 <andythenorth> photos are such a non-criteria
23:46:56 <TrueBrain> but our recruitment removed photos before it got to me, so I have no clue how many did send it ..
23:46:56 <andythenorth> I get triggered because people are putting their age and gender
23:47:23 <TrueBrain> Place of birth is a fun one to add
23:47:27 <TrueBrain> like ... who fucking gives a shit?
23:47:33 <TrueBrain> Country of birth, sure
23:47:34 <LordAro> we were supposed to put photos on our CVs for university year-in-industry placements
23:47:34 <TrueBrain> but place?
23:47:37 <LordAro> i refused
23:47:52 <andythenorth> was that for ID, or to enable affinity bias, structuralism racism etc?
23:48:11 <TrueBrain> in student housing we liked photos
23:48:15 <TrueBrain> it made screening go quicker :P
23:48:22 <TrueBrain> I AM NOT SAYING IT WAS FAIR
23:48:33 <andythenorth> oof, the wrong things I have done :)
23:48:37 <andythenorth> that I no longer do
23:48:44 <frosch123> andythenorth: how about marital status?
23:48:50 <TrueBrain> at-door-screening .. we were not allowed by the female housemates ..
23:49:02 <andythenorth> frosch123 another zero-fucks issue :)
23:49:30 <TrueBrain> frosch123: from 65% to 56% memory usage
23:49:34 <TrueBrain> from one simple patch
23:49:36 <TrueBrain> that is nice :D
23:50:18 <frosch123> andythenorth: anyway, a lot of old german templates contain all that weird stuff. so you will find that stuff on first-hires
23:53:01 <frosch123> TrueBrain: putting leisure activites onto your cv is recommended for initial smalltalk or something
23:53:39 <andythenorth> hiring is so weird
23:54:02 <frosch123> i also put foss on my cv. and someone once asked why ottd is so much C without objects :p
23:54:24 <frosch123> (i did not put "ottd" on the cv, they googled that themself)
23:54:29 <andythenorth> stalking!
23:55:40 <TrueBrain> Strictly seen not allowed to google people before an interview
23:56:09 <TrueBrain> Funny how many rules there are, and how many are broken :p
23:57:05 * LordAro has OTTD on his
23:57:31 <LordAro> think i mighr also have FRCT on it as well... i've not looked at it in about 4 years
23:57:51 <frosch123> TrueBrain: well, i also always got the name of the interviewer in advance
23:58:03 <frosch123> so, stalking was fair :)
23:58:40 <frosch123> oh, right... i found one interviewer on youtube, where they ranted about some political drama in their village
23:58:47 <frosch123> that was funny :p
23:59:01 <frosch123> esp. because it was the only video on their channel
23:59:19 <TrueBrain> Haha
23:59:35 <TrueBrain> I have to dust off my CV soon .. no clue what is on it tbh
23:59:42 <TrueBrain> Has been years :D