IRC logs for #openttd on OFTC at 2019-11-30
⏴ go to previous day
00:03:37 <Flo> idk if that tells you anything but...
00:03:43 <Flo> i get something like this:
00:03:44 <Flo> process_begin: CreateProcess(NULL, bin/git-info --num-id --version, ...) failed.
00:06:01 <LordAro> that's an unusual error message
00:06:08 <milek7> it seems to try start python script with CreateProcess?
00:06:29 <Flo> i never did python or make before...so i dont really know...
00:07:39 <LordAro> Flo: unlikely to be a question andy can answer anyway :p
00:07:53 <milek7> GIT_INFO = bin/git-info
00:08:01 <milek7> try GIT_INFO = python3 bin/git-info
00:08:08 <milek7> and ensure python3 is in $PATH
00:08:32 <glx> this suggestion makes sense
00:08:32 <LordAro> milek7: if python3 was in PATH, presumably there'd be no issues starting bin/git-info ?
00:08:53 <glx> shebang is not a thing on windows
00:09:03 <LordAro> pure windows environment would have trouble with it
00:09:12 <LordAro> ...but you're running make?
00:09:52 <glx> CreateProcess runs as a windows process
00:10:12 <LordAro> CreateProcess is the windows syscall for creating processes
00:10:27 <LordAro> Flo: how are you running make? what's your environment?
00:11:15 <glx> so make working in cmd/powershell
00:11:55 *** supermop_work_ has joined #openttd
00:11:55 <glx> I think in mingw it should work fine
00:12:45 <glx> anyway git-info just prints "1 1 1 1" it seems :)
00:12:58 <glx> (just looking at the code on github)
00:13:19 <Flo> i now get make: bin/git-info: Command not found
00:17:34 <glx> try REPO_INFO = $(shell $(PYTHON) $(GIT_INFO) --num-id --version) at line 29
00:17:49 <glx> (based on original firs repo)
00:19:06 <Flo> how do I do that? sorry if I seem dumb
00:19:22 <glx> hmm probably better to do GIT_INFO = "$(PYTHON) bin/git-info" on line 10
00:20:50 <glx> same on the following lines too
00:20:54 <Flo> looks like that made a difference
00:22:07 <Flo> i now get /bin/sh: bin/git-info: No such file or directory
00:27:11 <glx> what is your command line ? cmd, powershell or mingw ?
00:28:18 <Flo> i get the same error with mingw and cmd
00:28:35 <glx> ok I clone firs repo to try locally
00:29:20 <glx> will be easier than random guessing :)
00:29:46 <Flo> is git a program or something?
00:30:21 <glx> git is a program, but in this case bin/git-info is just a python script
00:31:38 <FLHerne> glx: Lots of pixels :P
00:31:51 <FLHerne> glx: --depth=1 is smaller if you don't care about history
00:34:59 <Flo> i might not have git-info?
00:35:08 <glx> it's included in the repo
00:40:44 <glx> ok need to install some libs it seems
00:41:15 <glx> but running make in mingw works
00:41:52 <glx> well it fails but not because of git-info
00:42:08 *** supermop_work_ has quit IRC
00:42:33 <Flo> i just ditn Florence@Flo-s-Laptop /c/Users/Florence/Downloads/firs-masterf/firs-master/bin
00:42:42 <Flo> however git-info is in that folder
00:43:11 <glx> current dir is never in the path
00:43:35 <LordAro> unlike native windows (like cmd, powershell), where it is
00:43:55 <glx> which is a good thing I think
00:44:46 <glx> in bin just type "./git-info"
00:46:23 <glx> hmm I should probably have skiped graphviz install
00:47:57 <Flo> Florence@Flo-s-Laptop /c/Users/Florence/Downloads/firs-masterf/firs-master/bin
00:48:19 <Flo> ./usr/bin/env: python3: No such file or directory
00:48:50 <glx> so no python, nor python3 from mingw command line I guess
00:49:13 <Flo> i get sh: /git-info: No such file or directory
00:49:18 *** supermop_work_ has joined #openttd
00:49:42 <glx> and if you type "python3 git-info" ?
00:50:20 <Flo> sh: python3: command not found
00:50:39 <Flo> python instead python3 seems to work
00:52:16 <LordAro> curious how you don't have a python3 executable
00:52:48 <glx> yeah I have both python and python3 on mingw (and they are the same)
00:53:47 *** HerzogDeXtEr has joined #openttd
00:58:08 <glx> but maybe it's because I have python2 installed too
01:00:29 <LordAro> wouldn't normally have that affect
01:00:56 <glx> nice a segfault when rendering the docs
01:02:59 <glx> let's uninstall graphviz then
01:05:21 <glx> better without graphviz :)
01:06:05 <glx> language warnings and finally nmlc ERROR: "generated/firs.nml", line 7342: Syntax error, unexpected token "waiting_cargo_1"
01:08:47 <glx> anyway for docs I needed to install python3-markdown
01:09:44 <glx> so using mingw it 'works'
01:10:06 <Flo> oh god...im so dumb...i just noticed i installed setuptools and cameleon but forgot markdowns...
01:10:13 <glx> nmlc errors, but makefile just works
01:11:35 <glx> with mingw it's often easier to install python module via pacboy (pip should work but sometimes fails if there's a module to compile)
01:12:11 <glx> anyway if pip works it's ok too
01:12:31 <glx> I think it failed for pillow
01:13:25 <Flo> there added markdown now :) i though ive put the 3...but ive only put 2...
01:15:25 <glx> oh of course it needs python3 exe
01:16:01 <Flo> sh: pacboy: command not found
01:16:21 <glx> oh you're on the old mingw
01:16:54 <glx> you should move to mingw-w64
01:16:57 *** Smedles has joined #openttd
01:17:24 <LordAro> i'm surprised you managed to get as far as you did
01:18:14 <glx> it's using the windows installed python
01:19:31 *** supermop_work_ has quit IRC
01:19:47 <glx> hmm typing python3 doesn't error in powershell, but it does nothing it seems
01:21:10 <glx> haha in cmd it opens windows store
01:21:11 <Flo> im installing the new mingw now
01:22:26 <Flo> buy python3, starting from only 199,99$. only available for windows proffesional
01:24:05 <Flo> when he said cmd took him to windows store
01:24:24 <glx> yeah it's free, but it can be annoying :)
01:24:50 *** supermop_work_ has joined #openttd
01:25:59 <glx> it's a shortcut they added in the last major update IIRC
01:34:18 <glx> follow initial pacman -Syu steps ?
01:35:43 <glx> you can close msys2 and open mingw
01:36:39 <glx> you'll need to do "pacboy -S python3"
01:38:06 <glx> and some other packages are needed too
01:38:47 <Flo> its doing python3 right now :)
01:40:00 <glx> probably "pacboy -S make" too
01:41:16 <glx> that should be a good start
01:41:40 <glx> of course you'll need to install required python modules too
01:42:18 <glx> via "pip install" or "pacboy -S python3-<module>" if pip fails
01:42:57 <LordAro> i'd start with the latter
01:43:24 <Flo> there, python3 and make added :)
01:45:18 <Flo> python3 command works but not make
01:46:15 <Flo> -bash: make: command not found
01:47:39 <glx> but it should be installed
01:52:11 <Flo> C:\msys64\var\lib\pacman\local\mingw-w64-x86_64-make-4.2.1-4
01:55:04 *** supermop_work_ has quit IRC
01:56:00 <glx> pacboy packages | grep install | grep make
01:56:08 <glx> you should see it there too
01:57:22 <DorpsGek_III> [OpenTTD/OpenTTD] MingweiSamuel dismissed a review for pull request #7595: Possible fix for #7430: when train visits station, only reset time_since_pickup if has room to load https://git.io/Jeiuh
01:57:23 <DorpsGek_III> [OpenTTD/OpenTTD] MingweiSamuel updated pull request #7595: Possible fix for #7430: when train visits station, only reset time_since_pickup if has room to load https://git.io/fjlA7
01:58:44 <glx> I can see I have the 3 variants installed mingw32 mingw64 and msys
02:00:04 *** supermop_work_ has joined #openttd
02:00:29 <Flo> theres nothing in C:\msys64\var\cache\pacboy
02:01:15 <Flo> idk how to check pacboy packages | grep install | grep make
02:01:30 <glx> just type it in mingw shell
02:03:07 <glx> my output for this command contains:
02:03:07 <glx> mingw32 mingw-w64-i686-make 4.2.1-4 [installé]
02:03:07 <glx> mingw64 mingw-w64-x86_64-make 4.2.1-4 [installé]
02:03:07 <glx> msys make 4.2.1-1 [installé]
02:03:53 <glx> (and cmake and many automake but that's useless for you)
02:05:07 <Flo> pacboy packages | grep install | grep make
02:05:07 <Flo> mingw32 mingw-w64-i686-make 4.2.1-4 [installed]
02:05:07 <Flo> mingw64 mingw-w64-x86_64-make 4.2.1-4 [installed]
02:05:35 <glx> so it's installed for mingw
02:06:26 <glx> I guess "which make" fails
02:07:44 *** WormnestAndroid has quit IRC
02:08:30 <glx> yeah it seems it's using msys one on my system
02:08:39 *** WormnestAndroid has joined #openttd
02:09:20 <Flo> yeah i added the msys one and now make is available
02:09:48 <glx> it's because mingw make are called mingw64-make and mingw32-make
02:12:02 <glx> so now you have make and python3
02:14:24 <glx> I don't know if pip is installed via python3
02:15:56 <glx> but it's easy to "pacboy -S python3-pip" if it's not
02:18:47 <glx> usual python modules are also available in packages
02:20:56 <glx> hmm first try "pacboy -S python3-<module>", I know pillow install fails with pip
02:22:20 <Flo> pacboy -S python3-<module>
02:22:21 <DorpsGek_III> [OpenTTD/OpenTTD] MingweiSamuel commented on pull request #7595: Possible fix for #7430: when train visits station, only reset time_since_pickup if has room to load https://git.io/Je1wQ
02:22:43 <Flo> $ pacboy -S python3-<module>
02:22:43 <Flo> -bash: syntax error near unexpected token `newline'
02:22:51 <glx> replace <module> with its name
02:23:17 <glx> pacboy -S python3-ply python-pillow
02:23:49 <glx> python3-markdown exists too
02:24:27 <glx> I think you'll need pip only for nml
02:24:50 <Flo> $ pacboy -S python3-pillow
02:24:50 <Flo> error: target not found: python3-pillow
02:26:11 <Flo> same with python3-markdown
02:27:41 <glx> it should show them (even uninstalled)
02:29:32 <glx> if you want to install something for msys you can do it from mingw by just adding :m to the package name (pacboy -S make:m)
02:30:18 *** supermop_work_ has quit IRC
02:31:27 *** supermop_work_ has joined #openttd
02:33:54 <glx> oh no sorry I miss read the help
02:34:16 <glx> For 64-bit MSYS2, name:i means i686-only
02:34:16 <glx> For 64-bit MSYS2, name:x means x86_64-only
02:34:16 <glx> For MSYS shell, name:m means mingw-w64
02:34:42 <glx> so to install stuff in msys you need to be in msys
02:35:03 <glx> but from msys you can install in mingw using :m
02:35:32 <glx> use "pacman help" to see more details
02:36:09 <glx> anyway you rarely need to install stuff in msys
02:36:54 <glx> unless like make it doesn't work as expected
02:37:17 <Flo> which: no dot in (/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/)
02:37:43 <glx> yeah, it's graphviz, ignore ite
02:38:18 <glx> doc generation crashed for me when I installed graphviz ;)
02:38:55 <Flo> Well then the last thing I get seems to be ModuleNotFoundError: No module named 'chameleon'
02:38:55 <Flo> make: *** [Makefile:75: generated/lang] Error 1
02:39:06 <Flo> But U got cameloan in my python... I think?
02:40:29 <Flo> Its in my python folder, but idk if it has to be in mingw
02:40:44 <glx> it's not the same python install
02:41:15 <glx> and it's not available as a mingw package, so pip install chameleon
02:41:33 <Flo> ok. how do I do pip install eacly?
02:45:34 <Flo> ok so I did pip install chameleon
02:45:44 <Flo> I still get no module called chameleon
02:45:45 <glx> usually for a missing python module you do "pacboy -S python3-<modulename>", and if it fails "pip install <modulename>"
02:46:28 <Flo> it said it installed chameleon with pip
02:46:41 <Flo> but I still get the module missing thing with make
02:47:19 <glx> "pip list" should show all installed modules
02:49:30 <Flo> ModuleNotFoundError: No module named 'chameleon'
02:49:30 <Flo> make: *** [Makefile:78: docs] Error 1
02:49:30 <Flo> Florence@Flo-s-Laptop MINGW64 /c/Users/Florence/Downloads/firs-masterf/firs-master2
02:49:32 <Flo> ---------- ---------------------
02:50:41 <glx> hmm and ugrading my msys/mingw install removed nml
02:51:20 <glx> oh it remove all stuff I previously installed via pip
02:52:25 <glx> so yeah using pacboy for python modules is a good idea :)
02:52:54 <glx> oh of course it upgrade python
02:53:05 <glx> I was on 3.7.4 and now it's 3.8
02:54:04 <Flo> so whats wrong with my chameleon?
02:54:56 <glx> reinstalling my local nmlc
02:58:53 <glx> I now get SyntaxWarnings during docs rendering (because python 3.8) but it works
03:01:36 *** supermop_work_ has quit IRC
03:08:08 <Flo> maybe because C:\msys64\usr\lib\python3.7?
03:08:17 <Flo> chameleon location is in there
03:15:10 <glx> for mingw it should be in C:\msys64\mingw64\lib\python3.8
03:15:32 <glx> but pip installs it there
03:15:56 *** Thedarkb-X40 has joined #openttd
03:16:41 <glx> you ran "pip install" in mingw ?
03:18:52 <Flo> Florence@Flo-s-Laptop MINGW64 ~
03:18:52 <Flo> Requirement already satisfied: chameleon in /usr/lib/python3.7/site-packages (3.6.2)
03:20:48 *** supermop_work_ has joined #openttd
03:21:16 <glx> pacboy find pip | grep install
03:21:48 <glx> I guess you have a pip in msys
03:22:49 <Flo> $ pacboy find pip | grep install
03:22:50 <Flo> An easy_install replacement for installing pypi python packages (mingw-w64)
03:22:50 <Flo> An easy_install replacement for installing pypi python packages (mingw-w64)
03:22:50 <Flo> An easy_install replacement for installing pypi python packages (mingw-w64)
03:22:50 <Flo> An easy_install replacement for installing pypi python packages (mingw-w64)
03:22:50 <Flo> The PyPA recommended tool for installing Python packages
03:22:52 <Flo> msys/python3-pip 19.3.1-1 [installed]
03:22:52 <Flo> The PyPA recommended tool for installing Python packages
03:23:33 <glx> pacboy -S python3-pip was done in the msys shell and not mingw shell :)
03:24:09 <Flo> im very bad at this sorry
03:25:03 <Flo> Atleast after that ill be able compile my firs fork as many times as I want :D
03:25:58 <glx> well the clean firs doesn't compile for me, but I use the dev version of nmlc
03:29:26 <glx> ah yes it's because 16 input/output cargo
03:31:10 <Flo> I get make: nmlc: Command not found
03:31:10 <Flo> make: *** [Makefile:93: generated/firs.grf] Error 127
03:32:22 <Flo> Running setup.py install for nml ... error
03:32:22 <Flo> ERROR: Command errored out with exit status 1:
03:33:17 <glx> I like when tools give very explicit messages :)
03:34:32 <glx> but I think I know why it fails
03:36:05 <Flo> ERROR: Command errored out with exit status 1: C:/msys64/mingw64/bin/python3.exe -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:/Users/Florence/AppData/Local/Temp/pip-install-dn_tymp4/nml/setup.py'"'"'; __file__='"'"'C:/Users/Florence/AppData/Local/Temp/pip-install-dn_tymp4/nml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
03:36:05 <Flo> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record C:/Users/Florence/AppData/Local/Temp/pip-record-ho7uyo_v/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
03:37:29 <Flo> Traceback (most recent call last):
03:37:29 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\cli\base_command.py", line 153, in _main
03:37:29 <Flo> status = self.run(options, args)
03:37:29 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\commands\install.py", line 446, in run
03:37:30 <Flo> installed = install_given_reqs(
03:37:30 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\req\__init__.py", line 58, in install_given_reqs
03:37:32 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\req\req_install.py", line 886, in install
03:37:34 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\utils\subprocess.py", line 271, in runner
03:37:36 <Flo> File "C:/msys64/mingw64/lib/python3.8/site-packages\pip\_internal\utils\subprocess.py", line 242, in call_subprocess
03:37:38 <Flo> raise InstallationError(exc_msg)
03:38:42 <glx> but I think "pacboy -S gcc" should do
03:40:41 <glx> it needs a compiler to build a lib
03:42:07 <glx> ahah and of course using this nml version fails too
03:42:19 <glx> because pillow is too recent
03:42:34 <glx> and that's fixed in dev version
03:44:47 <Flo> oh oof I had ModuleNotFoundError: No module named 'ply'
03:47:41 <Flo> I had Error: (AttributeError) "module 'PIL.Image' has no attribute 'VERSION'".
03:48:26 <Flo> how do I get an older pillow?
03:49:18 <Flo> or the version of firs that was fixed?
03:50:01 <glx> dev version of nmlc works with current pillow, but firs is not compatible with it
03:51:00 *** supermop_work_ has quit IRC
03:51:37 <Flo> should I just get an older pillow?
03:53:40 *** supermop_work_ has joined #openttd
03:55:08 *** Wormnest_ has joined #openttd
03:57:24 <Flo> what version is compatible with it?
03:58:17 <glx> only dev version is compatible with pillow >6.0
03:59:21 <Flo> ill install pillow earlier than 6.0
04:00:33 <glx> I'm trying "pip install pillow\<6.0"
04:03:32 <Flo> does not yet support python 3.7
04:04:29 <glx> it fails to compile some stuff but I don't know why
04:05:52 <Flo> oh you got it installed on 3.8?
04:06:21 <glx> I mean 6.2.1 package, the one from pacboy
04:07:11 <Flo> seems like we would need to use python 3.7
04:08:31 <glx> ah yes I use python 3.7 on windows for that reason, because there a precompiled pillow package compatible
04:09:06 <Flo> well then I guess ill do everything again but in python 3.7
04:09:17 <Flo> atleast now i should be able do most of it alone
04:09:26 <glx> it's hard to change version on mingw
04:10:17 <Flo> i mean, ill prob have to reinstall the packages
04:10:45 <glx> but without the pacman -Syu steps
04:11:34 *** ChanServ sets mode: +v tokai
04:11:43 <glx> because pacman -Syu upgrades the package list
04:12:07 <glx> but you could also use current nml
04:14:01 <Flo> ill get 3.7 python to start with, and try put all dependencies
04:14:15 <Flo> unless theres a way to do it in 3.8...
04:17:03 <glx> not yet, and I think it's a bug
04:23:50 *** supermop_work_ has quit IRC
04:29:00 *** supermop_work_ has joined #openttd
04:32:54 <glx> hmm will be easier to fix FIRS
04:34:05 <Flo> still trying to figure out how to install nml from that github
04:34:41 <glx> just git clone and "pip install ."
04:35:10 <glx> or better "pip install -e ."
04:49:44 <Flo> yeah, failed for me to...
04:49:57 <Flo> as you said, it does seems like a firs issue: nmlc ERROR: "generated/firs.nml", line 7342: Syntax error, unexpected token "waiting_cargo_1"
04:50:29 <glx> yes major change between nml 0.4 and 0.5
04:50:56 <glx> but that's fixable in firs
04:51:21 <Flo> yeah, seems more like a syntax thing...some rewriting could possibly fix that
04:51:40 <glx> just the doc is not fully updated ;)
04:52:04 <Flo> wait you updating the firs right now
04:53:12 <glx> but Industry variables is not
04:54:22 <Flo> if you manage to get a working firs, let me know :)
04:59:14 *** supermop_work_ has quit IRC
05:02:09 *** supermop_work_ has joined #openttd
05:31:26 *** Smedles has joined #openttd
05:32:20 *** supermop_work_ has quit IRC
05:33:09 *** supermop_work_ has joined #openttd
06:03:23 *** supermop_work_ has quit IRC
06:08:27 *** supermop_work_ has joined #openttd
06:38:34 *** supermop_work_ has quit IRC
06:40:44 *** supermop_work_ has joined #openttd
07:10:57 *** supermop_work_ has quit IRC
07:17:20 *** supermop_work_ has joined #openttd
07:43:26 *** sla_ro|master has joined #openttd
07:47:33 *** supermop_work_ has quit IRC
07:51:19 *** supermop_work_ has joined #openttd
08:03:28 *** Progman has joined #openttd
08:15:05 *** andythenorth has joined #openttd
08:21:31 *** supermop_work_ has quit IRC
08:24:09 *** supermop_work_ has joined #openttd
08:55:21 *** supermop_work_ has quit IRC
09:01:37 *** supermop_work_ has joined #openttd
09:01:48 <andythenorth> why is python import perfomance so poor?
09:01:56 <andythenorth> and what can I do about it ? :P
09:02:11 <andythenorth> is using modules wrong?
09:02:30 <andythenorth> I could just write everything in one python file
09:03:55 <Eddi|zuHause> andythenorth: it depends on loads of things, like what kinds of initialisations the module does
09:04:05 <Eddi|zuHause> or whether it needs to be converted to .pyc first
09:04:20 <andythenorth> it scales horribly
09:04:32 <andythenorth> there are some core modules that I import hundreds of times
09:04:46 <andythenorth> they're only 0.02s or such
09:04:55 <andythenorth> but they scale up to delays of seconds
09:05:12 <andythenorth> then I use a multiprocessing pool, and every pool instance scales that delay
09:05:13 <Eddi|zuHause> im sure there are import optimizers :)
09:05:28 <andythenorth> it's interestingly a new problem in 3.7 or 3.8
09:05:41 <andythenorth> 3.5 does not show this behaviour
09:05:55 <Eddi|zuHause> "pool instance" sounds like loads of initialisation code
09:06:05 <andythenorth> it is, and there's a known problem of overhead
09:07:48 <andythenorth> or 1.1s to import (import iron_horse)
09:08:32 <andythenorth> at least one problem appears to be that chameleon's PageTemplateLoader class is incredibly slow to import
09:09:03 <andythenorth> so I'll find a way around that :P
09:09:09 <Eddi|zuHause> i don't think i can help you here
09:09:15 <andythenorth> no, this is my mess
09:09:25 <andythenorth> but the 3.5 vs 3.8 degradation was shocking
09:09:31 <andythenorth> it's probably because 3.8 is faster
09:09:59 <andythenorth> so something in the implementation has changed, and is no longer caching modules or similar
09:14:24 <Eddi|zuHause> likely a side effect of some other fix
09:15:42 <andythenorth> 3.8 is generally faster for the cases I have, about 10%-20%
09:17:05 <andythenorth> also I learn some things this way :P
09:17:45 <planetmaker> moin, is it possibly worth reporting that as regression back to python? With the example as-is?
09:20:11 <andythenorth> not in the current form
09:20:20 <andythenorth> I'm using terrible python practices, like no main()
09:20:53 <andythenorth> I'll find out if the regression is legitimate, or just my bad style, then see :)
09:21:08 <andythenorth> shall we place a bet on who is more likely wrong? :P
09:26:09 <andythenorth> 3.8 vs 3.5 is measurably faster for nmlc
09:28:29 <Eddi|zuHause> andythenorth: step 1: make two test environments with 3.5 and 3.8 )and possibly some inbetween versions) to test whether the slowdown is real
09:28:45 <Eddi|zuHause> 2a) comment out code, until the difference disappears
09:29:16 <Eddi|zuHause> 2b) work through the changelog which changes would be likely to cause a slowdown
09:29:34 <andythenorth> that's not done yet :P
09:30:10 <andythenorth> I've fixed some self-inflicted problems, and now have parity for total run time between 3.5 and 3.8
09:30:25 <andythenorth> but as 3.8 is generally faster, that masks some things that are slower :)
09:31:24 <andythenorth> general observation, Iron Horse nmlc run time is about 28s for py3.8 vs 35s for py3.5
09:31:48 *** supermop_work_ has quit IRC
09:32:07 <andythenorth> oh if I let the caches prime, my total 3.8 run time is now a few seconds less than 3.5
09:33:10 <andythenorth> the remaining interesting problem
09:33:20 <andythenorth> is why a specific module takes 0.2s to just run
09:33:23 <andythenorth> and 1.1s to import
09:33:47 *** supermop_work_ has joined #openttd
09:34:01 *** arikover has joined #openttd
09:36:03 <andythenorth> hmm, that's same behaviour on 3.5 also, but after the first import, it's cached
09:36:18 <andythenorth> whereas 3.8 seems to incur the 1.1s every time the module is imported
09:41:13 <andythenorth> very interesting :P
09:41:19 <andythenorth> where's that xkcd on time saved? :P
09:53:07 <andythenorth> python3.5 doesn't trigger multiple imports when Pool is used, python3.8 does
09:53:47 <andythenorth> (I knew this already, I just added timings to verify it)
10:03:47 <andythenorth> python mp fork inherits from parent, mp spawn initialises clean
10:03:59 *** supermop_work_ has quit IRC
10:04:12 <andythenorth> fork is unreliable, changed on macos, consensus couldn't be achieved to change it on *nix for 3.8
10:04:30 <andythenorth> may be changed for *nix in 3.9
10:05:27 *** supermop_work_ has joined #openttd
10:07:52 <planetmaker> I hoped you were joking :D
10:08:56 <andythenorth> joke multithreading not do about I
10:19:18 <peter1138> "I didn't test my PR."
10:20:12 <peter1138> Is that how other developers do things?
10:20:48 <andythenorth> on certain days I think OpenTTD is well above the average on quality
10:35:39 *** supermop_work_ has quit IRC
10:40:49 *** supermop_work_ has joined #openttd
11:11:01 *** supermop_work_ has quit IRC
11:19:28 *** supermop_work_ has joined #openttd
11:49:42 *** supermop_work_ has quit IRC
11:58:40 *** HerzogDeXtEr has joined #openttd
12:02:01 *** supermop_work_ has joined #openttd
12:32:11 *** supermop_work_ has quit IRC
12:33:38 <nielsm> that would make diagnostics easier
12:34:40 *** supermop_work_ has joined #openttd
12:53:23 <Samu> but it's last gen :( and next year, new stuff is coming out
12:57:17 <Samu> with about €370 i can upgrade cpu, mb, ram and still add a 1 TB SSD
13:04:53 *** supermop_work_ has quit IRC
13:05:20 *** supermop_work_ has joined #openttd
13:35:33 *** supermop_work_ has quit IRC
13:39:32 *** andythenorth has joined #openttd
13:41:13 *** supermop_work_ has joined #openttd
13:49:25 <andythenorth> so why am I tickling "[Errno 24] Too many open files"
13:51:37 <andythenorth> ulimit reports 'unlimited'
13:51:37 <Wolf01> Close a file handler when you finish to use it?
13:51:56 <andythenorth> yes, I wonder where it is though
13:52:12 <andythenorth> and why this triggers with pypy but not python3
13:53:05 <Wolf01> I think that "unlimited" is like "free! (with a 4.99€ purchase)" or "you get unlimited call time (up to 200 minutes/month)"
13:53:49 <Wolf01> Maybe it depends on other things, like OS, ram, whatever
13:54:24 <Wolf01> "Too many open files" errors are always tricky – you not only have to twiddle with ulimit, but you also have to check system-wide limits and OSX-specifics"
13:55:26 <andythenorth> ulimit -n 4096 solves it
13:55:40 <andythenorth> but I wonder if I should be closing files
13:58:07 <andythenorth> well I wonder where, to be more accurate :P
14:00:15 <Wolf01> Just after you finished to write/read it
14:03:42 *** andythenorth has joined #openttd
14:11:27 *** supermop_work_ has quit IRC
14:13:19 *** supermop_work_ has joined #openttd
14:21:19 <supermop_Home> can an industry distribute cargo to only 2 stations, or is it 2 station per cargo?
14:23:28 <nielsm> when a cargo packet is about to be distributed it searches for all stations in catchment and sorts them by rating for that cargo, then picks the top two
14:24:25 <supermop_Home> with cdist i usually keep two stations, one for far destination and one for near
14:25:04 <supermop_Home> but in this case i think i want separate trains taking the vehicles vs the engineering supplies
14:42:20 * andythenorth closes some file handles :P
14:43:30 *** supermop_work_ has quit IRC
14:44:43 <andythenorth> closing file handles seems to have made the compile faster :P
14:45:30 <andythenorth> not sure why that would be
14:46:12 <andythenorth> plenty of RAM :P
14:48:09 *** supermop_work_ has joined #openttd
14:50:00 *** HerzogDeXtEr has joined #openttd
15:00:52 <andythenorth> the py3.8 syntax errors were me not nml
15:00:57 <andythenorth> but the file handles are nml :P
15:04:54 <supermop_Home> hmm this GS wants 'raw energy' once a town is bigger than 10k
15:05:19 <supermop_Home> but in steeltown that only means coal, which means a coke oven
15:06:09 <supermop_Home> and the only coal mine, oven, and steelmill are conveniently all near eachother in the far corner of the map
15:06:43 <supermop_Home> so setting up a new oven in the middle of my big city makes no sense
15:06:58 <supermop_Home> think globally act locally
15:08:26 <supermop_Home> nothing makes me want to move to a new city like huge trains of coal being hauled in to be baked in town, just to be carried back out to where they came from
15:08:44 <andythenorth> I am trying Migrations GS
15:08:57 <andythenorth> when you serve an industry, it spams a new town and houses round it
15:09:19 <andythenorth> neat concept, probably not going to be a keeper for me :)
15:10:26 <supermop_Home> i accidentally started this steeltown game in tropic, so now im setting up complex systems to serve vehicles to all towns to let them grow
15:10:45 <supermop_Home> because all but one town spawned in the desert
15:18:22 *** supermop_work_ has quit IRC
15:23:31 *** supermop_work_ has joined #openttd
15:41:30 * andythenorth wonders what happens to an open Image file when copy is called on it :P
15:41:34 <andythenorth> do I also need to close the copy?
15:53:42 *** supermop_work_ has quit IRC
15:55:32 *** supermop_work_ has joined #openttd
15:59:25 *** Progman has joined #openttd
16:02:42 *** WormnestAndroid has quit IRC
16:02:55 *** WormnestAndroid has joined #openttd
16:09:34 <andythenorth> hmmm when things work...
16:25:45 *** supermop_work_ has quit IRC
16:27:37 *** supermop_work_ has joined #openttd
16:31:13 *** snail_UES_ has joined #openttd
16:39:29 <andythenorth> Iron Horse compile is reliably ~10 seconds faster with python 3.8 compared to a week ago with python 3.5
16:39:56 <andythenorth> BUT some of the gain is due to me removing broken/slow things that worked under 3.5 and don't in 3.8 :P
16:41:11 <DorpsGek> andythenorth: 0.808823529412
16:57:50 *** supermop_work_ has quit IRC
16:59:43 *** supermop_work_ has joined #openttd
17:25:58 <Samu> it's 1958 in the debug test, 3 years away from the crash
17:29:11 <FLHerne> andythenorth: PyPy doesn't use reference-counted GC
17:29:53 <FLHerne> So file handles are only closed whenever the GC feels like it, whereas in CPython they're usually closed when leaving their scope
17:29:54 *** supermop_work_ has quit IRC
17:30:23 <FLHerne> (or, of course, when actually closing them like the spec says to :P)
17:30:31 <andythenorth> that makes sense
17:30:42 <andythenorth> nmlc might want to Image.close()
17:30:45 <andythenorth> I didn't look where
17:31:08 <andythenorth> it's possibly a cause of reported RAM bloat as well, but dunno
17:31:18 <andythenorth> not sure the 2 are connected?
17:34:12 *** supermop_work_ has joined #openttd
17:36:05 <Eddi|zuHause> i wouldn't count "GC can free it" under "bloat"
17:36:43 <Eddi|zuHause> however, "GC can't figure out that it can be freed" is different
17:38:08 *** Wormnest_ has joined #openttd
17:43:46 <andythenorth> I stopped testing with pypy, it's faster for some cases, but slower than others
17:49:00 <andythenorth> hmm overall faster with pypy though
17:57:31 <andythenorth> if I could switch py38 and pypy I could save another 8 seconds :P
17:57:58 <andythenorth> but 'source' to swtich virtualenv doesn't seem to work from a makefile
17:59:39 <Eddi|zuHause> i've no clue what you're saying
18:00:31 *** lastmikoi_ has joined #openttd
18:01:31 *** lastmikoi_ is now known as lastmikoi
18:02:20 <andythenorth> some python interpreters are faster than others
18:02:29 <andythenorth> which one is fastest depends on the code being run
18:02:58 <andythenorth> the compile is made up of discrete steps
18:03:14 <andythenorth> I could switch to the fastest interpreter for each step
18:03:26 <andythenorth> but it's unwise, and I can't get it to trivially work anyway
18:04:11 <snail_UES_> andythenorth: I just read your past messages… just curious, how long does it take to compile IH?
18:04:26 *** supermop_work_ has quit IRC
18:04:31 <andythenorth> roughly 1 minute
18:05:00 <snail_UES_> right… my set with m4nfo is much quicker
18:05:25 <Eddi|zuHause> CETS spends 1 minute in grfcodec alone
18:06:18 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh updated pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/fhho4
18:06:40 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/Je1QY
18:07:00 <andythenorth> I did consider switching to m4nfo
18:07:00 <Eddi|zuHause> it's about 300k lines of NFO
18:07:23 <Eddi|zuHause> and grfcodec cannot be parallelized
18:07:54 <Eddi|zuHause> so while it's only 20% of the work, it's 50% of the time
18:08:05 <snail_UES_> mine just compiled in 28 seconds, including something like $1A5F recoloring tables
18:08:11 <snail_UES_> without those, it’d take like 10 secs
18:08:39 *** supermop_work_ has joined #openttd
18:08:40 <andythenorth> how long does it take you to generate the m4nfo?
18:09:07 <snail_UES_> you mean, within those 28 seconds?
18:09:55 <andythenorth> no, I mean to get the input file?
18:10:08 <andythenorth> or you wrote it all by hand?
18:10:39 <snail_UES_> I wrote my code by hand… well, with lots of copy and paste :p
18:10:50 <snail_UES_> but all of the logic is done by hand
18:11:33 <DorpsGek> andythenorth: 0.690909090909
18:11:54 <andythenorth> ok so about 30% of the Horse compile is code generation, graphics generation, docs generation, overhead
18:12:01 <Eddi|zuHause> andythenorth: my generate.py takes 6 seconds
18:12:37 <andythenorth> you should have parellelised it :P
18:12:50 <andythenorth> use 16 processes, it would be done in 5 seconds
18:13:10 <snail_UES_> you mean you don’t write your nml code by hand? then how do you do it?
18:13:15 <LordAro> andythenorth: what bit of it is being parallelised?
18:13:55 <andythenorth> snail_UES_: I don't even know how to write nml :)
18:13:59 <andythenorth> I have to look it up every time
18:14:06 <Eddi|zuHause> andythenorth: how would you parallelize reading a giant table? from a single file? :p
18:14:08 <andythenorth> LordAro: just graphics generation
18:14:23 <andythenorth> Eddi|zuHause: divide it into 16 chunks :P
18:14:35 <andythenorth> the overhead will kill you :P
18:14:53 <snail_UES_> andythenorth: so how are you coding your set? :p
18:15:14 <LordAro> personally i'd like make deal with parallelising for me
18:15:32 <snail_UES_> you write in python, then python generates nml, and then from nml you compile the newgrf?
18:15:44 <snail_UES_> alright… sounds a bit convoluted to me :)
18:16:08 <andythenorth> I really don't like nml
18:16:21 <andythenorth> if I could template nfo, I would, but I'm not smart enough
18:16:41 <andythenorth> the nml is actually compiled to nfo
18:16:45 <andythenorth> then grfcodec is used
18:17:00 <snail_UES_> python —> nml —> nfo —> newgrf? :D
18:17:12 <Eddi|zuHause> snail_UES_: it probably is convoluted in andy's case, but if you have any kind of theoretical background it's pretty clean and straightforward to generate code.
18:17:39 <Eddi|zuHause> snail_UES_: basically, you write one vehicle, and it does all the copy-paste for you
18:18:02 <snail_UES_> yes, but each vehicle has its own quirks...
18:18:10 <andythenorth> that's why it gets convoluted
18:18:19 <andythenorth> and that's why Iron Horse vehicles are all pretty much the same
18:18:20 <snail_UES_> it’s rare that 2 vehicles are coded exactly the same in any trainset
18:20:07 <snail_UES_> right… well, if your trains are coded the same, it would be easy to create templates in nfo
18:20:21 <Eddi|zuHause> snail_UES_: there are ways out of that dilemma. like you can extend the system to handle all these tiny differences, or you can restrucure them to minimize the differences
18:20:50 <snail_UES_> Eddi|zuHause: yeah, like bringing some degree of standardization
18:20:58 <snail_UES_> I will totally do it for my next sets...
18:21:09 <snail_UES_> the narrow gauge one was my first, so I had crazy ideas
18:21:16 <andythenorth> I did that for HEQS :P
18:21:31 <andythenorth> Iron Horse is probably my 8th or 9th attempt at templating
18:21:56 *** gelignite has joined #openttd
18:22:02 <snail_UES_> templating helps, but then the result might lack variety...
18:22:17 <Eddi|zuHause> i found templates to be too restrictive
18:22:48 <Eddi|zuHause> that's where the code generation comes in, you have more ways to fit in the variations
18:22:53 <snail_UES_> you need to find a compromise between standardization and diversity
18:23:11 <andythenorth> there's a gameplay aspect as well
18:23:34 <Eddi|zuHause> code generation is essentially just fancy templates
18:23:35 <andythenorth> too much variation is bad for e.g. AIs, coop etc
18:23:44 <andythenorth> not enough variation is dull as hell
18:24:02 <snail_UES_> what do you mean by "coop"?
18:24:10 <andythenorth> coop style gameplay
18:24:47 <andythenorth> they like all trains to have same performance
18:25:08 <snail_UES_> all trains to have the same performance??
18:25:14 <snail_UES_> then just buy the same model :p
18:25:52 <andythenorth> it it tonne-miles?
18:26:56 * andythenorth wonders whether to try and make Iron Horse compile faster
18:27:07 <LordAro> much vs many is the same as less vs fewer
18:27:44 <Eddi|zuHause> LordAro: the problem is, if your native language uses the same word for both cases
18:28:16 <LordAro> Eddi|zuHause: true :)
18:28:18 <Eddi|zuHause> ... then you might lack the ability to distinguish these cases
18:28:36 <andythenorth> nmlc is so much faster under pypy
18:28:47 <andythenorth> maybe I do need to switch interpreter in the makefile
18:28:58 <LordAro> uncountable vs countable - how much water, there is less water. how many people, there are fewer people
18:30:04 <Eddi|zuHause> LordAro: that may seem logical to you...
18:30:10 <snail_UES_> LordAro: this might look a bit complicated to anyone who wasn’t born English ;)
18:30:35 <snail_UES_> even lots of native-born Americans have an incredible trouble with that
18:30:46 <LordAro> quite a lot of British people too :p
18:30:48 <andythenorth> lego or legos :P
18:30:48 <Eddi|zuHause> LordAro: it's like a japanese person confusing "r" and "l", or a french person randomly sprinking "h" into a word
18:31:19 <snail_UES_> what pisses me off is native-born speakers who confuse “it’s” with “its”
18:31:22 <snail_UES_> you see that all the time...
18:31:29 <andythenorth> I do that all the time :)
18:31:33 <andythenorth> it's really hard to not get wrong
18:31:57 <LordAro> "[wsc]ould of" gets me
18:31:58 <nielsm> and classic their/they're/there
18:32:05 <Eddi|zuHause> that's the reverse case, because in most foreign languages there is no way to confuse these two things
18:32:12 <LordAro> "off of" particularly gets me
18:32:15 <andythenorth> I know all the rules, but I still make the mistakes
18:32:17 <Eddi|zuHause> so non-native speakers are less likely to make that mistake
18:32:19 <andythenorth> the language is stupid
18:32:59 <Eddi|zuHause> what i never could wrap my head around was peopple using "of" instead of "have"... what?? why?
18:33:10 <andythenorth> nmlc run time Iron Horse: py38 39s; pypy 20s
18:33:16 <andythenorth> ^ substantial saving
18:33:53 <nielsm> heh, Aircraft::GetCurrentMaxSpeed() is not used anywhere at all
18:34:10 <andythenorth> so how do I change virtualenv in a makefile :P
18:34:11 <nielsm> and the point of GetSpeedOldUnits() is not explained anywhere as far as I can tell
18:34:29 <LordAro> andythenorth: you really don't
18:34:58 <LordAro> Eddi|zuHause: probably due to contractions - "should've" -> "should of"
18:35:49 <andythenorth> LordAro: the speed though :P
18:36:30 <LordAro> andythenorth: sure, but switching dev environments from within a makefile? ew ew ew
18:36:40 <Eddi|zuHause> andythenorth: excuse my ignorance, but the point of a virtualenv is that the program being run doesn't know about it?
18:36:47 <andythenorth> it's just an interpreter LordAro
18:36:58 <andythenorth> I used to have a mixed python3 / python2.7 compile :P
18:36:59 <LordAro> i'm guessing nml itself can't cope with pypy?
18:37:14 <andythenorth> Eddi|zuHause: somewhat yes
18:37:38 <LordAro> then call pypy as "pypy" ? why is the virtualenv a factor?
18:38:15 <nielsm> aircraft movement speed is deep magic
18:38:15 <andythenorth> I don't use system-wide python
18:38:21 <andythenorth> it's very fragile
18:38:28 <LordAro> sure, but your build system doesn't need to know that
18:38:36 <andythenorth> Makefile.local maybe
18:38:39 <LordAro> virtualenv sets up your environment
18:38:40 <Eddi|zuHause> yeah, i don't understand what the virtualenv has to do with it. you need a virtualenv with both interpreters in it
18:38:47 <nielsm> the virtualenv should have the python version used encoded in it
18:38:47 <LordAro> make runs in that environment
18:38:52 *** supermop_work_ has quit IRC
18:39:00 <Eddi|zuHause> and then just call the interpreter in the makefile
18:39:10 <nielsm> by having an appropriate symlink from the venv's bin directory to the intended interpreter
18:39:19 <andythenorth> what nielsm said
18:39:45 <Eddi|zuHause> one virtualenv, two interpreters.
18:39:50 <andythenorth> that's not how it's done
18:40:01 <Eddi|zuHause> but that's the only way to do it
18:40:03 <andythenorth> that's just a particularly inventive way to break things
18:40:26 <nielsm> this is one good line: v->subspeed = (t = v->subspeed) + (byte)spd;
18:40:29 *** supermop_work_ has joined #openttd
18:40:41 <andythenorth> actually one virtualenv, two interpreters might be the best way to break things so far :)
18:40:52 <andythenorth> it would take ages to figure it out :)
18:41:20 <Eddi|zuHause> nielsm: if i were to work in QA, i'd immeddiately ban = in expressions
18:41:40 <Eddi|zuHause> nielsm: so many ways for it to go wrong
18:42:02 <LordAro> there are certain cases where it's acceptable
18:42:08 <Eddi|zuHause> and "t = subspeed; subspeed += spd" isn't even any longer
18:42:08 <LordAro> but those are vanishingly rare
18:42:36 <LordAro> e.g. while((foo = read())) {...}
18:42:51 <Eddi|zuHause> LordAro: iterator.
18:43:16 <Eddi|zuHause> LordAro: that's a for-loop
18:43:19 <nielsm> static void MaybeCrashAirplane(Aircraft *v);
18:45:06 <Eddi|zuHause> nielsm: there's a good chance you can trace these kinds of expressions back to the original decompile
18:45:12 <glx> andythenorth: you can have many venv, but you can have only one enabled per session
18:45:38 <andythenorth> one has to switch
18:45:49 <glx> that's the main use of venv
18:46:00 <andythenorth> doing that in the Makefile might be considered unwise :P
18:46:22 <glx> Makefile doesn't have to know about venv
18:46:32 <glx> it's the responsability of the user
18:46:54 <andythenorth> rock, hard place
18:47:03 <andythenorth> that game-overs switching interpreter
18:47:06 <Eddi|zuHause> Makefile MUST NOT know about virtualenv
18:47:41 <andythenorth> we have a work Makefile that actually creates virtualenvs
18:47:52 <Eddi|zuHause> that's different
18:47:53 <andythenorth> but it's for creating a build environment, not for the compile
18:48:11 <Eddi|zuHause> that makefile lives on a meta-environment
18:48:24 <glx> projects themself don't care
18:48:35 <nielsm> @todo De-mystify the cur_speed values for helicopter rotors. <- yeah that too
18:49:06 <andythenorth> well I could just install all the deps in system-wide pypy
18:49:19 <andythenorth> but I hate system-wide python installs, they break at no-notice
18:49:41 <glx> or use venv from the global install
18:50:25 <glx> on windows I can't install modules system wide if not in an admin shell
18:50:33 <Eddi|zuHause> i stand by my opinion that the only way to do what you want is to have both interpreters in the same virtualenv
18:50:45 <glx> but pip supports that easily
18:51:01 <andythenorth> Eddi|zuHause: that might work because pypy is tracking py3
18:51:06 <andythenorth> but it's very prone to breaking
18:51:26 <andythenorth> which package should be installed in the env for which interpreter?
18:51:38 <andythenorth> it defeats the point of the env
18:52:06 <andythenorth> are there interpreter-specific packages?
18:52:13 <glx> venv/bin is put in front of the path
18:52:24 <andythenorth> I'm confused now
18:52:59 <glx> so if you use python2 -m pip you install for python2, and python3 -m pip to install python3
18:53:54 <andythenorth> well I could try it
18:54:03 <andythenorth> I can always just delete the env if it breaks
18:54:36 <glx> venv/bin/pip on the other hand will use the latest interpreter added to the venv
18:55:33 * andythenorth looks what's in bin/activate
18:55:48 <glx> bin/activate just adds the path IIRC
18:56:41 <andythenorth> so with two interpreters, I'm curious how it will know which one to activate
18:58:24 <glx> anyway it's probably easier to just do python2 -m venv venv2 and python3 -m venv venv3
18:59:01 <andythenorth> that results in 2 venvs though
18:59:13 <glx> but only one used at a time
18:59:50 <andythenorth> the only solution seems to be system-wide install
19:02:32 <Samu> {type=VKI_STATION (0) id={station=1746 town=1746 sign=1746 } center=-48128 ...}
19:02:36 <andythenorth> is there a convention for naming pypy?
19:02:49 <Samu> the id being equal to station, town and sign, is that the way it works?
19:03:18 <andythenorth> probably the makefile needs to test if pypy exists?
19:06:40 <DorpsGek_III> [OpenTTD/OpenTTD] SamuXarick commented on issue #7847: Crash - Assertion failed at line 213 of src\core\kdtree.hpp: next != INVALID_NODE https://git.io/Je1Eh
19:07:45 <nielsm> Samu: the id there is a union, not a struct, so it's only the id for the appropriate item type
19:08:19 <nielsm> (it's not a StationID, TownID, SignID all at the same time, only the one type given by VKI_STATION/etc.)
19:08:24 <Samu> btw, it also crashed on master version
19:08:48 <glx> of course, 1.10 is master ;)
19:09:24 <glx> not the more recent master, but still master
19:09:58 <glx> and no changes happened in this area since beta release
19:10:47 *** supermop_work_ has quit IRC
19:10:57 <glx> still no fast way to reproduce ?
19:11:12 <andythenorth> hmm maybe I can rewrite to be faster for pypy
19:11:14 <Samu> _viewport_sign_kdtree.Remove(ViewportSignKdtreeItem::MakeStation(this->index));
19:11:19 <andythenorth> seems that it needs code written for it
19:11:34 <andythenorth> or I could maybe switch template language
19:11:38 <Samu> it enters this part, and tries to find something, and reaches invalid_node
19:11:47 <glx> Samu: yes I know where the crash is, I posted the log in the issue ;)
19:12:27 <glx> at some point it takes the wrong branch of the tree I think
19:12:38 *** supermop_work_ has joined #openttd
19:13:05 <glx> now the question is why ?
19:13:48 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh updated pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/fhho4
19:14:19 <Samu> is there anything I can do to brute force a search?
19:15:34 <Samu> or maybe a failsafe when it fails to find, have it make some slower search
19:15:37 <andythenorth> py35: 59s; py38: 47s; pypy: 31s
19:15:56 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/Je17r
19:16:11 <glx> FIRS (at least the one on github) needs a big update for current nmlc
19:16:24 <andythenorth> there's a branch
19:16:29 <andythenorth> v4-development or something
19:16:43 <andythenorth> main reason I switched it to git :)
19:16:53 <glx> luckily as you did it only one file needs changes ;)
19:17:17 <andythenorth> has something else changed? :P
19:20:32 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh updated pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/fhho4
19:22:03 <glx> ha yes v4-development-track compiles
19:22:16 <glx> I forgot to check branches last night
19:22:35 <andythenorth> I wonder if FIRS is suitable for stored procedures :P
19:23:09 <andythenorth> there are some truly horrible things for drawing the correct ground tile: snow, slope etc
19:23:15 <andythenorth> they are endlessly repetitive
19:23:18 <glx> it is if switches are based on the same calculations
19:23:42 <andythenorth> I am hoping that stored procedures might work for Iron Horse
19:23:58 <andythenorth> optimistically, I might remove 20k lines of 170k total
19:25:14 <andythenorth> I don't know if childsprite can use a stored procedure result
19:27:22 <andythenorth> that would be a trip deep into nml :P
19:27:43 <andythenorth> there's also an advanced format using registers :P
19:32:35 <glx> I think childsprite can use a stored procedure result
19:32:59 <glx> I guesse the stuff is converted to a varact2 by nmlc
19:33:34 <andythenorth> maybe nml could consolidate them for me :P
19:34:06 <glx> it should be possible to use [STORE_TEMP(), procedure()]
19:34:54 <glx> and in the procedure you use LOAD_TEMP() to get the parameter
19:36:18 * andythenorth wonders how compilers optimise
19:37:15 <andythenorth> this expression is all keywords
19:37:20 <andythenorth> (climate != CLIMATE_TROPIC) || ((climate == CLIMATE_TROPIC) && (nearby_tile_terrain_type(0, 0) == TILETYPE_DESERT)) || ((climate == CLIMATE_TROPIC) && (nearby_tile_terrain_type(0, 0) == TILETYPE_NORMAL) && ((nearby_tile_terrain_type( 1, 0) != TILETYPE_DESERT) && (nearby_tile_terrain_type(-1, 0) != TILETYPE_DESERT) && (nearby_tile_terrain_type( 0, 1) != TILETYPE_DESERT) && (nearby_tile_terrain_type( 0,-1) !=
19:37:20 <andythenorth> TILETYPE_DESERT) ) )
19:37:26 <glx> and we could probably add something like function name(args..) { return <calculation>; } to let nmlc generate the varact2
19:37:31 <andythenorth> so presumably all instances of it could be resolved to one instance
19:37:36 <andythenorth> automatically :P
19:37:56 <andythenorth> do compilers construct tables of recurring patterns?
19:38:14 <glx> I don't think nmlc does :)
19:40:40 <Samu> + [2595] {element={type=VKI_STATION (0) id={station=1746 town=1746 sign=1746 } center=-48128 ...} left=2597 right=...} Kdtree<ViewportSignKdtreeItem,int (__cdecl*)(ViewportSignKdtreeItem const &,int),int,int>::node
19:40:49 <Samu> it's in the list of nodes
19:40:55 <Samu> it just fails to find it
19:41:24 <nielsm> does the cached position match the expected?
19:41:40 <nielsm> (cached position = the position stored in the tree)
19:42:16 <Samu> not sure how to check, but let me try
19:42:50 *** supermop_work_ has quit IRC
19:44:29 *** supermop_work_ has joined #openttd
19:44:53 <Samu> nop, doesn't seem to match
19:45:07 <Samu> sec, i take another screenshot
19:45:18 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/Je15f
19:46:52 <Samu> the top for one is 43872, the top for the other is 43904
19:46:59 <Samu> they should have the same value, right?
19:47:09 <glx> I think I know what happened
19:47:35 <glx> dock removed, land raised, sign moved
19:48:06 <nielsm> so changing land height below a ghost station sign can trigger bugs
19:48:53 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh updated pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/fhho4
19:48:59 <glx> let see if I can reproduce using these simple steps
19:49:41 <andythenorth> maybe I can speed up PIL :P
19:50:06 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on pull request #7353: Feature: Measure vehicle capacity utilisation efficiency https://git.io/Je15T
19:50:40 <glx> it's annoying to not be able the batch replies to reviews ;)
20:12:41 <nielsm> glx: you can if you go to the Files Changed tab first
20:14:43 *** supermop_work_ has quit IRC
20:20:06 *** supermop_work_ has joined #openttd
20:44:21 <Samu> i can't reproduce the crash glx
20:44:39 <Samu> raised land, lowered land, nothing seems to make it crash
20:50:19 *** supermop_work_ has quit IRC
20:51:31 *** supermop_work_ has joined #openttd
21:04:25 <Samu> strange, when I load an autosave from a month earlier
21:04:46 <Samu> i get top = 43904, and it's now node 171, instead of 2595
21:05:07 <nielsm> is the station already demolished at that time?
21:05:22 <glx> node changes when something is added removed, ignore the nod ;)
21:05:29 <nielsm> the tree is recalculated when the save is loaded
21:05:51 <Samu> so, gonna try load when the station is not yet demolished
21:06:06 <nielsm> so if you save and reload while the tree is in an inconsistent state, the state will be rebuilt to be consistent
21:06:13 <glx> the main issue is to find why the sign "moved"
21:06:49 <glx> or how we can move it without updating the tree
21:10:59 <Samu> top = 43872 with the station yet built
21:11:42 <Samu> if i delete the station, will i get the crash now?
21:11:43 <nielsm> how does the map look with the station in place?
21:11:59 <nielsm> compared to how it looks after it was deleted
21:12:10 <nielsm> sorry, demolished not deleted
21:13:42 <nielsm> so the AI demolishes the dock and also lower the land piece it sits on
21:13:59 <Samu> yes, but he's not demolishing now :(
21:14:18 <Samu> ais after load tend to do different than a straight run
21:14:41 <nielsm> yes they use the interactive random, not a game state random
21:21:42 *** supermop_work_ has quit IRC
21:24:56 *** supermop_work_ has joined #openttd
21:26:20 <Samu> yay, i think i found a more reproducible way to crash it
21:26:41 <Samu> gonna try again to confirm
21:31:55 <DorpsGek_III> [OpenTTD/OpenTTD] SamuXarick commented on issue #7847: Crash - Assertion failed at line 213 of src\core\kdtree.hpp: next != INVALID_NODE https://git.io/Je1Eh
21:34:23 <Samu> in autosave11 the top value is different than that in autosave12, because the dock still exists in 11, but not in 12
21:34:57 <Samu> if i load the autosave12 it won't crash
21:37:08 <nielsm> I wonder what the trigger for the sign moving it though
21:37:43 <Samu> if you don't lower the land after demolishing the docks, autosave11 won't crash either
21:37:50 <Samu> so it's something related to lowering land
21:38:07 * andythenorth wonders if grfcodec could be faster
21:38:08 <nielsm> normally ghost station signs don't move after modifying the land below them
21:38:28 <glx> the sign moves when the dock is removed but doesn't cause crash
21:39:37 <glx> and it moves because content changes
21:41:40 <glx> waiting for the sign to disappear at 7fps takes time :)
21:41:57 <nielsm> but I reproduce it too with that save
21:42:11 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on issue #7847: Crash - Assertion failed at line 213 of src\core\kdtree.hpp: next != INVALID_NODE https://git.io/Je1Eh
21:43:19 *** snail_UES_ has joined #openttd
21:48:14 <andythenorth> I need to save 5s on generating the nml
21:49:20 <andythenorth> the overhead of starting the template engine is 1s
21:49:37 <andythenorth> so I don't think parallelising will help :(
21:55:09 *** supermop_work_ has quit IRC
21:57:33 <nielsm> okay the bug seems to occur already in demolishing the station
21:57:39 <nielsm> lowering the land is not required
21:58:44 *** supermop_work_ has joined #openttd
22:04:17 <glx> I tried building something similar but that doesn't trigger
22:04:32 <nielsm> yeah it depends on the rest of the map and signs
22:05:03 <nielsm> it has to be sufficiently large and positioned "correctly" so the station sign that moves ends up in a different position in the tree
22:05:24 <glx> ah yes it's the only station I have when I try
22:05:33 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on issue #7847: Crash - Assertion failed at line 213 of src\core\kdtree.hpp: next != INVALID_NODE https://git.io/Je1Eh
22:09:53 <DorpsGek_III> [OpenTTD/OpenTTD] nielsmh commented on issue #7847: Crash - Assertion failed at line 213 of src\core\kdtree.hpp: next != INVALID_NODE https://git.io/Je1Eh
22:11:20 <nielsm> if you want to trace it, breakpoint in ClearTile_Station() and follow it into RemoveDock()
22:12:12 <nielsm> and stuff happens with the st->rect.AfterRemoveTile() until st->AfterStationTileSetChange() calls
22:15:11 <nielsm> the reason the sign moves is not because it gets moved, or the tile height changes, but because the way the tile z coordinate is calculated changes when the tile changes from a station tile to a clear tile
22:26:56 <andythenorth> oof can't commit this to a public repo eh :D
22:26:58 <andythenorth> ../../pypy3/bin/nmlc -l generated/lang --verbosity=4 --nfo=generated/iron-horse.nfo generated/iron-horse.nml
22:27:42 <glx> hardcoded path to an exe ?
22:28:58 *** supermop_work_ has quit IRC
22:30:48 *** supermop_work_ has joined #openttd
22:36:53 <andythenorth> Makefile.local and gitignore it? :P
22:43:56 <andythenorth> that seems to work fine
22:44:20 <DorpsGek> andythenorth: 0.573529411765
22:44:28 <andythenorth> not a bad speed up
22:56:04 <Samu> Point pt = RemapCoords2(TileX(this->xy) * TILE_SIZE, TileY(this->xy) * TILE_SIZE);
22:56:16 <Samu> Point pt = RemapCoords(TileX(st->xy) * TILE_SIZE, TileY(st->xy) * TILE_SIZE, GetTileMaxZ(st->xy) * TILE_HEIGHT);
22:56:27 <Samu> could this be the issue?
22:56:56 <nielsm> I'm fixing that too, but it isn't
22:57:00 <nielsm> I know exactly what's going wrong
22:57:04 <nielsm> and it's annoying to fix
22:58:01 <glx> I think it could be done in ViewportSign::UpdatePosition()
22:58:10 <supermop_Home> andythenorth: why do sliding wagons cost more than curtain sided?
22:59:54 <glx> but it first needs to check it's already in the tree before maybe remove
23:01:01 *** supermop_work_ has quit IRC
23:03:38 <andythenorth> supermop_Home: not sure :D cost in code is the same for both
23:05:07 <supermop_Home> i mean at this point i can afford the sliding side wagon, running cost is the same
23:05:26 <supermop_Home> i figure it might look better for shipping vehicles
23:06:33 *** supermop_work_ has joined #openttd
23:08:09 <supermop_Home> its like $2027 vs $1851 for the large wagon
23:10:33 <andythenorth> mine all cost same :)
23:11:09 <andythenorth> oh the tarp wagon is cheaper
23:12:15 <supermop_Home> well the metal sided looks more like a car carrier
23:12:31 <supermop_Home> so im paying the premium for it
23:13:58 <andythenorth> I should finish the vehicle wagons :P
23:14:39 <nielsm> glx no doesn't fix it, I suspected that
23:15:06 <supermop_Home> if im dropping a town effect cargo off on the outskirts of town (at a black hole industry) does the TE go to the town with the station name, or closest by mht distance?
23:15:13 <nielsm> thing is the expected position of the sign changes behind the station code when the tile is modified after it stops being a station tile
23:16:07 <nielsm> really ViewportSignKdtreeItem::MakeStation() ought to use BaseStation::sign.top and .center for the position
23:16:18 <nielsm> but those aren't always valid when it gets called first, I think
23:18:41 <glx> ah yes ViewportSign::UpdatePosition() can't know the object containing it
23:21:06 <nielsm> I think I need to make a StationViewportSign inheriting ViewportSign and keeping track of its kdtree state
23:35:57 <supermop_Home> how many tiles away until luxury passenger car makes sense?
23:37:15 *** supermop_work_ has quit IRC
23:38:28 <nielsm> depends on train speed too I think
23:39:06 <nielsm> since the effect of luxury passenger cars is to make the trip "feel shorter"
23:39:39 <nielsm> if it's sufficiently long (too slow) then you'll hit the max cap I think exists
23:39:51 <supermop_Home> 185 kph… still haven't figured out all the livery easter eggs in IH
23:41:27 <andythenorth> in tests it's about 64 tiles or so
23:42:05 <andythenorth> the luxury pax cars actually do ~nothing
23:42:16 <andythenorth> the standard pax cars have to have a nerf
23:42:44 <andythenorth> the decay rate modifier is mostly useless
23:43:15 <supermop_Home> these are just 4 tile push-pulls with a fury on the back
23:43:24 <supermop_Home> and mail emu up front
23:46:39 *** supermop_work_ has joined #openttd
continue to next day ⏵
