IRC logs for #openttd on OFTC at 2020-10-13
⏴ go to previous day
08:04:07 *** sla_ro|master has joined #openttd
08:54:40 *** andythenorth has joined #openttd
09:19:12 *** andythenorth has joined #openttd
09:33:27 *** longtomjr_ has joined #openttd
09:40:21 *** andythenorth has joined #openttd
09:53:04 <TrueBrain> awh, frosch123 did not include his export script :P
09:56:32 <TrueBrain> holy crap, gollum consumes 1GB of RAM
09:59:06 <TrueBrain> seems the caching is a bit overly eager :)
11:01:41 <TrueBrain> holy crap, gollum is slow
11:03:23 <LordAro> i feel like you've been saying that from the start
11:03:38 <TrueBrain> pretty sure I haven't judge gollum on speed
11:03:44 <TrueBrain> are you confusing me with frosch?
11:03:53 <TrueBrain> as that would be ... well .. I will judge you for that :)
11:04:13 <LordAro> i don't keep track :p
11:04:39 <TrueBrain> ... you do not keep track .....
11:04:39 <andythenorth> everyone here is just the same bot codebase, with a different nick configured, right?
11:04:43 <TrueBrain> different people is SO HARD, I know right :P
11:04:49 * andythenorth assumes everyone else is just software
11:06:41 <TrueBrain> ah, this is a bug in gollum / wikicloth it seems :)
11:06:49 <TrueBrain> it is escaping the escape of the escape
11:06:52 <TrueBrain> pretty funny tbh :)
11:07:13 <TrueBrain> pretty sure this is the reason this page took 5+ seconds to render :P
11:07:42 <TrueBrain> lets see how big this page is :D
11:08:23 <TrueBrain> it should be ~20 kB :)
11:08:23 <LordAro> my internet is terrible enough right now that i probably wouldn't notice
11:08:38 <TrueBrain> it made it ... 70 times bigger with all the amping up :)
11:09:42 <TrueBrain> on other pages it is equal in speed to mediawiki itself
11:09:52 <TrueBrain> you would think Ruby would be faster than PHP, but ala
11:13:39 <TrueBrain> what is odd, that gollum keeps and keeps increasing in memory, which I assumed was a cache, but pages are as quick to load the first time as the second ..
11:13:48 <TrueBrain> so what is it doing :P
11:14:21 <LordAro> (void)malloc(something_big);
11:14:46 * LordAro is not being helpful today
11:15:19 <TrueBrain> anyway, I have gollum now rendering pages < 300ms for the wiki itself, with the exception of a few weird pages :)
11:15:25 <TrueBrain> the gollum "overview" pages how-ever ...
11:15:54 <TrueBrain> making a listing for 1300 pages takes 3 seconds, and is 1.2MB :P
11:16:05 <TrueBrain> pretty sure those pages are not useful ;)
11:22:46 <TrueBrain> okay, the memory increase does have to do with some form of cache .. it does not happen if I reload pages :)
11:26:37 <TrueBrain> and basically, out mediawiki content is such a collection of .. "attempts", that structure-wise it is one big mess .. not meaning content-wise, but how things are defined, what syntax is used,etc
11:33:57 <TrueBrain> LOL! Turns out gollum was running in development mode ...
11:34:05 <TrueBrain> which was the reason for booming in memory
11:44:42 <TrueBrain> what is odd in the gollum implementation, there seems to be no cache-keys / etags / etc
12:01:45 <TrueBrain> okay, enough toying around with this .. speeded up browsing the wiki via gollum with a factor of 10 by adding nginx .. works-for-me :P
12:05:31 <TrueBrain> there are some things that need resolving, before we can push this to production
12:06:27 <TrueBrain> GitHub was using Gollum, not?
12:06:40 <TrueBrain> I would have expected more polishing in that case ..
12:06:51 <TrueBrain> it is too bad I really do not understand Ruby :p Or have the intention to understand ...
12:07:27 <andythenorth> varnish-cures-all-ills
12:07:53 <TrueBrain> as our use-case is really simple, I am so tempted to make a simple Python wiki :P
12:08:20 <andythenorth> it's always good to remind ourselves why we don't write our own tools :D
12:08:29 <TrueBrain> for some reason, these wikis we have all integrate two things that have nothing to do with each other
12:08:31 <andythenorth> with practical demonstrations
12:08:36 <TrueBrain> you have editing pages
12:08:36 <andythenorth> what are the two things?
12:08:38 <TrueBrain> and you have reading pages
12:08:46 <andythenorth> oh yeah that shit
12:08:47 <TrueBrain> wikis closely integrate those two .. but .. why?
12:08:52 <TrueBrain> the reading part is VERY static
12:08:58 <TrueBrain> they NEVER change, until someone makes an edit
12:09:00 <andythenorth> basically most of my companies products are CMS
12:09:04 <TrueBrain> so .. generate HTML files and serve those?
12:09:06 <andythenorth> we used to have read/edit integrated
12:09:18 <TrueBrain> and make an edit page, where you can edit them
12:09:24 <andythenorth> now we have hardline, filesystem+runtime split between edit and serve
12:09:26 <TrueBrain> like BaNaNaS .. the read and write part are two completely different parts
12:09:36 <andythenorth> admin / public, it's a pattern
12:09:43 <TrueBrain> I don't mind if it is done by a single process etc
12:09:53 <andythenorth> details details :)
12:09:58 <TrueBrain> but the re-implementing of the read part ...
12:10:06 <TrueBrain> why does it take ~300ms to render a static page?
12:10:09 <TrueBrain> I mean .. IT NEVER CHANGES :P
12:10:34 <TrueBrain> well, maybe more strongly worded: when a system has a rare-event, don't make that your main flow
12:10:38 <TrueBrain> edits are rare-events
12:10:51 <andythenorth> we should start a newsletter
12:10:55 <TrueBrain> varnish is nice, but you need to tell it to invalidate ;)
12:10:56 <andythenorth> or maybe do an XKCD
12:11:09 <TrueBrain> so that means someone needs to write enough Ruby to do so :P
12:11:44 <TrueBrain> but take Gollum .. checking who changed the page last is every time a few git cals to figure that out
12:12:16 <TrueBrain> I guess it is easy to forget storage is cheap :)
12:12:23 <andythenorth> because lots of us are terrible programmers
12:12:26 <andythenorth> just some of us know it
12:12:31 <TrueBrain> what I read people are doing with Gollum: run Gollum locally, and output the HTML external
12:12:35 <TrueBrain> like our Jekyll website :)
12:13:26 <TrueBrain> mediawiki is btw exactly the same, and also the reason it is this resource heavy
12:13:54 <TrueBrain> LordAro: you are kinda repeating yourself ... are you choking? do we need to call someone?
12:14:32 <TrueBrain> does anyone has his number? I am worried now :(
12:15:30 <andythenorth> was it a cry for help?
12:15:47 <andythenorth> I guess wiki -> GitHub Pages was not a viable route then :D
12:16:14 <TrueBrain> it got shot down :P
12:16:34 <andythenorth> I tried GH Pages, it's kind of fine, and kind of odd
12:16:40 <andythenorth> anyway, ship has sailed
12:16:43 <TrueBrain> they latest version is a lot better :)
12:18:08 *** markymark has joined #openttd
12:18:43 <TrueBrain> well, what might be a solution, is to have gollum for editing pages, and use GH Actions to publish the wiki or something
12:19:33 <andythenorth> how rapidly do edits need to show up in public?
12:19:36 <andythenorth> not very rapidly?
12:20:50 * andythenorth searched 'varnish for gollum' but it was all LOTR figure painting :P
12:29:33 <markymark> Hi, is there someone here who was on the OpenTTD project at the start?
12:33:56 <LordAro> there are a few people here who are suitably ancient, but i think you should just ask whatever question you have
12:34:48 <andythenorth> owen was nearly at the start
12:35:10 * andythenorth wants to hear the question now
12:42:57 <markymark> I want to start a project myself for another game (Jagged Alliance: Deadly Games) and would appreciate some getting started tips.
12:43:07 <markymark> Better learn from the best
12:44:15 <LordAro> the very general answer to that is "reverse engineering"
12:44:35 <LordAro> which is a huge topic
12:44:51 <markymark> Besides that, aren't there any legal issues? regarding IP
12:46:20 <LordAro> that's a very difficult topic without an awful lot of expensive lawyers :p
12:46:20 <markymark> if you want to use for instance all the graphics, sound, etc.
12:46:35 <LordAro> OTTD's initial reverse engineering was done in a country where it was legal
12:46:56 <LordAro> but even so, we don't distribute any of the original graphics/sounds/data
12:47:51 <markymark> yes, I was thinking about that but it would probably save a lot of work when you can reuse a lot of the original content.
12:48:43 <markymark> yeah ok, but isn't distributing the point :-)
12:48:52 <markymark> otherwise you can only use it yourself
12:49:05 <LordAro> you can (probably) distribute whatever you have made
12:49:22 <LordAro> everything else is "you must have a copy of the original game and put the files in <location>"
12:49:59 <markymark> ah right, so you could like rebuild the main "engine" part of the game and reference the use of the content from the original game
12:50:28 <markymark> I think you guys did the same in earlier versions of openttd right? Ive been around for a while also ;-)
12:50:43 <LordAro> a more recent example would be openrct2
12:51:19 <andythenorth> if you could identify who holds the copyright, your life would be easier
12:51:20 <markymark> ah that helps a lot
12:51:23 <andythenorth> but you probably can't
12:51:37 <markymark> no, I did some online searching, but it gets sketchy
12:51:39 <andythenorth> looks like sirtech no longer exists
12:51:45 <andythenorth> but there may be publishers holding rights
12:52:09 <andythenorth> and original authors may hold some copyright, depending on contract
12:52:53 <markymark> I will try to find a contact later on, knowing we could start without probably infringing on something
12:53:41 <markymark> ok, that helps for now. Having some early doubts out of the way :-)
13:32:11 <TrueBrain> to add to all of that, for OpenDUNE we reached out to what were most likely the copyright holders and people with distribution licenses, to figure out on how many toes we were going to step
13:32:19 <TrueBrain> but in games that are old, it is very very difficult
13:32:24 <TrueBrain> and companies start to point to each other
13:32:38 <TrueBrain> in the end, it is about taking a calculated risk, as least, for OpenDUNE it was
13:32:57 <TrueBrain> what still stands in that project: if any copyright holder complains that the project should not exist, it will be gone within that day
13:33:26 <TrueBrain> that is a calculated risk, that they won't sue you dry before giving you the chance to close shop :)
13:33:51 <TrueBrain> but still there, we do not distribute the graphics, as we could not get a distribution license for that
13:34:07 <TrueBrain> (nobody said no, but also nobody said yes :P)
13:36:09 <TrueBrain> the problem you will be facing with a game like Jagged Alliance, that there are still new versions coming out. So people still care. And that could possibly mean they don't appreciate 1-on-1 clones. So yeah, contacting the developers and publishers is the right first step :D
13:45:25 <andythenorth> set up a UK limited company with no assets :P
13:45:29 <andythenorth> let that get sued
13:46:00 <andythenorth> the downside is that if you are a director of a company and you wilfully and knowingly let the company do illegal things, you are on the hook :)
13:46:05 <andythenorth> and sometimes even if you didn't know
13:46:16 <TrueBrain> exactly; so that doesn't really help
13:46:41 * andythenorth could have gone to prison if we'd killed The Stig by accident
13:47:12 <TrueBrain> do ... we want to know that is the scenario you picked as example? :P
13:48:57 <andythenorth> that shot was quite worrying, I wasn't on the job, but I got a phone call after the shot to say "we didn't kill the Stig, you're not going to prison"
13:49:14 <TrueBrain> not sure it is funny or sad :P
13:49:49 <andythenorth> me going to prison is not funny
13:50:00 <andythenorth> I can clarify any confusion on that point :P
13:52:24 <andythenorth> is it lunchtime?
13:52:27 <andythenorth> bring back peter
13:52:37 <TrueBrain> start the summoning ritual!
13:52:38 <andythenorth> can we restore him from backups?
13:54:05 *** nielsm is now known as Guest3376
14:00:38 *** longtomjr_ has joined #openttd
14:02:04 *** snail_UES_ has joined #openttd
14:11:52 <TrueBrain> do you really want to rollback a running instance? That sounds horrible, if you put it like that :P
14:13:00 <Eddi|zuHause> that would depend on whether that requires killing the already running instance :p
14:13:36 <TrueBrain> do you really want to risk someone recovers backup of me? Can anyone handle more than one? :P
14:15:13 <Eddi|zuHause> that is a different question :p
14:17:08 <andythenorth> might get conflict errors
14:17:25 <andythenorth> how would we know which version is canonical?
16:05:56 <TrueBrain> @blame andythenorth
16:05:56 <DorpsGek> TrueBrain: blames andythenorth
16:06:04 <TrueBrain> learn something new about your own bot every day ...
16:12:29 <LordAro> TrueBrain: just think what new features we'll get when you upgrade it :p
16:12:45 <TrueBrain> well, I am removing many many many many of them, sorry :P
16:13:57 <TrueBrain> I am always shocked how much a default install of such bots allow :P
16:16:47 <TrueBrain> " Dec 3 2012 github" <- folder on the current installation
16:17:13 <LordAro> what happened that day? :p
16:18:05 <TrueBrain> guess in 2012 we already did something with GitHub
16:18:10 <TrueBrain> took 6 more years to migrate to there :p
16:18:22 <TrueBrain> I think I made a sync-copy on GitHub on that date
16:18:35 <TrueBrain> lol .. 1 person is ignored by DorpsGek .. just 1
16:18:43 <TrueBrain> I think that ignore is there for years and years :D
16:24:08 <TrueBrain> you are not an exception there :)
16:25:40 <TrueBrain> meh .. I remember was I was dreading doing this bot stuff ...
16:25:54 <TrueBrain> supybot is bringing a gun to a pillow fight
16:26:10 <TrueBrain> we want to use 1% of what it offers ...
16:26:35 <TrueBrain> him reporting anyone, that is cute :)
16:26:46 <LordAro> it's the first mention of "me" anywhere on irc
16:27:28 <TrueBrain> "<PeterT> He's been making images smaller so that they can fit HIS screen"
16:27:48 <LordAro> iirc i was using a 1024x768 screen at the time
16:28:17 <LordAro> and yeah, reducing the width by 3px made it fit next to the banner
16:29:31 <TrueBrain> that conversation is priceless
16:29:45 <TrueBrain> PeterT was this ADD kid everyone was wondering why he was running around for
16:31:16 <TrueBrain> <andythenorth_> PeterT: just go and play the game and get over it. life is too short for internet pissant nonsense <- at least andythenorth hasn't changed in 10 years :P
16:31:40 <TrueBrain> LordAro: your first mention on IRC was 2009-09-06 :)
16:31:45 <TrueBrain> so a few months before that line :P
16:31:59 <LordAro> i have been living a lie
16:32:22 <TrueBrain> not sure which channel .. I had to filter that out to get it sorted by date :D
16:32:56 <TrueBrain> ah, in a side-channel we had :)
16:33:24 <TrueBrain> <CENSORED> urgs... PeterT and LordAro are now both in #openttdcoop channel :-D
16:33:27 <TrueBrain> <CENSORED> currently he tries to help lordaro to get on one of our servers. Dunno which one actually :-)
16:33:52 <LordAro> well that must be PM :p
16:33:54 <TrueBrain> and I have this on record:
16:33:55 <TrueBrain> 2009-08-22T20:31:52 *** lordaro has joined #openttd
16:34:02 <TrueBrain> it is the smileys, isn't it? :)
16:34:45 *** longtomjr_ has joined #openttd
16:35:45 <TrueBrain> grepping anything in this massive archive takes FOR EVER :P
16:35:53 *** frosch123 has joined #openttd
16:36:01 <TrueBrain> what .. is frosch123 doing here this early?!
16:36:13 <TrueBrain> it is people-act-out-of-normal-day, I guess :D
16:36:25 <glx> <TrueBrain> grepping anything in this massive archive takes FOR EVER :P <-- so many join/part messages ;)
16:43:15 <TrueBrain> 2010-08-27T09:14:30 <lordaro> i suspect it's something to do with this: <url-removed>
16:43:23 <TrueBrain> a year and 5 days on IRC before you said something
16:45:25 <TrueBrain> @calc 970286 / 3465450
16:45:25 <DorpsGek> TrueBrain: 0.279988457487
16:45:41 <TrueBrain> glx: 27% of the logs are parts/joins
16:46:15 <glx> oh on some days it's probably almost 90%
16:46:49 <glx> and near 100% in some channels :)
16:46:59 <TrueBrain> this was only #openttd :P
16:47:28 <TrueBrain> @kick Xaroth make that 1000
16:47:28 *** Xaroth was kicked by DorpsGek (make that 1000)
16:48:07 <TrueBrain> that is all not that much
16:48:30 <glx> ban list is currently empty
16:48:34 <TrueBrain> (in 13.5 years btw)
16:49:55 <TrueBrain> 36k renames .. lot of people cannot make up their name :D
16:50:33 <TrueBrain> Eddi|zuHause is #3 on the list of nick renames :D Gratz!
16:50:43 <TrueBrain> andythenorth on a close #4
16:50:46 <glx> most of mine are because I lost connection for some reason
16:50:57 <TrueBrain> you are not even in the top 10
16:52:31 <TrueBrain> of those 1000 (hihi) kicks, 152 were because people tried openttdcoop commands in here
16:52:51 <glx> and that happens only when I'm here
16:52:55 <TrueBrain> so that has been worth implementing, I guess :P
16:53:43 <frosch123> i think that made it on some external irc quotes site
16:53:57 <TrueBrain> owh, bash .. bash was amazing :D
16:54:49 <TrueBrain> no, the quote site :)
16:55:02 <TrueBrain> it still exists, it seems :P
16:55:16 <TrueBrain> I knew that site before I knew the shell :)
16:55:28 <TrueBrain> you know you liked being kicked!
16:56:18 <DorpsGek> TrueBrain forced me to do it
16:56:29 <Xaroth> Oh you love kicking me, DorpsGek.
16:56:57 <TrueBrain> Supybot is really idiotic in how it maintains databases etc :P
16:57:19 <TrueBrain> the seen database is 705KiB of CSV :)
16:57:40 <TrueBrain> glx: when ever I say supybot, just read Limnoria
16:57:41 <frosch123> luckily it's not an excel database
16:57:43 <glx> hey it could be XML and twice as big
16:57:56 <TrueBrain> that is how their docs work too, and I fucking cannot remember that other fucking name
16:58:05 <TrueBrain> well, that seen database is rewritten every N minutes
16:58:20 <TrueBrain> I am surprised it never got corrupted tbh :)
16:59:09 <TrueBrain> and we stopped with the "any" seen database, as that was even worse ... 3 times as big, I believe :)
17:03:17 <TrueBrain> frosch123: seen my PR? Not sure what to think of gollum .. have been reading their issue-tracker .. they don't really seem to care about performance
17:03:39 <TrueBrain> a dude with 4k pages and page-load times of 2 seconds, their reply was: at least it isn't as slow in the last version
17:04:02 <TrueBrain> s/slow in/slow as in/
17:05:03 <TrueBrain> owh, and if a page has no translations it shows something about the translation cache being invalid :D But that is work from your diff, so I assumed you knew about that already :)
17:06:37 <frosch123> i also know about the case-sensitivity
17:06:56 <frosch123> my conversion scripts replaces all page, image and template links
17:07:09 <frosch123> as you saw, the pages are now sorted into languages
17:07:19 <TrueBrain> some templates were also broken, like "warning" instead of "en/Warning", but I was sure you would tackle them over time :)
17:07:25 <TrueBrain> yeah, it looks pretty nice
17:07:31 <frosch123> next i want to move the crap pages into archive, so the "many files in one directory" should reduce
17:07:36 <TrueBrain> the gollum-test-data layout looks really good tbh
17:07:47 <TrueBrain> but I am worried about the gollum performance
17:08:09 <frosch123> so, render all pages to disk, and only serve /gollum via gollum?
17:08:16 <frosch123> i.e. only edit+preview via gollum
17:08:17 <TrueBrain> I have been considering exactly that, yes
17:08:23 <TrueBrain> the main issue there is templates
17:08:29 <TrueBrain> difficult to see the full tree of what you need to invalidate
17:08:36 <TrueBrain> (and rendering all pages on every edit .. well, no :P)
17:08:46 <frosch123> TrueBrain: oh, that part is easy
17:08:57 <frosch123> i already generate .category and .translation
17:09:14 <TrueBrain> which I was hoping you also weren't going to do every edit? :D
17:09:15 <frosch123> which can also be put into a some git hook to update based on diffs
17:09:22 <TrueBrain> btw, category was broken, as in, they are always empty :)
17:09:47 <frosch123> there are many things broken :)
17:09:57 <TrueBrain> but if we use gollum only for editing .. what is left of gollum? That isn't really a lot of code, is it?
17:10:22 <frosch123> well, searching (no idea how good is that), and preview and stuff
17:10:36 <TrueBrain> search make the memory go BOOOOOOOOOMMM :D
17:11:08 <TrueBrain> yeah :D I tried as much functionality as I could to get a feeling for gollum to run it in production
17:11:47 <TrueBrain> btw, if you rebase, use "--committer-date-is-author-date" :)
17:11:58 <TrueBrain> looks prettier in GitHub :D
17:12:01 <TrueBrain> (totally not important)
17:12:32 *** gelignite has joined #openttd
17:12:45 <frosch123> yeah, i only noticed that after push
17:12:53 <frosch123> local log does not show the commiter date
17:12:57 <TrueBrain> you can always do that, even after you "broke" it :)
17:13:11 <TrueBrain> it is one of the quirks in GitHub
17:13:19 <TrueBrain> they changed it a few times from commit to author to commit date, I believe :)
17:14:25 <frosch123> do you want to try some of the python mediawiki-parser ?
17:14:35 <frosch123> to see how good they are at templates and stuff
17:14:45 <frosch123> then we could use them to render the html to disk
17:15:06 <TrueBrain> yeah, I think that is a good road to explore
17:15:22 <TrueBrain> I was looking for Python implementations of a wiki, but they are .. euh .. well, I couldn't find any decent ones :D
17:16:22 <frosch123> haha, yeah, i did that research before. gollum is pretty much the only thing out there, that does git-based-data and mediawiki.-syntax
17:16:48 *** blathijs has joined #openttd
17:19:51 <frosch123> TrueBrain: btw. custom.css and custom.js are controlled in config.rb. i enabled them for now, since we probably use them both at some point
17:20:05 <TrueBrain> it was slowing everything down like a mofo :P
17:20:19 <frosch123> i did not notice, since loading 20+ flag icons took even longer
17:20:21 <TrueBrain> as they are js/css, the page was not really rendering before they returned .. 600ms later :P
17:20:31 <TrueBrain> nginx really speeds shit up :P
17:21:10 <TrueBrain> now I found out how simple it is to have nginx as sidecar, it might also help with BaNaNaS etc :)
17:21:15 <andythenorth> are most of my renames me trolling? :P
17:22:54 <frosch123> TrueBrain: are you sure nginx is the sidecar, and not the other way around?
17:23:13 <TrueBrain> nginx links to the gollum containers, and inherits the volumes
17:23:29 <TrueBrain> gollum runs fine without nginx, nginx not without gollum
17:23:33 <TrueBrain> that makes nginx the sidecar :)
17:24:04 <TrueBrain> you can skip using nginx if you like btw, just add the portforward on the "docker run" for gollum again :)
17:37:24 <frosch123> "Compared with mwparserfromhell" <- haha, looks like it's the reference standard :)
17:39:15 <frosch123> oh, and it lists the same wiki misfeatures as unsupported that i also broke when patching wikicloth
17:41:45 <TrueBrain> mwparserfromhell is used by 462 projects on GitHub
17:41:51 <TrueBrain> so yeah, it is a bit of a standard :)
17:42:10 <LordAro> "!!! Do not set RVS_TEMP to 'C:\' to test this !!!"
17:42:31 <LordAro> only the files older than 12 hours!
17:42:38 <TrueBrain> hahahahahahahahahaa
17:42:44 <TrueBrain> hahahahahaha :D :D :D
17:42:45 *** Wormnest has joined #openttd
17:42:48 <TrueBrain> what a terrible way to leave your company :P
17:43:36 <LordAro> well the desktop would be getting wiped anyway :p
17:44:39 <TrueBrain> the worst part is, someone left that comment because .. it is not the first time :P
17:45:12 <LordAro> luckily this is a comment on a bug report
17:45:47 <LordAro> we think the code that deletes the files was written before RVS_TEMP was made to be overridable
17:50:58 <TrueBrain> frosch123: I guess this means I have to learn what all this stuff means in mediawiki ... ugh ... :P
17:51:27 <LordAro> part of the translation stuff
17:51:44 <frosch123> it includes the template "Template/en/-.mediawiki"
17:52:06 <frosch123> just that some people like to name templates by using pictures
17:52:10 <TrueBrain> who the ... made a template named - :P
17:52:20 <frosch123> i think there were no emoticons at that time
17:52:26 <LordAro> someone who was adding it to every single page, i suspect
17:52:39 <LordAro> the shorter the better does make sense in that case
17:53:11 <TrueBrain> and now I need to understand how it works :D
17:54:50 <TrueBrain> owh, okay, that is what the [[Translation: syntax is doing
17:55:17 <LordAro> well [[]] are internal wiki links
17:55:40 <LordAro> so linking to a page in the Translation namespace? :p
17:56:00 <TrueBrain> so all translation pages do "[[Translation:Main/en/Main Page]]{{en/-}}"
17:56:16 <TrueBrain> and that generates the language bar for every page :P
17:56:33 <TrueBrain> but I see frosch123 already wrote some python to do this :D
17:56:34 <frosch123> the "-" template is a line flush or something
17:56:51 <TrueBrain> it only contains "<div style="clear:both"></div>"
17:57:06 <TrueBrain> and it can be translated :P
17:57:15 <frosch123> "Translation:" is now supposed to work like a category, it's jsut rendered differently
17:57:51 <frosch123> TrueBrain: i put all pages witout language-tag into "en", what else should i do?
17:58:08 <TrueBrain> I think that is fine
17:58:17 <TrueBrain> Main/pl/Szablon:-.mediawiki
17:58:27 <TrueBrain> that file is weird .. it is not really the - template, but it acts like it?
17:58:42 <TrueBrain> also .. why did they translate even "Template" into different languages? :D
17:58:54 <frosch123> TrueBrain: yeah, i have list of pages to just delte
17:59:23 <frosch123> some people create pages with wrong name, and then cannot delete them
17:59:27 <frosch123> so keep repeating to create new pages until they fixed all their typos
17:59:32 <TrueBrain> I fully understand :)
17:59:45 <frosch123> like that RTL char yesterday :p
18:00:11 <TrueBrain> so much dirt got into the wiki :)
18:00:12 <frosch123> or those two busstop graphics that only differ in company colours and in capitalisation
18:00:44 <TrueBrain> so .. I wonder .. does gollum store the file in the right language depending on the language tag?
18:00:53 <TrueBrain> or how does it know?
18:01:03 <frosch123> no, i invented those languages
18:01:32 <frosch123> {{Translate:foobar}} is a custom extension i invented :)
18:02:01 <TrueBrain> but so how do you see editing working?
18:02:06 <frosch123> well, it's [[Translate:foobar]]
18:02:40 <TrueBrain> owh, in the original wiki it lists all the translations?
18:02:44 <frosch123> TrueBrain: validate the names of all new pages to fit into the naming schema
18:03:02 <TrueBrain> so if you add a language, you have to alter that for all languages too?
18:03:16 <frosch123> TrueBrain: yes, in the current wiki all translators have to reference their translation from all other translations. O(n^2) edits for n languages :)
18:03:42 <TrueBrain> okay, good to know you invented this, fine by me, etc :)
18:03:46 <frosch123> oh, you have not seen the worst by far :)
18:03:47 <TrueBrain> why did you make it a link, not a template?
18:04:20 <frosch123> i had it a template first :) but the [[ ]] solution is better
18:04:26 <frosch123> it's not a link, but a category
18:04:55 <frosch123> categories work the same, you do [[Category:foobar]] somewhere in the page, and wiki renders it as something special at the bottom
18:05:20 <TrueBrain> or [[:Category:foobar]], I see in some cases .. lol
18:05:20 <frosch123> [[Translation:foobar]] can also be anywehre in the source, it is always rendered at top, but you can also put it into a sidebar or similar
18:05:24 *** Progman has joined #openttd
18:05:50 <frosch123> basically [[Category::foo]] and [[Translation:bar]] tag the page, and do not render anything in-place
18:05:54 <TrueBrain> so the difference between template and innerlinks is a bit weak in mediawiki, I see :)
18:06:04 <TrueBrain> as I also see [[File:]]
18:06:21 <frosch123> haha, you want a complete list? :p
18:06:29 <TrueBrain> eventually, I guess I have to :P
18:06:36 <TrueBrain> but for now I am trying to get a feeling what I am getting myself into :)
18:07:22 <TrueBrain> okay, so {{en/-}} exists because - doesn't have a translation, so it is put in "en"
18:07:31 <TrueBrain> which feels a bit weird I guess, but I understand
18:07:32 <frosch123> [http://external link] [[internal link]] [[Category:add page into category]] [[::Category:link to the category page]] [[File:include image as img]] [[Media:just link to the image]] ...
18:07:54 <frosch123> {{include template}} {{functioncall:othermagic}}, ...
18:08:26 <frosch123> TrueBrain: what is your goal? i consider the data migration a WIP
18:08:39 <frosch123> next thing i want to fix are the differences in capitalisation
18:08:42 <TrueBrain> well, a Python-based "view" part :P
18:08:42 <frosch123> which breaks some pages
18:09:05 <frosch123> TrueBrain: sure, but i would recommend against writing yuor own paser
18:09:14 <TrueBrain> I was not planning to
18:09:33 <TrueBrain> I was, how ever, looking into how to make a renderer
18:09:44 <TrueBrain> but that was before I was told about the tons of odd stuff going on :)
18:12:35 <frosch123> oh, i forgot... i should have asked the 3px expert
18:15:39 <TrueBrain> okay, wikitextparser is a nice library to manipulate medawiki syntax :)
18:16:00 <frosch123> why do you want to manipulate it?
18:16:01 <andythenorth> frosch123 I like the red one
18:16:12 <andythenorth> TBH it's potato / potato
18:16:40 <TrueBrain> frosch123: I see 2 possibilities: 1) we wrap something around gollum to make static exports of the files .. means that after an edit, it takes N minutes to update the live site. 2) we write a Python render to HTML that does this based on the mediawiki files; most likely a lot quicker, and we can work in a language we know a bit more :)
18:17:06 <TrueBrain> frosch123: I just wrote a very simple mediawiki renderer in Python that replaces templates with some text .. hence: manipulate
18:17:35 <frosch123> ok. you sounded like you wanted to add a second conversion step for the whole data
18:17:51 <TrueBrain> nope; that is your department :)
18:18:27 <frosch123> i won't tell you more horror stories, to not distract you more :)
18:18:27 <TrueBrain> possibly you want to put everything that doesn't have an explicit language in a folder like "default", and still allow {{-}}
18:18:40 <TrueBrain> as in: if no language used, use default
18:18:55 <frosch123> "common" is somewhere further down on my list
18:18:58 <TrueBrain> but I guess that depends a bit how we want to solve it :)
18:19:08 <TrueBrain> do we want to write our own viewer or not, I guess
18:19:19 <TrueBrain> it seems like it is not -that- complicated tbh
18:19:47 <frosch123> as i see it, Main containly like 1k pages now (down from 10k with all translations, user pages and shit), which can be sorted manually into: keep or move to "archive"
18:23:41 <TrueBrain> okay, translations of templates is just weird :P
18:23:45 <TrueBrain> that went all places, it seems
18:24:37 <TrueBrain> so frosch123 , how are you building your migration-scripts .. lot of seds? :)
18:24:46 <frosch123> sometimes it is valid, when templates contain plain text. sometimes translators only wanted to translate the "how to use this template" part
18:24:58 <glx> frosch123: I prefer .png because the building is not hidden in the grass
18:25:35 <frosch123> TrueBrain: yes, python with lots of regexp. maybe it would have been smarter to use some existing mediawiki->ast thingie. but too late :)
18:26:05 <TrueBrain> "<noinclude>'''Broken translation link'''{{en/-}}</noinclude>{{ambox"
18:26:31 <TrueBrain> guess that is the {{Other_languages}} where your script failed?
18:26:37 <TrueBrain> (just trying to get a feeling; no judgement :D)
18:26:40 <frosch123> i guess i can just strip that template from templates
18:27:22 <TrueBrain> what a mess what a mess
18:27:27 <TrueBrain> how are we going to make soup out of this? :)
18:27:35 <TrueBrain> well, after dinner, I will see how difficult a Python frontend really is
18:27:43 <TrueBrain> if it is easy, that might work better than wrapping Gollum
18:28:22 <TrueBrain> so far, wikitextparser is really nice, as it allows you to easily modify the resulting document
18:29:02 <TrueBrain> okay, that is my queue to get to dinner :)
18:34:09 <frosch123> 9 cases of these duplicate pages
18:34:20 <frosch123> i guess i'll ask the dice
18:34:39 <glx> Coca-cola is not even fully translated
18:35:12 <frosch123> lol, yeah, the last sentence was too much
18:38:22 <glx> anyway I don't get why they added "coca-"
18:40:03 <glx> and in game the translation is "cola"
19:05:47 *** HerzogDeXtEr has joined #openttd
19:43:40 <frosch123> TrueBrain: i see "Coding Style" breaks gollum a lot :)
19:43:51 <TrueBrain> it escapes the escaping of the escape
19:43:56 <TrueBrain> your .translation HTML is invalid btw :D
19:44:02 <TrueBrain> took me a while to figure out why it failed to render .. :P
19:44:13 <TrueBrain> <img src="/uploads/zh/Flag.png"</img> <- it ismissing a > :)
19:44:32 <frosch123> oh, my browser fixed that
19:47:27 <TrueBrain> I found a bug in wikitextparser :D
19:47:59 <frosch123> oh dear... we ended up considering gollum unusable, and now you also find bugs in the next thing? :p
19:48:13 <TrueBrain> at least this is a language in which we can fix stuff :)
19:48:42 <frosch123> oh, i concluded that ruby is a troll language
19:48:54 <frosch123> of all the language design choices, it always picks the worst combination
19:49:43 <TrueBrain> basically, the length of a string is only decided at init; if you update it later, it never applies that update :D
19:50:29 <TrueBrain> it is pretty nice, it tries to not copy strings as much as possible
19:50:39 <TrueBrain> so subclasses have: it is in this "object" from byte N to M
19:52:05 <DorpsGek_III> - Update: Translations from eints (by translators)
19:52:28 *** tokai|noir has joined #openttd
19:52:28 *** ChanServ sets mode: +v tokai|noir
19:52:44 <TrueBrain> or maybe I am replacing these objects wrong .. always a possibility too, ofc :D
20:01:37 <frosch123> pushed new test-data
20:01:48 <frosch123> the capitalisation issues should be solved now
20:16:40 <frosch123> so many people join github for us :)
20:18:32 <TrueBrain> frosch123: ah, you moved the {{en/-}} into the translation thingy
20:19:04 <frosch123> TrueBrain: the final goal for the translation thingie is to get put into a sidebar, or the header or something, but not in the main content
20:19:23 <TrueBrain> okay, that should be doable
20:19:37 <frosch123> it groups pages, it's no page content
20:20:38 <TrueBrain> hmm ... the problem seems to be with inline styles, I am having
20:20:43 <TrueBrain> not so much with what I was doing
20:23:26 <frosch123> 4 failed invitations meanwhile :) 8%
20:38:56 <TrueBrain> ha, found the bug .. it only goes wrong when the text STARTS with [[ :)
20:43:39 <TrueBrain> see, I can debug this in Python ! :D
20:46:47 <LordAro> 100% line coverage is impressive
20:49:16 <TrueBrain> that whole repo is impressive
20:49:20 <TrueBrain> I wish every repo was like this
20:49:46 <TrueBrain> funny, if in mediawiki you make a link to your own page, it doesn't make it a link, it seems :P
20:50:55 <frosch123> yes, that's an often used feature
20:51:38 <frosch123> that box is used on all pages, yet the current page is not a link
20:51:38 <TrueBrain> that was where I noticed it :)
20:52:00 <frosch123> i did not support that in gollum btw :p
20:52:18 <TrueBrain> a single if-statement in Python, easy :P
20:56:19 <TrueBrain> wtf is ''' ... I need to learn mediawiki :P
20:57:01 <frosch123> '' italic, ''' bold, '''' no idea
20:57:12 <TrueBrain> because, why not ... lol
20:57:21 <TrueBrain> ''''' bold and italic
20:57:39 <frosch123> is it bold first or italic first? :p
21:00:13 <frosch123> andythenorth: sorry, i meant to say "idiomatic"
21:04:35 <TrueBrain> frosch123: italic first, it turns out
21:05:23 <frosch123> i guess browsers do not care about <i><b>foo</i>bar</b> :)
21:06:39 <TrueBrain> hmm .. middot is replaced by :
21:08:43 <TrueBrain> guess a : also has some sorts of meaning
21:09:00 <frosch123> but only in first position
21:09:16 <TrueBrain> what was wrong with markdown again? :P
21:09:32 <frosch123> doesn't markdown do the same?
21:09:57 <TrueBrain> this was just throwing me off
21:10:04 <TrueBrain> the parser library does the right thing btw
21:13:47 <andythenorth> hmm the tank game is quite full of silly racists
21:13:58 <andythenorth> probably woke twitter says I should stop playing
21:14:09 <andythenorth> but why should they silly racists get to dictate my life?
21:14:20 <andythenorth> I don't think cancelling wins much
21:20:57 <TrueBrain> mediawiki is fun .. so much fucked up shitty details ..
21:21:07 <TrueBrain> who in his right mind makes ":" mean something else depending on the context
21:22:06 <TrueBrain> owh, funny enough mediawiki itself implements it as a list ... haha
21:23:52 <TrueBrain> not following your own formatting guide
21:28:25 *** arikover has joined #openttd
21:29:51 <TrueBrain> you have includeonly and onlyinclude
21:30:04 <TrueBrain> never knew mediawiki was this hilarious :)
21:31:05 <frosch123> apparently they mean different things
21:31:22 <frosch123> but only if you transclude non-template pages
21:36:24 <TrueBrain> why not have EVERYTHING on a single page I am trying to render ..
21:37:15 <TrueBrain> frosch123: you replaced Image with File, I assume?
21:37:37 <TrueBrain> why? (honest question)
21:37:47 <frosch123> "Image" is deprecated
21:38:13 <frosch123> no idea why they decided for one or the other
21:38:31 <frosch123> "Media" is different though :p
21:38:32 <TrueBrain> I like that you changed wikipedia: for decent links :)
21:38:43 <TrueBrain> however, please make it https :D
21:39:06 <frosch123> should we do that with all external links?
21:39:13 <frosch123> (some may not support it)
21:39:40 <TrueBrain> if they support it, yes
21:40:22 <TrueBrain> I good bugfinder yes yes? :D
21:42:27 <andythenorth> frosch123 less casually racist than tanks I think
21:45:52 <TrueBrain> okay, now I need CSS ...
21:46:47 <andythenorth> you never need CSS
21:48:35 <TrueBrain> where do I find the CSS gollum/mediawiki uses .. :D
21:49:31 <frosch123> probably not useful :)
21:50:11 <TrueBrain> just going to "borrow" the gollum one
21:57:47 <TrueBrain> okay, this renders pretty easily
21:58:02 *** blathijs has joined #openttd
21:59:33 <TrueBrain> that is the only namespace really, isn't it?
21:59:36 <TrueBrain> will you remove it?!
21:59:36 <frosch123> i'll probably rename those
21:59:51 <frosch123> and no, it's not a namespace
21:59:57 <frosch123> that's the main problem with it
22:00:36 <TrueBrain> I can render the Coding Style in 0.08s, including Python startup
22:01:09 <TrueBrain> owh, it got cut off at the end .. let me fix that :D
22:01:53 <TrueBrain> ah, no, the page just ends really odd
22:02:33 <TrueBrain> what I currently do: I resolve all templates, embed them in the main document
22:02:46 <TrueBrain> after that, I parse bold, italic, lists, sections, external links and wikilinks
22:02:50 <TrueBrain> not sure that is the right way around
22:03:02 <TrueBrain> (in other words: a template with a wikilink will be relative from the page that uses the template)
22:03:15 <TrueBrain> (wikilink == [[ ]] )
22:03:33 <frosch123> relative? they are all absolute paths
22:04:00 <TrueBrain> well, I mean I read somewhere [[Category::PATH]] or something
22:04:20 <TrueBrain> [[Category:en/Manual|{{PAGENAME}}]]
22:04:26 <TrueBrain> owh, its a template
22:04:36 <TrueBrain> well, it is now first embedded in the main page
22:04:39 <TrueBrain> then it is resolved
22:05:19 <frosch123> i think that's the correct order :)
22:05:55 <frosch123> does it support those #ifexpr ?
22:06:03 <TrueBrain> which page uses it?
22:06:54 *** jottyfan has joined #openttd
22:10:57 <TrueBrain> it does parse them correctly; but this is just a lexer I think
22:12:25 <TrueBrain> yeah, so it parses them, but I would have to implement them :)
22:13:38 <TrueBrain> and wtf does {{{ mean in this context :P Lets find out!
22:13:46 <frosch123> those are the parameters
22:14:23 <frosch123> {{Foobar|Param1|Param2}} includes Template:Foobar with {{{1}}} = "Param1" etc
22:14:45 <TrueBrain> but I also have {{{type|}}}
22:14:45 <frosch123> looks like you need to implement a lot then :(
22:15:14 <frosch123> TrueBrain: yes, named parameter {{Foobar|type=yolo}} and default value
22:31:47 <TrueBrain> oef, ifexpr is nasty :P
22:33:28 <TrueBrain> well, it does render at least :)
22:51:45 <frosch123> meh, why does outlook use ctrl+q for "mark as read". i close my private mailtool so often...
23:12:29 <TrueBrain> frosch123: {{# stuff mostly works
23:12:38 <TrueBrain> at least the Page you gave renders correctly
23:12:42 <TrueBrain> 1 thing .. I use eval() atm :P
23:12:44 <TrueBrain> that .. needs fixing :D
23:12:55 <TrueBrain> was only 50 lines of code
23:20:39 <frosch123> let's see whether those turn into 5000 by the end of the week :)
23:20:50 <TrueBrain> in total it is now 200 lines
23:21:02 <TrueBrain> and it renders most things
23:21:06 <TrueBrain> so doubling is likely
23:23:10 <TrueBrain> same person made them or something? :D
23:23:39 <frosch123> just random pages i visited
23:25:29 <frosch123> though "berlin" is funny :p
23:25:45 <TrueBrain> This message box is using an invalid "type={{{type|}}}" parameter and needs fixing.
23:28:39 <TrueBrain> there is a bug in the ambox template
23:28:43 <TrueBrain> and it is an annoying one
23:28:58 <TrueBrain> }} padding: 4px 8px; background:#efefef; min-height: 44px;">}}
23:29:00 <TrueBrain> the > is inside the template
23:29:14 <TrueBrain> I am not sure how mediawiki can render this
23:29:33 <TrueBrain> or if you use style, you have to add a >
23:30:52 <TrueBrain> NO clue if there are any side-effects
23:31:38 <frosch123> pretty sure it's not intentional
23:31:46 <frosch123> it's okay to break anything that relies on it
23:33:00 <TrueBrain> nothing SEEMS to be using "style"
23:42:00 <TrueBrain> [File:en/. Content.png\n\n|40px]] (I replaced newlines with \n)
23:42:03 <TrueBrain> wtf happened there :P
23:42:15 <TrueBrain> I think frosch's converter failed there :D
23:44:03 <TrueBrain> cannot blame him, it is in reality a switch-case
continue to next day ⏵
