Static Wiki Website Archive #5

New issue

Closed

opened 2020-07-13 04:09:17 +00:00 by kuschelyagi · 3 comments

kuschelyagi commented

2020-07-13 04:09:17 +00:00

Owner

Create a static HTML/CSS archive of the wiki. This archive is HTML/CSS only (no javascript) and can be hosted anywhere.

kuschelyagi commented

2020-08-24 05:55:31 +00:00

Author

Owner

Progress was made. Main pain points right now:

Template transclusion is half-baked. Note that the library we're using (mwparserfromhell) doesn't actually do this so we have to roll our own.
- Magic words (e.g. PAGENAME, NUMBEROFARTICLES etc) are not yet implemented - they would be implemented like any other template
Category parsing/generation is also half-baked, doesn't seem to take into account template transclusion (which it should, since transclusion happens before we look for categories)
Images are not yet included. We could make a script to pull images or just use the dump from archive.org for this
Page name normalization is funky. Notable in that sometimes a slash (/) indicates a subpage (e.g. GlitchDex/RB:000) and sometimes it's just a slash (e.g. Gold/Silver/Crystal). We have some special logic in both the archive builder and redirector to know the difference.
- nginx normalizes URLs before passing them into our handlers so e.g. Gold%2FSilver%2FCrystal (which is what we would generate) is interpeted as Gold/Silver/Crystal (i.e. a directory structure), which obviously fails. It's probably possible to configure nginx to get around this but the goal with these archives is to avoid having special server configuration for them, and they should be hostable anywhere. Most likely we need to come up with a new character to represent / (+ is the current candidate)

Progress was made. Main pain points right now: * Template transclusion is half-baked. Note that the library we're using (mwparserfromhell) doesn't actually do this so we have to roll our own. * Magic words (e.g. PAGENAME, NUMBEROFARTICLES etc) are not yet implemented - they would be implemented like any other template * Category parsing/generation is also half-baked, doesn't seem to take into account template transclusion (which it should, since transclusion happens before we look for categories) * Images are not yet included. We could make a script to pull images or just use the dump from archive.org for this * Page name normalization is funky. Notable in that sometimes a slash (/) indicates a subpage (e.g. GlitchDex/RB:000) and sometimes it's just a slash (e.g. Gold/Silver/Crystal). We have some special logic in both the archive builder and redirector to know the difference. * nginx normalizes URLs before passing them into our handlers so e.g. Gold%2FSilver%2FCrystal (which is what we would generate) is interpeted as Gold/Silver/Crystal (i.e. a directory structure), which obviously fails. It's probably possible to configure nginx to get around this but the goal with these archives is to avoid having special server configuration for them, and they should be hostable anywhere. Most likely we need to come up with a new character to represent / (+ is the current candidate)

kuschelyagi commented

2020-09-03 21:57:22 +00:00

Author

Owner

URL issue was resolved by replacing '/' with '+' as in: https://archives.glitchcity.info/wiki/Mimic_glitch_(Japanese_Diamond+Pearl).html

I'll make separate tickets for other issues with the wiki archival script.

URL issue was resolved by replacing '/' with '+' as in: https://archives.glitchcity.info/wiki/Mimic_glitch_(Japanese_Diamond+Pearl).html I'll make separate tickets for other issues with the wiki archival script.