Want to take part in these discussions? Sign in if you have an account, or apply for one below
Vanilla 1.1.9 is a product of Lussumo. More Information: Documentation, Community Support.
Following on from Urs' announcement, here's where to record anything regarding the migrated nlab.
As was noticed on the blog, currently the migrated site does not respond. Hopefully this doesn't mean that the migration alone doesn't solve the responsiveness problem.
If it does, would you know a way out?
I keep getting smoke at http://nlab.mathforge.org/nlab/show/diff/Sandbox.
Also check out http://nlab.mathforge.org/nlab/show/exercise+in+groupoidification+-+the+path+integral, where Urs put in a picture with raw HTML tags; it looks fine at http://ncatlab.org/nlab/show/exercise+in+groupoidification+-+the+path+integral.
Also compare http://ncatlab.org/nlab/new/3%26%238260%3B2-colimit, http://ncatlab.org/nlab/new/3%26%238260%3B2-colimit, http://ncatlab.org/nlab/new/3%2F2-colimit, and http://nlab.mathforge.org/nlab/new/3%2F2-colimit; the last of these does not work (404).
Hmm.
Regarding your first set, this is not a problem with the migration per se. The problem code, the raw diagram, is the same in both sets. Something deep within the code is parsing them differently. This may be due to different versions of ruby on the two systems. The logs are complaining about &s in the code.
This might pertain to your second as well. Again, this is a code issue rather than a migration one. For the first two then I see no difference - what am I supposed to see? Ah, I see why I see no difference! You've put the same link twice. Nope, I still see no difference.
For the second, again it's in the code. What may be happening is that the new way of serving the pages adds another layer to the whole process. This might strip off one level of encoding, resulting in the %2F getting converted to a / before it ought to.
Maybe my dire warnings of putting / in page names were not so unrealistic after all!
But both need investigating. Are they killer problems, though? It does seem to be something about the actual version of ruby used rather than within Instiki which makes it harder to fix, and questionable as to whether they should be fixed or whether we should learn the newer way of doing things.
Right, I've tracked down the 3/2-colimit issue. That's a bug in one of apache or passenger, depending on how you read the details here:
http://code.google.com/p/phusion-passenger/issues/detail?id=113
Seems as though the fix suggested there has made it in to the main code since putting 'AllowEncodedSlashes on' in the config file made it work. I should probably read up on that to be sure that I'm happy with us having that directive ...
I don't know if these are killers or what, but they're differences between the two sites. Why is it that they work on the old site but don't work on the new site? If we know why, and we're OK with it, then that's one thing; but it seems to me that we need to know.
And as for the link that Urs put in exercise in groupoidification - the path integral, well, we haven't used that sort of thing much, but I would sure like to use it more; it's extremly convenient. So we really should have some way to make it work.
Sorry for repeating the link in (4); of course the second one should have been http://nlab.mathforge.org/nlab/new/3%26%238260%3B2-colimit. The point (which I think you got) is that the first three work, but the last one doesn't (or didn't). (They're all links from the Sandbox, by the way.)
Okay, I was expecting to see a difference between the first two links so when I got the same on both then I wasn't sure what your point was.
I agree that the link Urs put in should work, but I disagree that it should be used more. I'd much rather go down the SVG path. If for no other reason than it makes the n-lab self-contained. What happens to pictures like this when someone downloads a copy of the n-lab? Unless they're careful, they'll just get a blank box.
But nonetheless, it's important to know why it doesn't work. I'll investigate.
I would like to use phpLaTeX, but since codecogs actually works now …. You're right about being self-contained, though, which is easy enough to arrange by storing a copy of the picture locally.
The problem with the codecogs syntax is the "non-safe" characters. It appears to be an incompatibility between markdown (aka maruku) and REXML. What I think happens is that maruku ships the page off to REXML to parse but REXML now wants its input in strictest safe mode. Whether or not it's possible or even desirable to patch either of these is moot, and I'm not even sure if I'm reading the error files correctly.
The quick fix is to encode all &s. Annoying, but if one uses as decent editor (say, via the "It's all text" plugin) then easy enough. Here's an example.
Even if people are going to use services like this, it'd be better to download the picture from codecogs and upload it again to the nLab, as you say.
I'm still confused as to why the two sites behave differently: the bad HTML works at ncatlab.org but not at nlab.mathforge.org.
You're quite right that the HTML that Urs put in (and that I left uncorrected) is improper. Since this was simply the HTML that codecogs offered the user to paste into a web page, I'd be inclined to report it to them as a bug. But in fact, it looks like they've already fixed that bug, as well as a related one that broke the link back to their site. So this problem should never happen again.
All the same, I still think that we should figure out why the two sites behave differently.
Incidentally, the best URIs to compare their behaviour now are probably http://nlab.mathforge.org/nlab/revision/exercise+in+groupoidification+-+the+path+integral/11 and http://ncatlab.org/nlab/revision/exercise+in+groupoidification+-+the+path+integral/11. I fixed the production version.
Incidentally, while follow a bunch of links at once on nlab.mathforge.org, I just got two ‘Application error \ Rails application failed to start properly’ errors in a row, at 15:52 UTC, from http://nlab.mathforge.org/nlab/revision/diff/latest+changes/151 and … erm … another one. Is this the sort of reporting that helps? (Both of these were fixed by hitting reload, and this is better than long waits and timing out at ncatlab.org.)
Yes, that's the sort of reporting that helps. I'll take a look at the logs.
The two sites behave differently because they have slightly different versions of ruby underneath. The old site uses ordinary ruby 1.8. The new site uses ruby enterprise edition. That's a version of ruby 1.8 that's been modified to use less memory (hence why we're using it). It was recommended by someone at Rails Playground. Since the issue seems to be in the libraries, rather than Instiki itself, that could well be the fault.
So the old version does some sort of HTML safety encoding? (Damn, I can't think of the right term for ‘safety encoding’ here; hopefully, you know what I mean.)
I suspect it's slightly more complicated in that's is most likely that some routine has slightly changed its expected input as to whether it was encoded or not and so the implication in "the old version does ... safety encoding" is probably wrong - the intention wasn't to make anything particularly safer, just to ensure that (to put it mathematically) the codomain of the first function was the domain of the second, rather than (as is so often the case in these languages) just something that looked like it in the right light.
But I would expect Markdown to correctly escape the URL before passing it off. That will need a little investigating to see how deep and how annoying the change is. However, in thinking about resource allocation (i.e. how much time I have) how highly do you rate it? (You've a much better global view of the 'lab than I do).
I don't think that this change affects anything else currently on the Lab (I've looked at obvious candidates, like other images), so I don't think that it's a high priority.
However, it will probably annoy Zoran whenever he posts a complicated link as <a href>, so we should track it down eventually. (And by ‘we’, I mean ‹you›, unless you tell me specifically what I could do to help.)
But don't delay the move over this.
More important is that this sort of think seems to break diffs: http://nlab.mathforge.org/nlab/show/diff/Sandbox. That we really should fix.
(I forgot to say that you should find two hits to that in a row in the ‘I smell smoke.’ error logs, just now.)
Okay, so what's happening is that on the old version it changes all bare &s to &s in URLs. In the new version it doesn't do this. I think that this is actually a bug in the new version because even if you do something like:
<p markdown="1">
Here's an ampersand: &
</p>
then it ought to parse the & inside and convert it to a & but it doesn't.
On the other hand, if the markdown tag isn't supplied then I'm not sure that it ought to change ampersands. What it ought to do is complain that what you've written isn't valid XHTML and refuse to accept it. It isn't doing that, though, it's complaining and then accepting it. I don't think that either behaviour is correct:
If you put in your own XHTML then it is your responsibility to ensure that it is valid. To ask the filter to interpret it is asking for trouble - the reason you put in XHTML directly is because you aren't happy with what the filter does! So the program should validate it and then accept it only if it is valid, otherwise reject it (with useful error messages, of course).
If you explicitly say that you want the filter to parse the contents, then it should do so in the same way as it does normally. However, this should apply to the contents of the tags, not their attributes. The validity of the extra syntax is still your responsibility.
That's just MHO!
(I've just been trying to use the  syntax to get this particular diagram to work, but I can't figure it out)
Little more testing: the markdown="1" test fails on both servers but the ampersand-in-url test only fails on the new server. It works on my course installation of Instiki as well so I find it hard to believe that it is something in the Instiki installation. That means that it must be in the version of ruby that's being used. Darn. That makes it harder to track down. Mind you, it also makes it more likely that someone else has spotted it.
Possibly you know about this, but I'm getting ‘Internal Error \ An application error occurred while processing your request.’ consistently (that is, thrice in a row) in http://nlab.mathforge.org/nlab/revision/diff/Sandbox/22. Presumably for the same reason as in my comment #17&18 above, but with a different error message.
Yes, and revision diffs 23–25 also produce this error, with the current diff producing smoke.
That's another reason why the wiki should reject bad XHTML rather than accepting it and wrapping it. I'm extremely reluctant to try it, but I would be willing to bet that if you did the <p markdown="1">&</p> test on the original lab and then looked at the diff then you'd also see smoke. Although the parser for showing a page can cope with malformed XHTML, the diff routine seems a little less robust and croaks.
Both the original installation and the new one seem to handle that just fine: http://ncatlab.org/tobybartels/show/diff/Sandbox and http://nlab.mathforge.org/tobybartels/show/diff/Sandbox. (There's still a bug for not parsing, but it doesn't crash.)
You're a braver man than I am, Gunga Din!
That's a little odd, I must say. From the error messages it did seem to be the unescaped ampersand that was causing the smoke. It's going to take a little more investigating to figure this out. I think I might add a line to the migration script that escapes ampersands to ensure that any old stuff that relies on them getting escaped goes through okay. Then any new input will trigger the warning and (hopefully) whoever put it in will notice and fix it.
But I think that to figure out what's going on, and fix it, I really need to get some familiarity with ruby to get an idea of how all the pieces fit together. Look out for a "hello world" program any time soon!
Actually, I've got an idea on that ... but it's not to do with migration so I'll start a new discussion ...
The email address in your error message indicates that the problem is first being noticed at your end. Whether or not it's being provoked by the nlab is something that's hard to figure out without more information (time of request, for example).
One thing that might be the reason is that Urs is away at the moment. I'm not sure how widely it's known that the only reason the current nlab works at all is because Urs and Toby keep restarting it. As Urs is offline, the burden of this falls on Toby and he has to sleep sometimes.
Bah, sleep is for the weak!
But Andrew, surely you restart the Lab sometimes too?
I freely confess to my weakness! Three kids have taught me the true value of sleep!
Sometimes I stoop to such menial tasks as restarting the lab, but only when others can't find the lock ...
(I certainly don't restart it at anything like the rate you and Urs do! I think I'll grep through the logs to find out just how often the two of you have to do it.)
Had to do it.
THANKS! Andrew. For the migration!!
Ah, you noticed.
The DNS change is still propagating through the system (it hasn't changed on my machine yet) so it's a bit obvious that you get redirected from ncatlab.org to nlab.mathforge.org. Hopefully that'll be in the system before all the stateside folks wake up.
Please let me know of any glitches! I'm monitoring the error log as well, but that also gets filled with debugging stuff from Instiki (which aren't actual errors just information about what's going on), so it's useful to know when something significant has happened. Remember that a useful bug report includes the time (and timezone!), the link, the ip (or if you don't know that, something that might help me figure out what computer you were on), and a description of what happened.
I have posted an announcement of the migration
http://golem.ph.utexas.edu/category/2009/09/nlab_migration_done.html
Here is a
BUG REPORT
Could it be that the case-sesnitivity of the entry titles has been lost?
I had (maybe a bad idea, but for "historical reason") two different entries on my personal web whose title differed only in spelling
But this link
http://nlab.mathforge.org/schreiber/show/differential+nonabelian+cohomology
takes me to the capitalized-named entry. And I can't access the lower case entry at all.
(Not much harm done, as I have a backup copy of that entry on my other, "private" web).
Sorry: "differed only in capitalization", of course :-)
Another
BUG REPORT
long entry titles have apparently been truncated.
This one here for instance:
Yet another
BUG REPORT
German time 11:50, I repeatedly get
Application error Rails application failed to start properly
While working on my personal web.
And now I get no reaction at all...
Here is something that is not a bug report:
I have now worked intensively with the new Lab for a longer while and it works great. I am so glad that we finally migrated...
(I had to go and teach for a couple of hours - I was dreading getting back and finding that it had crashed again, but fortunately it seems to be okay.)
The 11:50 I think was due to a rogue process. It seems that if you change the settings and reload apache while instiki is doing something then it might go a little mad. So I should post a note for anyone who has to restart things to leave some time to monitor processes to make sure that they die properly.
Losing capitalisation is irritating. It could be an issue with the database, in which case it'll be extremely annoying to fix but given the other problem then I'll have to do it anyway so may as well fix that. If it's within the ruby code then it could take a bit to track down. However, if it's in the ruby code then I can manually go into the database and rename one of the pages for you. Actually, I can do that anyway. I suspect that there may be a few things that require me to hack the database so I might save them up for a single shot, unless there are urgent ones.
The truncation is almost certainly a database issue. I strongly suspect that it is the same issue as with the stylesheets (see above). MySQL is a bit more rigid in its enforcement of database column types than sqlite3 so whereas sqlite3 says "You want to put 80 characters in a 60 character pot? Fine, go ahead!", MySQL says "Okay, I'll take the first 60 and ditch the rest." When we came across this for the stylesheets, I looked at the values for the other fields but thought they were enough. Little did I reckon with 'Background fields in twisted differential nonabelian cohomology"!
I suspect that it is only the official title that is truncated. Redirects shouldn't be covered by this, so we could easily have a 60 character limit on titles but then put in a more suitable redirect. Jacques will have his own opinion on whether or not page titles should be any length or should have a fixed upper length. A quick search shows that a practical upper limit on URLS is about 2000 characters. Please no-one take this as a challenge! (For one, I haven't extended the limit yet).
We have 14 pages with titles longer than 60 characters (unless anyone's added some since I did the migration). The longest is a whopping 95 characters. I can "up" the limit to something long enough to accommodate these (say, 100) or we can rename them. I think that allowing completely arbitrary lengths is probably a Bad Idea, and I would say that 100 is probably long enough. However, the fact that this is now being enforced should be made plain. It's probably a good idea if I add this to the FAQ or somewhere like that.
Ah, just found a possible for the case sensitivity here. This implies that the comparisons (foreach (page) {is page equal.to page.you.want}) are case insensitive meaning that the lowercase one just happens to be first in the list. But what's most significant about this is that the data itself is case sensitive which is great as it means I don't have to manually fix that (phew!).
The bit I don't understand is that the default character set is latin1_swedish_ci. Swedish??!!??
Thanks, Andrew.
I tried to have a truancated "official" entry title with a longer redirect name. But that didn't work either. (The redirect command had no effect).
Don't worry about renaming my personal entries, I have taken care of that, or will still.
What are these very long titles? Possibly those with multiple occurence of "infinity" in them, right? In that case, since we agreed not to use the single character infinity-symbol instead I would say we shoul "up" the limit to accomodate entries about infinity-categories of infinity-somethings of infinity otherthings, since that may occur again.
The overlong titles are:
I'm surprised that the redirect didn't work. Maybe redirects are stored somewhere as well - makes sense, I suppose. Ah, yes. They are in a table called 'wiki_references' and sure enough, they have a character length of 60. Since the redirect name becomes part of the URL, it makes as much sense to put a limit on them as on page names (i.e. some, but not so it becomes excessively inconvenient).
I'll ask Jacques for a quick opinion on this as well.
I am just this minute getting consistently ‘Application error \ Rails application failed to start properly’ from http://ncatlab.org/nlab/show/reality+check, after 5 or so reloads in a row.
Yeah, I think that it just crashed.
And it's back now.
I just had to restart the web server. It's too late for me to figure out what exactly happened, but I'll try and chase it down in the morning. One 'instiki' process had gone rogue and was eating up all the memory, but why it was doing that I've no idea. Killing it brought down the memory usage, but then the web server didn't like the fact that a process had gone from underneath it so I had to restart that as well.
I've said all along that being on the new server isn't a magic bullet that will instantly speed everything up, but is rather giving us the opportunity to speed everything up, mainly by giving us better control and better access to monitor what's going on. It's not perfect, but we have a better chance of getting it closer now.
Recently Revised is back, and using it doesn't slow everything down to a crawl. That alone makes this worth while to me!
Now if we can just fix the field size restrictions, then I'll be happy.
By the way, I just noticed that the new server is in a different Time Zone??? UTC would be nice. (For the record, it is now 23:17 UTC.)