FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » Serious NNTP Import bug in FUDforum 3.0.3
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
Serious NNTP Import bug in FUDforum 3.0.3 [message #166924] Fri, 30 March 2012 10:59 Go to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
OK, I previously thought this was a bug with importing split attachments, but I have now discovered this is far worse problem and can happen with any post in any newsgroup.

When the NNTP import process hits certain posts, it is thrown into an infinite loop. the php and mysql processes go to 100% and the import never proceeds any further. Next time the cron job comes round, another php process is started, which immediately goes into an infinite loop. After a few minutes, I end up with hundreds of PHP processes, and a virtually unusable server. The only way out is to disable the cron job and "killall php" to recover control of the server.

My investigations so far seem to point to it being something to do with 'umlauted' characters in the User Name portion of the 'From:' field. Normally, FUDforum just truncates the name at the first umlauted character, but imports the post successfully. However, when combined with other aspects of the 'From:' field structure (which I have yet to establish), and if the e-mail address has not been seen before by FUDforum, this will cause the php script to enter the infinite loop.

I can provide an example of a 'From:' field which reliably triggers the bug if required, but I cannot post this on the public forum.

However, this is not the only trigger! Some other characters or character sequences occurring within the message can also trigger this problem, as I was hitting this problem with some Uuencoded attachments. Some detailed further investigation is needed.

I am seeing the exact same effect on two different servers:
First server is running Red Hat Enterprise Linux Server release 5.6 (Tikanga), PHP version 5.3.3, MySQL version 5.0.77.
Other server is running Debian 6.0.4, PHP version 5.3.3, MySQL version 5.1.49

In both cases, the NNTP server is running locally on the same box, so FUDforum is talking to 127.0.0.1 to sync newsgroups. First server is running INN 2.4.3, other server is running INN 2.5.2, but it seems to be specific posts in the news spool that are triggering the bug in FUDforum, rather than the NNTP server itself.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166929 is a reply to message #166924] Sat, 31 March 2012 06:20 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
For info, this bug is NOT present in 3.0.0 or 3.0.1, but IS present in 3.0.2 and 3.0.3.

The bug was introduced at some stage between 3.0.1 and 3.0.2.

[Updated on: Sat, 31 March 2012 10:54]

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166945 is a reply to message #166929] Fri, 06 April 2012 09:45 Go to previous messageGo to next message
Multitool is currently offline  Multitool
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
Any news on whether this will be fixed in the next release?

I've built the forum on 3.0.1 for the time being, and all is working, but it would be nice to upgrade to the latest release if possible.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166965 is a reply to message #166945] Tue, 10 April 2012 08:51 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
I can only fix problems that I can reproduce.
Mail me a problem message (with full headers), and I'll look into it.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166986 is a reply to message #166965] Fri, 13 April 2012 08:23 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
Thanks Frank!

I have now uninstalled 3.0.3 as we're running with 3.0.1 now to avoid this bug, but I will set up an instance of 3.0.3 and send you a few example messages that trigger the bug.

However, I think if you set FUDforum to mirror any newsgroup containing split attachments, you will hit the bug very soon. It seems to be certain text strings occurring within the message headers or body that are triggering it.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167033 is a reply to message #166986] Mon, 16 April 2012 18:30 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
Frank - I have set up a test server with FF3.0.3, and can now provide you with an NNTP message which reliably and repeatably triggers this bug.

How do I get in contact with you? I can't see an option to send a Private Message?

(I cannot post the message on the public forum).

Thanks.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167037 is a reply to message #167033] Tue, 17 April 2012 02:04 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
I'll drop you a mail.
Re: Serious NNTP Import bug in FUDforum 3.0.3 - FIXED! [message #167210 is a reply to message #167037] Fri, 18 May 2012 07:03 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
Just to update in case anyone else was tracking this, Frank has produced a patch that fixes the infinite loop issue.

http://fudforum.svn.sourceforge.net/viewvc/fudforum?view=revision&revis ion=5471

Usernames containing "invalid" characters are still truncated, but at least the import script doesn't crash now Smile

[Updated on: Fri, 18 May 2012 07:05]

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 - FIXED! [message #167881 is a reply to message #167210] Wed, 31 October 2012 08:09 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
Multitool wrote on Fri, 18 May 2012 12:03
Just to update in case anyone else was tracking this, Frank has produced a patch that fixes the infinite loop issue.

http://fudforum.svn.sourceforge.net/viewvc/fudforum?view=revision&revis ion=5471

Usernames containing "invalid" characters are still truncated, but at least the import script doesn't crash now Smile


Unfortunately this fix breaks the code that appends [n] to the end of duplicate usernames.

The next number is appended after the previous number, so instead of "James[5]" the name becomes "James[1][2][3][4][5]".
If there are a lot of duplicates, or the original name is long, this ends up making the string exceed the max length, gets truncated, and add_user fails with a duplicate entry.
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167892 is a reply to message #166924] Thu, 01 November 2012 14:31 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
A better fix...
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167893 is a reply to message #167892] Fri, 02 November 2012 02:10 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Great, thanks!
Can you commit it or should I?
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167894 is a reply to message #167893] Fri, 02 November 2012 05:37 Go to previous messageGo to next message
Multitool is currently offline  Multitool   United Kingdom
Messages: 47
Registered: February 2012
Karma: 1
Member
add to buddy list
ignore all messages by this user
naudefj wrote on Fri, 02 November 2012 06:10
Great, thanks!
Can you commit it or should I?


Can you commit it? I don't have the necessary accounts Embarassed

Thanks!
Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167895 is a reply to message #167894] Fri, 02 November 2012 05:38 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Done @ http://fudforum.svn.sourceforge.net/fudforum/?rev=5560&view=rev

PS: Let me know if you need access.
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Cache problem when logging in and pressing "Back"
Next Topic: Newsgoups and XML aggregation
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Dec 13 06:21:48 EST 2017

Total time taken to generate the page: 0.00849 seconds