FUDforum: Bug Reports » Serious NNTP Import bug in FUDforum 3.0.3

Home » FUDforum Development » Bug Reports » Serious NNTP Import bug in FUDforum 3.0.3

Show: Today's Messages :: Polls :: Message Navigator

Serious NNTP Import bug in FUDforum 3.0.3 [message #166924]

Fri, 30 March 2012 14:59

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

OK, I previously thought this was a bug with importing split attachments, but I have now discovered this is far worse problem and can happen with any post in any newsgroup.

When the NNTP import process hits certain posts, it is thrown into an infinite loop. the php and mysql processes go to 100% and the import never proceeds any further. Next time the cron job comes round, another php process is started, which immediately goes into an infinite loop. After a few minutes, I end up with hundreds of PHP processes, and a virtually unusable server. The only way out is to disable the cron job and "killall php" to recover control of the server.

My investigations so far seem to point to it being something to do with 'umlauted' characters in the User Name portion of the 'From:' field. Normally, FUDforum just truncates the name at the first umlauted character, but imports the post successfully. However, when combined with other aspects of the 'From:' field structure (which I have yet to establish), and if the e-mail address has not been seen before by FUDforum, this will cause the php script to enter the infinite loop.

I can provide an example of a 'From:' field which reliably triggers the bug if required, but I cannot post this on the public forum.

However, this is not the only trigger! Some other characters or character sequences occurring within the message can also trigger this problem, as I was hitting this problem with some Uuencoded attachments. Some detailed further investigation is needed.

I am seeing the exact same effect on two different servers:
First server is running Red Hat Enterprise Linux Server release 5.6 (Tikanga), PHP version 5.3.3, MySQL version 5.0.77.
Other server is running Debian 6.0.4, PHP version 5.3.3, MySQL version 5.1.49

In both cases, the NNTP server is running locally on the same box, so FUDforum is talking to 127.0.0.1 to sync newsgroups. First server is running INN 2.4.3, other server is running INN 2.5.2, but it seems to be specific posts in the news spool that are triggering the bug in FUDforum, rather than the NNTP server itself.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166929 is a reply to message #166924]

Sat, 31 March 2012 10:20

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

For info, this bug is NOT present in 3.0.0 or 3.0.1, but IS present in 3.0.2 and 3.0.3.

The bug was introduced at some stage between 3.0.1 and 3.0.2.

[Updated on: Sat, 31 March 2012 14:54]

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166945 is a reply to message #166929]

Fri, 06 April 2012 13:45

Multitool
Messages: 47
Registered: February 2012

Karma: 1

Member

Any news on whether this will be fixed in the next release?

I've built the forum on 3.0.1 for the time being, and all is working, but it would be nice to upgrade to the latest release if possible.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166965 is a reply to message #166945]

Tue, 10 April 2012 12:51

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

I can only fix problems that I can reproduce.
Mail me a problem message (with full headers), and I'll look into it.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #166986 is a reply to message #166965]

Fri, 13 April 2012 12:23

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

Thanks Frank!

I have now uninstalled 3.0.3 as we're running with 3.0.1 now to avoid this bug, but I will set up an instance of 3.0.3 and send you a few example messages that trigger the bug.

However, I think if you set FUDforum to mirror any newsgroup containing split attachments, you will hit the bug very soon. It seems to be certain text strings occurring within the message headers or body that are triggering it.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167033 is a reply to message #166986]

Mon, 16 April 2012 22:30

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

Frank - I have set up a test server with FF3.0.3, and can now provide you with an NNTP message which reliably and repeatably triggers this bug.

How do I get in contact with you? I can't see an option to send a Private Message?

(I cannot post the message on the public forum).

Thanks.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 [message #167037 is a reply to message #167033]

Tue, 17 April 2012 06:04

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

I'll drop you a mail.

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 - FIXED! [message #167210 is a reply to message #167037]

Fri, 18 May 2012 11:03

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

Just to update in case anyone else was tracking this, Frank has produced a patch that fixes the infinite loop issue.

http://fudforum.svn.sourceforge.net/viewvc/fudforum?view=revision&revis ion=5471

Usernames containing "invalid" characters are still truncated, but at least the import script doesn't crash now Smile

[Updated on: Fri, 18 May 2012 11:05]

Report message to a moderator

Re: Serious NNTP Import bug in FUDforum 3.0.3 - FIXED! [message #167881 is a reply to message #167210]

Wed, 31 October 2012 12:09

Multitool

Messages: 47
Registered: February 2012

Karma: 1

Member

Multitool wrote on Fri, 18 May 2012 12:03

Just to update in case anyone else was tracking this, Frank has produced a patch that fixes the infinite loop issue.

http://fudforum.svn.sourceforge.net/viewvc/fudforum?view=revision&revis ion=5471

Usernames containing "invalid" characters are still truncated, but at least the import script doesn't crash now

Unfortunately this fix breaks the code that appends [n] to the end of duplicate usernames.

The next number is appended after the previous number, so instead of "James[5]" the name becomes "James[1][2][3][4][5]".
If there are a lot of duplicates, or the original name is long, this ends up making the string exceed the max length, gets truncated, and add_user fails with a duplicate entry.

Report message to a moderator