FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » How To » maillist debug (how to debug the maillist import)
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
maillist debug [message #163014] Thu, 16 September 2010 18:03 Go to next message
lbrigman124 is currently offline  lbrigman124   United States
Messages: 26
Registered: March 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
I have maillist.php running well enough during my testing
that we have turned it on for production.

The problem is that right after we have done that I find
that maillist.php is randomly dropping import request on the
floor without anything being logged.

I have this configured from a procmail filter so the
maillist.php has multiple forums to import and does so as
messages arrive. I also have procmail logging each delivery
to know that the message is being delivered to the maillist.php
script.

I am running 3.0.1 but have merged all the changes from
the svn tree into it.

What would be your recommendation for debugging this random
occurring bug?
Re: maillist debug [message #163015 is a reply to message #163014] Fri, 17 September 2010 02:09 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
Log every single message. If one is not imported, you will be able to zoom in on the code that casing the problem.

Note that FUDforum 3.0.2 will show mailing list errors in the Log Manager ACP (different file).

To log additional messages is easy, just call:
fud_logerror('New message that will appear in log', 'mlist_errors');

Re: maillist debug [message #163040 is a reply to message #163014] Mon, 20 September 2010 13:37 Go to previous messageGo to next message
lbrigman124 is currently offline  lbrigman124   United States
Messages: 26
Registered: March 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
adding the fud_errorlog call to maillist.php in spots that should be executed is not showing anything getting logged.

It seems that all my test cases work and then real messages
get dropped.

It looks like the real messages have a subject length >100 chars
long.

Now that I have tested this, it seems to be the case.

What is the limit on the subject/topic title?
Re: maillist debug [message #163043 is a reply to message #163040] Mon, 20 September 2010 16:05 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
The subject length rings a bell. Someone else complained about it, but I don't think it was ever fixed.

We need to trim the subject line down to 100 characters of increase the column's length in fud30_msg. Maybe both! Are there any RFCs that state that subjects cannot be longer than X chars?

Would you be able to prepare a patch for us?

Regards.

Frank
Re: maillist debug [message #163044 is a reply to message #163043] Mon, 20 September 2010 16:17 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
OK, here's the details:

We fixed a similar problem for Usenet posts as per http://fudforum.svn.sourceforge.net/viewvc/fudforum/trunk/install/forum_dat a/include/nntp.inc?r1=4899&r2=4915
The same fix must be applied to the Mailing list and XML importers.
Re: maillist debug [message #163045 is a reply to message #163043] Mon, 20 September 2010 16:51 Go to previous messageGo to next message
lbrigman124 is currently offline  lbrigman124   United States
Messages: 26
Registered: March 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
naudefj wrote on Mon, 20 September 2010 13:05
The subject length rings a bell. Someone else complained about it, but I don't think it was ever fixed.

We need to trim the subject line down to 100 characters of increase the column's length in fud30_msg. Maybe both! Are there any RFCs that state that subjects cannot be longer than X chars?

Would you be able to prepare a patch for us?

Regards.

Frank


It looks like the RFC in question is RFC2822.
The info on that seem to state that it MUST be no more than
998 chars and SHOULD be 78 or less not including CR/LF.
http://stackoverflow.com/questions/1592291/what-is-the-email-subject-length -limit

Now that I know for sure that the current limit is 100, I'll see
what I can do for a patch.

What I am really suprised about is that it is silently dropping these messages.
Re: maillist debug [message #163047 is a reply to message #163014] Mon, 20 September 2010 17:34 Go to previous messageGo to next message
lbrigman124 is currently offline  lbrigman124   United States
Messages: 26
Registered: March 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Here is a patch for limiting the subject to 100 chars (the size
of the field). It really should be done against a variable
that depends on the size of the table/field in question but I
am not good enough with php/sql to get that done.

--- maillist.php 2010-09-16 09:17:18.000000000 -0700
+++ maillist.php.new 2010-09-20 14:24:24.000000000 -0700
@@ -526,7 +527,7 @@
}

fud_wordwrap($msg_post->body);
- $msg_post->subject = htmlspecialchars(apply_custom_replace($emsg->subject));
+ $msg_post->subject = htmlspecialchars(substr(apply_custom_replace($emsg->subject),0,100)); //limit subject line to MAX field lenght of 100)
if (!strlen($msg_post->subject)) {
mlist_error_log('Blank Subject', $emsg->raw_msg);
$msg_post->subject = '(no subject)';
Re: maillist debug [message #163048 is a reply to message #163014] Mon, 20 September 2010 17:37 Go to previous messageGo to next message
lbrigman124 is currently offline  lbrigman124   United States
Messages: 26
Registered: March 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
wow, that htmlized version of the patch is really messed up.
I have attached the patch as a file.
  • Attachment: maillist.zip
    (Size: 0.44KB, Downloaded 156 times)
Re: maillist debug [message #163132 is a reply to message #163048] Fri, 01 October 2010 16:46 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
Thanks!
A variation of your patch was applied.
Details @ http://fudforum.svn.sourceforge.net/fudforum/?rev=5013&view=rev
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Require approval of all posts
Next Topic: setting up user groups and permissions
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Oct 21 11:36:20 EDT 2017

Total time taken to generate the page: 0.00735 seconds