FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » NNTP Import Stalls
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
NNTP Import Stalls [message #164945] Thu, 07 April 2011 04:57 Go to next message
jm2morri is currently offline  jm2morri   
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Hello,

...similar issue...

The nntp import dies on a specific message of a news group that I am importing. No error message on command line or in error logs that I can see. mysqld is running at close to 100% CPU usage.

Cheers,

James.
Re: NNTP Import Stalls [message #164947 is a reply to message #164945] Fri, 08 April 2011 05:33 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
OK, a bit more info. I put in some debugging echo statements in the script nntp.php. It stalls on the line where the function match_user_to_post() is called.

I printed out the inputs to that function and noticed two things that may be issues:

1) The from_name field contains a Scandinavian name with an accented character. When it gets echoed it shows up as a question mark in bash but I can see (using Thunderbird) that it actually is the lower case "o" with a slash through it (not sure what that character is called.

2) The user_id field is blank. Not sure if that is normal or not.

Also, the nntp_opt is 106 though I assume that is OK.

Does that help nail down what might be the problem? I really need to get this fixed as my forums are getting out of sync and my users won't like that....

James.
Re: NNTP Import Stalls [message #164948 is a reply to message #164947] Fri, 08 April 2011 05:38 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
OK, I managed to figure out that it was indeed that Scandanavian character that was causing problems. I added this hack to nntp.php right before the call to match_user_to_post()

if ($msg_post->post_stamp == ID )
{
  $emsg->from_name = "name without accent";
}


And it then imported the message (and all the others that had queued up) just fine. So that function didn't like that character. That should be fixed, though I have no idea how to do that.

James.
Re: NNTP Import Stalls [message #164954 is a reply to message #164948] Fri, 08 April 2011 07:54 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
I've setup a test forum and imported the post without problem (from details in your mail).

Your DB may not accept UTF-8 characters.
Mail me access details if you want to to check it out.
Re: NNTP Import Stalls [message #164955 is a reply to message #164954] Fri, 08 April 2011 08:42 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
naudefj wrote on Fri, 08 April 2011 07:54

Your DB may not accept UTF-8 characters.


It should I think. There are lots of german language text in other parts of the forum.

Is there a way I can check this?

James.


Re: NNTP Import Stalls [message #164956 is a reply to message #164955] Fri, 08 April 2011 09:43 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Good question, maybe:
SQL> SHOW VARIABLES LIKE 'character_set_%';

More about the ΓΈ character: http://en.wikipedia.org/wiki/%C3%98
Re: NNTP Import Stalls [message #164957 is a reply to message #164956] Fri, 08 April 2011 09:49 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Here is the result from that command. JPG attached.

Variable_name Value
character_set_client utf8
character_set_connection utf8
character_set_database latin1
character_set_filesystem binary
character_set_results utf8
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/

Does that help?

James.
Re: NNTP Import Stalls [message #164958 is a reply to message #164957] Fri, 08 April 2011 10:04 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
It looks pretty much UTF-8 to me.

You need to debug match_user_to_post() and possibly create_new_user() in include/scripts_common.inc.
It's just a couple of lines - should only take a couple of minutes of your time.
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Problems with MySQL 5.5
Next Topic: E-mail to admin not working
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Dec 13 15:49:28 EST 2017

Total time taken to generate the page: 0.00706 seconds