FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » USENET sync breaks and goes into endless cycle when it encounters attachment
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
Re: USENET sync breaks and goes into endless cycle when it encounters attachment [message #158322 is a reply to message #158321] Wed, 18 February 2009 02:39 Go to previous messageGo to next message
kerryg is currently offline  kerryg   Canada
Messages: 157
Registered: September 2008
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Hi Frank. With the help of tech support at the server we ran through the checklists; the tables are fine, they reckon the db shouldn't need resetting, the only errors visible alternate between the two errors I've previously posted; I've bumped up the memory and timeouts in php.ini to pretty high levels, checked consistency a million times, compacted messages, rebuilt themes, nothing seems to be affecting it. Here's one slightly more detailed error message that happened as I was rebuilding the search index (I think it was successful but it's hard to tell as it didn't give me a confirmation) -

Wed Feb 18 7:15:51 GMT 2009	(/home/kerrygal/public_html/ParisForums/index.php:75
/home/kerrygal/public_html/ParisForums/index.php:135
/home/kerrygal/public_html/ParisForums/theme/default/selmsg.php:727
/home/kerrygal/public_html/ParisForums/index.php:1188
) 1053: Server shutdown in progress
Query: SELECT count(*) FROM fud26_msg m INNER JOIN fud26_thread t ON m.thread_id=t.id INNER JOIN fud26_forum f ON t.forum_id=f.id INNER JOIN fud26_cat c ON f.cat_id=c.id INNER JOIN fud26_group_cache g1 ON g1.user_id=0 AND g1.resource_id=f.id LEFT JOIN fud26_mod mm ON mm.forum_id=f.id AND mm.user_id=0 WHERE m.apr=1 AND f.id=21 AND t.id=6348 AND t.replies=0 AND (mm.id IS NOT NULL OR (g1.group_cache_opt & 2) > 0)
_GET: array ( 't' => 'selmsg', 'frm_id' => '21', 'th' => '6348', 'reply_count' => '0', )
Server Version: 


Since one single post is breaking the sync of an entire forum and rendering it null and void - is there any way to get an emergency "killswitch" somewhere, a way to say "just skip this post if it's a problem, increment one and carry on"?

[Updated on: Wed, 18 February 2009 02:51]

Report message to a moderator

Re: USENET sync breaks and goes into endless cycle when it encounters attachment [message #158323 is a reply to message #158322] Wed, 18 February 2009 02:51 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
You can easily skip messages (and import them afterwards) by changing the group's pointer file. This file contains the next message number to import and is located in your data/errors/.nntp directory.
Re: USENET sync breaks and goes into endless cycle when it encounters attachment [message #158325 is a reply to message #158323] Wed, 18 February 2009 03:01 Go to previous messageGo to next message
kerryg is currently offline  kerryg   Canada
Messages: 157
Registered: September 2008
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
EXCELLENT news! Thanks, I'll get on it!

UPDATE: Well, that would be my "killswitch" right there, I'm 1000 posts past that one and getting closer to current date as we speak. Success at last.

I can't tell you how many times since last summer I've come *this close* to throwing in the towel and deleting the whole thing and writing off the time invested over this issue - so many, many thanks again, Frank.

[Updated on: Wed, 18 February 2009 03:46]

Report message to a moderator

Re: USENET sync breaks and goes into endless cycle when it encounters attachment [message #158343 is a reply to message #158325] Wed, 18 February 2009 16:30 Go to previous messageGo to next message
kerryg is currently offline  kerryg   Canada
Messages: 157
Registered: September 2008
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
I have now completed the sync of all three forums; even the largest one with the most problems is now syncing - up to and including today's posts.

I'm going to rip it all down now and reload everything from scratch to make sure all the problems are ironed out, and at that point I'll post it as resolved. But as far as I'm concerned, it's looking like a major "win" for the good guys - enormous thanks, Frank!

[Updated on: Wed, 18 February 2009 16:32]

Report message to a moderator

Re: USENET sync breaks and goes into endless cycle when it encounters attachment [message #158359 is a reply to message #158343] Thu, 19 February 2009 15:52 Go to previous message
kerryg is currently offline  kerryg   Canada
Messages: 157
Registered: September 2008
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Followup: I've created new subforums and started a fresh sync. Two out of three previously broken groups synced fast and flawlessly again, and the third (the biggie with 47,000 posts) is still bringing them across but appears to be working just fine (I'll update this when it's done), flying through previous "problem posts" with no errors. The MIME stuff junks the posts up a lot (as does the *ridiculous* level of >>> quotes this group can get up to) but we'll worry about that in the future; as long as the forums sync, I know I can re-sync later if the MIME encoding issue is tackled and clean it all up then.

I've got max posts = 100 per sync. At that speed I get the occasional "server has gone away" error between every say three to seven syncs, but importantly they're now "benign". I can tell in advance when it's a benign error because it refers to the first post of a given block of 100 posts being transferred - those errors cause no cycling problems, sync just continues as the db catches up.

Being able to maintain a speed of 100 per sync (one sync per minute, 6000 posts per hour) is *very* good - at that speed that's still eight solid hours of syncing to get the posts across. But I'm starting to feel like I can take my eyes off it and trust it.

[Updated on: Thu, 19 February 2009 15:53]

Report message to a moderator

Quick Reply
Formatting Tools:   
Pages (2): [ «    1  2]  Switch to threaded view of this topic Create a new topic
Previous Topic: Small bug - "Members" list does not sort by join date.
Next Topic: Small issues probably related to the new search box
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Tue Oct 17 16:45:38 EDT 2017

Total time taken to generate the page: 0.00720 seconds