FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » nntp import reloads (duplicates) posts randomly (possible bug - fix provided - your call)
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
nntp import reloads (duplicates) posts randomly [message #36066] Fri, 02 March 2007 00:12 Go to next message
rdthoms is currently offline  rdthoms   United States
Messages: 12
Registered: January 2007
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Had a problem where the newsgroup link would seem to randomly reload (duplicate) a bunch of posts into the forum. Here is what I found and changed:

in the parse_msgs function in include/nntp.inc there is some logic where the function compares the last read message id ($start_id) with the first and last message ids available from your news server. It looks like this:

if ($start_id && $start_id > $this->group_first && $start_id <= $this->group_last) {
  $this->group_first = $start_id;
}


or what is really happening in psuedo code:

if (last_read > first_available && last_read <= last_available) {
  start reading from last_read
} else {
  start reading from first_available
}


The "random" problem would occur when the nntp server would somehow "lose" the last few posts. For example I would get a good sync reading messages 900-1000 from the nntp server and then start_id would be 1000. So far so good.

Then when the next sync happened sometimes for some reason the last message was 998 instead of the expected 1000 (maybe they deleted some spam or something).

Anyway, in that case the logic in the above snippet of code fails to start reading from last_read and instead starts reading from first_available! So now messages 900-998 get read again!

So I changed the code to:

if ($start_id && $start_id > $this->group_first) {
  $this->group_first = $start_id;
}


I don't see any reason to check that the local start_id is less than the group_last. If it happens that that is not the case then it seems to safely run and not read any messages.

I hope you can understand the issue and consider making the change permanent.

Thanks,
Richard

Re: nntp import reloads (duplicates) posts randomly [message #36094 is a reply to message #36066] Sun, 04 March 2007 12:39 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Thanks for the detailed analysis, I've made the suggested change to CVS. Originally it was designed to prevent duplicate importing, but I can see how removal of messages could confuse the code.


FUDforum Core Developer
Re: nntp import reloads (duplicates) posts randomly [message #36209 is a reply to message #36066] Thu, 08 March 2007 06:49 Go to previous messageGo to next message
greg is currently offline  greg   China
Messages: 18
Registered: February 2007
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
yes, i have the same problem, hope it's can fixed soon.
Re: nntp import reloads (duplicates) posts randomly [message #36225 is a reply to message #36209] Thu, 08 March 2007 19:33 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
it was already fixed in cvs.

FUDforum Core Developer
Re: nntp import reloads (duplicates) posts randomly [message #36231 is a reply to message #36066] Thu, 08 March 2007 19:43 Go to previous messageGo to next message
greg is currently offline  greg   China
Messages: 18
Registered: February 2007
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
what is CVS?
Re: nntp import reloads (duplicates) posts randomly [message #36233 is a reply to message #36231] Thu, 08 March 2007 22:26 Go to previous message
rdthoms is currently offline  rdthoms   United States
Messages: 12
Registered: January 2007
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
CVS is the code repository. It means he's made the change permanent and any new builds after this date should have the fix.

However, if you think you're running into the same exact problem as me you can make this fix yourself on your installation. Even if you are not seeing the same exact problem, this fix should not harm anything.

In the data directory area edit the file include/nntp.inc.

Look for a piece of code (in my version it's around line 350-400) like:

if ($start_id && $start_id > $this->group_first && $start_id <= $this->group_last) {
  $this->group_first = $start_id;
}


and change it to:

if ($start_id && $start_id > $this->group_first) {
  $this->group_first = $start_id;
}


So make that edit and see if that fixes your problem...


Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Database or file to store message
Next Topic: Can not display online user name
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sun Oct 22 04:31:37 EDT 2017

Total time taken to generate the page: 0.00727 seconds