FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » Problem with pruning topics
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
Problem with pruning topics [message #26907] Sat, 20 August 2005 06:59 Go to next message
falcon15121978 is currently offline  falcon15121978   United Kingdom
Messages: 5
Registered: August 2005
Karma: 0
Junior Member
Hi,

This is my first post to the forum, and please excuse me if this is the wrong place to post it.

To start off with i had FUDForum version 2.3.x. My basic problem was i could not prune old threads. When select prune threads and select threads older than 5 months it show me another page saying there are 4000 entries to be deleted, are you sure.

I press the yes to confirm the delete. The browser(firefox) is waiting for some information back from the forum. After a while. the browser just stops / disconnects from the server.

Then i hit a back and go back to thread pruning and select threads older than 5 months, it shows me approx 3960. So manages to delete a very few.

Assuming that it was a problem or bug which was present in the old version of the forum, I upgraded it to the current version. There did not seem to be any obvious issues during the upgrade process. I did not get any sort of feedback on to the browser after the upgrade, nor did it start the consistency check automatically.

After got about 3 or 4 hours the browser disconnected. and when i tried reconnecting the forum seemed to have updated.

With the updated forum, i still have the same issue with pruning threads. I have approx 800000 messages and 52000 topics. So i would like to clean up the forum and make it light weight.

Below is my system configuration.

PHP built On: 2.4.18-1-686-smp #1 SMP Sun Feb 1 04:12:27 MST 2004 i686
Database Version: 3.23.49-log
PHP Version: 4.3.3
Web Server: Apache/1.3.26 (Unix) Debian GNU/Linux PHP/4.3.3
WebServer to PHP interface: apache
Forum Version: 2.6.15
Relavent PHP Settings:
Safe Mode: OFF
Open basedir: none
Display Errors: OFF
File Uploads: ON
Magic Quotes: ON
Register Globals: ON
Output Buffering: No
Disabled Functions:none
PSpell Support: No
Zlib Support: Yes

Thanks a lot in advance. I have just recently started to maintain this fudform that i am talking about. So if i am forseeing something which is pretty obvious please excuse me.

Bye the way, every time i try to do a consistency check it does come up with some erros and says deleted so many rows etc. but nothing critical/failed as such.

regards
Re: Problem with pruning topics [message #26909 is a reply to message #26907] Sat, 20 August 2005 15:14 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Topic prunning is not a fast process, your browser times out because while the script is running it is not sending output and the browser incorrectly assumes the page is broken. In 2.7.0 topic pruning should be a lot faster.

FUDforum Core Developer
Re: Problem with pruning topics [message #26916 is a reply to message #26909] Sat, 20 August 2005 20:04 Go to previous messageGo to next message
falcon15121978 is currently offline  falcon15121978   United Kingdom
Messages: 5
Registered: August 2005
Karma: 0
Junior Member
Hi,

Yes i know topic pruning could be a long process. While the pruning is going on, i have been monitoring the database activity by issuing a

mysql -e 'show processlist'

After a while nothing seems to be happening at the database level either. That is why i am worrried.

May i take this opportunity to also ask if FudForum has got any issues running and locking files from a NFS mounted location. All my forum files, are located on an NFS mount. I have roughly 4 machines mounting the same location and serving the forum for load balancing.

When users try to make a post(with the new version 2.6.15), their replies get mixed up with other user's replies. About 2 years back when we had originally set the forum using version 2.3 our old administrator said there were issues with locking files on nfs mount and he wrote a C module to overcome this. Are you aware of any such issues with the forum. If so has it been rectified.

If it has been rectified, what do you think, is causing my users to have their post mangled with other user's postings.

Hope i am not asking for too much. Thanks in advance. And thanks a lot for the earlier response. That was a quick response.

I have had to roll back to the old version of the forum (2.3) as i had so many complaints.. And another strange thing that happened after the upgrade was that my users all of a sudden started getting emails, for new postings even though they never subscribed.

cheers.

regards
Re: Problem with pruning topics [message #26917 is a reply to message #26916] Sat, 20 August 2005 20:16 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
FUDforum does use any file locks, so location of files on NFS would not pose a problem, it instead uses a special table in the database to implement write locks for messages file.

The problem maybe the fact that NFS is caching the file, subsequently when the forum tries to get the latest position in the file, it actually gets an old, cached value and results in the error that you are seeing.

How often does this problem occur?

As far as e-mail notificiations that is something I've never seen before or have heard reports about. What are you forum settings when it comes to notifications, are you certain that people who claim to be not-subscribed, really aren't?


FUDforum Core Developer
Re: Problem with pruning topics [message #26918 is a reply to message #26917] Sat, 20 August 2005 21:12 Go to previous messageGo to next message
falcon15121978 is currently offline  falcon15121978   United Kingdom
Messages: 5
Registered: August 2005
Karma: 0
Junior Member
That was a good pointer you gave me on NFS.. caching. That will keep me busy for a couple of days.. till i get to the bottom of it. Thanks for that.

But does not fudforum, write to any of the files, under normal user operation, session info or anything at all ?. If not that is cool.

You mentioned about catching of files.. and getting the latest position in the file. Which file are you referring to. I am a bit confused. My mysql database is not on NFS. It is on a dedicated mysql server.

The problem with user posting getting truncated or getting mangled is intermittent. I personally did not experience it, but i saw some of the postings. There was a link for "report message to moderator" which appears on the last line of the message far right. It was out of position. and appeared along with the user posting. It looked as if a bit of code was missing, and the table structure was lost or something.

Regarding the sudden influx of emails being sent out. i had users complain about receiving 20 emails in half an hour or so. I personally went into their user profiles and they didn't not seem to have any email notification enabled or subscriptions.

That is about when i decided to rollback, i didn't know how extensive this problem was, and i didn't want it to crash my email server and brings thing to a halt.

Also some of them had their default view changed from flat with message list to some other view, so that they had to click on each message header to see the entry.

Do you think it will be possible to create a new FudForum 7.0 installation, and then import the user details onto it? I am not desparate about the postings. I can get rid of them.

2.6.15 seemed to use a lot of temporary tables and was quite fast. Do you thing it is work upgrading it to 2.7 and then prune all the messages. Or just install a new version and migrate the user details on to the new installation of fudforum.

By the way, can i use the template for displaying threads from the old forum, on to the new forum. My users are so fussy about the old look of the forum. Atleast the displaying of the thread.

My posting has now moved on from a bug report to more casual questions. If you want me to move over to another forum other than the bugs report please let me know.

thanks.. i am very impressed by your response times..

regards
a new system admin.
(i am into my second month of employment)
Re: Problem with pruning topics [message #26922 is a reply to message #26918] Sun, 21 August 2005 15:49 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
The forum stores sessions in the database, aside from messages no other concurrent write operations are possible that could cause problems.

Well, you cannot directly import data from 2.3 into 2.7, you need to run the upgrade script, which will the do the job.


FUDforum Core Developer
Re: Problem with pruning topics [message #26926 is a reply to message #26922] Mon, 22 August 2005 06:53 Go to previous messageGo to next message
falcon15121978 is currently offline  falcon15121978   United Kingdom
Messages: 5
Registered: August 2005
Karma: 0
Junior Member
Again writing of messages even though it is concurrent, it is stored in the database, so i don't see a problme in that.

Correct me if i am wrong.

If i may ask a last question.

at the moment i have fudforum 2.3. I am monitoring mysql operations using a mysql -e 'show processlist'

Say if I try to prune roughly about 1000 threads. Initially i seem some mysql database activity going on.

After some time, all mysql activity stops. and my messages do now get deleted. Is there anything i can do about it. It is one of the join commands which it attempts to execute before it dies.

my version of mysql is 3.23. Any suggestions ?

I will investigate into the possible chaching problem you mentioned about NFS. by the way.. if you want to see one disfigured posting i can always attach an image of it.. or copy paste it.

Thanks for all your help..

Thanks..
Re: Problem with pruning topics [message #26932 is a reply to message #26926] Mon, 22 August 2005 13:08 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Well, it could be that when the activity stops the browser had timed out an the message deletion process is effectivly stopped.

FUDforum Core Developer
Re: Problem with pruning topics [message #26934 is a reply to message #26932] Mon, 22 August 2005 13:18 Go to previous messageGo to next message
falcon15121978 is currently offline  falcon15121978   United Kingdom
Messages: 5
Registered: August 2005
Karma: 0
Junior Member
So what do you reccomend that i do, so that the pruning process carries on and deletes the threads i requested it to delete.
Re: Problem with pruning topics [message #26936 is a reply to message #26934] Mon, 22 August 2005 13:23 Go to previous message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Well, there is a quick hack that will allow you very quickly prune messages. You do the following:

DELETE FROM messages_table WHERE post_stamp < given_date;

Once you do that, run the consistency checker that will perform the "cleanup", this will allow you to quickly remove any number of messages.


FUDforum Core Developer
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: Moving a topic twice creates a counter error
Next Topic: Critical error in fudforum?
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Mon Sep 16 20:31:32 GMT 2024

Total time taken to generate the page: 0.02729 seconds