FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » Slow posting
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
Slow posting [message #11695] Mon, 14 July 2003 11:52 Go to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Sometimes, when users submit their messages, the response from the server takes a lot of time (minutes) to be sent to the user.

This problem is unrelated to the CPU load, and when it happens the forum isn't slow or anything, in fact the post the user is waiting for is inserted at once in the topic. The user is kept waiting for nothing, apparently.


~
~
:wq
Re: Slow posting [message #11696 is a reply to message #11695] Mon, 14 July 2003 11:59 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Are you using compression or output buffering?

FUDforum Core Developer
Re: Slow posting [message #11697 is a reply to message #11696] Mon, 14 July 2003 12:00 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
prottoss wrote on Mon, 14 July 2003 17:59

Are you using compression or output buffering?

Compression with mod_gzip in apache. No buffering.


~
~
:wq
Re: Slow posting [message #11698 is a reply to message #11697] Mon, 14 July 2003 12:32 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Hmm... just about the same thing I use on this forum here, except I also use php's output buffering.

The problem sounds awfully like a rare compression problem that happens with certain versions of IE that occasionally experience problems with zlib compressed pages.

I'll keep a look on out for the problem, but if the message appeared immidiately then it is unlikely the FUDforum is the cause of the problem.


FUDforum Core Developer
Re: Slow posting [message #11699 is a reply to message #11698] Mon, 14 July 2003 12:36 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Mmmh, don't know if it's a IE problem, the bug was reported by some user (using IE) but was experienced by me too, and I use Konqueror.

Maybe the problem is the same, but it's strange how it happens only when users are submitting their messages, and not when they are browsing the site or the forum.


~
~
:wq

[Updated on: Mon, 14 July 2003 14:17]

Report message to a moderator

Re: Slow posting [message #11700 is a reply to message #11699] Mon, 14 July 2003 12:40 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Posting of messages is the slowest part of nearly all forum operations. Because many things need to happen, caches need to be updated, topic view needs to be rebuilt etc...
And if you told me that the message that the user tried to post did not appear right away I would start asking different questions and possibly see if I can spot any bottlenecks in the posting process based on your setup. However, since the message appears right away, I am pretty confident the problem is not FUDforum related.

Since you said you can replicate the behaviour could you tell me approximately how often does it happen % wise.


FUDforum Core Developer
Re: Slow posting [message #11701 is a reply to message #11700] Mon, 14 July 2003 12:56 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
prottoss wrote on Mon, 14 July 2003 18:40

However, since the message appears right away, I am pretty confident the problem is not FUDforum related.

Since you said you can replicate the behaviour could you tell me approximately how often does it happen % wise.
I see your point.
I'm not a heavy poster on our forum but ppl say it's quite common, I'd say 5%. On a forum with 4000 posts per day, it means that in a day someone is annoyed 200 times a day...

The problem isn't so serious, but the effect is. The impression user have is to navigate in a slow site, when the CPU isn't used at all Smile


~
~
:wq
Re: Slow posting [message #11702 is a reply to message #11701] Mon, 14 July 2003 13:23 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
How big is you forum (# of messages, # of users, avg # of concurrent users), basic hardware specs?

FUDforum Core Developer
Re: Slow posting [message #11703 is a reply to message #11702] Mon, 14 July 2003 14:14 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Messages: 794914

Topics: 39067

20.35 messages per topic

Forums: 125

6359.31 messages per forum
312.54 topics per forum

Categories: 17

46759.65 messages per category
2298.06 topics per category
7.35 forums per category
Private Messages: 59894

Users: 14937

53.22 messages per user
2.62 topics per user
4.01 private messages per user
Moderators: 130

0.87% of all users
1.04 per forum
Administrators: 4

0.00% of all users
User Groups: 130

2.98 members per group

It's distribuited on 2 different machines: apache on a Athlon 1800+ with 512Mb ram and MySQL on a dual XP 1800+ with 2Gb Ram.
Raid 1 on both machines with 2 IDE disks.
Those servers manage squid (1st machine) and Zope (2nd machine) too.
Load avg is low on both.

With timeout set on 15 minutes, # of contemporary users is stable between 150 and 200 during the day.


~
~
:wq

[Updated on: Mon, 14 July 2003 14:16]

Report message to a moderator

Re: Slow posting [message #11704 is a reply to message #11703] Mon, 14 July 2003 15:35 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Hmm... are you by any chance importing messages from NNTP or a mailing list?

FUDforum Core Developer
Re: Slow posting [message #11705 is a reply to message #11704] Mon, 14 July 2003 15:37 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
prottoss wrote on Mon, 14 July 2003 21:35

Hmm... are you by any chance importing messages from NNTP or a mailing list?
Nope.


~
~
:wq
Re: Slow posting [message #11706 is a reply to message #11705] Mon, 14 July 2003 15:53 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Do you have search engine enabled?

FUDforum Core Developer
Re: Slow posting [message #11707 is a reply to message #11706] Mon, 14 July 2003 15:57 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Yes, and reindexing all the forum takes quite a lot. But searches are quite fast.

~
~
:wq
Re: Slow posting [message #11713 is a reply to message #11707] Mon, 14 July 2003 19:20 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Yeah, re-indexing should be avoided if possible.

You can try temporarily disabling the search, which would make message posting much quicker especially for longer messages. That 'may' solve the problem you are seeing, or that may at least be a likely cause of the problem.


FUDforum Core Developer
Re: Slow posting [message #11818 is a reply to message #11713] Fri, 18 July 2003 04:12 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Disabling searches it means new posts will not indexed?
I've got to reindex the forum if I want to re-enable it?

Thanx


~
~
:wq
Re: Slow posting [message #11823 is a reply to message #11818] Fri, 18 July 2003 09:10 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
That is correct.

FUDforum Core Developer
Re: Slow posting [message #12130 is a reply to message #11695] Tue, 29 July 2003 16:57 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
I don't think it's a gzip problem.

Tonight, while I was browsing the forum (and it was fast as usual) I performed a search, and noticed how the results page wasn't delivered at all. I tried another search: same behaviour. I suspected some load problem, but after some checking I excluded it (in effect the forum browsing and the other non-forum parts of the site were fast, as I said): after logging I could see the CPUs not loaded at all.
Then I tried to post, and the usual bug presented itself: no response from the site, but the post was inserted in the right place.

Taking advantage of this situation, I checked the status of db tables with mysqladmin processlist, and I couldn't help noticing how many processes (maybe 30-40) where waiting to write in the index tables, those tables were locked.

I stopped and restarted mysql, and the problem disappeared: fast searches, posting without problems.

I think it's a deadlock problem, occurring while index/search tables are updated.
The more users post at the same time, the more likely the bug is triggered.


~
~
:wq

[Updated on: Tue, 29 July 2003 17:00]

Report message to a moderator

Re: Slow posting [message #12134 is a reply to message #12130] Tue, 29 July 2003 17:47 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
That could be the case, except that your CPU would be loaded by MySQL who would be trying to process the search. If your CPU is unused then there is a problem...

FUDforum Core Developer
Re: Slow posting [message #12135 is a reply to message #12134] Tue, 29 July 2003 17:58 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
prottoss wrote on Tue, 29 July 2003 23:47

That could be the case, except that your CPU would be loaded by MySQL who would be trying to process the search. If your CPU is unused then there is a problem...
Well, it wasn't completely unused, but it wasn't loaded to the excess either. Let's say the system wasn't loaded enough to justify a timeout when performing a search. The problem wasn't the CPU (who in the meantime nicely generated normal pages and "plain" forum-ones as thread, msg, global indexes) but were the locked tables.


~
~
:wq
Re: Slow posting [message #12137 is a reply to message #12135] Tue, 29 July 2003 18:01 Go to previous messageGo to next message
ctbk is currently offline  ctbk   Italy
Messages: 142
Registered: April 2002
Location: Milan, Italy
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
ctbk wrote on Tue, 29 July 2003 23:58

prottoss wrote on Tue, 29 July 2003 23:47

That could be the case, except that your CPU would be loaded by MySQL who would be trying to process the search. If your CPU is unused then there is a problem...
Well, it wasn't completely unused, but it wasn't loaded to the excess either. Let's say the system wasn't loaded enough to justify a timeout when performing a search. The problem wasn't the CPU (who in the meantime nicely generated normal pages and "plain" forum-ones as thread, msg, global indexes) but were the locked tables.
And I don't think my searches were performed at all, I think their queries were waiting for the tables to unlock before being run.


~
~
:wq
Re: Slow posting [message #12138 is a reply to message #12135] Tue, 29 July 2003 18:01 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
On the 2 systems I've tried while the search was doing it's thing 1 MySQL thread was eating the CPU completely according to top.

FUDforum Core Developer
Re: Slow posting [message #18931 is a reply to message #12138] Thu, 17 June 2004 18:28 Go to previous messageGo to next message
andy_scouser is currently offline  andy_scouser   United Kingdom
Messages: 76
Registered: June 2003
Karma: 1
Member
add to buddy list
ignore all messages by this user
im seeing the same on my site. It has only just started in the last few days.

If I restart mysql or reindex the site, i get the usual very quick response. 10 minutes later, back to v e r y slow posting again.

PHP built On: SunOS forum 5.8 Generic_117000-03 sun4u
Database Version: 4.0.13-max
PHP Version: 4.3.2
Web Server: Apache/2.0.47 (Unix) PHP/4.3.2
WebServer to PHP interface: apache2handler
Forum Version: 2.5.2

sun ultra 10, 256mb ram. When slow, the server is 99% idle with no disk i/o to speak of, no smtx or paging and im on a 100mb fdx connection to it.

Messages: 23082
Topics: 2327 9.92 messages per topic
Forums: 17 1357.76 messages per forum
136.88 topics per forum
Categories: 4 5770.50 messages per category
581.75 topics per category
4.25 forums per category
Private Messages: 2542
Users: 392 58.88 messages per user
5.94 topics per user
6.48 private messages per user
Moderators: 2 0.51% of all users
0.12 per forum
Administrators: 6 0.02% of all users
User Groups: 19 1.84 members per group

[Updated on: Thu, 17 June 2004 18:33]

Report message to a moderator

Re: Slow posting [message #18936 is a reply to message #18931] Thu, 17 June 2004 21:08 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Try upgrading the forum.


FUDforum Core Developer
Re: Slow posting [message #18981 is a reply to message #18936] Sat, 19 June 2004 14:55 Go to previous messageGo to next message
andy_scouser is currently offline  andy_scouser   United Kingdom
Messages: 76
Registered: June 2003
Karma: 1
Member
add to buddy list
ignore all messages by this user
i upgraded, same issue. I also lost my theme, have reverted back. The slow post issue is still there as well.

It must be a mysql issue ?!?!?! Is there anything I can test or do to check. ???

Ive disabled searches as suggested and posts are now fine, any tips on where I should look to resolve this issue please ?

[Updated on: Sat, 19 June 2004 16:26]

Report message to a moderator

Re: Slow posting [message #18992 is a reply to message #18981] Mon, 21 June 2004 09:02 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
In MySQL there is an option that would allow you to log slow queries and define how many seconds does it take to constitute a slow query. Set the slow query limit to 1 second and then analyze the "slow" queries with EXPLAIN command.

FUDforum Core Developer
Re: Slow posting [message #19027 is a reply to message #18992] Tue, 22 June 2004 13:05 Go to previous messageGo to next message
andy_scouser is currently offline  andy_scouser   United Kingdom
Messages: 76
Registered: June 2003
Karma: 1
Member
add to buddy list
ignore all messages by this user
No Message Body

[Updated on: Tue, 22 June 2004 13:09]

Report message to a moderator

Re: Slow posting [message #34803 is a reply to message #18992] Sat, 18 November 2006 11:05 Go to previous messageGo to next message
viktor is currently offline  viktor   United States
Messages: 12
Registered: November 2004
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
I have the similar problem. Forum is working pretty fast, new postings are fast too, but any reply gives a delay up to minutes...
FUDforum 2.7.3

I have another forum based on FUDforum 2.7.5RC2, but it has the same problem

You can test it right now:

http://www.ulitka.com/forum/
login: mark
passwd: 123123



mysql was optimized, restarted an so on...

Please advise, how to fix it.

[Updated on: Sat, 18 November 2006 12:43]

Report message to a moderator

Re: Slow posting [message #34816 is a reply to message #34803] Sun, 19 November 2006 11:30 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Try the 2.7.6 version.

FUDforum Core Developer
Re: Slow posting [message #36754 is a reply to message #34816] Fri, 06 April 2007 12:30 Go to previous messageGo to next message
angryowl   United States
Messages: 7
Registered: October 2006
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Nope, it's not FUDForum at all.

My install of FUDforum had this problem too. Posts were taking FOREVER (like minutes, literally) and sometimes timing out. Our forum is frequented by about two dozen people, and many of them are subscribers to topics and forums so email notifications are a huge part of why we like FUDforum.

But these slow posts, it was making us insane. We tried all sorts of tricks to speed things up... thinking that it had something to do with the install, the config of the host machine, and so forth.

Then, in an act of desperation, I started climbing around through the FUDforum code looking for some way (ANY way) to speed up posts. Our best working theory was that the email notifications were slow. So I popped open iemail.inc.t in the FUDforum source and found the line where FUDforum calls the PHP function "mail()" in a loop to send out the notifications.

Thanks to a bit of astute undertanding on the part of a man named Vikas Jayna, we discovered that mail() was calling sendmail() on our host machine, and the Hosting Service that owned the machine had a rate limiter/anti-spam delay on sendmail to prevent drive-by spammers from setting up shop on their servers. There was an enforced transaction delay per mail() call. Since we were calling mail() in a loop for up to 24 users, this was taking a long long time.

The fix is to send mail deferred. You MAY be only able to do this if your host is using sendmail, I don't know. But here's the magic spell:

from PHP documentation:

bool mail ( string $to, string $subject, string $message [, string $additional_headers [, string $additional_parameters]] )


Quote:


I think passing the following string "-O DeliveryMode=d" in the $additonal_parameters variable would do the job

Vikas Jayna



In the fudforum source, in iemail.inc.t, I added the "-O DeliveryMode=d" as the last parameter of the mail command here:

        foreach ((array)$to as $email) {
                $mail_func($email, $subj, $body, $header, "-O DeliveryMode=d" );
        }


And OH MY GOD is it FAST when posting messages now!

If you're experiencing the same issues I had (rate limited ISP/Hosting provider) this will be your magic beans.
Re: Slow posting [message #36773 is a reply to message #36754] Sun, 08 April 2007 11:33 Go to previous message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Wow, thanks for the detailed analysis, I am certain this with be helpful to many people and not just people who use FUDforum.

FUDforum Core Developer
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: 2.7.6: Link to "new in topic" from topic list often only shows first post...
Next Topic: forum rank system - inconsistent rank display
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Fri Dec 15 18:47:55 EST 2017

Total time taken to generate the page: 0.01173 seconds