| Re: FUD Question [message #660 is a reply to message #659] |
Thu, 21 February 2002 17:03   |
Ilia
Messages: 13241
Registered: January 2002
Karma:
|
Senior Member
Administrator
Core Developer |
|
|
| B@rt wrote on Thu, 21 February 2002 11:48 AM |
Hi Protoss,
thanks for your answer! I tried your forum and it's quite fast indeed. Still I'm a bit worried: Am I correct in assuming then that on a system with 100,000 threads there will be 100,000 threads stored in *one* directory? That sounds like a 'dirty' solution to me..
B@rt
|
Yes, all of threads will be stored in a single directory. This won't be a problem, unless you intend to run ls -l in that directory all the time 
It may seem "dirty" but consider the alternative SQL approach. 1st there will be a limit to a length of the message, because you'll have to define the maximum length of the field. MySQL does not cache the TEXT/BLOB and like fields because it would be a waste of memory, and since all the data is kept in one file kernel will not cache that file unless you have a large amount of available RAM (won't happen in most cases).
When doing a MySQL insert MySQL always sets a lock on the table. So, since all the messages are stored in one table, if you have many people posting at the same time they'll have to wait for all the previous inserts to go through. On a very busy forum that may cause serious delayes during message posting.
Another problem is that when you get data from MySQL there is a lot of in-between overhead. MySQL needs to get the data and allocate memory for it, then php's mysql module needs to get the data parse it and allocate memory for it. Then when you fetch the data for yourself another copy of the data is made this time for the php script. In the end you have 3 copies of the same data in memory in addition to various php wrappers around the data.
This as you can imagine takes quite a bit of memory & cpu.
When working with files there is no MySQL step, the data is fetched directly into php, and absolutely no parsing of the data is done real time.
Another problem you may encounter is that a file system (at least most old ones) had a limitation that allowed a file to be no larger then 2 gigabytes. On a large forum that would restrict growth and cause all kinds of problem.
I have a question, which file system do you use? I myself have a few FreeBSD boxes with a stock file system which came with the FreeBSD 3.3 release, albeit it is slower the Linux's ReiserFS when there are lots & lots of files but it is still quite fast. I have a directory structure for storing images with ~10 million images split across 1024 directories. I've had absolutely no problems with speed, while reading files from those directories.
The bottom line to my rant  , is that it is MOST UNLIKELY that you will suffer any performance loss over using FUDforum's way to store messages.
FUDforum Core Developer
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
) 1 Vote
]