| Re: Asynchronous FTP Upload [message #172085 is a reply to message #172082] |
Sun, 30 January 2011 14:20   |
Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma:
|
Senior Member |
|
|
On 1/30/2011 8:41 AM, Luuk wrote:
> On 28-01-11 20:48, Jerry Stuckle wrote:
>> On 1/28/2011 10:56 AM, duderion wrote:
>>> On Jan 28, 4:29 pm, Jerry Stuckle<jstuck...@attglobal.net> wrote:
>>>> On 1/28/2011 10:12 AM, duderion wrote:
>>>>
>>>> > hi guys,
>>>>
>>>> > could anyone tell me how i can handle an ftp upload to several servers
>>>> > at once?
>>>>
>>>> > I found fb_nb_put, but i dont know how to combine 5 connections with
>>>> > this.
>>>>
>>>> > any help would be nice :)
>>>>
>>>> > dude
>>>>
>>>> You'll need to open 5 different streams and start each transfer. Keep
>>>> track of the status of each transfer in an array and loop while any of
>>>> them need to continue. In the loop, continue those which have not
>>>> finished.
>>>>
>>>> Not sure what this is going to do though for you though, other than take
>>>> a lot of unnecessary CPU because you're effectively polling constantly.
>>>> Why don't you just upload each file individually?
>>>>
>>> Hi Jerry
>>> thanks for the quick and nice reply,
>>>
>>> i need to do this, because i have to trasfer videos to around 500
>>> hosts during one night. i have a 1gb upload line, and thats why i want
>>> to run those uploads simultaniously....
>>>
>> <Top posting fixed>
>>
>> That doesn't mean you'll get anywhere near 1gb upload. Your limit in
>> this case is likely going to be disk access speed (assuming the other
>> hosts are replying in a timely manner, of course). And forcing the disk
>> to jump around to fetch data from different areas of the disk is likely
>> to be slower then accessing the data in a contiguous file.
>>
>> The point being - even if you open 5 parallel connections, you are not
>> going to get 5x the speed; in fact, depending on what you're doing, you
>> may actually slow down the processing. And error recovery becomes much
>> harder.
>
> You might be true, but most of it depends on the download speed at the
> receiving site. If its lower then 1/5 of your uploadspeed than you
> should not worry ;)
>
Not really. You're not going to get 1gb/sec. or even 200mb/sec. from
the disk drive, especially not continuously. So even if the download
speed on the other end is 200mb/sec, that's still not going to be a
limiting factor.
And forcing the disk to pull data from several different files on the
disk will slow disk overall disk access even more, especially if the
files are contiguous.
>>
>> You need to test and find out. The "sweet spot" may be anywhere from 1
>> to 500 parallel connections (although I highly doubt the latter :) ).
>> And it may vary depending on exactly which hosts you're currently
>> accessing and how quickly they respond.
>>
>> P.S. Please don't top post. Thanks.
>>
>
>
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex(at)attglobal(dot)net
==================
|
|
|
|
Calendar
Search
Help
Members
Register
Login
Home
]