FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » How To » Newsgroup import
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
Newsgroup import [message #6791] Thu, 24 October 2002 16:42 Go to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Hi, all.

I tryed make gate to newsgroup.

To new forum I asign newsgroup "ru.linux" from server "news3.fido7.ru".

I write command to cron, but cron send to me error:

Fatal error: Call to undefined function: decode_header_value() in /home/linuxportal/fud/include/nntp.inc on line 220.

What I must do for correct this error?


sorry for my poor english Smile
Re: Newsgroup import [message #6794 is a reply to message #6791] Thu, 24 October 2002 16:57 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Replace your nntp.inc with the one attached, it should address the problem.
  • Attachment: nntp.inc
    (Size: 18.73KB, Downloaded 644 times)


FUDforum Core Developer
Re: Newsgroup import [message #6795 is a reply to message #6794] Thu, 24 October 2002 17:08 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Thanks, Ilia

All is work Smile


sorry for my poor english Smile
Another error [message #6811 is a reply to message #6795] Fri, 25 October 2002 05:06 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
<br />
<b>Fatal error</b>: Call to undefined function: decode_string() in <b>/home/linuxportal/fud/include/nntp.inc</b> on
line <b>551</b><br />


sorry for my poor english Smile
One more question [message #6812 is a reply to message #6811] Fri, 25 October 2002 05:14 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Ilia,

I make my FUD in win-1251 charset, because my old forum (WBB) has same charset.

I wonna import messages from newsgroup. His messages in KOI-8r charset.

Can nntp.php convert (KOI-8R --> Win-1251) messages from newsgroup on the fly?

And reversed convert when my users post messages from FUD to newsgroup.

Can you, Ilia, make this modification for us?

Thanks Smile


sorry for my poor english Smile

[Updated on: Fri, 25 October 2002 06:29]

Report message to a moderator

Start importing messages from asigned date [message #6814 is a reply to message #6812] Fri, 25 October 2002 06:26 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Ilia,

I successfuly start import messages from newsgroup.

But FUD take very old messages (2001 year).
This messages today is not actually and use more disk space.
I dont need this old messages.

How I can set up date, from wich I wont start import messages?


sorry for my poor english Smile
Re: Another error [message #6827 is a reply to message #6811] Fri, 25 October 2002 12:54 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
Fixed in CVS.

FUDforum Core Developer
Re: One more question [message #6828 is a reply to message #6812] Fri, 25 October 2002 12:57 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
belmax wrote on Fri, 25 October 2002 01:14


Can nntp.php convert (KOI-8R --> Win-1251) messages from newsgroup on the fly?



No.

Quote:


Can you, Ilia, make this modification for us?



The type of modification you request is a custom mod, these modifications are only made for commercial customers who are willing to pay for them. If you want the functionality write it yourself Wink I can give you a tip though. PHP has a 'recode' module, which can convert text between various charsets, although it does not work too well, it may be able to do what you need still.


FUDforum Core Developer
Re: Start importing messages from asigned date [message #6830 is a reply to message #6814] Fri, 25 October 2002 12:59 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
belmax wrote on Fri, 25 October 2002 02:26

Ilia,

I successfuly start import messages from newsgroup.

But FUD take very old messages (2001 year).
This messages today is not actually and use more disk space.
I dont need this old messages.

How I can set up date, from wich I wont start import messages?


Well, there are two alternatives, you can allow the messages to be imported and then use the 'Thread Pruning' option to remove all messages inside the imported forum older then X days/weeks/etc...
The other possibility is more complex but non-the-less doable. FUDforum keeps a file where it contains the last NNTP id of the message it imported last, if you modify this id to an id of a message older then certain date then only messages newer then that message will be imported.


FUDforum Core Developer
Re: Start importing messages from asigned date [message #6831 is a reply to message #6830] Fri, 25 October 2002 13:39 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
prottoss wrote on Fri, 25 October 2002 16:59

FUDforum keeps a file where it contains the last NNTP id of the message it imported last, if you modify this id to an id of a message older then certain date then only messages newer then that message will be imported.


Thanks Smile

What name of this file, Ilia?
What name of variable contained last NNTP id?


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6834 is a reply to message #6831] Fri, 25 October 2002 14:45 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
belmax wrote on Fri, 25 October 2002 09:39

prottoss wrote on Fri, 25 October 2002 16:59

FUDforum keeps a file where it contains the last NNTP id of the message it imported last, if you modify this id to an id of a message older then certain date then only messages newer then that message will be imported.


Thanks Smile

What name of this file, Ilia?
What name of variable contained last NNTP id?



The file can be found inside errors/.nntp/ and it's file name depends on the name of the newsgroup & server which is being imported.
For example 'News.CIS.DFN.DE-comp.lang.php'


FUDforum Core Developer
Re: Start importing messages from asigned date [message #6884 is a reply to message #6834] Mon, 28 October 2002 06:58 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Quote:

The file can be found inside errors/.nntp/ and it's file name depends on the name of the newsgroup & server which is being imported.
For example 'News.CIS.DFN.DE-comp.lang.php'


I have found this file, but it has zero length


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6888 is a reply to message #6884] Mon, 28 October 2002 12:21 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
That's probably because you've aborted the import process?

FUDforum Core Developer
Re: Start importing messages from asigned date [message #6898 is a reply to message #6888] Mon, 28 October 2002 13:28 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Yes, Ilia, I have interrupted process of import because import began from messages at 2001. Import of messages this date would occupy huge volume of a database and time.

To change a variable "start ID" I can only completely download newsgroup?


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6899 is a reply to message #6898] Mon, 28 October 2002 13:33 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
No, connect to the news group manually and find out an id of a message that you want to start with. Then put the id of that message inside that file.
When the import process will run it'll import messages who's IDs are greater then the id of the message you've entered.


FUDforum Core Developer
Re: Start importing messages from asigned date [message #6900 is a reply to message #6899] Mon, 28 October 2002 13:38 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Thanks, Ilia Smile

I did not know, that this file contains only ID number.



sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6901 is a reply to message #6900] Mon, 28 October 2002 15:08 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
It is impossible....

I have source code of needed message from newsgroup:

Quote:

Path: news.rt.ru!rt.ru!newsfeed.rt.ru!rt.ru!news-out.nuthinbutnews.com!propagator 2-sterling!news-in-sterling.newsfeed.com!news-in.nuthinbutnews.com!mtu.ru!n ews3.cnt.ru!srcc!Gamma.RU!ddt.demos.su!f400.n5020!f4441.n5020!f52.n5020!f23 9.n465!not-for-mail
Newsgroups: fido7.ru.linux
Distribution: fido7
X-Comment-To: Igor Kaminskiy
Approved: gateway(at)fido7(dot)ru
From: Mihail Kapitanov <Mihail(dot)Kapitanov(at)p1(dot)f239(dot)n465(dot)z2(dot)fidonet(dot)org>
Date: Thu, 17 Oct 2002 16:24:22 +0400
Subject: Re^2: ProFTPD
Message-ID: <1034864805(at)p1(dot)f239(dot)n465(dot)z2(dot)ftn>
References: <2377752244(at)mpeks(dot)tomsk(dot)su>
Organization: Storm
X-FTN-FLAGS: K/S
X-FTN-AREA: RU.LINUX
X-FTN-REPLY: mpeks.tomsk.su 8db99eb4
X-FTN-MSGID: 2:465/239.1 3daec8a5
X-FTN-PID: GED+W32 1.1.5-021004
X-FTN-CHRS: CP866 2
X-FTN-TID: hpt 0.9.7d-stable/w32 02-01-01
X-FTN-Tearline: GoldED+ 1.1.5-021004 (WinNT 5.0.2195-Service_Pack_3 i686)
X-FTN-Origin: Storm (2:465/239.1)
X-FTN-SEEN-BY: 50/203 361 421 424 438 523 610 913 51/7 400/333 462 465 520 450/102
X-FTN-SEEN-BY: 450/166 208 452/25 100 454/9 460/15 461/33 74 106 132 435 640
X-FTN-SEEN-BY: 5083/13 5085/13 51 75 5090/2 67 91 104 105 108 113 188 5091/10
X-FTN-SEEN-BY: 5092/1 5093/4 20 27 63 5095/1 45 78 5096/9 5097/28 64 5099/4 11
X-FTN-SEEN-BY: 5100/113 6009/8 6023/1 6028/1 6053/1 6083/1 11 6096/10
X-FTN-PATH: 465/239 5020/52 4441
X-FTN-PATH: 5020/400
Lines: 20
X-Received-Date: Tue, 22 Oct 2002 17:53:34 MSD (news.rt.ru!rt.ru)
Xref: news.rt.ru fido7.ru.linux:280283


I tryed put string "1034864805" to file "news3.fido7.ru-fido7.ru.linux.lock".
Unsuccessfully.
Import begun since 2001

I tryed put string "1034864805(at)p1(dot)f239(dot)n465(dot)z2(dot)ftn" to file "news3.fido7.ru-fido7.ru.linux.lock".
Unsuccessfully.
Import begun since 2001


Server of news group is news3.fido7.ru
Name of newsgroup is fido7.ru.linux

Can you help me to choice string "start ID"?


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6902 is a reply to message #6901] Mon, 28 October 2002 15:14 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
You are taking a wrong approach. To get the above mentioned id, you need to manually (via telnet) make a request to the NNTP server and request a list of articles in the news group you are trying to import. That will return a few number such as 'starting id' and number of entries. Using those two numbers you can specify a proper starting position.

FUDforum Core Developer
Re: Start importing messages from asigned date [message #6910 is a reply to message #6902] Mon, 28 October 2002 17:06 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
I have found these numbers:
223 234712 < 2t3bf1bec0i42bn3ex(at)alien(dot)observ(dot)univ(dot)kiev(dot)ua >

I Has written in "news3.fido7.ru-fido7.ru.linux" file string "223 234712"

It has not helped.

What file open NNTP.PHP:
"news3.fido7.ru-fido7.ru.linux"?
or existing
"news3.fido7.ru-fido7.ru.linux.lock"?

May be I use wrong file-names?

What should be syntax for this string?
What divider must be used between this two numbers?
What EOL must have this string?
When I upload this file through "File Manager", its owner become "www". The owner of other files is "linuxportal."
CHMOD is 666

Who should be the owner of the file?

Sorry for my multiply questions, Ilia...


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6911 is a reply to message #6910] Mon, 28 October 2002 17:07 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
There should be just 1 number in there not 2. Do not modify the .lock file.

FUDforum Core Developer
Re: Start importing messages from asigned date [message #6917 is a reply to message #6911] Mon, 28 October 2002 17:46 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
I in despair.... Nothing it turns out.....

I was tryed put in "news3.fido7.ru-fido7.ru.linux" file string "223"..... no results....

I was tryed put in "news3.fido7.ru-fido7.ru.linux" file string "234712"..... no results....

Near to this file there is another - "news3.fido7.ru-fido7.ru.linux.lock" with zero lenght....

I wont to stop my attempts....


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6920 is a reply to message #6917] Mon, 28 October 2002 18:02 Go to previous messageGo to next message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
The lock file is and always be 0 bytes, it is used for something else entirely, primarily to prevent more then 1 nntp import script running on the same newsgroup.

The other file should contain just the number from which to start importing, no new lines or anything like that, just a number.

To get this number you must follow a VERY basic procedure:

telnet news_server 119
GROUP your_news_group

The response will look something like this:
211 2001 6063 8089 your_news_group

211 - Return Code
2001 - # number of messages
6063 - 1st message id
8089 - last message id

I think it should be pretty obvious what you need to do from this point on...


FUDforum Core Developer
Re: Start importing messages from asigned date [message #6922 is a reply to message #6920] Mon, 28 October 2002 18:47 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Uffffff Smile

We make it! Smile

I was received from another man wrong start ID.
I haven't access to 119 port in my network and ask another man about it.

Thanks for you for last message Smile

FUD forever! Smile


sorry for my poor english Smile
Re: Start importing messages from asigned date [message #6934 is a reply to message #6922] Tue, 29 October 2002 13:33 Go to previous messageGo to next message
belmax is currently offline  belmax   Russian Federation
Messages: 114
Registered: October 2002
Location: Saint-Petersburg, Russia
Karma: 0
Senior Member
Ilia, i wont use FUD on my second site.

But those hosting-provider don't give access to PHP-binary...

Is there any ways to use nntp.php without PHP-binary?

May be there is perl-version of nntp.php?


sorry for my poor english Smile

[Updated on: Tue, 29 October 2002 13:49]

Report message to a moderator

Re: Start importing messages from asigned date [message #6935 is a reply to message #6934] Tue, 29 October 2002 13:59 Go to previous message
Ilia is currently offline  Ilia   Canada
Messages: 13241
Registered: January 2002
Karma: 0
Senior Member
Administrator
Core Developer
no, the forum is only in PHP and that is all it'll use.

FUDforum Core Developer
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: Display of Usergroup
Next Topic: NNTP.PHP without PHP-binary
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Nov 27 18:55:36 GMT 2024

Total time taken to generate the page: 0.02500 seconds