FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum » How To » NNTP Issue (New User) (New User)
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
NNTP Issue (New User) [message #162382] Tue, 04 May 2010 20:44 Go to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi Folks!

New user here, just finished doing an install. I've used IPB 1.3.1 for close to a decade now and first impressions with FUD are very good!

My issue is NNTP. I created a proper rule and now want to get it to retrieve posts.

I edited the first line in nntp.php to read like this;

#!C:\Progra~1\PHP -q

That's my path to the PHP executable. Is it OK? (programmatically)

My next issue is, how do I trigger an update? The docs don't show anything for Windows/IIS. They just mention Task Scheduler.

Sorry for the newby questions - can anyone help?

NNTP Issue [message #162384 is a reply to message #162382] Tue, 04 May 2010 20:51 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi Folks!

Brand new user here Shocked

I created an NNTP profile and changed the first line in nntp.php to;

#!C:\Progra~1\PHP -q

Is that OK? (it is the right path to my PHP directory)

The next question is how exactly do I trigger an update? The docs show a Cron job (for Unix) but nothing for Windows. I understand that I need to use task manager but what exactly am I trying to run?

Thanks!
Re: NNTP Issue [message #162394 is a reply to message #162384] Wed, 05 May 2010 02:57 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
See Windows example at http://www.php-cli.com/php-cli-tutorial.shtml

Feel free to expand the documentation at Newsgroup Manager
Re: NNTP Issue [message #162403 is a reply to message #162394] Wed, 05 May 2010 14:50 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Thank you very much - I managed to get it working from the command line and as soon as figure out how to schedule it with Task Scheduler I'll be happy to update the docs accordingly as it's no difficult thing Smile

Question though, during testing I began to pull down a newsgroup and after some time got this error;

Importing microsoft.public.windowsxp.general message 2037635
Importing microsoft.public.windowsxp.general message 2037636
(C:\Inetpub\fudforum\include\theme\default\db.inc:72<br />
C:\Inetpub\fudforum\include\theme\default\db.inc:176<br />
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112<br />
C:\Inetpub\fudforum\include\nntp.inc:577<br />
C:\Inetpub\fudforum\scripts\nntp.php:94<br />
) 1366: Incorrect string value: '\xBB Alte...' for column 'subject' at row 1<br
/>
Query: INSERT INTO fud30_msg (
thread_id,
poster_id,
reply_to,
ip_addr,
host_name,
post_stamp,
subject,
attach_cnt,
poll_id,
icon,
msg_opt,
file_id,
foff,
length,
file_id_preview,
offset_preview,
length_preview,
mlist_msg_id,
poll_cache,
flag_cc,
flag_country
) VALUES(
274,
350,
1662,
'79.78.172.66',
NULL,
1266356162,
'Re: Sound Schemes ╗ Altering, etc. (XP Professional)',
0,
0,
NULL,
3,
20000,
6631745,
1915,
0,
0,
0,
'uVL93$0rKHA(dot)3944(at)TK2MSFTNGP06(dot)phx(dot)gbl',
NULL,
NULL,
NULL
)<br />
Server Version: 5.1.42-community<br />

Any ideas on why or how to avoid it?
Re: NNTP Issue [message #162404 is a reply to message #162403] Wed, 05 May 2010 15:13 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
One of the messages you are importing has a control character in the subject line that MySQL doesn't like. You may want to change the encoding of the table to accept it or filter it our in the PHP script.
Re: NNTP Issue [message #162405 is a reply to message #162404] Wed, 05 May 2010 16:12 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hmmm.... the easiest way (for me) would be to change the encoding of the table in MySQL to accept it.

What table would that be and what should the encoding be changed to?
Re: NNTP Issue (New User) [message #162409 is a reply to message #162382] Wed, 05 May 2010 21:33 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Alternatively, how should the code be changed?

This issue is stopping me from importing data from almost every feed I set up Sad
Re: NNTP Issue (New User) [message #162412 is a reply to message #162409] Thu, 06 May 2010 01:38 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
Run the forum's MySQL Charset Changer to change the encoding of your tables to UTF-8. If they are already UTF-8, we can fix the script.

PS: I would need a test case or copy of the message to prepare a fix.

Best regards.

Frank
Re: NNTP Issue (New User) [message #162414 is a reply to message #162382] Thu, 06 May 2010 02:15 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Remarkable - being able to change the character set through the forum admin! I find a lot of things about this forum very interesting.

I checked the properties of fud30_index and found that it is already UTF8 Shocked I guess that means a programmatic solution?

The news pull stopped again and I'm sure it was also due to a non-standard character in the subject line. This time it might be a Spanish post that caused it to cough.

A search on Google for some of the text (Id� del suceso: 36871) seems to show that it was a Spanish post.

That caused me to think, maybe he's using a spanish-specific character such as an upside down exclamation mark or a letter with a hat Laughing

So, my theory is, if a foreigner made a post with a subject line that used non-standard characters maybe that will also cause the NNTP pull to cough up an error and stop?

This is the most recent error that seems to indicate a Spanish post;

Importing microsoft.public.exchange2000.general message 7475
(C:\Inetpub\fudforum\include\theme\default\db.inc:72
C:\Inetpub\fudforum\include\theme\default\db.inc:176
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112
C:\Inetpub\fudforum\include\nntp.inc:577
C:\Inetpub\fudforum\scripts\nntp.php:94

) 1366: Incorrect string value: '\xB7 del ...' for column 'subject' at row 1

Query: INSERT INTO fud30_msg (
thread_id,
poster_id,
reply_to,
ip_addr,
host_name,
post_stamp,
subject,
attach_cnt,
poll_id,
icon,
msg_opt,
file_id,
foff,
length,
file_id_preview,
offset_preview,
length_preview,
mlist_msg_id,
poll_cache,
flag_cc,
flag_country
) VALUES(
0,
2776,
0,
'200.16.138.46',
NULL,
1134395206,
'Id� del suceso: 36871',
0,
0,
NULL,
3,
110000,
13212695,
428,
0,
0,
0,
'OidmbJy$FHA(dot)3944(at)TK2MSFTNGP10(dot)phx(dot)gbl',
NULL,
NULL,
NULL
)

When I check Google for that post it shows many similar posts - all of them in Spanish.

Best;
Marvin

Re: NNTP Issue (New User) [message #162415 is a reply to message #162382] Thu, 06 May 2010 02:19 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hmmmm... found another one;

'Outlook doesn�t work'

I would assume that one was just an apostrophe?
Re: NNTP Issue (New User) [message #162416 is a reply to message #162415] Thu, 06 May 2010 11:56 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
Doesn't help me much. As explained above, I need a test case or copy of the message to prepare a fix.
Re: NNTP Issue (New User) [message #162423 is a reply to message #162382] Fri, 07 May 2010 22:49 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
OK, I got one Laughing

This is the error;

(C:\Inetpub\fudforum\include\theme\default\db.inc:72
C:\Inetpub\fudforum\include\theme\default\db.inc:176
C:\Inetpub\fudforum\include\theme\default\imsg_edt.inc:112
C:\Inetpub\fudforum\include\nntp.inc:577
C:\Inetpub\fudforum\scripts\nntp.php:94

) 1366: Incorrect string value: '\xBB Alte...' for column 'subject' at row 1


'Re: Sound Schemes � Altering, etc. (XP Professional)',


and this is the message (note the arrows in the subject)

Subject: Re: Sound Schemes » Altering, etc. (XP Professional)
Did you save the sound scheme after you'd modified it?

"thricipio" <thricipio(at)discussions(dot)microsoft(dot)com[/email]> wrote in message
news:1BEFD643-F393-487E-8B9C-2F81B39A23FF(at)microsoft(dot)com...[/email]
> So. I've been trying to do something on XP Pro, that I know I've done
> before
> with no great difficulty on XP Home: that is, change a couple of the
> sound
> scheme settings - specifically, the "Windows Logoff" and "Windows Logon"
> settings.
>
> On XP Home, I recall having to go into the "Sounds and Audio Devices
> Properties" dialogue and selecting the desired item (e.g., Windows Logoff)
> and then using the [Browse] button to find the desired .wav file. I can
> do
> that, and I can check that I have the correct .wav file by clicking the
> [>]
> button. So far, so good. But when I actually Logoff/on, I don't get the
> newly selected .wav files.
>
> I've even renamed the desired .wav files to "Windows Logoff(/on)" and
> placed
> them in the default directory {<root_drive>\Windows\Media} and tested them
> with the [>] button. And that works (as one would expect). But again,
> when
> I logoff and logon, I still get the default Win logoff/on .wav file
> sounds.
>
> Obviously, I'm missing something basic here. just can't figure out what
> that
> is. Any help you can give will be appreciated. Thanks.
>
> Color me stymied,
> -Thri

[Updated on: Fri, 07 May 2010 22:54]

Report message to a moderator

Re: NNTP Issue (New User) [message #162426 is a reply to message #162423] Fri, 07 May 2010 23:16 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
I actually need a full message with complete headers to test with. I'll try to construct one from your post, however, I suspect it won't help as the Subject line is encoded.

Is this message available on a public newsgroup that I can use for testing? If so, what is the NNTP server name and message number?
Re: NNTP Issue (New User) [message #162428 is a reply to message #162426] Sat, 08 May 2010 02:23 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi Frank;

It came from news server msnews.microsoft.com
The group was microsoft.public.windowsxp.general
It was posted on Feb 16, 2010 by thricipio

Here's a link? to it;
news:1BEFD643-F393-487E-8B9C-2F81B39A23FF(at)microsoft(dot)com

Hopefully that will help?
The one thing I'm concerned about though is that it seems any message with goofy characters in the subject line causes the pull to fail.
So I'm thinking that if someone posts in Spanish or Japanese or something like that will also cause it to fail?
I'm just guessing in an effort to try and help Smile

Best & Thanks!
Marvin
Re: NNTP Issue (New User) [message #162429 is a reply to message #162428] Sat, 08 May 2010 03:59 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
Great this is exactly what I need. Looking at the message's source, the subject is:

Subject: =?Utf-8?Q?Sound_Schemes_=C2=BB_Altering=2C_etc._=28?=
=?Utf-8?Q?XP_Professional=29?=
Re: NNTP Issue (New User) [message #162430 is a reply to message #162429] Sat, 08 May 2010 09:32 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
I'm unable to reproduce the problem. I've tried on my Linux box and on Windows with FUD2Go. Both systems were able to import the message without a problem.

To proceed, we need to do some debugging. Please let me know if you would prefer 1 or 2 below:

1) If the forum is on the Net, create an admin user for me and mail me the password.

2) I can prepare a debugging patch for you. You will need to apply it, run the import and report back on the output.

Re: NNTP Issue (New User) [message #162431 is a reply to message #162430] Sat, 08 May 2010 12:51 Go to previous messageGo to next message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi Frank;

Number 2 would be easier for me as the site is not available from the 'net yet.
Re: NNTP Issue (New User) [message #162433 is a reply to message #162431] Sun, 09 May 2010 05:12 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
For starters, open include/nntp.inc, and search for:
$this->subject = htmlspecialchars(trim(decode_header_value($this->headers['subject'])));

Replace this line with:
echo "FORUM CHARSET=[". $GLOBALS['CHARSET'] ."]\n";
echo "RAW SUBJ=[". $this->subject ."]\n";
$this->subject = htmlspecialchars(trim(decode_header_value($this->headers['subject'])));
echo "NEW SUBJ=[". $this->subject ."]\n";
die();


Reset the NNTP tracker to the problematic message by running this statement from the SQL Manager:
UPDATE fud30_nntp SET tracker = 2037610 WHERE id = 1;

Run the import script and you should see something like this:

Importing microsoft.public.windowsxp.general message 2037610
FORUM CHARSET=[utf-8]
RAW SUBJ=[=?Utf-8?Q?Sound_Schemes_=C2=BB_Altering=2C_etc._=28?= =?Utf-8?Q?XP_Professional=29?=]
NEW SUBJ=[Sound Schemes » Altering, etc. (XP Professional)]


You may also want to add extra echo statements to decode_header_value() in include/scripts_common.inc.
Re: NNTP Issue (New User) [message #162434 is a reply to message #162433] Sun, 09 May 2010 13:12 Go to previous message
mmiller is currently offline  mmiller   Canada
Messages: 27
Registered: May 2010
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi Frank;

I did as you asked and this is what I got when I ran it;

Output of file C:/Inetpub//fudforum/scripts/nntp_1.log:
Importing microsoft.public.windowsxp.general message 2037610
FORUM CHARSET=[utf-8]
RAW SUBJ=[]
NEW SUBJ=[Sound Schemes » Altering, etc. (XP Professional)]

Does that help?
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Can't Log in to Admin
Next Topic: PDF and code page
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Wed Oct 18 08:56:49 EDT 2017

Total time taken to generate the page: 0.00945 seconds