FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » NNTP import fails with charset issue (Different charset for the sender than for the message contents)
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
NNTP import fails with charset issue [message #184912] Sat, 15 February 2014 04:38 Go to next message
StephenKing is currently offline  StephenKing   
Messages: 10
Registered: August 2012
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi,

I'm not sure if this is because of a limitation of the news reader, MySQL or FUDforum - but the effect was that NNTP import stopped working.

Path: lists.typo3.org!not-for-mail
From: J�rg Schaller <typo3@rasc....com>
Newsgroups: typo3.german
Subject: Re: [TYPO3-german] Mit Powermail news MEHRSPRACHIG
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Newsreader: Forte Agent 6.00/32.1186


The From: header of the email seems to be ISO-8859-1 encoded, while the body of the message is declared as UTF-8.
Even Thunderbird has trouble with this, when I show the message details (it shows the ö correctly in the list of mails).

The bad thing with FUDforum is that the import always stops with a fatal error, because of a duplicate key entry, as there is already another user 'J':
1062: Duplicate entry 'J' for key 'fud30_users_i_l'
Query: INSERT INTO fud30_users ( login, alias, ... ) VALUES ( 'J�rg Schaller', 'J�rg Schaller',......' )

So FUDforum seems to pass the name as is to MySQL, which then truncates the string.

I'm not sure, what a good relief could be.. and I'm also not an expert in mail header affairs - but it is valid that header fields are of a different charset then the body, isn't it?
Or is more a mistake of the Forte Agent mailer (that I never heard of).

Thanks
Steffen

P.S: Entry in our issue tracker: http://forge.typo3.org/issues/55997
Re: NNTP import fails with charset issue [message #185468 is a reply to message #184912] Wed, 02 April 2014 04:11 Go to previous messageGo to next message
naudefj is currently offline  naudefj   
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
Hi Steffen,

As far as I understand headers should ALWAYS be US-ASCII (special characters should be encoded). So, this is not a valid message! That said, I've managed to load it into my forum without a problem:
$ ~/web/lamp/php/bin/php ./nntp.php 1 ./mail.txt 
Importing article from file ./mail.txt

$ ~/web/lamp/php/bin/php ./nntp.php 1 ./mail.txt 
Importing article from file ./mail.txt - previously loaded


If MySQL truncates the text, you may need to review the table's character set.

Best regards.
Frank
Aw: Re: NNTP import fails with charset issue [message #185469 is a reply to message #185468] Wed, 02 April 2014 05:52 Go to previous messageGo to next message
Fladi is currently offline  Fladi   Germany
Messages: 19
Registered: April 2011
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hi!

I think the problem occurs first when a second user is added to the database. (e.g. a message from a second user is imported and his name starts with the same "J�"). To reproduce another mail would be needed.

best regards
tim
Re: Aw: Re: NNTP import fails with charset issue [message #185470 is a reply to message #185469] Wed, 02 April 2014 06:05 Go to previous message
naudefj is currently offline  naudefj   
Messages: 3632
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
add to buddy list
ignore all messages by this user
If you can provide a proper test case, I'll be more than willing to look into it.
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Importing #n from a.b.cCannot fetch #n: 430 No such article: n
Next Topic: Registered users change view and get error message
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Dec 16 21:36:16 EST 2017

Total time taken to generate the page: 0.00666 seconds