FUDforum: Bug Reports » NNTP import fails with charset issue

Home » FUDforum Development » Bug Reports » NNTP import fails with charset issue (Different charset for the sender than for the message contents)

Show: Today's Messages :: Polls :: Message Navigator

NNTP import fails with charset issue [message #184912]

Sat, 15 February 2014 09:38

StephenKing

Messages: 10
Registered: August 2012

Karma: 0

Junior Member

Hi,

I'm not sure if this is because of a limitation of the news reader, MySQL or FUDforum - but the effect was that NNTP import stopped working.

Path: lists.typo3.org!not-for-mail
From: J�rg Schaller <typo3@rasc....com>
Newsgroups: typo3.german
Subject: Re: [TYPO3-german] Mit Powermail news MEHRSPRACHIG
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Newsreader: Forte Agent 6.00/32.1186

The From: header of the email seems to be ISO-8859-1 encoded, while the body of the message is declared as UTF-8.
Even Thunderbird has trouble with this, when I show the message details (it shows the ö correctly in the list of mails).

The bad thing with FUDforum is that the import always stops with a fatal error, because of a duplicate key entry, as there is already another user 'J':

1062: Duplicate entry 'J' for key 'fud30_users_i_l'
Query: INSERT INTO fud30_users ( login, alias, ... ) VALUES ( 'J�rg Schaller', 'J�rg Schaller',......' )

So FUDforum seems to pass the name as is to MySQL, which then truncates the string.

I'm not sure, what a good relief could be.. and I'm also not an expert in mail header affairs - but it is valid that header fields are of a different charset then the body, isn't it?
Or is more a mistake of the Forte Agent mailer (that I never heard of).

Thanks
Steffen

P.S: Entry in our issue tracker: http://forge.typo3.org/issues/55997

Attachment: mail.txt
(Size: 2.03KB, Downloaded 811 times)
Attachment: Screen Shot 2014-02-15 at 10.28.49.png
(Size: 136.20KB, Downloaded 797 times)

Report message to a moderator

Re: NNTP import fails with charset issue [message #185468 is a reply to message #184912]

Wed, 02 April 2014 08:11

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

Hi Steffen,

As far as I understand headers should ALWAYS be US-ASCII (special characters should be encoded). So, this is not a valid message! That said, I've managed to load it into my forum without a problem:

$ ~/web/lamp/php/bin/php ./nntp.php 1 ./mail.txt 
Importing article from file ./mail.txt

$ ~/web/lamp/php/bin/php ./nntp.php 1 ./mail.txt 
Importing article from file ./mail.txt - previously loaded

If MySQL truncates the text, you may need to review the table's character set.

Best regards.
Frank

Report message to a moderator

Aw: Re: NNTP import fails with charset issue [message #185469 is a reply to message #185468]

Wed, 02 April 2014 09:52

Fladi

Messages: 19
Registered: April 2011

Karma: 0

Junior Member

Hi!

I think the problem occurs first when a second user is added to the database. (e.g. a message from a second user is imported and his name starts with the same "J�"). To reproduce another mail would be needed.

best regards
tim

Report message to a moderator

Re: Aw: Re: NNTP import fails with charset issue [message #185470 is a reply to message #185469]

Wed, 02 April 2014 10:05

naudefj

Messages: 3771
Registered: December 2004

Karma: 28

Senior Member
Administrator
Core Developer

If you can provide a proper test case, I'll be more than willing to look into it.

Report message to a moderator

Previous Topic:	Importing #n from a.b.cCannot fetch #n: 430 No such article: n
Next Topic:	Registered users change view and get error message

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Fri Nov 01 00:08:26 GMT 2024

Total time taken to generate the page: 0.03851 seconds