FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » NNTP Import of Messages without Content-Type in Header
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
NNTP Import of Messages without Content-Type in Header [message #161824] Thu, 11 February 2010 12:49 Go to next message
jm2morri is currently offline  jm2morri   Canada
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Hello everyone,

For months I've had a few messages here or there be imported and still show the black diamond with the question mark in place of some german characters. I did a bit more digging and it seems to happen only on nntp articles that don't have "Content-Type" in the header.

Of course it should have that, but if it doesn't it should default to the encoding of the forums, in my case UTF-8. That doesn't appear to be happening.

Something else that seems common to the few I've tracked down is the following in the header:

X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2900.3598
X-RFC2646: Format=Flowed; Original
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350



Cheers,

James.
Re: NNTP Import of Messages without Content-Type in Header [message #161826 is a reply to message #161824] Thu, 11 February 2010 14:13 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
It's impossible to verify your claim without looking at the message body. Please upload it or forward to my e-mail address.
Re: NNTP Import of Messages without Content-Type in Header [message #161831 is a reply to message #161826] Thu, 11 February 2010 18:09 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   Canada
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
naudefj wrote on Thu, 11 February 2010 14:13
It's impossible to verify your claim without looking at the message body. Please upload it or forward to my e-mail address.


I sent it via email. It had someone's personal information in it so I didn't want to make it public without their permission.

It follows the same pattern. I used another few messages to come up with my theory before and the one I sent is another example that follows the pattern.

Let me know if you need anything more...

Cheers,

James.
Re: NNTP Import of Messages without Content-Type in Header [message #161836 is a reply to message #161831] Fri, 12 February 2010 09:17 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
I've looked at the code (charset_convert() in scripts_common.inc) and I don't think this is going to work. If we convert from X to Y. But if X==Y noting is going to happen.

For example, forum charset is UTF-8; post doesn't specify charset, so we assume it is UTF-8. No conversion is implied.

Any other ideas?
Re: NNTP Import of Messages without Content-Type in Header [message #161840 is a reply to message #161836] Fri, 12 February 2010 10:50 Go to previous messageGo to next message
jm2morri is currently offline  jm2morri   Canada
Messages: 225
Registered: January 2009
Location: Stratford, Ontario, Canad...
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
naudefj wrote on Fri, 12 February 2010 09:17
I've looked at the code (charset_convert() in scripts_common.inc) and I don't think this is going to work. If we convert from X to Y. But if X==Y noting is going to happen.

For example, forum charset is UTF-8; post doesn't specify charset, so we assume it is UTF-8. No conversion is implied.

Any other ideas?


Can we scan the actual text and deduce what the charset is? Somehow Thunderbird displays these messages fine so it must be doing something.

Or how about you look for the Outlook Express signature (not sure if the full Outlook client has the same issue) and then pick the assume that the charset is the windows one and convert from that to charset of forum?

Every case I've seen of this has Outlook Express as the client.

Cheers,

James.
Re: NNTP Import of Messages without Content-Type in Header [message #161841 is a reply to message #161840] Fri, 12 February 2010 12:13 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3624
Registered: December 2004
Karma: 17
Senior Member
Administrator
Core Developer
remove from buddy list
ignore all messages by this user
It might work. But it's more than what I'm willing to take on right now. Patches, however, are always welcome.
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: XML Aggregation :: Variable not Defined Warning
Next Topic: Query failed: ERROR: value too long for type character varying(100)
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Oct 21 23:01:01 EDT 2017

Total time taken to generate the page: 0.00623 seconds