FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Bug Reports » Bugs in V.2.8.1 RC1
Show: Today's Messages :: Polls :: Message Navigator
Switch to threaded view of this topic Create a new topic Submit Reply
Re: Bugs in V.2.8.1 RC1 [message #158968 is a reply to message #158963] Mon, 20 April 2009 07:46 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
naudefj wrote on Sun, 19 April 2009 21:57
Strange, I can register other users on my test forum. What is on line 78?


Yip, "russian" is now UTF-8. Like all the other translations.



Strange, yes you are right, I should have looked at that line myself.
I checked now it only happens when you use danish as default language.

It is caused by my danish translation file, the registration form where I did use ' instead of " which breaks the php logic. I will send in a corrected msg-file later, (looking for more mistakes first)


I still only can compile the foreign utf-8 characters when i use my "patch" in compiler.inc line 187 to 194:

// if (stristr($GLOBALS['char_set'], 'utf-8') !== false && substr(PHP_OS, 0, 3) == "WIN") {
// Windows doesn't have UTF-8 locales
$func = '\'.utf8_encode(strftime("';
$close = ')).\'';
// } else {
// $func = '\'.strftime("';
// $close = ').\'';
// }

My system runs on Opensuse 11.0

Re: Bugs in V.2.8.1 RC1 [message #158990 is a reply to message #158968] Mon, 20 April 2009 19:10 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
That explains why the condition is not triggered (substr(PHP_OS, 0, 3) != "WIN").

You need to use a UTF-8 locale. For example, try da_DK.UTF-8.
Re: Bugs in V.2.8.1 RC1 [message #158997 is a reply to message #158990] Wed, 22 April 2009 09:32 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
Ok, DA_da.UTF-8 works with compiler.inc, but in Wiki you write:

" Locale is the setting FUDforum will export to PHP making php automatically format various strings such as dates according to certain language/country. This field is automatically filled in using JavaScript when you select the language, however you can change the value manually if you believe you have a better alternative to the one offered.

If you are unsure of what this option does, it is better to leave the default value as the locale setting."

I've never seen that field automatically filled with any UTF-8 versions, but should be from the below list, would be nice if it was. Should the UTF-8 part even be necessary as all msg-files are in utf-8? It is mandatory now for the international characters to appear in dates.


af_ZA.UTF-8, am_ET.UTF-8, an_ES.UTF-8, ar_AE.UTF-8, ar_BH.UTF-8, ar_DZ.UTF-8, ar_EG.UTF-8, ar_IN.UTF-8, ar_IQ.UTF-8, ar_JO.UTF-8, ar_KW.UTF-8, ar_LB.UTF-8, ar_LY.UTF-8, ar_MA.UTF-8, ar_OM.UTF-8, ar_QA.UTF-8, ar_SA.UTF-8, ar_SD.UTF-8, ar_SY.UTF-8, ar_TN.UTF-8, ar_YE.UTF-8, as_IN.UTF-8, ast_ES.UTF-8, az_AZ.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, bn_BD.UTF-8, bn_IN.UTF-8, br_FR.UTF-8, bs_BA.UTF-8, byn_ER.UTF-8, ca_AD.UTF-8, ca_ES.UTF-8, ca_FR.UTF-8, ca_IT.UTF-8, cs_CZ.UTF-8, cy_GB.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_BE.UTF-8, de_CH.UTF-8, de_DE.UTF-8, de_LU.UTF-8, el_CY.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_BE.UTF-8, en_BW.UTF-8, en_CA.UTF-8, en_DK.UTF-8, en_GB.UTF-8, en_HK.UTF-8, en_IE.UTF-8, en_IN.UTF-8, en_NZ.UTF-8, en_PH.UTF-8, en_SG.UTF-8, en_US.UTF-8, en_ZA.UTF-8, en_ZW.UTF-8, es_AR.UTF-8, es_BO.UTF-8, es_CL.UTF-8, es_CO.UTF-8, es_CR.UTF-8, es_DO.UTF-8, es_EC.UTF-8, es_ES.UTF-8, es_GT.UTF-8, es_HN.UTF-8, es_MX.UTF-8, es_NI.UTF-8, es_PA.UTF-8, es_PE.UTF-8, es_PR.UTF-8, es_PY.UTF-8, es_SV.UTF-8, es_US.UTF-8, es_UY.UTF-8, es_VE.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fa_IR.UTF-8, fi_FI.UTF-8, fo_FO.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, fr_LU.UTF-8, ga_IE.UTF-8, gd_GB.UTF-8, gl_ES.UTF-8, gv_GB.UTF-8, he_IL.UTF-8, hi_IN.UTF-8, hr_HR.UTF-8, hsb_DE.UTF-8, hu_HU.UTF-8, id_ID.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, iw_IL.UTF-8, ja_JP.UTF-8, ka_GE.UTF-8, kk_KZ.UTF-8, kl_GL.UTF-8, ko_KR.UTF-8, ku_TR.UTF-8, kw_GB.UTF-8, lg_UG.UTF-8, lt_LT.UTF-8, lv_LV.UTF-8, mg_MG.UTF-8, mi_NZ.UTF-8, mk_MK.UTF-8, ml_IN.UTF-8, mn_MN.UTF-8, mr_IN.UTF-8, ms_MY.UTF-8, mt_MT.UTF-8, nb_NO.UTF-8, ne_NP.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, nn_NO.UTF-8, no_NO.UTF-8, oc_FR.UTF-8, om_ET.UTF-8, om_KE.UTF-8, pa_IN.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, ru_UA.UTF-8, se_NO.UTF-8, sh_YU.UTF-8, sid_ET.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, so_DJ.UTF-8, so_ET.UTF-8, so_KE.UTF-8, so_SO.UTF-8, sq_AL.UTF-8, st_ZA.UTF-8, sv_FI.UTF-8, sv_SE.UTF-8, ta_IN.UTF-8, te_IN.UTF-8, tg_TJ.UTF-8, th_TH.UTF-8, ti_ER.UTF-8, ti_ET.UTF-8, tig_ER.UTF-8, tl_PH.UTF-8, tr_CY.UTF-8, tr_TR.UTF-8, tt_RU.UTF-8, uk_UA.UTF-8, ur_PK.UTF-8, wa_BE.UTF-8, xh_ZA.UTF-8, yi_US.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_SG.UTF-8, zh_TW.UTF-8, zu_ZA.UTF-8, aa_DJ.UTF-8, aa_ER.UTF-8, aa_ET.UTF-8,
Re: Bugs in V.2.8.1 RC1 [message #158998 is a reply to message #158963] Wed, 22 April 2009 09:36 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
about russian-utf8, I hope you fetched it is not in the 2.81RC1 distribution.
Re: Bugs in V.2.8.1 RC1 [message #159011 is a reply to message #158997] Wed, 22 April 2009 14:02 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
The documentation is slightly dated. We copied it to the wiki so it can be updated and kept up-to-date by the community. You are welcome to hit "Edit" to fix it up for us.

The correct locale for Danish is "danish" on Windows and "DA_da.UTF-8" on others. No matter what we make the default, someone is going to complain. If you feel serious about this, you are welcome to develop a patch to set the default based on the OS FUDforum is installed on.

The "russian-utf8" translation was permanently removed and will not be available with any future FUDforum version. You need to start using "russian" instead.

Best regards.

Frank
Re: Bugs in V.2.8.1 RC1 [message #159012 is a reply to message #159011] Wed, 22 April 2009 16:35 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
But there is no russian in 2.8.1RC1 either.

About what I like to get fixed most of all is the import of international characters from maillist and newslist. Do you think that this problem is on Linux systems only? I see the same problem in other forums like http://www.bythedrop.com/forum2/index.php/f/3/
(newlist import)

In all cases, if I should find the reason for the characters either not appearing at all or with wrong coding I need some way to debug the fudforum source, starting with maillist.php and nntp.php

Do you know of a good debugger one could use for this, open source and possible to use for a beginner?
Re: Bugs in V.2.8.1 RC1 [message #159014 is a reply to message #159012] Wed, 22 April 2009 18:16 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
The Russian translation is in CVS - I have no idea why it's not in the package. I'll ask Ilia to investigate.

The strange characters can also be due to misconfiguration or badly encoded mails. If you have a reproducible test case, we can look into it.
Re: Bugs in V.2.8.1 RC1 [message #159017 is a reply to message #159014] Wed, 22 April 2009 20:03 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
I can give you acess to my system, as I deleted the 2.8.0 install with som private messages and now are the only user on a 2.8.1RC1 install.
Re: Bugs in V.2.8.1 RC1 [message #159026 is a reply to message #159017] Fri, 24 April 2009 11:24 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
Character conversion goes to Windows locals, should go to Linux locales.

In the source, I find this sections doing all conversion. Is this true?




from maillist.php,v 1.77

line 407-410:
list($GLOBALS['usr']->lang, $locale) = db_saq("SELECT lang, locale FROM ".sql_p."themes WHERE theme_opt=1|2 LIMIT 1");

/* set locale */
$GLOBALS['good_locale'] = setlocale(LC_ALL, $locale);

line 472-486:
$msg_post->body = apply_custom_replace($msg_post->body);
if (!($mlist->mlist_opt & 16)) {
if ($frm->forum_opt & 16) {
$msg_post->body = tags_to_html($msg_post->body, 0);
} else {
$msg_post->body = nl2br($msg_post->body);
}
}

fud_wordwrap($msg_post->body);
$msg_post->subject = htmlspecialchars(apply_custom_replace($emsg->subject));
if (!strlen($msg_post->subject)) {
mlist_error_log("Blank Subject", $emsg->raw_msg);
$msg_post->subject = "(no subject)";
}
--------------------------------------------------------------
from replace.inc.t,v 1.20:

function apply_custom_replace($text)
{
if (!defined('__fud_replace_init')) {
make_replace_array();
}
if (empty($GLOBALS['__FUD_REPL__'])) {
return $text;
}

return preg_replace($GLOBALS['__FUD_REPL__']['pattern'], $GLOBALS['__FUD_REPL__']['replace'], $text);
}

function make_replace_array()
{
$GLOBALS['__FUD_REPL__']['pattern'] = $GLOBALS['__FUD_REPL__']['replace'] = array();
$a =& $GLOBALS['__FUD_REPL__']['pattern'];
$b =& $GLOBALS['__FUD_REPL__']['replace'];

$c = uq('SELECT with_str, replace_str FROM {SQL_TABLE_PREFIX}replace WHERE replace_str IS NOT NULL AND with_str IS NOT NULL AND LENGTH(replace_str)>0');
while ($r = db_rowarr($c)) {
$a[] = $r[1];
$b[] = $r[0];
}
unset($c);

define('__fud_replace_init', 1);
}
Re: Bugs in V.2.8.1 RC1 [message #159027 is a reply to message #159026] Fri, 24 April 2009 14:22 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
The default locale is defined the "locale" file (thm/default/i18n/danish directory). The installer or Theme Manager will then populate the fud28_themes table.

So, if you want to write a fix, we probably need to have 2 locale files, say "locale" and "locale_win".

Good luck with your patch.

Best regards.

Frank
Re: Bugs in V.2.8.1 RC1 [message #159043 is a reply to message #159027] Mon, 27 April 2009 09:32 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
A short update,

I found a partly solution, whitch works for the body part of the message.
The problem is not part of the former in this thread posted source code (which shows up to deal with word mangling), nor is caused by bad initiaton of locales.
The problem is, that the decoded charset from the parsing af headers is not used at all. That results in only emails coded in same charset-code as the system are handelt correctly.

We need to put the decoded charset in function decode_message_body(), like this:

maillist.php ver 1.77, line 215-218:
function decode_message_body()
{
$this->body = decode_string($this->body, $this->headers['content-transfer-encoding'], $this->headers['__other_hdr__']['content-type']['charset']);
}


Now I need to find a similar solution for the subject part of the message.
Re: Bugs in V.2.8.1 RC1 [message #159047 is a reply to message #159043] Mon, 27 April 2009 20:05 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
The subject must be Q-encoded. Here is an example (PHP5 only, unfortunately):

iconv_mime_encode("Subject", $subject,
array('scheme'=>'Q', "input-charset"=>$GLOBALS['CHARSET'],"output-charset"=>$GLOBALS['CHARSET']))."\r\n");
Re: Bugs in V.2.8.1 RC1 [message #159051 is a reply to message #159047] Tue, 28 April 2009 19:38 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
naudefj wrote on Mon, 27 April 2009 22:05
The subject must be Q-encoded. Here is an example (PHP5 only, unfortunately):

iconv_mime_encode("Subject", $subject,
array('scheme'=>'Q', "input-charset"=>$GLOBALS['CHARSET'],"output-charset"=>$GLOBALS['CHARSET']))."\r\n");



using iconv_mime_decode in stead of decode_header_value in maillist.php solves the problem, but I don't know if ther are other effects.

2 testmails I use for this testing imported without problems now, one coded in ISO-8859-1, the other in UTF-8.


change maillist.php ver 1.77 like this:
(together with earlier patch)
line 56:
if (!isset($this->headers[$hk])) {
$this->headers[$hk] = iconv_mime_decode($hv,2,'UTF-8');
} else { $this->headers[$hk] .= ' '.iconv_mime_decode($hv,2,'UTF-8');
}
Re: Bugs in V.2.8.1 RC1 [message #159052 is a reply to message #159051] Tue, 28 April 2009 19:47 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
another solution would be trying to repair decode_header_value from scripts_common.inc,v 1.36

In my solution, I used int $mode=2 in iconv_mime_decode, could also be 0 ore 1, don't know what is best.
Re: Bugs in V.2.8.1 RC1 [message #159057 is a reply to message #159052] Wed, 29 April 2009 20:42 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
Looks like you solved this problem. Can you please prepare a patch that we can apply to CVS?
Re: Bugs in V.2.8.1 RC1 [message #159060 is a reply to message #159057] Thu, 30 April 2009 14:34 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
Hey, I think i need to know a diff system, made this by hand, hope it works.



diff -u -r1.77 maillist.php
--- maillist.php 2009/02/28 11:47:18 1.77
+++ maillist.php
@@ -215,3 +215,3 @@


function decode_message_body()
{
- $this->body = decode_string($this->body, $this->headers['content-transfer-encoding']);
+ $this->body = decode_string($this->body, $this->headers['content-transfer-encoding'], $this->headers['__other_hdr__']['content-type']['charset']);
}


then I think we just change function decode_header_value($val) in scripts_common.inc to be iconv_mime_decode($val,2,'UTF-8') for php >= 5

diff for scripts_common.inc

??????????????????????
Re: Bugs in V.2.8.1 RC1 [message #159071 is a reply to message #159060] Sat, 02 May 2009 12:40 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
You can make a patch by running one of the following commands:
$ diff -u oldfile newfile >patch

or if you have CVS installled:
$ cvs diff -u filename


Please check if you still need the "iconv_mime_decode" if you apply this patch to scripts_common.inc:

diff -u -r1.36 scripts_common.inc
--- scripts_common.inc  15 Feb 2009 20:01:03 -0000      1.36
+++ scripts_common.inc  2 May 2009 12:36:48 -0000
@@ -147,9 +147,9 @@
                        $ec_type = strtolower($m[4][$i]);

                        if ($ec_type == 'q') {
-                               $newval .= decode_string(str_replace('_', ' ', $m[5][$i]), 'quoted-printable');
+                               $newval .= decode_string(str_replace('_', ' ', $m[5][$i]), 'quoted-printable', $m[3][$i]);
                        } else if ($ec_type == 'b') {
-                               $newval .= decode_string($m[5][$i], 'base64');
+                               $newval .= decode_string($m[5][$i], 'base64', $m[3][$i]);
                        }

                        if (!empty($m[5][$i])) {


Best regards.

Frank
Re: Bugs in V.2.8.1 RC1 [message #159076 is a reply to message #159071] Sat, 02 May 2009 18:35 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
Yes, that works much better then (no change), but there still is a problem, wich does not occur with "iconv_mime_decode":

In case where the Q-encoding switches back and force, the 'space' char is lost between the last to undecoded words, example subject send by Kmail:

original subject text:
nr.4 med æ å ø og flere og mange
decoded with your latest patch:
nr.4 med æ å ø og flere ogmange
coding:
nr.4 med =?iso-8859-1?q?=E6_=E5_=F8_og_flere_og?= mange


We see the last space is lost. How can this get repaired without going back to "iconv_mime_decode"?


Thank yu for explaining for me the fundamentals of using diff, so next time I will be better prepaired

Peter
Re: Bugs in V.2.8.1 RC1 [message #159088 is a reply to message #159076] Sun, 03 May 2009 18:46 Go to previous messageGo to next message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
Thanks, this should fix it: http://cvs.prohost.org/c/index.cgi/FUDforum/chngview?cn=11920
Re: Bugs in V.2.8.1 RC1 [message #159102 is a reply to message #159088] Mon, 04 May 2009 12:13 Go to previous messageGo to next message
Peter Vendike is currently offline  Peter Vendike   Denmark
Messages: 65
Registered: February 2009
Location: Denmark
Karma: 0
Member
Translator
Yes, this works for my example messages, also without "iconv_mime_decode", wonder why that "space" was taken out originally.

Thanks, no more problems left for me in this thread.
Re: Bugs in V.2.8.1 RC1 [message #159248 is a reply to message #159102] Fri, 15 May 2009 14:30 Go to previous message
naudefj is currently offline  naudefj   South Africa
Messages: 3771
Registered: December 2004
Karma: 28
Senior Member
Administrator
Core Developer
Feedback from Ilia:

Quote:
The russian bit is now fixed, it was a bug in the packaging script (the one that I use to push final releases) it was removing legacy russian/ directory which is once again being used.


So, I guess it's all systems GO for the final 2.8.1 release.

Best regards.

Frank
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: NNTP Errors
Next Topic: Message Navigator: No logical search function
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Thu Apr 25 22:32:36 GMT 2024

Total time taken to generate the page: 0.02502 seconds