FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » FUDforum Development » Plugins and Code Hacks » Multi-Bytes Language full text search engine Hack..
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
Multi-Bytes Language full text search engine Hack.. [message #8615] Tue, 11 February 2003 04:40 Go to next message
hightman is currently offline  hightman   China
Messages: 4
Registered: February 2003
Location: China
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user

This code Hack make it is possible to do a full text search with multi-byte languages support such as chinese...

You can try it by visiting http://forum.hightman.net/

the following is code: [please rebulid the theme and rebuild the search index after modifing the php code.

-----------------------------------
/path/to/data/src/isearch.inc.t
-----------------------------------
1. add two functions befor 'function index_text($subj, $body, $msg_id)'

[code]
function valid_chr($ch)
{
return ( ($ch >= 97 && $ch <= 122) ||
($ch >= 65 && $ch <= 90) ||
($ch >= 48 && $ch <= 57) ||
($ch & 0x80) );
}

function split_word($str)
{
$arr = array();
$len = strlen($str);
$j = 0;
$cat_mode = 0;

for ( $i = 0; $i < $len; $i++ )
{
$ia = ord($str[$i]);

if ( !valid_chr($ia) )
{
if ( $cat_mode == 1 )
$j++;

$cat_mode = 0;
}
else if ( $ia & 0x80 )
{
if ( $cat_mode == 1 )
{
$j++;
$cat_mode = 0;
}

$arr[$j] = substr($str, $i, 2);
$i++;
$j++;
}
else
{
$cat_mode = 1;
$arr[$j] .= $str[$i];
}
}
return $arr;
}
[/code]

2. change function index_text();
/* build full text index */
//$w = explode(' ', trim($subj.' '.$body));
$w = split_word(trim($subj.' '.$body));

... ...
for ( $i=0; $i<$a; $i++ ) {
//if ( strlen($w[$i]) > 50 || strlen($w[$i])<3 ) continue;
if ( strlen($w[$i]) > 50 || strlen($w[$i])<2 ) continue;
... ...
/* build subject only index */
//$w = explode(' ', $subj);
$w = split_word($subj);

for ( $i=0; $i<count($w); $i++ ) {
//if ( strlen($w[$i]) > 50 || strlen($w[$i])<3 ) continue;
if ( strlen($w[$i]) > 50 || strlen($w[$i])<2 ) continue;

... ...

3. change function search()

function search($str, $fld, $start, $count, $forum_limiter='', &$total)
{
//$w = explode(" ", $str);
$w = split_word($str);

... ...
Re: Multi-Bytes Language full text search engine Hack.. [message #8616 is a reply to message #8615] Tue, 11 February 2003 04:41 Go to previous message
hightman is currently offline  hightman   China
Messages: 4
Registered: February 2003
Location: China
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user

If you want to make the keyword highligh like http://forum.hightman.net you should change another file:

/path/to/data/src/search.php.t
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: Need a hack.
Next Topic: Newsgroup Import Hack?
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Sat Oct 21 10:15:21 EDT 2017

Total time taken to generate the page: 0.00704 seconds