summaryrefslogtreecommitdiff
path: root/inc/utf8.php
Commit message (Collapse)AuthorAge
* Last fixes for Japanese Romanization. Now all 22893 tests succeed.Andreas Gohr2008-06-08
| | | | darcs-hash:20080608113523-7ad00-81e25091d59c2333f4f82f1cf61321155b03f895.gz
* Japanese romanization updateAndreas Gohr2008-05-08
| | | | | | Down to 57 fails darcs-hash:20080508212444-7ad00-16286e9f5be2bbbd3069d5c22ab8c270b2e1b23e.gz
* Updates for Japanese romanization support FS#1363Andreas Gohr2008-05-06
| | | | | | | This patch adds some fixes for the romanization lookup table and a test case for more than 20000 phrases and their correct romanization. About 2100 tests currently fail. darcs-hash:20080506203707-7ad00-9d95b8af459fa44c8d3e95560c7e1c116b8ffc48.gz
* Fixes for Japanese romanization FS#1363Denis Scheither2008-04-07
| | | | darcs-hash:20080407174238-84fef-88cae1548503760595a19f00e03060604303b934.gz
* utf8_trim bugfixAndreas Gohr2007-11-02
| | | | | | Fixes the utf8_trim() function when a charlist is given darcs-hash:20071102181430-7ad00-4160d3d47b53e9c0db76328004c1f95c76d590e6.gz
* fixed Thai romanizationAndreas Gohr2007-10-15
| | | | darcs-hash:20071015170603-7ad00-cce18a874fa1857af1717519cac14e86f986c7f2.gz
* removed unnessary UTF-8 replacement functionsAndreas Gohr2007-07-19
| | | | darcs-hash:20070719130041-7ad00-84d00f6385973e6f2f9499374c3c1d475eecb715.gz
* several speed improvements in UTF-8 libAndreas Gohr2007-07-19
| | | | darcs-hash:20070719110142-7ad00-1192e190c62637ed68e2c2c0a0b3135abfd6ecb5.gz
* Escape Ctrl-Z so darcs stops treating utf8.php as binary.Tom N Harris2007-03-23
| | | | darcs-hash:20070323030243-6942e-836105b95078b213df8497386ae9b0418fcf29be.gz
* Encode/Decode numeric HTML entities correctly.Tom N Harris2007-02-02
| | | | | | | utf8_tohtml handles all codepoints, and the inverse function, utf8_unhtml, is added. darcs-hash:20070202070509-6942e-09ed9dc37f1469055a7c04d44044768e160d60e6.gz
* tf_rename_lower.patchhenning.noren2007-01-03
| | | | | | | Name the TRUE/FALSE-constants consistently as lowercase everywhere. This might also be an tiny optimization in some environments. darcs-hash:20070103205700-d2a3e-e7ec0aedb938d563f583116a2d5b17f3a3fea36c.gz
* Indexer asian language fixes and speed-upsTom N Harris2006-11-17
| | | | | | | | | Make Chinese and Japanese work better with the new indexer. Some missing punctuation added to utf8_stripspecials. Misc. other changes to make indexing faster. The indexes will expire on backend upgrades, so you don't have to delete *.indexed darcs-hash:20061117123032-6942e-774b38e08234928c49b37e40addba375acf67ac0.gz
* do not transliterate cyrillic soft sign #958Andreas Gohr2006-10-28
| | | | darcs-hash:20061028113426-7ad00-f1d6b3b919c3aadd2bd7585fb772071b81b4b42d.gz
* more utf8_substr improvements (re FS#891 and yesterday's patch)chris2006-09-28
| | | | | | | - rework utf8_substr() NOMBSTRING code to always use pcre - remove work around for utf8_substr() and large strings from ft_snippet() darcs-hash:20060928165122-9b6ab-0eefc216f07f9d7e7d8eb62ce26605c28ee340fa.gz
* utf8_substr fix for FS#891chris2006-09-27
| | | | darcs-hash:20060927033713-9b6ab-4b35e0a85b6d11d5a3a98858cd2f860b383ff153.gz
* utf8_stripspecials optimizationchris2006-09-23
| | | | | | | | | | | | | | | Add preconverted utf-8 string of special characters. The (once only) conversion of the special character unicode array into utf-8 occurs on every DokuWiki page view, irrespective of action or caching, and takes about one third of the time involved in delivering a wiki page straight from cache. The original unicode array has been left in place in the file to make any future amendments easier. darcs-hash:20060923151937-9b6ab-cae0340a95d9596415ef71d7b7e67ef9daca84ef.gz
* search improvementschris2006-08-31
| | | | | | | | | | | | | | | | | | | ft_snippet() - make utf8 algorithm default - add workaround for utf8_substr() limitations, bug #891 - fix some indexes which missed out on conversion to utf8 character counts - minor improvements idx_lookup() - minor changes to wildcard matching code to improve performance (changes based on profiling results) utf8 - specifically set mb_internal_coding to utf-8 when mb_string functions will be used. darcs-hash:20060831003413-9b6ab-712021eda3c959ffe79d8d3fe91d2c9a8acf2b58.gz
* further update to global memory cache arrayschris2006-08-29
| | | | | | | | | | | | | - remove initialisation of caches in inc/pageutils.php - add global declaration to init.php to support init.php being included from within a function, e.g. unit testing ;-) - minor change to utf8_substr, remove non-essential brackets added as part of an earlier patch darcs-hash:20060829134806-9b6ab-ab15191344a83be664c412403dc84a24fa2253a2.gz
* utf8_substr() fix, it wasn't using mb_substr results when availablechris2006-08-28
| | | | darcs-hash:20060828092029-9b6ab-f76c94b76ce1ada49e2fefde11af824bb98b99c7.gz
* utf8_correctIdx bounds checking and more unittestschris2006-08-27
| | | | darcs-hash:20060827153254-9b6ab-3c76fde7cb5534ca12628e9aa6e6d59d9bb02f45.gz
* ft_snippet() update, fix utf8 problemschris2006-08-26
| | | | darcs-hash:20060826095311-9b6ab-9a6f272cc7c7532eb2bad8f7b4404c5a16b71109.gz
* code to remove bad UTF-8 bytes addedAndreas Gohr2006-08-26
| | | | | | | This adds code to remove or replace invalid UTF-8 bytes and uses it in the ft_snippets function. darcs-hash:20060826082919-7ad00-a94004de159ae93ff5b7270fd3e631ff467233cd.gz
* cleanID unit testschris2006-08-09
| | | | | | | + fix missing utf8 deaccent character mapping + set utf-8 charset for HTMLReporter (unit tests) darcs-hash:20060809160209-9b6ab-26c80a4830643b9790536f6d3a4adee0f451e4f0.gz
* make sure UTF8 lookup tables are always globalAndreas Gohr2006-06-11
| | | | darcs-hash:20060611173240-7ad00-9bc775163fd9fc65917ffe10f78f872a302bdbcf.gz
* fix for utf8_strpos #827Andreas Gohr2006-06-08
| | | | darcs-hash:20060608200438-7ad00-05fbb18c15df64725ca4ef1ffdc0aa817a508ea4.gz
* more efficient mb_string checking in utf8.phpAndreas Gohr2006-04-03
| | | | darcs-hash:20060403194930-7ad00-499940017f74cfe297f2aa4e65d441243f8572a1.gz
* better utf8_substr functionAndreas Gohr2006-04-03
| | | | darcs-hash:20060403192537-7ad00-72b129ce494066bce491821a0396db7576873ec2.gz
* nicer russian romanizationDenis Simakov2006-03-03
| | | | darcs-hash:20060303032557-3c565-36015a29e83f000f0a23d8ea039c954766c1223e.gz
* hebrew romanization fixDenis Simakov2006-03-03
| | | | darcs-hash:20060303031656-3c565-2458122a2481ea3acfbf772e4faae883808cbf71.gz
* fixes for utf-8 to/from unicode conversionAndreas Gohr2006-02-21
| | | | | | | | The functions utf8_to unicode and unicode_to_utf8 didn't work correctly with some 3 and 4 byte strings. This exchanges those functions against two more sophisticated ones. It also adds unit testing for them. darcs-hash:20060221212605-7ad00-7bfefe8c9615d5a7f3b33c279ce79d4200d4778c.gz
* file cleanupsAndreas Gohr2006-02-17
| | | | | | | | | | This patch cleans up the source code to satisfy the coding guidelines (see http://wiki.splitbrain.org/wiki:development#coding_style) It converts files to UNIX lineendings and removes tabs and trailing whitespace. Not all files were cleaned yet. darcs-hash:20060217222040-7ad00-bba3d2bee3b5aa7cbb5184258abd50805cd071bf.gz
* romanization support in utf8 libraryAndreas Gohr2006-02-10
| | | | | | | | | | | | | | | | | This patch addes basic romanization support to the utf-8 library. It converts non-latin languages to ASCII. The transliteration tables used where gathered from various places on the net. I do not speak any of those languages so I can't say how good they are. Any recommendations and fixes are welcome! This can be enabled for ID cleaning by setting the deaccent option to 2. It is also used in the XHTML renderer to generate section ids based on the header titles. Leading digits and any remaining non-ASCII chars are removed as well. This is the first step to make section ID always XHTML compatible. Making sure they are unique is not implemented yet. darcs-hash:20060210200627-7ad00-61a633563bb92a00ef4a3f699d73117139cbf367.gz
* Wildcardsearch added #552 #632Andreas Gohr2005-11-27
| | | | | | | | | | | Now searching for word parts is possible by adding or prepending a * character to the searchword: 'foo*' searches for words beginning with 'foo' eg. 'foobar' '*foo' looks for words ending in 'foo' eg. 'barfoo' '*foo*' gets anything with 'foo' in it eg. 'barfoobaz' darcs-hash:20051127180723-7ad00-1eb29e812ddaf38d9812697bb1cffffe9a5fb330.gz
* fixed two bugs reported by Timo FalkAndreas Gohr2005-11-08
| | | | darcs-hash:20051108195013-7ad00-f2b2b6d955e64fe2d2abfc1d9421ed619f70abd3.gz
* get_tests_running_1hfuecks2005-11-07
| | | | darcs-hash:20051106233800-e96b6-90328f1b6cc5c8b25c6e02f113cf80475789d297.gz
* removed mb_string requirement in JSON.php #592Andreas Gohr2005-10-17
| | | | darcs-hash:20051017200431-7ad00-c9ed74326a44b078cc516e08515375d9d0fba811.gz
* fixed $UTF8_UPPER_ACCENTS arrayAndreas Gohr2005-10-06
| | | | darcs-hash:20051006175402-7ad00-f0bccfd8a14f231aab7c15a571628f9320e4fead.gz
* small fix for utf8.phpAndreas Gohr2005-09-18
| | | | | | utf8_strtoupper called mb_strtolower darcs-hash:20050918094308-7ad00-c595455bcb6e312f29c0d27fa83d0905cd2303cc.gz
* a first step for search indexing - nothing to see yetAndreas Gohr2005-08-07
| | | | darcs-hash:20050807203322-7ad00-6db6733f8fcd861366856635ba3d205fd3bb54da.gz
* relicensed utf8 lib under LGPL to make it available to a broader audienceandi2005-06-28
| | | | darcs-hash:20050628201920-9977f-91d30cb4391f49979415edf31b798e31177d76c2.gz
* spellchecker fixes for Konqerorandi2005-06-11
| | | | | | Konqeror seems to ignore the charset darcs-hash:20050611092916-9977f-311b99e63b66a83c9f9022c468ba61d0687822c2.gz
* AJAX spellchecker #29andi2005-06-07
| | | | | | | | | | | | | | | | | | | | This is nearly a complete rewrite of the gmail like AJAX spellchecker from http://www.broken-notebook.com/spell_checker/index.php Here are the differences and features * seemless integrated into DokuWiki * no need for the pspell extension * needs GNU aspell installed (not sure about the version I guess 0.60+ for UTF8) * needs PHP 4.3.0+ * uses SACK for AJAX * gets errors and suggestions in one transfer So far only tested in Firefox. It should work in IE, Safari and Opera 8, too. Please test and report back. darcs-hash:20050607194456-9977f-f699144d1fd28359742b2ce0f28c839a1f4cefbb.gz
* fixed problem with utf8_substr and feed generation #340andi2005-05-20
| | | | darcs-hash:20050520203448-9977f-4381d98899e730a301c655b5aae59e83b5580a88.gz
* performance enhancement for utf8_stripspecialsandi2005-04-29
| | | | darcs-hash:20050429211602-9977f-70bda6c598e239d227663a6c2ff1c19fbf069bb8.gz
* optimized count calls in for loopsandi2005-04-27
| | | | darcs-hash:20050427182243-9977f-4c1ec6ce278925147379430540eec723d109444a.gz
* removed closing php tag for includesandi2005-04-13
| | | | darcs-hash:20050413175338-9977f-c969db6f86df5df175db9a39b4fabb9f072f8f67.gz
* UTF8 performance enhancementsandi2005-03-19
| | | | darcs-hash:20050319141940-9977f-480f5b4d61060e1c36774859677671cb7cb32e4b.gz
* better mailheader handling #168andi2005-02-27
| | | | darcs-hash:20050227094102-9977f-a481ce76ccdb40f8eaa762a332b93093de8f7a87.gz
* fix for #164andi2005-02-25
| | | | darcs-hash:20050225164823-9977f-c45bc34160b06a1bc1aa41c1f238daaeaae4c7ff.gz
* added init.php - may have broken something! (related to #153)andi2005-02-19
| | | | darcs-hash:20050219102055-9977f-575d654e742934c911ffab855d82aa91f198b5cf.gz
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-06 20:43:48 +0000