The fact that MS-word and some other sources use CP-1252, and that it is so close to Latin1 ('ISO-8859-1') causes a lot of confusion. What confused me the most was finding that mySQL uses CP-1252 by default.
You may run into trouble if you find yourself tempted to do something like this:
<?php
$trans[chr(149)] = '•'; // Bullet
$trans[chr(150)] = '–'; // En Dash
$trans[chr(151)] = '—'; // Em Dash
$trans[chr(152)] = '˜'; // Small Tilde
$trans[chr(153)] = '™'; // Trade Mark Sign
?>
Don't do it. DON'T DO IT!
You can use:
<?php
$translationTable = get_html_translation_table(HTML_ENTITIES, ENT_NOQUOTES, 'WINDOWS-1252');
?>
or just convert directly:
<?php
$output = htmlentities($input, ENT_NOQUOTES, 'WINDOWS-1252');
?>
But your web page is probably encoded UTF-8, and you probably don't really want CP-1252 text flying around, so fix the character encoding first:
<?php
$output = mb_convert_encoding($input, 'UTF-8', 'WINDOWS-1252');
$ouput = htmlentities($output);
?>
get_html_translation_table
(PHP 4, PHP 5)
get_html_translation_table — Returns the translation table used by htmlspecialchars() and htmlentities()
Description
$table = HTML_SPECIALCHARS
[, int $flags = ENT_COMPAT | ENT_HTML401
[, string $encoding = 'UTF-8'
]]] )get_html_translation_table() will return the translation table that is used internally for htmlspecialchars() and htmlentities().
Note:
Special characters can be encoded in several ways. E.g. " can be encoded as ", " or ". get_html_translation_table() returns only the form used by htmlspecialchars() and htmlentities().
Parameters
-
table -
Which table to return. Either
HTML_ENTITIESorHTML_SPECIALCHARS. -
flags -
A bitmask of one or more of the following flags, which specify which quotes the table will contain as well as which document type the table is for. The default is ENT_COMPAT | ENT_HTML401.
Available flagsconstantsConstant Name Description ENT_COMPATTable will contain entities for double-quotes, but not for single-quotes. ENT_QUOTESTable will contain entities for both double and single quotes. ENT_NOQUOTESTable will neither contain entities for single quotes nor for double quotes. ENT_HTML401Table for HTML 4.01. ENT_XML1Table for XML 1. ENT_XHTMLTable for XHTML. ENT_HTML5Table for HTML 5. -
encoding -
Encoding to use. If omitted, the default value for this argument is ISO-8859-1 in versions of PHP prior to 5.4.0, and UTF-8 from PHP 5.4.0 onwards.
The following character sets are supported:
Supported charsets Charset Aliases Description ISO-8859-1 ISO8859-1 Western European, Latin-1. ISO-8859-5 ISO8859-5 Little used cyrillic charset (Latin/Cyrillic). ISO-8859-15 ISO8859-15 Western European, Latin-9. Adds the Euro sign, French and Finnish letters missing in Latin-1 (ISO-8859-1). UTF-8 ASCII compatible multi-byte 8-bit Unicode. cp866 ibm866, 866 DOS-specific Cyrillic charset. cp1251 Windows-1251, win-1251, 1251 Windows-specific Cyrillic charset. cp1252 Windows-1252, 1252 Windows specific charset for Western European. KOI8-R koi8-ru, koi8r Russian. BIG5 950 Traditional Chinese, mainly used in Taiwan. GB2312 936 Simplified Chinese, national standard character set. BIG5-HKSCS Big5 with Hong Kong extensions, Traditional Chinese. Shift_JIS SJIS, SJIS-win, cp932, 932 Japanese EUC-JP EUCJP, eucJP-win Japanese MacRoman Charset that was used by Mac OS. '' An empty string activates detection from script encoding (Zend multibyte), default_charset and current locale (see nl_langinfo() and setlocale()), in this order. Not recommended. Note: Any other character sets are not recognized. The default encoding will be used instead and a warning will be emitted.
Return Values
Returns the translation table as an array, with the original characters as keys and entities as values.
Changelog
| Version | Description |
|---|---|
| 5.4.0 |
The default value for the encoding parameter was
changed to UTF-8.
|
| 5.4.0 |
The constants ENT_HTML401, ENT_XML1,
ENT_XHTML and ENT_HTML5 were added.
|
| 5.3.4 |
The encoding parameter was added.
|
Examples
Example #1 Translation Table Example
<?php
var_dump(get_html_translation_table(HTML_ENTITIES, ENT_QUOTES | ENT_HTML5));
?>
The above example will output something similar to:
array(1510) {
[" "]=>
string(5) "	"
["
"]=>
string(9) "
"
["!"]=>
string(6) "!"
["""]=>
string(6) """
["#"]=>
string(5) "#"
["$"]=>
string(8) "$"
["%"]=>
string(8) "%"
["&"]=>
string(5) "&"
["'"]=>
string(6) "'"
// ...
}
See Also
- htmlspecialchars() - Convert special characters to HTML entities
- htmlentities() - Convert all applicable characters to HTML entities
- html_entity_decode() - Convert all HTML entities to their applicable characters
Be careful using get_html_translation_table() in a loop, as it's very slow.
I wrote a quick little function for converting something like '·' into '·':
$to_convert = '·';
$table = get_html_translation_table(HTML_ENTITIES);
$equiv = '&#'.ord(array_search($to_convert,$table)).';';
If you have troubles (like me) getting data from ISO-8859-1 encoded forms where user copy and paste from word, this routine could be useful.
It adds to the standard get_html_translation_table the codes of the characters usually M$ Word replacs into typed text.
Otherwise those characters would never be displayed correctly in html output.
function get_html_translation_table_CP1252() {
$trans = get_html_translation_table(HTML_ENTITIES);
$trans[chr(130)] = '‚'; // Single Low-9 Quotation Mark
$trans[chr(131)] = 'ƒ'; // Latin Small Letter F With Hook
$trans[chr(132)] = '„'; // Double Low-9 Quotation Mark
$trans[chr(133)] = '…'; // Horizontal Ellipsis
$trans[chr(134)] = '†'; // Dagger
$trans[chr(135)] = '‡'; // Double Dagger
$trans[chr(136)] = 'ˆ'; // Modifier Letter Circumflex Accent
$trans[chr(137)] = '‰'; // Per Mille Sign
$trans[chr(138)] = 'Š'; // Latin Capital Letter S With Caron
$trans[chr(139)] = '‹'; // Single Left-Pointing Angle Quotation Mark
$trans[chr(140)] = 'Œ '; // Latin Capital Ligature OE
$trans[chr(145)] = '‘'; // Left Single Quotation Mark
$trans[chr(146)] = '’'; // Right Single Quotation Mark
$trans[chr(147)] = '“'; // Left Double Quotation Mark
$trans[chr(148)] = '”'; // Right Double Quotation Mark
$trans[chr(149)] = '•'; // Bullet
$trans[chr(150)] = '–'; // En Dash
$trans[chr(151)] = '—'; // Em Dash
$trans[chr(152)] = '˜'; // Small Tilde
$trans[chr(153)] = '™'; // Trade Mark Sign
$trans[chr(154)] = 'š'; // Latin Small Letter S With Caron
$trans[chr(155)] = '›'; // Single Right-Pointing Angle Quotation Mark
$trans[chr(156)] = 'œ'; // Latin Small Ligature OE
$trans[chr(159)] = 'Ÿ'; // Latin Capital Letter Y With Diaeresis
ksort($trans);
return $trans;
}
htmlentities includes htmlspecialchars, so here's how to convert an UTF-8 string :
htmlentities($string, ENT_QUOTES, 'UTF-8');
There have been issues when hispanic websites or other websites dont use the corrent collision in mysql.
Some problems result that the accents (éä ... ) result in weird characters when a backup is done and restored later on. Or when database is changed to another one.
To fix this try something like this
function accents($text){
foreach(get_html_translation_table(HTML_ENTITIES) as $a=>$b){
$text = str_replace($a,$b,$text);
}
return $text;
}
and use as accents("Hello ....... WITH ACCENTS") and it will return the escaped string.
Alans version didn't seem to work right. If you're having the same problem consider using this slightly modified version instead:
function unhtmlentities ($string) {
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
$trans_tbl = array_flip ($trans_tbl);
$ret = strtr ($string, $trans_tbl);
return preg_replace('/&#(\d+);/me',
"chr('\\1')",$ret);
}
get_html_translation_table
It works only with the first 256 Codepositions.
For Higher Positions, for Example ф
(a kyrillic Letter) it shows the same.
A lot of quite common characters (or at least not rare, like oelig, euro or minus) are missing from the table unfortunately.
Here are some, if you want to make your translation table more complete and your xml data less error-prone. Not sure why some characters have 2 codes, just use one. Here goes: '''=>''', '−'=>'-', 'ˆ'=>'^', '˜'=>'~', 'Š'=>'Š', '‹'=>'‹', 'Œ'=>'Œ', '‘'=>'‘', '’'=>'’', '“'=>'“', '”'=>'”', '•'=>'•', '–'=>'–', '—'=>'—', '˜'=>'˜', '™'=>'™', 'š'=>'š', '›'=>'›', 'œ'=>'œ', 'Ÿ'=>'Ÿ', 'ÿ'=>'ÿ', 'Œ'=>'Œ', 'œ'=>'œ', 'Š'=>'Š', 'š'=>'š', 'Ÿ'=>'Ÿ', 'ƒ'=>'ƒ', 'ˆ'=>'ˆ', '˜'=>'˜', 'Α'=>'Α', 'Β'=>'Β', 'Γ'=>'Γ', 'Δ'=>'Δ', 'Ε'=>'Ε', 'Ζ'=>'Ζ', 'Η'=>'Η', 'Θ'=>'Θ', 'Ι'=>'Ι', 'Κ'=>'Κ', 'Λ'=>'Λ', 'Μ'=>'Μ', 'Ν'=>'Ν', 'Ξ'=>'Ξ', 'Ο'=>'Ο', 'Π'=>'Π', 'Ρ'=>'Ρ', 'Σ'=>'Σ', 'Τ'=>'Τ', 'Υ'=>'Υ', 'Φ'=>'Φ', 'Χ'=>'Χ', 'Ψ'=>'Ψ', 'Ω'=>'Ω', 'α'=>'α', 'β'=>'β', 'γ'=>'γ', 'δ'=>'δ', 'ε'=>'ε', 'ζ'=>'ζ', 'η'=>'η', 'θ'=>'θ', 'ι'=>'ι', 'κ'=>'κ', 'λ'=>'λ', 'μ'=>'μ', 'ν'=>'ν', 'ξ'=>'ξ', 'ο'=>'ο', 'π'=>'π', 'ρ'=>'ρ', 'ς'=>'ς', 'σ'=>'σ', 'τ'=>'τ', 'υ'=>'υ', 'φ'=>'φ', 'χ'=>'χ', 'ψ'=>'ψ', 'ω'=>'ω', 'ϑ'=>'ϑ', 'ϒ'=>'ϒ', 'ϖ'=>'ϖ', ' '=>' ', ' '=>' ', ' '=>' ', '‌'=>'‌', '‍'=>'‍', '‎'=>'‎', '‏'=>'‏', '–'=>'–', '—'=>'—', '‘'=>'‘', '’'=>'’', '‚'=>'‚', '“'=>'“', '”'=>'”', '„'=>'„', '†'=>'†', '‡'=>'‡', '•'=>'•', '…'=>'…', '‰'=>'‰', '′'=>'′', '″'=>'″', '‹'=>'‹', '›'=>'›', '‾'=>'‾', '⁄'=>'⁄', '€'=>'€'
Not sure what's going on here but I've run into a problem that others might face as well...
<?php
$translations = array_flip(get_html_translation_table(HTML_ENTITIES,ENT_QUOTES));
?>
returns the single quote ' as being equal to ' while
<?php
$translatedString = htmlentities($string,ENT_QUOTES);
?>
returns it as being equal to '
I've had to do a specific string replacement for the time being... Not sure if it's an issue with the function or the array manipulation.
-Pat
In case you want a 'htmlentities' function which prevents 'double' encoding of the ampersands of already present entities (> => &gt;), use this:
<?php
function htmlentities2($myHTML) {
$translation_table=get_html_translation_table (HTML_ENTITIES,ENT_QUOTES);
$translation_table[chr(38)] = '&';
return preg_replace("/&(?![A-Za-z]{0,4}\w{2,3};|#[0-9]{2,3};)/","&" , strtr($myHTML, $translation_table));
}
?>
This function will convert get_html_translation_table from a ISO-8859-1 string to UTF-8 string.
<?php
function translation_table_to_utf8($arTranslationtable)
{
//loop through the array and convert everything both keys and values
foreach($arTranslationtable as $charkey => $char)
{
$charkey = utf8_encode($charkey);
$arUTFchars[$charkey]= utf8_encode($char);
}
return $arUTFchars;
}
//get the translation table
$arSpecialchar = get_html_translation_table(HTML_ENTITIES);
//call the function to convert to utf-8
$arUTFchars = translation_table_to_utf8($arSpecialchar);
print_r($arUTFchars);
?>
If you want to display special HTML entities in a web browser, you can use the following code:
<?
$entities = get_html_translation_table(HTML_ENTITIES);
foreach ($entities as $entity) {
$new_entities[$entity] = htmlspecialchars($entity);
}
echo "<pre>";
print_r($new_entities);
echo "</pre>";
?>
If you don't, the key name of each element will appear to be the same as the element content itself, making it look mighty stupid. ;)
If you want to decode all those { symbols as well....
function unhtmlentities ($string) {
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
$trans_tbl = array_flip ($trans_tbl);
$ret = strtr ($string, $trans_tbl);
return preg_replace('/\&\#([0-9]+)\;/me',
"chr('\\1')",$ret);
}
"rafael at phpit dot com dot br" your solution only works for the ISO-8859-1 encoding, I mean, it works but only for that encoding and that's because get_html_translation_table won't let you specify the charset... it uses the default one, that is ISO-8859-1
The solution from "olito24 at gmx dot de" does work for UTF-8, I just modified it a bit specifying the UTF-8 charset, also the $str parameter wasn't being used at all, I just renamed it to $string
Note:
Change ENT_NOQUOTES to ENT_QUOTES to convert both double and single quotes
These are the functions to encode html but tags using UTF-8 and ISO-8859-1
<?php
class Html
{
/*by olito24 at gmx dot de*/
function htmlButTags($string) {
$pattern = '<([a-zA-Z0-9\. "\'_\/-=;\(\)?&#%]+)>';
preg_match_all ('/' . $pattern . '/', $string, $tagMatches, PREG_SET_ORDER);
$textMatches = preg_split ('/' . $pattern . '/', $string);
foreach ($textMatches as $key => $value) {
$textMatches [$key] = htmlentities ($value, ENT_NOQUOTES, 'UTF-8');
}
for ($i = 0; $i < count ($textMatches); $i ++) {
$textMatches [$i] = $textMatches [$i] . $tagMatches [$i] [0];
}
return implode ($textMatches);
}
/*by "rafael at phpit dot com dot br" */
function htmlButTags_iso($str){
// Take all the html entities
$caracteres = get_html_translation_table(HTML_ENTITIES,ENT_NOQUOTES);
// Find out the "tags" entities
$remover = get_html_translation_table(HTML_SPECIALCHARS,ENT_NOQUOTES);
// Spit out the tags entities from the original table
$caracteres = array_diff($caracteres, $remover);
// Translate the string....
$str = strtr($str, $caracteres);
// And that's it!
return $str;
}
}
?>
Searching for a fast replacement of the MS WORD special characters which are not covered by get_html_translation_table() , I think the following function might help someone
<?php
function clean_up($str){
$str = stripslashes($str);
$str = strtr($str, get_html_translation_table(HTML_ENTITIES));
$str = str_replace( array("\x82", "\x84", "\x85", "\x91", "\x92", "\x93", "\x94", "\x95", "\x96", "\x97"), array("‚", "„", "…", "‘", "’", "“", "”", "•", "–", "—"),$str);
return $str;
}
?>
It replaces all types of quotes (single and double), horizontal ellipsis (...), bullet, en dash and em dash.
and a few more :
'ℑ'=>'ℑ', '℘'=>'℘', 'ℜ'=>'ℜ', '™'=>'™', 'ℵ'=>'ℵ', '←'=>'←', '↑'=>'↑', '→'=>'→', '↓'=>'↓', '↔'=>'↔', '↵'=>'↵', '⇐'=>'⇐', '⇑'=>'⇑', '⇒'=>'⇒', '⇓'=>'⇓', '⇔'=>'⇔', '∀'=>'∀', '∂'=>'∂', '∃'=>'∃', '∅'=>'∅', '∇'=>'∇', '∈'=>'∈', '∉'=>'∉', '∋'=>'∋', '∏'=>'∏', '∑'=>'∑', '−'=>'−', '∗'=>'∗', '√'=>'√', '∝'=>'∝', '∞'=>'∞', '∠'=>'∠', '∧'=>'∧', '∨'=>'∨', '∩'=>'∩', '∪'=>'∪', '∫'=>'∫', '∴'=>'∴', '∼'=>'∼', '≅'=>'≅', '≈'=>'≈', '≠'=>'≠', '≡'=>'≡', '≤'=>'≤', '≥'=>'≥', '⊂'=>'⊂', '⊃'=>'⊃', '⊄'=>'⊄', '⊆'=>'⊆', '⊇'=>'⊇', '⊕'=>'⊕', '⊗'=>'⊗', '⊥'=>'⊥', '⋅'=>'⋅', '⌈'=>'⌈', '⌉'=>'⌉', '⌊'=>'⌊', '⌋'=>'⌋', '⟨'=>'〈', '⟩'=>'〉', '◊'=>'◊', '♠'=>'♠', '♣'=>'♣', '♥'=>'♥', '♦'=>'♦'
Another way of converting HTML entities into numeric entities to please XML parsers is using two arrays as conversion tables in a preg_replace function. The conversion table mechanism is based on Ryan's examples above.
<?php
function xmlEntities($s){
//build first an assoc. array with the entities we want to match
$table1 = get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
//now build another assoc. array with the entities we want to replace (numeric entities)
foreach ($table1 as $k=>$v){
$table1[$k] = "/$v/";
$c = htmlentities($k,ENT_QUOTES,"UTF-8");
$table2[$c] = "&#".ord($k).";";
}
//now perform a replacement using preg_replace
//each matched value in array 1 will be replaced with the corresponding value in array 2
$s = preg_replace($table1,$table2,$s);
return $s;
}
?>
The existance of html entities such as " inside an xml node causes most xml parsers to throw an error. The following function cleans an input string by converting html entities to valid unicode entities.
<?php
function htmlentities2unicodeentities ($input) {
$htmlEntities = array_values (get_html_translation_table (HTML_ENTITIES, ENT_QUOTES));
$entitiesDecoded = array_keys (get_html_translation_table (HTML_ENTITIES, ENT_QUOTES));
$num = count ($entitiesDecoded);
for ($u = 0; $u < $num; $u++) {
$utf8Entities[$u] = '&#'.ord($entitiesDecoded[$u]).';';
}
return str_replace ($htmlEntities, $utf8Entities, $input);
}
?>
So, an input of
Copyrights © make "me" grin ®
outputs
Copyrights © make "me" grin ®
In XML, you can't assume that the doctype will include the same character entity definitions as HTML. XML authors may require character references instead. The following two functions use get_html_translation_table() to encode data in numeric references. The second, optional argument can be used to substitute a different translation table.
function xmlcharacters($string, $trans='') {
$trans=(is_array($trans))? $trans:get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
foreach ($trans as $k=>$v)
$trans[$k]= "&#".ord($k).";";
return strtr($string, $trans);
}
function xml_character_decode($string, $trans='') {
$trans=(is_array($trans))? $trans:get_html_translation_table(HTML_ENTITIES, ENT_QUOTES);
foreach ($trans as $k=>$v)
$trans[$k]= "&#".ord($k).";";
$trans=array_flip($trans);
return strtr($string, $trans);
}
I found this useful in converting latin characters
<?php
function convertLatin1ToHtml($str) {
$allEntities = get_html_translation_table(HTML_ENTITIES, ENT_NOQUOTES);
$specialEntities = get_html_translation_table(HTML_SPECIALCHARS, ENT_NOQUOTES);
$noTags = array_diff($allEntities, $specialEntities);
$str = strtr($str, $noTags);
return $str;
}
?>
Here is a simple way to convert named character entities to numeric character entities:
<?php
function numeric_entities($string){
$mapping = array();
foreach (get_html_translation_table(HTML_ENTITIES, ENT_QUOTES) as $char => $entity){
$mapping[$entity] = '&#' . ord($char) . ';';
}
return str_replace(array_keys($mapping), $mapping, $string);
}
?>
Quite disappointingly, get_html_translation_table() only gives the characters for ISO-8859-1, making it quite useless for UTF-8 or anything else like that (as a previous commenter noticed).
to display the mapping on a webpage no matter what the server encoding is, this can be used
echo "<pre>\n";
echo htmlentities(print_r((get_html_translation_table(HTML_SPECIALCHARS)), true));
echo htmlentities(print_r((get_html_translation_table(HTML_ENTITIES)), true));
since get_html_translation_table() actually gives the special chars in iso-8859-1 (Latin-1) encoding, so to see the tables correctly using
print_r(get_html_translation_table(HTML_ENTITIES));
your server needs to give a HTTP header as iso-8859-1, unless you use header() or manually set the browser's encoding setting to iso-8859-1. And you need to view the source of the page to see the mapping. (except English version of IE 7 outputs the page source as iso-8859-1 anyway).
without heavy scientific analysis, this seems to work as a quick fix to making text originating from a Microsoft Word document display as HTML:
<?php
function DoHTMLEntities ($string)
{
$trans_tbl = get_html_translation_table (HTML_ENTITIES);
// MS Word strangeness..
// smart single/ double quotes:
$trans_tbl[chr(145)] = '\'';
$trans_tbl[chr(146)] = '\'';
$trans_tbl[chr(147)] = '"';
$trans_tbl[chr(148)] = '"';
// Acute 'e'
$trans_tbl[chr(142)] = 'é';
return strtr ($string, $trans_tbl);
}
?>
