Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Mojibake is specifically the "garbage" from getting an encoding wrong—treating a sequence of bits as if they were a valid sequence of characters in some encoding, when they're either not from that encoding, or not from any textual encoding (because you're e.g. trying to render the contents of an executable binary as text.)

"Tofu" is a more recent phenomena, that didn't really have a specific name until now: when there are Unicode code-points—correctly decoded—in a document, but you have no font installed that offers a glyph to represent them.

Mojibake results from "I did this wrong and didn't notice"; tofu results from "I know what this is, and it's something I don't have a visual signifier for."

Mind you, often mojibake will result in "tofu"; the garbage code-points you get from bad encoding detection will turn out to be ones that don't have a currently-defined Unicode character, so they'll show up as U+FFFD REPLACEMENT CHARACTER (�). But that's a coincidence, rather than an equivalence.



Thanks! That was a super helpful response!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: