A Phonetic Encoding

Talk is a phonetic front end and tokenization layer for speech and language models. Its machine encoding turns text or IPA into a single code point per sound, so a model gets a finite, normalized phone vocabulary instead of raw IPA or byte-pair fragments that split a pronunciation across several tokens. If you build TTS, ASR, G2P, or any multilingual pipeline that has to reason about how words sound, this gives you one clean token per phone that converts losslessly to and from IPA, plus syllabification for free.

It is also a readable, keyboard-typable romanization. Talk uses the Latin script with diacritics to encode most of Earth's natural language features, enough that you can write every language in the same Latin-oriented system and land close to a realistic pronunciation, including nasalized vowels, tense consonants, clicks, and tones. It reads like the pronunciation guides people already know, so a non-linguist can get the gist without special training.

There are two forms:

• ASCII: For writing in a text editor without fancy symbols.
• Machine: A single Hangul code point per sound, for compact tokenization (useful for AI models and lookups).

Talk is not trying to compete with or replace IPA. It is for writing pronunciations in plain ASCII and tokenizing them in a standard way for tools like NLP, AI, and databases. It aims for about 95% accuracy, which is plenty for a tokenizer or a learner, and it speaks IPA fluently for the cases where you need the last 5%.

Implementations of Talk, one per language. Each converts between IPA, Talk, and the machine (token) encoding. The TypeScript library also splits words into syllables.

The machine column is the machine encoding: talk.machine(...) packs each sound into a single Hangul code point (base plus all its modifiers), so a whole word becomes a compact, one-code-point-per-sound string.

Here are the modifiers on consonants, vowels, and symbols (like punctuation), and how they look in ASCII.

There are a few affixes on consonants and vowels. There are 5 tone affixes (extra low, low, neutral, high, and extra high), which can be combined in standard ways.

This is the full set of IPA consonants we map, in ASCII, and machine (Hangul) form. Every consonant, however modified, is a single Hangul code point.

Note: GitHub markdown doesn't really render the diacritics that nicely, some are misaligned. We will have a font to remedy this for websites.

This is the full set of IPA vowels we map. Every vowel, with any tone, length, stress, or nasalization, is a single Hangul code point.

Every vowel takes the same modifier set (stress, length, nasal, tone, etc.). Here are all the combos for the letter a, and the same pattern applies to all vowels.

Note: Exact tone sequences can be represented with sequences like a+a++a--, where one vowel is spread across multiple tones. But common tones, across languages, can take advantage of the shortened syntax/encoding.

The IPA defines a set of diacritics that modify a base symbol, grouped by type below. The state column says whether Talk maps the diacritic to a phonetic feature, or ignores it, and talk is the resulting Talk affix when it is mapped.

On the ignored ones. IPA itself is not perfectly exact. Real speech has far more subtle variation than any discrete character set can capture, and the ignored diacritics above (relative articulation, breathy/creaky voice, tongue-root position, half-long, and so on) mark distinctions so fine-grained that they are hard to even hear reliably, hard to transcribe consistently, and hard to reproduce the same way across recordings. For a discrete model of sound aimed at AI and coding, they are mostly unnecessary, so Talk lets them fall through rather than inventing symbols for detail it cannot use.

Talk also splits a word into syllables, each a sequence of onset, nucleus, and coda clusters. IPA and X-SAMPA give you symbols and stop there, so this comes built in, which the TTS, speech-recognition, and language-learning use cases all need.

Some rarer sounds are not yet handled by the syllable system. The current gaps are listed in base/syllable/unsupported.csv (dentals, retroflex plosives, implosives, ejectives, and clicks).

Which Chinese characters actually render in the base fonts that ship on ordinary computers, rather than showing the "missing-glyph" box basically (so you can actually see the glyphs not missing font glyphs, most of the time). Coverage is read from each font's cmap (the ground truth for whether a glyph exists), then intersected per script: Simplified with Simplified, Traditional with Traditional.

The base list is 84,107 Han ideographs used in Chinese, taken from the Unihan database by Chinese source or reading.

Characters that render on every common desktop of a script (the intersection):

Per-font and intersection lists live in base/symbol/chinese/.

MIT

Made by ClueSurf, meditating on the universe ¤. Follow the work on YouTube, X, Instagram, Substack, Facebook, and LinkedIn, and browse more of our open-source work here on GitHub.