Conjuncts, and vowels that move

The storage-order problem

Every encoding system has to answer the same question: in what sequence do you record a syllable? For Latin text the answer is trivially the left-to-right reading order, because the characters are independent and sit side by side in a line. Indic scripts refuse that convenience. A syllable in Devanagari, Tamil, or Malayalam is not a sequence of atoms — it is a unit whose constituent parts can migrate around one another depending on which consonants and which vowel signs are present.

Take a standard Devanagari syllable: a consonant followed by the vowel sign for the short i sound. In Unicode, the consonant's code point comes first in the byte stream, and the matra — the dependent vowel sign — follows it. Yet the matra for short i renders to the left of the consonant it belongs to. A renderer that naïvely paints glyphs in storage order will produce nonsense: the vowel drawn on one side of a character it must visually precede. This is not a corner case. It is the normal behaviour of one of the most common vowels in the script.

The gap between storage order and visual order is not an accident or an oversight. Unicode inherited its logical ordering from ISCII, the Indian national standard that preceded it, and ISCII made a deliberate choice to record every element of a syllable in phonological order — consonant first, then vowel — regardless of where the glyph would ultimately appear. The logic is clean: if you store text in phonological sequence, searching, sorting and cursor movement all behave predictably. You pay for that predictability in the rendering layer, which must know how to reorder before it paints.

What the shaping layer actually does

The component that performs this work is called a shaping engine. Its job is to take a sequence of Unicode code points and produce a sequence of positioned glyphs — the actual outlines a font will draw. For Indic scripts, this involves several distinct operations, and reordering is only one of them.

Conjunct consonants are the other major obligation. When two or more consonants appear without a vowel between them, the vowel of the first is suppressed: in Devanagari, the halant (the virama sign, U+094D) marks the suppression. A sequence like KA + halant + TA is not three separate characters on screen; it is a single conjunct form, kta, whose visual shape is determined by which consonants are involved and what the font provides. Some conjuncts are formed by stacking the constituent consonants vertically. Others use a half-form, where the first consonant sheds its vertical stem and attaches horizontally to its neighbour. A few have entirely idiosyncratic shapes that bear little resemblance to either component drawn alone.

The shaping engine must consult the font's lookup tables — stored in the OpenType GSUB (glyph substitution) and GPOS (glyph positioning) tables — to discover what forms are available and in what order to apply them. The sequence of operations for Devanagari, as specified in the OpenType documentation, proceeds through pre-defined feature tags: nukt for nukta attachment, akhn for mandatory conjuncts that must not be broken, rphf and blwf for the above- and below-base forms of the consonant RA, and so on through roughly a dozen ordered steps before any reordering of the matra occurs. The order of those steps is not advisory — applying them out of sequence produces incorrect output.

Tamil and Malayalam impose their own variants of this logic. Tamil has no stacked conjuncts; it uses the pulli (virama equivalent) to produce a pure consonant, and its combining vowel signs include several that split across the consonant, appearing both before and after it in a single glyph cluster. Malayalam historically used a very large set of conjunct ligatures, many of which were pruned in the 1971 script reform by the Kerala government; fonts targeting the reformed orthography carry a different set of GSUB lookups than those targeting the traditional pazhaya lipi. The shaping engine has no way to know which regime a document expects unless the font and the locale data agree.

Take a standard Devanagari syllable: a consonant followed by the vowel sign for the short i sound.

Where C-DAC and the standards bodies fitted in

India's Centre for Development of Advanced Computing — C-DAC, based in Pune — was involved in Indic script rendering before OpenType existed as a mature standard. C-DAC developed the GIST (Graphics and Intelligence-based Script Technology) system in the late 1980s and 1990s, and its internal encoding schemes preceded Unicode deployment in Indian software. When Unicode and OpenType became the target platform, the shaping rules that had been embedded in GIST had to be re-expressed as font feature tags and engine logic. That translation was not always lossless: some conjunct forms that GIST rendered through custom code paths had to wait for OpenType feature sets to catch up before they could be represented cleanly.

The Unicode Consortium's Indic blocks were influenced by ISCII's structure, which mapped ten Brahmi-derived scripts onto a single logical layout. Because ISCII had already committed to phonological ordering, Unicode kept it, which is why the matra-before-consonant rendering problem exists at all at the storage level. The shaping specification then became the place where the rendering community had to write down, exhaustively, what the encoding standard had chosen not to enforce. Tamil Nadu's state-level standardisation efforts and Kerala's own computing localisation work both fed into practical experience of where the specifications were underspecified — particularly around edge cases in syllable boundary detection and in the handling of the anusvara and visarga signs that appear across multiple scripts.

The consequence for fonts

A font that carries no OpenType Indic features will not render these scripts correctly, regardless of how complete its glyph coverage is. The glyphs must be present, but so must the lookup logic that assembles them. A font developer building a Devanagari typeface must write GSUB substitution rules for every conjunct the typeface supports, ordered to match the shaping engine's expected feature-processing sequence. Miss one step, and a consonant cluster that should produce a ligature will instead render as a consonant followed by a visible halant — correct in storage, wrong on screen.

This interdependency between font and engine is precisely why the vowel-reordering problem and the conjunct problem are treated separately in OpenType documentation even though both arise from the same structural property of the scripts: each requires its own class of lookup, and failures in one do not resemble failures in the other. The shaping layer is, in the end, a formal grammar for the script's syllable structure, compiled into lookup tables and executed at render time — every time a word is drawn.