More free FontLab encoding files for type designers »

Last week I wrote about posting five FontLab encoding files for Adobe Latin character sets.

Today I posted in the same Github repository three FontLab encoding files for Adobe Cyrillic character sets, and updated the five Latin files with a few added currency symbols and glyph name changes (as I expected I might).

The character set definitions underlying these files were built on a bunch of research I did at Adobe back in 2006–08, with additional work by Miguel Sousa. The headers include much detail on the differences between each set, and the languages covered. Both of these character sets reflect the latest data from Adobe on how they name glyphs and what they are including in current fonts (not including OpenType alternates and features, mind you). The headers of the files have some interesting details and history, especially on the Cyrillic side.

Thanks as always to my old friends at Adobe, including Miguel and David Lemon, for their willingness to share production information with the type design community.

I dedicate this post and my work on the Cyrillic encoding files to the memory of Emil Yakupov, CEO of the ParaType type foundry in Moscow, who passed away just a month ago at the age of 56. His advice and feedback on Cyrillic character sets—among many things—was invaluable to me. I remember one of our first meetings, when Emil gave me a pair of ParaType catalogs as I was first becoming involved with Cyrillic type design. I still consult them to this day when trying to internalize what forms different Cyrillic characters can take in different font styles.

Also, Emil remembered by Adam Twardoch.

Прощай, Эмиль.

 

 

Greek Support in Fonts: Truth & Lies »

A recent thread over on Typophile prompts me to explain why one sometimes sees OpenType CFF fonts that don’t actually support Greek, claiming that they do (by means of the Unicode Range and Codepage bit settings).

Originally, when Adobe converted the Adobe Type Library to OpenType, in the early stages we were thinking we wanted to be as compatible as possible with the Type 1 versions of the fonts.

In the absence of codepage bits and the like in Type 1 fonts, Windows GDI used to do a test for specific codepoints in the font to determine whether given codepages were supported, I believe one codepoint per codepage. I believe the codepoint to determine Greek support was the one for the “mu”… but it was definitely a test for a character that was present in the basic ISO-​Adobe character set (now Adobe Latin 1). This even though the character set didn’t really support Greek, it just had a few Greek characters because of their use as math symbols.

So, the Type 1 fonts were (arguably erroneously) detected as supporting Greek by GDI. The idea at the time was that the OpenType fonts with the equivalent character set should behave the exact same way, and therefore the determination of whether to give them the flag saying they support the Greek codepage should be based on the same test… so basically all the fonts would claim to support Greek, even though they really didn’t.

Somehow, even though the idea of near-​perfect compatibility between the OpenType fonts and their Type 1 predecessors was abandoned, this principle stuck during the initial conversion of the Adobe Type Library (“Alchemy”). Unicode ranges were set more-​or-​less in compatibility. Additionally, the AFDKO “makeOTF” tool used to build fonts would automatically do this, unless specifically over-​ridden. So you could see third-​party fonts doing this as well if they were made with older versions of Adobe tools.

I thought this was a mistake, and eventually convinced other folks of my opinion, so this decision was changed in the revision of the Adobe Type Library (“Facelift”) a couple of years ago, released about October 2007. The AFDKO was changes as well, to match this new preferred behavior.

So, you’ll find that the 1.x version of Adobe Caslon Pro built in 2001 has bits set to claim it supports the Windows Greek codepage and the Greek Unicode range, but this claim was dropped in the 2.0 version of 2007.

AFDKO 2.5 released »

Adobe has just released a new version of the Font Development Kit for OpenType (AFDKO). This is an important milestone for font developers.

Why care?

Even if you don’t use it yourself, the AFDKO code is licensed at no cost to developers of other font tools such as FontLab and DTL FontMaster. It forms the basis of the OpenType generation and OpenType features support in those tools. So very often new functionality in the AFDKO is a preview of new functionality in these other tools, and hence indirectly of capabilities of both Adobe and other vendors’ fonts. It’s reasonable to suspect that features in the AFDKO will preview new features in future versions of FontLab Studio (maybe 6.0?) and DTL FontMaster (look at the DataMaster tool in particular).

So, with the new FDK, and one hopes in future versions of FontLab Studio and DTL FontMaster, users are able to build fonts which support complex scripts (Arabic, Indic, etc.), without using Microsoft’s VOLT as a separate process. All (yes all) OpenType GSUB and GPOS lookup types are supported. There’s support for Unicode Variation Sequences, especially useful for Japanese and ideographic languages. Plus one can give “friendly” names to stylistic sets, although this last will also need to be supported by applications, and I wouldn’t hold my breath waiting for that. Plus there are various bug fixes and lesser features.

What is the AFDKO, exactly?

The AFDKO is a set of command-​line (yes, really) font development tools available for Mac, Windows and Unix. They cover compiling, proofing, testing and some editing of OpenType fonts. Compiling a font requires a basic TrueType, Type 1 or OpenType font as input; what the AFDKO does is add OpenType layout features to an existing font (as in, you still need a font editor).

I’m particularly fond of the “CompareFamily” tool, which is unusual in that it not only does tests on individual fonts, but looks at how information and key values are coodinated across a family. This is very useful for checking things such as style linking, or making sure your trademark statement is identical in all members of the family.

Because this stuff is command-​line driven, it may not be for the average user. Besides which, many folks may prefer to use functionality which is integrated with their font editing environment. But for those who need reliability and a controlled, repeatable build process, the FDK is a nice option to have.


\

World-​Ready Composer in Adobe CS4 »

This is a guide to options and tools for laying out global text in the CS4 versions of InDesign, Photoshop and Illustrator. None of them are obvious or documented in the regular versions of the application, but there are a dizzying variety of options: templates; scripts; InDesign plug-​ins; and special “ME” versions of applications. Prices range from free to more expensive than the base version of the application. This will help you figure out which might be right for your needs, and even provide some basic tools to help you get started, if your needs are not too extensive.

Why would you even need something special for global text layout? For most basic left-​to-​right languages, if the fonts you are using have all the right glyphs, the regular version of the Adobe application will do an adequate job out of the box. However, many left-​to-​right languages of south and south-​east Asia (such as Thai, Lao and the Indic languages) require additional language-​specific processing to get the right glyph output given the incoming character stream. Many Indic languages assemble multiple characters into a single visual “cluster” (sort of like a syllable), using complicated shaping rules. Some languages, notably Thai and Lao, do not even have spaces between words, and therefore need special dictionaries just to get correct line breaking. Then there are right-​to-​left languages such as Arabic and Hebrew, which require further capabilities. (Note that InDesign added Thai layout functionality in its regular composition engine back in CS3, although with some limitations.)

Standards such as Unicode only provide a framework around which such additional processes must be built—they don’t provide the code. Winsoft has long offered special “ME” versions of Adobe applications (with full support for Arabic and Hebrew, though not the Indic or other Asian languages), but none of this functionality was in the standard versions of Adobe’s Creative Suite applications before CS4.

One cool thing Adobe did in the Creative Suite 4 product cycle was to work on global text support across several products, including InDesign, Illustrator and Photoshop. The CS4 versions of these apps have an alternate composition engine, the World-​Ready Composer, which enables support for “complex script” languages such as Arabic, Hebrew, Thai, and the Indic languages. One of the goals of this move was to unify file formats and code between western, CJK, and ME versions of the applications. But unless you have an ME version of an application, the World-​Ready Composer isn’t directly accessible in the CS4 applications as shipped.

Why not? Well, the World-​Ready Composer was not fully tested and debugged, and hyphenation dictionaries and spell checkers aren’t available for the extra languages. Therefore, the World-​Ready Composer is neither documented nor officially supported by Adobe in CS4, and no user interface was provided for the added features in the apps (like selecting the composer, or choosing right-​to-​left text). Although many people assume this work will be finished in CS5, the last time I checked Adobe was making no promises as to when these capabilities will be finished and formally released.

Native CS4 Capabilities

Now, the capabilities above might seem not very useful, but there are several handy things one can do with the CS4 versions of these applications, right off the bat:

• In CS4 applications, one can now open and print Hebrew and Arabic documents created with Winsoft’s ME versions of the Adobe applications.
• If one opens a document that has text frames, paragraphs, and/​or styles which use the World-​Ready Composer, one can then copy and paste those into another document, or delete other content and use the original document as the basis of something else, thereby gaining access to the World-​Ready Composer.
• If right-​to-​left text direction is part of the formatting of the frame/​paragraph/​style, it comes along for the ride.
• In InDesign CS4, all these features are accessible to scripting, and the scripting interface is documented! These features are also open to plug-​ins. This decision by the InDesign team opened the way for third-​party developers to make scripts and plug-​ins to ease access to the added functionality. See below for more details on both.

Options for More Support

There are many ways to get more access to the World-​Ready Composer than you get out of the box with the CS4 applications. Further details on each are in the sections below. In order of increasing functionality, they are:

• templates (free, see below)
• scripting (InDesign only, there are free existing scripts or you can make or modify them yourself, see below)
• special plug-​ins ($19.99 – $110, InDesign only for now, see below)
• Winsoft’s ME versions of the applications, starting at €270/$270 to upgrade another version of InDesign to CS4 ME, or €978/$945 for a stand-​alone copy of InDesign CS4 ME. These are also available in the US from InTools for $350 for the InDesign upgrade, or $1169 for the stand-​alone InDesign CS4 ME, including shipping.

FontShop has a nice explanation of the various right-​to-​left features and related functionality in InDesign ME; it was written for CS3, but is equally applicable to CS4.

Also, if you want to use the World-​Ready Composer for Indic languages, Thai, Lao, or others not mentioned previously, be aware that none of these solutions (not even the ME versions, to date) offer spell checking or dictionaries. However, there are some third-​party solutions, notably MetaDesign’s SpellPlus for spell-​checking some of the Indic languages (currently only for InDesign CS2 and CS3, $149).

Languages (Writing Systems)

Which languages are enabled by the World-​Ready Composer? Currently, there are two tiers. First, these writing systems have been implemented, but not fully tested:

• Arabic
• Devanagari (Hindi)
• Cyrillic (Russian, etc.)
• Greek
• Hebrew
• Latin (European and American languages, but also Vietnamese)
• Lao (but without line breaking)
• Telugu
• Thai (including line breaking)

These additional writing systems have been at least partially implemented, but not tested:

• Bengali
• Canadian Aboriginal Syllabics
• Georgian
• Gujarati
• Gurmukhi
• Kannada
• Khmer
• Malayalam
• Oriya
• Sinhala
• Syriac
• Tamil
• Tibetan
• Thaana
• Yi

Because it’s only the UI that is missing in regular InDesign CS4, documents created using special plug-​ins, scripts, or templates should be fine to open and print from InDesign CS4 (as much as they are with the plug-​ins, anyway). It’s just that the UI for changing things is lacking—editing is possible, for sure, but control over right-​to-​left directionality vs left-​to-​right may be troublesome, and access to tweak additional options (like numbering styles) is lacking.

Limitations of CS4 Apps Not Using ME Versions

These limitations apply to anything one does with the templates, scripts and plug-ins.

Issues affecting all CS4 applications:

• Non OpenType “smart font” technologies, such as Apple’s AAT/​GX and SIL’s Graphite, are not supported. This means that Apple system fonts for complex scripts don’t work, but Microsoft’s do. (This is also an issue for Winsoft’s ME apps, as far as I know.)
• If you fill a text block with placeholder text, Winsoft’s ME apps automatically select appropriate text based on language, while the Adobe CS4 apps do not (at least, not for the “unsupported” languages, not sure about languages they officially support, such as French or Japanese).
• Currently only the Winsoft ME versions of the applications offer Arabic spell-​checking and Hebrew hyphenation.

InDesign-​specific issues:

• The Story Editor and the Notes panel do not render RTL text correctly
• InCopy compatibility is an issue
• Importing Word files is tricky if you want complex scripts to be handled correctly. You need to set “World Ready Composer” in the “No Paragraph Style” style.

Templates for ID, Ai & PS

Unfortunately, Photoshop CS4 doesn’t expose the World-​Ready Composer to scripting or plug-​ins, and Illustrator CS4 exposes the APIs to plug-​ins (only), but nobody has made anything for Illustrator yet. But these two applications do open documents from their ME counterparts, which makes it possible to get the World-​Ready Composer and/​or RTL text active by opening existing documents with appropriately-​formatted text blocks and using copy-​paste to transfer the text to new documents. You can also copy-​paste text between Illustrator and Photoshop and it retains the World-​Ready composer and paragraph direction formatting from one to the other.

Where would you find a document to get at such text? Here are some template documents to get you started, for all three major Adobe applications (see below for the template license terms: by downloading these templates you are agreeing to the terms below):

• InDesign
• Illustrator
• Photoshop

Note the styles used in the InDesign document. If opening the template gives a missing plug-​in warning, just dismiss it.

The templates are a nice option for InDesign folks who don’t want to mess with scripts, and the only option short of an ME application for people needing this functionality in Illustrator or Photoshop.

InDesign Scripts & Scripting

Here are some simple scripts, which you may download under the license terms below (don’t download unless you read and agree to the terms). These scripts can help anybody access both the World-​ready Composer and basic right-​to-​left text features for a few sentences or paragraphs. Anybody can use InDesign scripts that are already written, and it is not hard to make minor edits as well. These scripts, by Peter Kahrel, with some minor additions and edits from me, are written in JavaScript, and should work on both Mac and Windows versions of InDesign CS4. Any errors or glitches were probably introduced by me, however. 🙁 

All the scripts in the set start with the “r2l” name so they will sort together.

• r2l Character Direction Flip (reverses default character direction for selection)
• r2l Character Direction r2l (sets default character direction r2l for selection
• r2l Paragraph Direction Flip (reverses paragraph direction for selected paragraphs)
• r2l Paragraph Direction r2l (sets paragraph direction to r2l for selected paragraphs)
• r2l Assign World-​Ready Paragraph Composer (to selection)
• r2l Assign World-​Ready Single-​line Composer (to selection)
• r2l Assign World-​Ready Paragraph Composer to Paragraph Style (edits current style(s) to change the assigned composer)
• r2l Paragraph Style Arabic (creates a paragraph style suitable for Arabic)
• r2l Paragraph Style Hebrew (creates a paragraph style suitable for Hebrew)

Note: The scripts linked above are ready to be installed. If you were taking a script which wasn’t already a separate file, you would copy the script into a plain text file, and save it giving it an appropriate extension: .applescript, or .jsx for JavaScript or .vbs for Visual Basic/​VBScript. AppleScript and VBScript are for Mac and Windows, respectively, while JavaScript is cross-platform.

Follow these simple rules for how/​where to install InDesign scripts:

If you want to install scripts for all users on the computer, put them here:

• Mac: Hard Drive/​Applications/​Adobe InDesign CS4/​Scripts/​Scripts Panel
• Windows XP or Vista: C:\\Program Files\Adobe\Adobe InDesign CS4\Scripts\Scripts Panel (Note: If you’re on a 64-​bit Windows system, that would be “Program Files (x86)” instead of just “Program Files.”)

If you want to install scripts only for a single user, put them here:

• Mac: Hard Drive/Users/<username>/Library/Preferences/Adobe InDesign/​Version 6.0/Scripts/Script Panel
• Windows XP: C:\\Documents and Settings\<username>\Application Data\Adobe\InDesign\Version 6.0\Scripts\Scripts Panel
• Windows Vista: C:\\Users\<username>\AppData\Roaming\Adobe InDesign\Version 6.0\Scripts\Scripts Panel

If you want to edit these scripts or write your own, you’ll benefit from some reference material:

• Adobe’s scripting info can be found on the main Adobe InDesign CS4 scripting page (click on “Scripting Resources”), particularly the Scripting Tutorial and the JavaScript, Windows VBScript and Mac AppleScript guides. Also look at sample scripts and the like.
• Peter Kahrel put together this great PDF guide to the properties and enumerations you’ll need to use for scripting right-​to-​left and World-​ready Composer settings in InDesign CS4.

End-​User License for Scripts & Templates

The scripts and templates (“Software”) provided above are licensed to you under a BSD-​style open source license, as described below.

Copyright 2008, 2009, Peter Kahrel & Thomas Phinney.
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/​or other materials provided with the distribution.
• Neither Thomas Phinney’s nor Peter Kahrel’s names may be used to endorse or promote products derived from this software without specific prior written permission.

This Software is provided “as is” and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall Thomas Phinney or Peter Kahrel be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this Software, even if advised of the possibility of such damage.

InDesign Plug-​ins Available

As discussed on InDesignSecrets.com, some third parties have already taken advantage of the scripting and plug-​in access, and released plug-​ins which give a UI for the World-​Ready Composer in InDesign:

• WorldTools for InDesign CS4, $49 (new lowered price), by Harbs at InTools. Compare functionality vs InDesign alone and InDesign ME.
• idRTL for InDesign CS4, $19.99 through Feb 14, $39.99 thereafter, by Steven F. Bryant. Compare functionality of idRTL vs InDesign ME. Currently Windows-​only, but Mac version promised Feb 1 with same pricing.
• IndicPlus, $110 by MetaDesign, for InDesign CS2 and CS3. Note that unlike the other solutions discussed here, it is not based on Adobe’s World-​Ready Composer. For this plug-​in only, ignore all the discussion here about compatibility, limitations, languages and so on; it is listed for comparison and reference. IndicPlus supports Hindi, Bengali, Gujarati, Kannada, Sanskrit, Tamil, Punjabi, Nepali, Kashmiri, Assamese, Manipuri, Sindhi, Marathi, Konkani, Telugu (with limitations), and Tibetan.

There are several notable differences between current versions of World Tools and idRTL. Broadly, World Tools has more functionality, and idRTL has a more convenient interface. As both products are in active development, one might expect improvements and new features to be added to each, but some further differences are:

• idRTL is a bit cheaper, but World Tools has a free 30-​day trial
• idRTL can switch the direction of an existing document
• idRTL has a modeless floating panel approach, well suited to “inspecting” text and style formatting as well as applying it; World Tools settings appear as a sub-​menu of the new “API” menu
• idRTL supports Arabic, Hebrew, Hindi and Farsi digit styles; World Tools supports all 20 number formats supported by the World-​Ready Composer, as well as CJK numbering options. Numbering can matter for various auto-​numbering situations, including page numbers, footnotes, numbered lists, etc.
• idRTL has an installer, while World Tools is installed manually (though it’s not difficult) 
• World Tools has a nice “tree” view dialog for setting paragraph and character styles
• World Tools can search specifically for existing Hebrew or Roman text and set a user-​specified character style on that text—useful for fixing existing documents or collaborating with someone who doesn’t have World Tools
• InTools offers an upgrade path from World Tools to InDesign ME, so that if you find World Tools doesn’t meet your needs, you can upgrade to ID ME for the same total cost as just buying the ME product in the first place

Broadly speaking, the plug-​ins offer a significant degree of functionality in InDesign CS4. If you are doing entire documents in right-​to-​left or complex scripts, and you don’t need the additional features and bug fixes of InDesign CS4 ME, then the plug-​ins may be your best choice. If a document was created using a plug-​in, opening it without the plug-​in may yield a warning, but the document should be fine.

Bugs & Comments

I am not offering technical support for the scripts and templates, nor for Adobe products. However, I may fix bugs in the scripts and templates, and I welcome discussion of them in comments to this post. Note that Adobe does not officially support the World-​Ready Composer in CS4, so I am taking bug reports and problems on the composer itself as comments to a separate post, to make sure Adobe engineers have a place to go to see such reports in one place.

Conclusion

If your needs are basic, the free templates and scripts provided here might do the trick, even for Photoshop and Illustrator. If your concern is strictly InDesign, the idRTL plug-​and WorldTools plug-​ins offer a bunch more functionality at bargain prices. For folks doing serious work in Arabic or Hebrew, including Photoshop and Illustrator, the ME versions of Adobe applications are the way to go, particularly if you need the built-​in dictionaries.

Special thanks to: Peter Kahrel, Harbs, Steven Bryant, and Diane Burns for blazing the way in how to tackle these problems, and reviewing this article. Extra-​special thanks to all the engineers at Adobe who did the hard work that made this possible, and shared their expertise with me when I worked at Adobe, including Joe, Margie, Eric, Zak and Niti. Finally, I’d like to thank the good folks at WinSoft who created the foundations this is all built on: I don’t know any of you so well, but without you this wouldn’t be here.

Revision history:

• 27 Jan 2009: new lower price for World Tools, possibility of Illustrator plug-​in, minor corrections
• 28 Jan 2009: US pricing for Winsoft, tried to fix plug-​in warning with InDesign template (cosmetic but irritating)
• 29 Jan 2009: Corrected that it’s idRTL that has the installer, not WorldTools
• 30 Jan 2009: Fixed some typos, and missing backslashes in Windows path names
• 05 Feb 2009: Updated the InDesign scripts that create Arabic and Hebrew paragraph styles so they set the text to right-​justified (thanks to Peter Kahrel for catching that)
• 18 Apr 2009: Fixed description of auto-​fill with placeholder text (thanks to Roy McCoy for catching the bug)



Share this:

• Email
• Print
• Twitter
• Facebook
• More

• Reddit
• LinkedIn
• Pocket
• Tumblr
• Pinterest

Bugs in World-​Ready Composer »

If you have bug reports on the underlying World-​Ready Composer capabilities in Adobe Creative Suite 4 applications, log them as comments on this post, and I’ll make sure they get seen by the right people.

If you have feedback on scripts, templates, plug-​ins, or my big article on the World-​Ready Composer, please make comments to that post instead.

Thanks!

T