Recently in Macintosh Category

Understanding the Character Viewer in OS X 10.7, Lion

|

For Unicode fans, one of the bigger and more useful changes in the Mac 10.7 Lion operating system is the updated Character Viewer. Despite improvements though, it's different enough that I think some documentation is work posting.

How to Access

As with previous versions Character Viewer is activated in the Language & Text section of the System Preferences menu. See http://tlt.its.psu.edu/suggestions/international/keyboards/charpalosx.html for details.

How to Find Characters

By default, the Viewer only gives options for Symbols (e.g. "Arrows, Punctuation, Currency Symbols, Emoji"), I actually like how the symbols have been organized into semantic groups rather than by numeric block. However, the list does not include all the scripts I need to access.

Fear not though - the other block are availables. To view other blocks click the Gear icon, then select Customize List. This opens a pop-up window which provides a list of all available symbol lists, including Unicode, which is the entire list organized by Unicode block.

Character Viewer with Customize options open and multiple scripts checked

How to Insert

The basic mechanism is to select the character you want to highlight. In previous versions of this tool, there was an Insert button, but this has disappeared.

In this version, you need to do the following.

  1. Place your cursor at an appropriate insertion point in your document.
  2. Open the Character Viewer.
  3. Find and highlight your character.
  4. Double click the character. It will be inserted into your document.
    Note: You can also drag and drop characters into your document.

Favorites

A feature that I am now using is the Favorites list, a place to list commonly used characters to insert. This puts everything in one list which you can order as needed.

You can add Favorites by selecting a character and either either clicking the Add to Favorites button or dragging it into the Favorites list.

Conclusion

Although I was a little confused by the new Lion Character Viewer, I actually think it was a good overhaul. It suits the average user who needs access to common symbols and emoji, but allows us more dedicated users access to what we need.

A final improvement worth noting is that the Font Variation feature is much more stable than in previous features. This is perfect for times when you need to debug a weird character/font combination.

Categories:

i18n Enhancements Announced for Mac OSX 10.7 (Lion)

|

They're kind of scattered, but it looks like the next version of Mac OSX will be bringing lots of good enhancements for those working outside of English.

Asian Fonts and Text Input

Support for many scripts from South Asia has been lagging behind Windows, so I am personally pleased to see fonts for Bengali, Kannada, Malayalam, Oriya, Telugu and Sinhala being added (especially since I took 12 credits of Sinhala back in the day). New fonts for Tamil, Devanagari, Gujarati and Urdu are also scheduled to be added as well as for Lao, Khmer and Myanmar.

Those working with East Asian languages should be able to access improved utilities for Chinese (filtering by tones, ordering radical/stroke), Japanese Kotoeri and Vietnamese (old and new orthography). The Chinese handwriting recognition software is also scheduled to include more support for Simplified Chines and Roman characters. Finally, Apple announced that Lion will support vertical text (typing and display)

Everyone will also be able to a new color emoji font.

In Safari

Improvements for Safari included:

  • Math ML support in Safari
  • Improved CSS3 support including vertical text, East Asian emphasis, auto hyphenation

Non-English Accessibility

Accessibility options for those not using 100% English are not available include Voiceover speech in 23 languages and expanded Braille options.

Categories:

iPad & Unicode: B-

|

I'm not first to announce that iPad is not as Unicode savvy as it could be, but hopefully I can give some clear instructions.

I will state up front that I 100% (emoji for "love") my iPad, but Unicode support is one area that could use some improvement.

Reading: B+

The good news is that you can read a lot of scripts if the text comes in via Safari or Mail. The ability to read a script depends on having a font with glyphs for that block and Apple does try to hit many of the modern scripts. So coverage is decent, extending to most phonetic , Braille, math as well scripts such as Lao, Tibetan, Cherokee Armenian, Georgian, Hebrew, Arabic, CJK, Cyrillic, Greek and so forth.

However, there are gaps such as Ethopic, the Canadian United Aboriginal symbols, most Southeast Asian scripts outside of Thai and Lao, and many Indic scripts except for Devanagari/Gujarati/Gurmukhi. There is basic display of Tamil, Oriya and some South Indic scripts, but I am not sure if the vowel marks are placed correctly (it appears not). There are also missing "extended" characters in the Latin, Cyrillic and Greek blocks - not great for some scholarly work. And if you were hoping to catch up on your Phonecian, Coptic or Ogam reading, you may need to go elsewhere (or ask for a PDF).

That would be fine to begin, but it has been difficult for users to add extra fonts without extensive jailbreaking. I haven't tried it, and don't recommend it unless you are doing it with a test iPad.

Typing with Virtual Keyboard: D

I do appreciate that the virtual keyboard is larger on the iPad than the iPhone, but the default way to type an accent is to hold down a vowel key (or appropriate consonant) and wait for a set of accented letters to pop up. I've not been thrilled with the iPhone version of this, and still am not thrilled with the iPad version of it...but it is there.

Still you can't input a Welsh ŵ this way, and that is an annoyance if you happen to work with Welsh. You also can't input ă, ḍ, ş and a host of other characters used outside of mainstream European languages. Bummer

In theory you can switch the virtual keyboard to a non-English setting, but there are surprisingly few options - far less than even the iPhone. You can't even enter Korean or Greek characters. Ouch!

Typing with a Physical Keyboard: C+

One thing that you can now do with an iPad is to plug it in to a keyboard dock, and this solves a few problems. First you get access to a real keyboard with an Option key. So those pesky accents can be types with familiar option codes. It also lets you stand the iPad next to your laptop so you can test Unicode sites will composing a blog entry.

The dock also allows you to access a few more keyboard configurations including the U.S. Extended keyboard (hello ŵ,ḍ,ă,ş). And here's how:

Keyboard Setup

Basic Setup/Virtual Keyboard

  1. On your iPad, click the Settings icon.
  2. In the left column, click General.
  3. Click Keyboard on the right, then International Keyboards in the next window. A list of activated keyboards opens.
  4. Click Add New Keyboard to see all the options. Click on a language name to add it.

  5. Exit Settings.
  6. When you launch the international keyboard in an app, you should see a globe icon similar to the one below. Click on it to open a list of available keyboards and select an appropriate one.
    globe icon

Instructions with images are available from PadGadget.

Options for Keyboard Dock

If you want to activate the U.S. Extended, U.S. International option (PC), Dvorak, other extended option, do the following:

  1. Plug in the iPad into the dock (you can now type in Safari, Mail, Pages).

  2. Follow steps #1-3 above to see a list of activated keyboards.
  3. Click English to see layout options. The second Hardware Keyboard Layout includes options such as Dvorak, U.S. International-PC and U.S. Extended.
  4. Exit Settings.

Depending on which layout you select, you have access to different/additional option codes. You can also activate a Phonetic (i.e. QWERTY) layout for Russian which is also handy.

That does take care of a lot of my needs, but not all of them by any means. I still need to enter phonetic symbols and math symbols. I would like either a character palette, or even better, a way to build and upload additional keyboard files, like you can in OS X. I'd also like to be able to safely install new fonts, and I know I'm not the only one.

Categories:

What did that font switch to in FileMaker (Mac)?

|

Prelude about the Problem

In terms of handling non-English characters, apps come in two types (at least on the Mac). There are apps which switch fonts behind the scenes without telling you, and those which don't...but then you have to guess which font to use.

To take a concrete example, if I switch from the English keyboard to Japanese input in FileMaker, the font will automatically switch to one of the Japanese fonts. In theory, once I switch back to English, I should return to the original font (except when I don't...we'll get to that). The same principle applies in most apps including TextEdit, FileMaker and so forth. In contrast, if I switch to Japanese input in Adobe Photoshop, I also have to change fonts.

In theory, the automatic font switching sounds nice except when 1) when the font doesn't change back after typing the exotic character (this happens a lot in phonetic transcription and elsewhere) or 2) you're trying to figure if font X actually has that glyph (or whether it's the illusion of font switching in action. With the Adobe products, the manual font switching means you know exactly which font you are using at all times, which is important in desktop publishing.

FileMaker

For instance...I uploaded a version of the UCD Unicode files into FileMaker so I would have a searchable reference locally. An additional function is that I can display glyphs in different fonts for comparison. I have most of the mega fonts selected, but few fonts have everything, so I know there are gaps.

However, because FileMaker switches fonts behind the scenes, I can't always be sure if font X actually has that glyph. If I see a bunch of boxes with identical glyphs, I can suspect an unannounced font switch...but to what?

Solution

The best solution now is to copy and paste the text into TextEdit then open up the font formatting palette (Command+T), and see what it says. Kind of dorky, but still more information than I had.

For the record, I understand why FileMaker is set up this way. For most purposes, you don't want your data entry operators to fidget with fonts. However, you can get inconsistent results if you are not careful. For instance, once I do switch to Japanese, I get the Japanese font, but if I return to English...I still get the Japanese font. I know Japanese fonts contain Latin characters, but the formatting is almost always NOT the one I intended.

It would be nice if FileMaker and the other apps (including Microsoft Office) could return you to your original English font formatting after your exotic sidetrip to the higher code points of Unicode.

Categories:

Unicode 33 Presentation Files

|

Announced i18n Enhancements for Mac Snow Leopard (10.6)

|

New operating systems often mean new i18n toys to play with and even through the upgrade from Apple 10.5 (Leopard) to 10.5 (Snow Leopard) is not supposed to be full of new features, there are, in fact, new features scheduled for the upgrade.

According to the Apple Snow Leopard Enhancement page, 10.6 will include:

  • Redesign of Pinyin Chinese input with faster speed and enhanced dictionary
  • Improvements to handwritten Chinese input
  • New Asian fonts - Heiti SC, Heiti TC, Hiragino Sans B.
  • New generic monospace font Menlo to be used in applications such as Terminal
  • Enhanced RTL support including split cursor option to show text direction in documents with bidirectional text
  • General Text substitution (e.g. (c) to ©) across applications. Could be handy for a lot of situations when you need to enter an unusual symbol. This already exists in Microsoft Office (Mac/PC).

But I almost missed the big one - the International pane in the System Preferences has been redesigned and will now be the Language and Text pane, presumably with more features. There may be other enhancements in the works that are too minor to be announced (or at least too minor for most people), but there may be more things to find out.

How will they work? Alas, no details from Apple yet. I guess we won't know until we know....

Categories:

Enter Plane 1 (Phonecian/Linear B...) on Mac Unicode Hex Keyboard

|

A useful utility on the Mac is the Unicode Hex keyboard which allows you to press Option plus any four digit Unicode code to get that character.

For instance, if you need to enter the rarely seen archaic Roman numeral symbol for 5,000 (), you could look up its Unicode character number (U+2181), then activate this keyboard then type Option+2181 and generate the code (assuming the correct font is loaded).

But a lot of ancient scripts are in Plane 1, meaning they have Unicode values with five digits (i.e. U+10000 or higher). In Unicode world, adding the fifth digit means that some processes go slightly awry, and the Unicode Hex keyboard is one of them. Suppose I want to input Phonecian character Alf (Aleph) (𐤀 or an A on its side), which is U+10900. If I enter Option+10900 on the Unicode Hex keyboard, I will not get Alf, but ႐ instead.

Note: Characters U+0000 to U+FFFF are in Plane 0 or the BMP (Basic Multilingual Plane). A lot of systems are set up to deal with BMP only, but need special support for codes beyond U+FFFF. The four-digit restriction corresponds to 16-bytes which a constraint in older systems. If you're not a programmer, let's just say it's a long story and leave it at that.

It turns out that the Unicode Hex keyboard has a four-digit limit. To get around it, you can break U+10900 into two 16-byte (i.e. 4-digit) sequences, also known as as a UTF-16 Surrogate Pair. For U+10900, the surrogate pair is D802+DD0C. So in the Unicode Hex utility, you can now do this.

  1. Hold down the Option key.
  2. Type D802+DD0C, where the + means type the Plus sign.
  3. Release the Option key.

I bet you're asking - how did she get from U+10900 to D802+DD0C? There is an algorithm, but in this case I got it by opening the Character Palette, finding the character I wanted and mousing over it. When you do that, the Unicode code point appears along with its surrogate pair in parentheses.

Of course, you could also directly Insert the character with the palette, but actually there are times when the Insert doesn't quite work (at some points in the careers of my laptops, I have corrupted my Character Palette so badly, it refused to play with me anymore).

Although this utility seems a little limited at the moment, if there's one thing I have learned is that Unicode no trick has ever gone to waste.

Categories:

iPhone Support for Romanian (Sorin Sbarnea)

|

This is sort of a guest column entrty. A while ago, I wrote about iPhone 3 Unicode support, and a reader Sorin Sbarnea sent me this following as a comment.


Sorin Sbarnea

I was interested in adding a comment to your article from http://www.personal.psu.edu/ejp10/blogs/gotunicode/2009/06/iphone-30-unicode-support-stil.html but I was not able to find any way of adding a comment on the page.

IPhone 3.0 does have full support for all Romanian characters, previous versions did not. In order to be able to enter special characters you need to use a Romanian keyboard. Due to ergonomics they decided to not include all character on one keyboard because it would be too hard to use. Also I want to say that the drag model is genuine and very practical - after few days you'll see that is much smarter and more precise than the old click model.

I remember that I submitted 3 bugs on Apple for Romanian support, they solved one in 2.1 and the rest in 3.0 - so all you have to do is to complain to them ;)

Just to give you an example: in the first implementation of the Romanian keyboard the Ă character was the last on the right of the list of accented A characters after characters that are not used in Romanian - I complained to them explaining the reasons and now they solved it. I don't know if I was the cause or the only one complaining but they solved it anyway. Also - I wasn't able to find *any* error in Romanian translation of the iPhone - this is something very good - let's say I wasn't expecting this level of quality for Romanian translation.


Comments

The lesson here is that Apple does listen...if you know where to send input. Still not convinced about dragging on the iPhone in general (except for solitaire), but I can be stubborn. I'm glad Sorin is a satisfied Apple customer.

As to the question about submitting comments - I disabled mine because about 99% of them were offers for land in Florida or pharmaceuticals I cannot use. For now, please feel free to contact me at ejp10@psu.edu. If I hear an outcry for commenting, I may change my mind.

Categories:

iPhone 3.0 Unicode Support (Finding the ŵ)

|

This week I upgraded my iPhone (actually iPod Touch) software to version 3.0, and although I noted the copy/paste and enhanced landscape display, of course I zoned in on the note saying there was increased character support. Hmmm.

As a warning, I have to admit that I'm a little behind the times in mobile computing, so bear with me if I repeat something you already know. Still, I'm not seeing this information all in one place it it may be a good over (at least for me).

The good news is that there does appear to be more character support, but the feature is still too well-hidden (I really had to work hard to find Welsh support). The iPhone also fails my test for general Unicode readiness because I am not able to yet enter phonetic characters like /ŋ,ɛ,ʃ/ (if nothing else that would kill the iPhone as a remote data entry device). However I doubt the iPhone is really not alone in that area.

So if you are wondering what I am talking about, let me discuss in context:

Baseline Support

Unicode data and display for major languages is generally supported. If Safari can display your Unicode Webpage, it will appear correctly on your iPhone...assuming that the built in fonts support the character. Further, if you have entered/purchased an exotic title in iTunes, it will appear correctly in your synched iTunes list on the iPhone.

Entering Accents

The next challenge is entering some exotic characters into e-mail or a notes application. If you are dealing with Roman characters, iPhone does have some support, but not as much as I would like. The easiest non-English characters to find are foreign currency symbols like £ (pound), ¥ (yen) and € (euro). You typically access these by clicking the the symbol set (often right after the numerals).

While I was able to figure that out, I admit to being stumped as to how to enter accented letters such as Spanish ñ or French è. Fortunately a quick Google search turned up some help sites including this blog entry from Pixelcoma. As you can see, the trick is to hold down a base key such as N or E to see the options for accented characters.

The trick though is that you have to drag your finger across to the right character. You can't hold and double tap as I tried to do. Oops

As stated earlier, there are more options in the palette than in previous earlier versions. For instance, the Pixelcoma A options show A,À,Á,Ä,Æ,Ã,Å,Ą which already covers lots of Western and Central European languages, but Version 3 does add Ā (macron) which is good for Japanese Romaji, Hawaiian, Maori and Latin with long marks (I know there are Latin users out there). I assume that there are other important additions at the other base letters.

However, there are still apparent gaps such as Welsh accented W and Icelandic þ,ð/Ð as well as Romanian Ă, Turkish Ğ,Ş and İ, Latvian Ņ and other really exotic accented letters. It turns out that many are actually in keyboard options installed on the iPhone with additional characters. It still can feel like these languages are "second" class in comparison to Spanish, French and German (at least Polish, Czech and Hungarian have been "mainstreamed" which is a plus).

Before I leave this section though, I do have a comment for future devlopers:

Future developers - if you want to wow your audience with global accent support, you may want to start here at the Wikipedia Latin palette.

WikipediaLatinPal.png

That way we can avoid the agonizing incremental addition of accented letters as individual user communities step forward. Why not be comprehensive at the start - like the Apple U.S. Extended keyboard (which is major reasons I still love Apple).

As much as I kvetch though, I don't think the iPhone is worse than any other U.S. mobile device. A forum post for Blackberry mentions holding down a vowel and moving a trackball. ¡Qué divertido!

Other Keyboards

As mentioned previously, if your character is not available in the accent palette, you may need to activate the keyboards (just like in the laptop/desktop). On the iPhone, you access these by clicking the Settings app, then going to General Settings then International. A number of keyboards for languages like Chinese, Japanese, Russian, Hebrew, Arabic as well as Icelandic, Turkish, Latvian are available (still no Welsh, unless it's hiding under the U.K. keyboard (yes it is !)).

This adds a globe icon (like the one below) to the usual iPhone keyboard and allows you to switch between keyboard modes. I just switched to the U.K. keyboard and behold, I found the ŵ under the W key (but now the ¥ key is missing).

GlobeIcon.png
Icon for International Keyboards on iPhone

What I Really Want...

Actually it's not necessarily more accented letters as I hold down a key. My thumb is shuddering at how the potential pain of dragging or trackballing additional accents on top of the other precision maneuvers required for English texting. I actually want several things

First, slightly better keyboard designs. The iPhone Google keyboard has the right idea when it makes the @ sign and .com extension basic keys. We already have options for switching on canned keyboards, but what if we had options for customizable keyboards. Maybe one with a "symbol" dock into which you drag the characters or phrases you need from a master slot (this way Americans learning Welsh CAN have their accented W's). Maybe you can reshuffle as well (like killing the \ key if you only synch with a Mac).

But I have to confess that I really want to be able to plug my iPhone into a keyboard. IThe touch interface is fine short small tasks on the run (like looking up movie times or weather by zip code), but still not so great for longer data entry or note taking tasks. I know it's Palm Pilot, but I am at a stage where I would like to ditch the laptop for short meetings and only carry a mobile device and take notes. I note that there are there are hacks out there already...despite the useful shortcuts provided. That should be a sign for Apple and other makers of mobile devices that the need is out there (bummer dudes).

It goes without saying that if true Mac keyboard integration comes, it should come with support for the U.S. Extended and other keyboard variations Apple and the user community have concocted (Windows users can use the U.S. International keyboard for the Mac).

A final wish though is better documentation. The Unicode support for iPhone is decent, but it's quite a chore tracking it all down through numerous user blogs and guessing. I know Apple relies somewhat on it's "intuitive" interface to help users through, but, for whatever reason, Unicode support is rarely intuitive. You just have to know where things are. I'm glad there's a user community out there but from the lack of documentation (especially in comparison to Microsoft) it seems like Apple doesn't care about these issues (when I think they really do).

Microsoft has various Globalization sites (in English), so why can't Apple (or at least one I can find)? Is it because we're in the U.S? To me, It's a little condescending to me to assume that just because I live in the U.S. I will rarely need to enter non-English text. In fact, I type something "non-English" nearly every day.

Categories:

Unicode Tool in Mac Calculator

|

Every so often you notice a brand new feature in an old tool and wonder where it's been all these years. One of these was the Unicode Character decoder in the built in Mac calculator. (I'm not the first to notice, but the last reference I found was from 2004, when the latest OS X was 10.3 (Panther).

If you open the Calculator app (it's in your Applications folder), you can switch over to the Programmer view (Command-3 or View » Programmer) which is where you can convert between decimal and hexadecimal number using the tabs at the right.

On the left though there is an ASCII/Unicode tab. This shows the characte corresponding to whatever number you've entered. For instance hex x562 (or 1378 in decimal) is an Armenian character բ (Armenian Small Letter Ben).

Calculator with Unicode tab on showing Armenian character and Hex tab on for x562

I have to confess that I wish it could do the reverse - you paste a character and it shows the Unicode code point, but this is good for testing odd character references you may find in a source file.

However a student in a recent seminar pointed out a site which does convert a character to a decimal code reference at http://www-atm.physics.ox.ac.uk/user/iwi/charmap.html (from Alan Iwi at the Rutherford Lab at Oxford). Just enter or paste the character and click the the Make HTML button to see a decimal entity code.

Although I don't generally recommend entity codes, there are times when your environment is so antique you need to convert some text into entity codes.

Categories:

About The Blog

I am a Penn State technology specialist with a degree in linguistics and have maintained the Penn State Computing with Accents page since 2000.

See Elizabeth Pyatt's Homepage (ejp10@psu.edu) for a profile.

Comments

The standard commenting utility has been disabled due to hungry spam. If you have a comment, please feel free to drop me a line at (ejp10@psu.edu).

Powered by Movable Type Pro

Recent Comments