Unicode — Supplementary Planes

Back to Unicode. Even Unicode needs room to expand. The Basic Multilingual Plane (BMP, also known as Plane 0) has codepoints 0 through FFFF hex (0 through 65,535) and contains, ignoring various special mathematical characters, historical quirks, and just plain oddities, most of the current alphabets and symbols in use today, even including the vast array of Chinese kanji. But of course there’s more room needed once historical alphabets and other special character sets are also considered. Supplementary planes such as Plane 1 are simply sets of 65,535 code points following Plane 0 (some of us refer to these as astral planes ). Using hexadecimal notation, it’s easy to spot Plane 1: all Plane 0 codepoints use four hexadecimal digits; all Plane 1 use five, etc.

Here is an example of a Plane 1 character: 𝔊 which renders as 𝔊 (marks a septaugint reference). In case anyone viewing this is having trouble, here’s an image:

To set things up to view this can be an interesting process. First of all it’s necessary to find a unicode font that supports plane 1 characters. Not all such will actually show the above, for example Code2001, an otherwise excellent unicode font, does not include historical Greek musical notation however Cardo works nicely. Just because a font is unicode based does not mean it will be “complete” (such a font would be staggeringly large). In particular, when choosing among unicode fonts, pay attention to which version of unicode is being supported, and what the target languages are. Plane 1 characters start to appear in Unicode versions 3.2 and up.

Second, the operating system might need slight adjusting to see Plane 1. I’m talking, of course, about Windows, XP and earlier versions. This page discusses how to do the registry edits necessary both for the operating system and for IE6 .

Linux/*nix, Mac, and Vista are all presently able to handle Plane 1 without modifications. My understanding is that for Windows Me and anything prior to NT, it’s hopeless.

Third, applications in general and browsers in particular may need slight tweaking to see Plane 1 characters. As usual, IE6 is the worst offender in this regard requiring not just setup but also a registry edit. In general, once the browser is set up with a Plane 1 aware font, it’s good to go. Firefox, Safari, Opera, and Konqueror all fall under this category.

A few notes: IE6 needs to be set to user-defined encoding plus the extended font needs to be listed under User Defined (rather than any of the specific regions listed, which refer back to non-unicode encodings as discussed in part 1). Opera actually reverses from the general MO of other browsers: under Tools->Preferences->Advanced->International Fonts, it’s possible to set a particular unicode font to a particular language (going by Unicode’s code blocks). This is the direction other browsers should no doubt take in the future.

Overview

So in Windows and Mac, there’s this so-called regional settings that only “allows” you to change your regional settings five times before “locking” it in, courtesy of the entertainment industry determined to dictate every choice to their customers and squeeze every penny out of them into the bargain. Ah, the fresh smell of capitalism in the morning.

The way it works is that DVDs which are “Region 1″ (2, 3, etc) are CSS (content scramble system) encrypted. The settings on the computer indicate which “region” it is, and that will refuse to play those DVDs not matching the set region. This is a result of collusion between OS companies (eg, Microsoft, Apple), DVD drive companies (Panasonic, Sony, etc), and assorted software programs. It’s an ongoing legal tangle, and also when OpenSource is tossed in, gets interesting fast, as OpenSource will not generally “respect” the encryption and simply play DVDs regardless, although such a user generally will need to install a few extra programs and such to make it all work so in the end, everyone winds up having to work around this issue.

Oddly enough, it’s almost simpler when discussing DVD players. For these, just google up the brand name of the player, along with the terms ‘region free” etc, and it’s possible to find, for most players, the codes necessary to convert the player into multiregional or region free players. Once that’s done and it’s hooked up to the TV set, playing any DVD is not a problem.

Approaches

But back to DVD drives in computers: there are several ways around regional restrictions. One has been to hack the firmware on the DVD drive. This isn’t an especially good idea since such hacking can render the drive dead a certain percentage of time. So I’d toss that out.

A second option is to copy the DVD first, stripping out the CSS, and burning the altered copy to a blank disk and using that DVD instead. This works, but is a little cumbersome — having to copy each encrypted DVD before viewing it.

Finally, there are software programs that simply bypass the operating system restrictions. These can be built into the dvd players themselves, or be a separate program functioning on its own, allowing the user to use whichever favorite software for viewing the disc’s contents.

Windows

For Windows users, most of the options center around the third one, although a number include the ability to use the second approach as well. For example DVD Region + CSS Free and DVD Region Free Master (both very reasonably priced shareware). The programs work at the software level to bypass any restriction on the computer’s dvd drive, whether or not it has been locked (changed more than five times) and without altering its allowable changes.

DVD Shrink allows users to copy commercial dvds to blanks. The program allows for copying and compressing and/or eliminating parts of the original dvd, including stripping the CSS encryption.

Mac

For Mac users, the picture is a little different. There are some programs out there but there don’t seem to be that many (and which seem to be for older versions), which puzzles me because OSX is Linux based, more or less. Some of the issues seem to revolve around which Mac one has, and which dvd drivers it has. VLC may work for some Mac users, but does not work for all. I did find this article which summed up the issues for mac users and described one solution the author worked out. I have also seen speculation that Mac’s new Bootcamp software might allow Mac users to run the Windows region free software + dvd playback to obtain similar results (but I did not find anything written up by anyone who’d tried it to see).

There’s a program called MacTheRipper which allows copying of DVD’s minus the encryption. Also check out DVDBackup to back the DVD up on the computer.

Of course with the new Intel chips, a dual boot WinXP and mac, or dual boot with Linux and Mac and move back and forth as needed.

Linux

As an Open Source operating system, Linux’s difficulties with CSS encryption stem from the proprietary nature of the software. In most cases, what a Linux user needs to do is download the additional software and codecs to view DVD’s. Most distributions will have instructions available somewhere on how to do this: googling on the distribution name and region free will in most cases uncover instructions specific to the distribution. For example, googling on “ubuntu dvd region free” (without the quotes) will uncover tutorials such as this.

There are a wide variety of players, such as mplayer, Xine, Kaffeine, Ogle, and Totem. Most distributions will come with one or more of these, the trick is enabling the play for encrypted DVDs as covered in the previous paragraph. Of these, Kaffeine is generally considered the best of the lot. The rest may not have a full gui interface, or sync properly, be somewhat buggy, or have other limitations.

As for creating region free DVD’s: one method is outlined here and another nice overview is here. The basic idea is as with the mac: create a copy or backup on the computer, and then burn selected portions to the blank disc.

Cross Platform

One name that kept popping up was VLC which has versions for Linux, Windows, and Mac. I have not tried this myself, but the user feedback/ratings look pretty good.

Further Information

Check wiki for more overall information on DVDs, DVD software, and dealing with CSS.
For complete informatin overload (but well written and easy to skim for specific questions), check out the DVD faq.

Legal Issues

Notes about legality. It’s becoming a bit more common to find notices such as this one even outside the U.S. Not only do laws vary from country to country but they appear to be in the process of changing, as more attention is paid to this issue. I should stress that this article is provided for informational purposes.

Also, I am not a lawyer, but this is my view: When I purchase a book, I am entitled to read it, whenever and whereever I want to. I can read it as many times as I want to. I can do anything to the book itself — write in it, tear pages out, laminate it, store it, throw it away, give it away, or sell it. I may not make copies of it, with the exception of small excerpts and such under “Fair Use” doctrine (eg, quoting passages in a review, that kind of thing). Extending that concept to a DVD, I personally find the encryption and regional restrictions problematic, as these may prevent me from enjoying something I have paid for the right to view. I therefore view getting around the encryption to be a legitimate exercise in being able to make rightful use of a product I’ve paid for.

The gray area is, obviously, when DVD’s are copied. And here it’s unclear: for example it’s been established as acceptable for users to make personal copies of audio tapes or CD’s — for example making a cassette tape for personal use, because one’s car does not have a CD player, or “timeshifting” where one makes a VHS copy of a broadcast to view later. No reselling is involved. So making a copy of a DVD for personal viewing ought to fall into similar categories (watch a DVD on laptop), but the companies are nervous because at this point, redistribution of copies is ridiculously easy over the Internet (not as easy to make VHS tape copies and distribute, nor audio cassette copies and distribute). That, in a nutshell, is why DVD copying is an “issue” and why the conventions are inconsistent between book, tape, and CD/DVD.