Entries Tagged as 'fdo'

Layout editor keyboard

This is a screenshot of the keyboard for the layout editor. The keyboard is a widget which is composed of individual widgets of each key.

I did not use glade-3 for the keyboard at this time. Although it is possible to create custom widgets in Python and install them in Glade, the current distributed packages are missing something, thus it would be messy when others try to use the editor. It’s a good experience to do all by hand anyway.

When creating a layout, you drag and drop characters on the keyboard. The editor shows a table with characters though it would be possible to drag characters from gucharmap as well.

The next step is to get an intuitive UI so that when you drop a character on a key, the key expands (a popup appears) showing the available four positions to receive the character.

Keyboard layout editor UI concept

(click for bigger image)

At the top we select the keyboard layout file, the variant, and set the corresponding verbose name.

The keyboard layout editor shows a standard keyboard, where each keyboard key can show up to four levels. When you select a key, the bottor-left window shows the characters that have been set (here we use four levels). In this bottom-left window we can drag and drop characters (from Unicode blocks) and dead keys that are found from the right of the image. Dead keys are shown in red boxes.

The user is also able to include existing keyboard layout files in the current layout.

At this stage I am thinking how to easily draw the keyboard in a PyGTK application. It would be important not to draw it manually. It would be cool to have a GTK+ keyboard key widget, that you can specify the size, and the text that appears on it, then build a keyboard in Glade. Another option would be to have the basic keyboard as an SVG file (already exists), then draw over it with Cairo. I am inclined for the second option.

Converting between XKB and XML

I completed the stage that takes keyboard layout files from XKB (X.Org) and converts them to XML documents, based on a keyboard layout Relax NG schema. Then, these XML documents can also be converted back to keyboard layout files.

Here is an imaginary example of a keyboard layout file.

// Keyboard layout for the Zzurope country (code: zz).
// Yeah.

partial alphanumeric_keys alternate_group hidden
xkb_symbols "bare" {
   key <AE01> { [        1, exclam,      onesuperior,  exclamdown      ] };
};

partial alphanumeric_keys alternate_group
xkb_symbols "basic" {
   name[Group1] = "ZZurope";

   include "zz(bare)"

   key <AD04> { [        r, R,           ediaeresis,   Ediaeresis      ] };
   key <AC07> { [        j, J,           idiaeresis,   Idiaeresis      ] };
   key <AB02> { [        x, X,           oe,           OE              ] };
   key <AB04> { [        v, V,           registered,   registered      ] };
};

partial alphanumeric_keys alternate_group
xkb_symbols "extended" {
    include "zz(basic)"
    name[Group1] = "ZZurope Extended";
    key.type = "THREE_LEVEL"; // We use three levels.
    override key <AD01> {   type[Group1] = "SEPARATE_CAPS_AND_SHIFT_ALPHABETIC",
[ U1C9, U1C8], [  any,   U1C7 ]   }; // q
    override key <AD02> {   [ U1CC, U1CB, any,U1CA ],
type[Group1] = "SEPARATE_CAPS_AND_SHIFT_ALPHABETIC" }; // w
    key <BKSP> {
        type[Group1]="CTRL+ALT",
        symbols[Group1]= [ BackSpace,   Terminate_Server ]
    };
    key <BKSR> { virtualMods = AltGr, [ 1, 2 ] };
    modifier_map Control { Control_L };
    modifier_map Mod5   { <LVL3>, <MDSW> };
    key <BKST> { [1, 2,3, 4] };
};

When converted to an XML document, it looks like

<?xml version="1.0" encoding="UTF-8"?>
<layout layoutname="zz">
  <symbols>
    <mapoption>hidden</mapoption>
    <mapoption>xkb_symbols</mapoption>
    <mapname>bare</mapname>
    <mapmaterial>
      <tokenkey override="False">
        <keycodename>AE01</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>1</symbol>
            <symbol>exclam</symbol>
            <symbol>onesuperior</symbol>
            <symbol>exclamdown</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
    </mapmaterial>
  </symbols>
  <symbols>
    <mapoption>xkb_symbols</mapoption>
    <mapname>basic</mapname>
    <mapmaterial>
      <tokenname name="ZZurope"/>
      <tokeninclude>zz(bare)</tokeninclude>
      <tokenkey override="False">
        <keycodename>AD04</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>r</symbol>
            <symbol>R</symbol>
            <symbol>ediaeresis</symbol>
            <symbol>Ediaeresis</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>AC07</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>j</symbol>
            <symbol>J</symbol>
            <symbol>idiaeresis</symbol>
            <symbol>Idiaeresis</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>AB02</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>x</symbol>
            <symbol>X</symbol>
            <symbol>oe</symbol>
            <symbol>OE</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>AB04</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>v</symbol>
            <symbol>V</symbol>
            <symbol>registered</symbol>
            <symbol>registered</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
    </mapmaterial>
  </symbols>
  <symbols>
    <mapoption>xkb_symbols</mapoption>
    <mapname>extended</mapname>
    <mapmaterial>
      <tokenname name="ZZurope Extended"/>
      <tokeninclude>zz(basic)</tokeninclude>
      <tokentype>THREE_LEVEL</tokentype>
      <tokenmodifiermap state="Control">
        <keycode value="Control_L"/>
      </tokenmodifiermap>
      <tokenmodifiermap state="Mod5">
        <keycodex value="LVL3"/>
        <keycodex value="MDSW"/>
      </tokenmodifiermap>
      <tokenkey override="True">
        <keycodename>AD01</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>U1C9</symbol>
            <symbol>U1C8</symbol>
          </symbolsgroup>
          <symbolsgroup>
            <symbol>any</symbol>
            <symbol>U1C7</symbol>
          </symbolsgroup>
          <typegroup value="SEPARATE_CAPS_AND_SHIFT_ALPHABETIC"/>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="True">
        <keycodename>AD02</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>U1CC</symbol>
            <symbol>U1CB</symbol>
            <symbol>any</symbol>
            <symbol>U1CA</symbol>
          </symbolsgroup>
          <typegroup value="SEPARATE_CAPS_AND_SHIFT_ALPHABETIC"/>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>BKSP</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>BackSpace</symbol>
            <symbol>Terminate_Server</symbol>
          </symbolsgroup>
          <typegroup value="CTRL+ALT"/>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>BKSR</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>1</symbol>
            <symbol>2</symbol>
          </symbolsgroup>
          <tokenvirtualmodifiers value="AltGr"/>
        </keysymgroup>
      </tokenkey>
      <tokenkey override="False">
        <keycodename>BKST</keycodename>
        <keysymgroup>
          <symbolsgroup>
            <symbol>1</symbol>
            <symbol>2</symbol>
            <symbol>3</symbol>
            <symbol>4</symbol>
          </symbolsgroup>
        </keysymgroup>
      </tokenkey>
    </mapmaterial>
  </symbols>
</layout>

When we convert the XML document back to the XKB format, it looks like

hidden xkb_symbols "bare"
{
	key <AE01> { [ 1, exclam, onesuperior, exclamdown ] };
};

xkb_symbols "basic"
{
	name = "ZZurope";
	include "zz(bare)"
	key <AD04> { [ r, R, ediaeresis, Ediaeresis ] };
	key <AC07> { [ j, J, idiaeresis, Idiaeresis ] };
	key <AB02> { [ x, X, oe, OE ] };
	key <AB04> { [ v, V, registered, registered ] };
};

xkb_symbols "extended"
{
	name = "ZZurope Extended";
	include "zz(basic)"
	key.type = "THREE_LEVEL";
	modifier_map Control { Control_L };
	modifier_map Mod5 { <LVL3>, <MDSW> };
	override key <AD01> { [ U1C9, U1C8 ], [ any, U1C7 ], type = "SEPARATE_CAPS_AND_SHIFT_ALPHABETIC"  };
	override key <AD02> { [ U1CC, U1CB, any, U1CA ], type = "SEPARATE_CAPS_AND_SHIFT_ALPHABETIC"  };
	key <BKSP> { [ BackSpace, Terminate_Server ], type = "CTRL+ALT"  };
	key <BKSR> { [ 1, 2 ], virtualMods = AltGr  };
	key <BKST> { [ 1, 2, 3, 4 ] };
};

Some things are missing such as partial, alphanumeric_keys and alternate_group, which I discussed with Sergey and he said they should be ok to go away.

In addition, we simplify by keeping just Group1 (we do not specify it, as it is implied).

I performed the round-trip with all layout files, and all parsed and validated OK (there is some extra work with the level3 file remaining, though).

Some issues that are remaining, include

  • Figuring out how to use XLink to link to documents in the same folder (+providing a parameter; the name of the variant), and how to represent that in the Relax NG schema.
  • Sort the layout entries by keycode value.

ANTLR grammar for XKB, and Relax NG schema (draft)

I completed the ANTLRv3 grammar for symbols/ configuration files of XKB. The grammar can parse and create the abstract syntax tree (AST) for all keyboard layouts in xkeyboard-config.

ANTLRv3 helps you create parsers for domain specific languages (DSL), an example of which is the configuration files in XKB.

Having the ANTLRv3 grammar for a configuration file allows to generate code in any of the supported target lagnuages (C, C++, Java, Python, C#, etc), so that you easily include a parser that reads those files. Essentially you avoid using custom parsers which can be difficult to maintain, or parsers that were generated with flex/bison.

On a similar note, here is the grammar to parse Compose files (such as en_US.UTF-8/Compose.pre). I am not going to be using in the project for now, but it was fun writing it. The Python target takes 18s to create the AST for the >5500 lines of the en_US.UTF-8 compose file, on a typical modern laptop.

I am also working on creating a RelaxNG schema for the XKB configuration files (those under symbols/). There is a draft available, which needs much more work.The Relax NG book by Eric van de Vlist is very useful here.

The immediate goal is to use the code generated by ANTLR to parse the XKB files and create XML files based on the Relax NG schema. I am using Python, and there are a few options; the libxml2 bindings for Python, and PyXML. The latter has more visible documentation, but I think that I should better be using the former.

Update: lxml appears to be the nice way to use libxml2 (instead of using directly libxml2).

Looking into the symbol files

In the previous post, we talked about the ANTLR grammar that parses the XKB layout files.

The grammar is available at http://code.google.com/p/keyboardlayouteditor/source/browse. I’ll rather push to the freedesktop repository once the project is completed. Now it’s too easy for me, just doing svn commit -m something.

Below you can see the relevant layout files for each country (and in some cases, language), and how the grammar deals with them. First column is filenames from the CVS XKB symbols subdirectory (to be moved eminently to GIT). Last’s week discussion with Sergey helped me figure out issues with the symbol files, simplify what information is needed, and what can be eliminated. Second column has Not OK if something is wrong. Third column tries to explain what was wrong.

ad
af
al
altwin
am
ara
az
ba
bd
be
bg
br
braille
bt
by
ca
capslock
cd
ch
cn
compose
ctrl
cz
de
dk
ee
epo
es
et
eurosign
fi
fo
fr
gb NOK Non-UTF8
ge
gh
gn
gr
group NOK virtualMods= AltGr
hr
hu NOK Non-UTF8
ie
il NOK key.type=”FOUR_LEVEL” (typically: key.type[something]=….)
in NOK key.type=”FOUR_LEVEL” (typically: key.type[something]=….)
inet
iq
ir
is
it
jp NOK key <BKSP> {
type=”",   // empty?
symbols[Group1]= [ bracketright, braceright ]
};
keypad NOK overlay1=<KO7> }; // what’s “overlay”?
kg
kh
kpdl
kr
kz
la
latam
latin
level3 NOK virtual_modifiers LAlt, AlGr; virtualMods= Lalt
level5
lk
lt
lv
ma
mao
me
mk
mm
mn
mt
mv
nbsp NOK Non-UTF8
ng
nl
no
np
olpc
pc NOK key <AA00> { type=”SOMETHING” } instead of { type[Group1]=”SOMETHING” }
pk
pl
pt
ro
rs
ru
se
shift NOK actions [Group1] = [
si
sk
srvr_ctrl NOK key <AA00> { type=”SOMETHING” } instead of { type[Group1]=”SOMETHING” }
sy
th
tj
tr
ua

Non-UTF-8 are the files that have characters that are not UTF-8 (are iso-8859-1).

Some layouts have key.type = “something” and others key.type[SomeGroup] = “something”. Apparently, the format allows to infer which is the group that the type acts upon? That’s weird. Would it be better to put the group information? Is it required that the group is not set?

Some files have virtualMods, which I do not know what it is. Is it used?

Parsing XKB files with antlr

antlr (well, antlr3) is an amazing tool that replaces lex/flex, yacc/bison.

One would use antlr3 if they want to deal with Domain-Specific Languages (DSL), an example of which are the text configuration files.

In our case, we use antlr3 to parse some of the XKB configuration files, those found in /etc/X11/xkb/symbols/??.

Our aim is to be able to easily read and write those configuration files. Of course, once we have them read, we do all sorts of processing.

The stable version of antlr3 is 3.0.1, which happened to give lots of internal errors. It has not been very useful, so I tried a few times the latest beta version 3.1b, and eventually managed to get it to work. If I am not mistaken, 3.1 stable should be announced in a few days.

When using antlr, you have the choice of several target languages, such as Java, C, C++ and Python. I am using the Python target, and the latest version that is available from the antlr3 repository.

Here is the tree of the gb layout file,

tree = (SECTION (MAPTYPE (MAPOPTIONS partial default alphanumeric_keys xkb_symbols) (MAPNAME “basic”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 quotedbl twosuperior oneeighth)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior sterling)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign onequarter)) (TOKEN_KEY (KEYCODEX AC11) (KEYSYMS apostrophe at dead_circumflex dead_caron)) (TOKEN_KEY (KEYCODEX TLDE) (KEYSYMS grave notsign bar bar)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign asciitilde dead_grave dead_breve)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar bar brokenbar)) (TOKEN_INCLUDE “level3(ralt_switch_multikey)”))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “intl”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom - International (with dead keys)”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 dead_diaeresis twosuperior onehalf)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior onethird)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign onequarter)) (TOKEN_KEY (KEYCODEX AE06) (KEYSYMS 6 dead_circumflex NoSymbol onesixth)) (TOKEN_KEY (KEYCODEX AC11) (KEYSYMS dead_acute at apostrophe bar)) (TOKEN_KEY (KEYCODEX TLDE) (KEYSYMS dead_grave notsign bar bar)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign dead_tilde bar bar)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar bar bar)) (TOKEN_INCLUDE “level3(ralt_switch)”))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “dvorak”)) (MAPMATERIAL (TOKEN_INCLUDE “us(dvorak)”) (TOKEN_NAME Group1 (VALUE “United Kingdom - Dvorak”)) (TOKEN_KEY (KEYCODEX BKSL) (KEYSYMS numbersign asciitilde)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 quotedbl twosuperior NoSymbol)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling threesuperior NoSymbol)) (TOKEN_KEY (KEYCODEX AE04) (KEYSYMS 4 dollar EuroSign NoSymbol)) (TOKEN_KEY (KEYCODEX LSGT) (KEYSYMS backslash bar)) (TOKEN_KEY (KEYCODEX AD01) (KEYSYMS apostrophe at)))) (SECTION (MAPTYPE (MAPOPTIONS partial alphanumeric_keys xkb_symbols) (MAPNAME “mac”)) (MAPMATERIAL (TOKEN_INCLUDE “latin”) (TOKEN_NAME Group1 (VALUE “United Kingdom - Macintosh”)) (TOKEN_KEY (KEYCODEX AE02) (KEYSYMS 2 at EuroSign)) (TOKEN_KEY (KEYCODEX AE03) (KEYSYMS 3 sterling numbersign)) (TOKEN_INCLUDE “level3(ralt_switch)”)))

When traversing the tree, we can then pretty-print the layout at wish:

partial default alphanumeric_keys xkb_symbols “basic” {
name[Group1] = “United Kingdom”;
include “latin”
include “level3(ralt_switch_multikey)”
key <AE02> = { [ 2 , quotedbl , twosuperior , oneeighth ] };
key <AE03> = { [ 3 , sterling , threesuperior , sterling ] };
key <AE04> = { [ 4 , dollar , EuroSign , onequarter ] };
key <AC11> = { [ apostrophe , at , dead_circumflex , dead_caron ] };
key <TLDE> = { [ grave , notsign , bar , bar ] };
key <BKSL> = { [ numbersign , asciitilde , dead_grave , dead_breve ] };
key <LSGT> = { [ backslash , bar , bar , brokenbar ] };
};
… snip …

The code is currently hosted at code.google.com (keyboardlayouteditor) and I intend to move it shortly to FDO.

Keyboard Layout Editor GSOC project

I got accepted for a GSOC project with the X.Org Foundation. My mentor is Sergey Udaltsov and I look forward working with him.

The project is about creating a Keyboard Layout Editor, that can be used to edit XKB files with a nice GUI.

I will be blogging about these from here (fdo category at this blog).