Textual Editing Framework (TEF)

The Textual Editing Framework (TEF) allows you to create textual editors for your EMF-based languages. TEF provides a syntax definition language called TSL. In this language, you, the language engineer, can describe a textual notation for a given Ecore meta-model. From this TSL description, the framework can automatically generate a TEF editor. Generated TEF editors provide an extensive set of modern text editor features: syntax highlighting, content assist (code completion), intelligent navigation, or visualisation of occurrences. TEF is based on Eclipse. TEF itself is provides as a series of eclipse plug-ins. Using TEF, you can create your own TEF editors within your Eclipse plug-ins

Two Different Strategies

We have learned that there are two different strategies to approach the textual editing of models. Each of them has its own advantages and disadvantages. We have tried both strategies as a foundation for TEF, and eventually selected the one strategy that has, from out perspective, the higher potential.

Strategy I: Model View Controller

The Model View Controller strategy is named after the model view controller pattern. This pattern, rooted in graphical user interface design, separates each model element from its representation, and thus allows to use multiple representations on the same model element. We have prepared a small video that exemplifies this approach:

This is a presentation video featuring an example editor for EMF ecore models written with TEF. The video demonstrates core TEF features and gives you a general idea about what you can expect from a TEF editor.

This ECore example editor showed how the model view controller pattern allowed to edit the same model from different views and how the different representations of the same model element are connected. The MVC priciple automatically keeps representation and model in synchronisation by only allowing changes to the model, which are immediately translated into according changes in the representations. This only allows model representation of possible models. This sounds like an advantage, but it is, at the same time, the biggest problem with the MVC strategy.

When editing with a textual notation you have invalid intermediate representations. When you type a correct model character by character you will have invalid representations within the process, even if the end result is a valid model representation. In the example editor we solved this problem by forbidding typing and only allowing model changes by atomic actions like "add a element", "rename a element", etc. This worked well for the ECore editor, but there are often structures that can be edited that way. An example are expressions, for example in OCL. If you want to create an expression like "(1+2)*3", you would have to think like a parser and create "...*3" first and then "(1+2)". This strategy is completely inappropriate for such structures.

Strategy II: Background Parsing

Well, if you don't like to think like a parser, you acutally might use a parser to do that for you. This strategy lets the user edit arbitrary text and tries to interpret the text in a background process. An example: this approach has successfully been used for the eclipse Java editors. This background process maintains a model at all times; this model, of course, is only up-to-date in the moments the textual representation is valid. It is possible to show the edited model in another view, but the model should not be changed from other views.

In the MVC approach it was irrelevant from which view we changes a model element. This was possible because representations and model are in sync all the time. In the background parsing approach, you can have invalid text and a model that only reflects the an earlier version of the text. It is not possible to inflict outside model changes under these conditions. The MVC advantage of representation/model separation is lost.

In return, background parsing uses normal text editors which allow you to use features and plug-ins that are based on such editors. For example, copy/paste, undo/redo. This strategy also allows to use arbitrary white-spaces and text layout. It is also possible to only handle the text as the main model, and use the actual EMF model just as a background representation for the text needed to create semantic editor features like code-completion and error annotations: you can decide if you want to store models as text or models.

The following video shows an example TEF editor using the background parsing strategy. It is an editor for OCL and allows to create constraints for EMF models.

The Final TEF Strategy is Background Parsing

We choose background parsing to be the actual TEF strategy. The limitations that model view controller implies to both the possible notations and the users editing capabilities outweigh the advantages of having multiple views on the same model elements. Background parsing, the strategy used for example in the Java editors, allows editors similar to those that users are used to.

TEF in Combination with Graphical Notations

TEF allows to define TEF editors as embedded editors. This means, textual TEF editors can be embedded into graphical or tree-based host editors. You select an element in the host editor, e.g. a generated GMF editor, and a TEF editor for the select element pops up in an small overlay window. You can change the model element with the text editor. When you close the text editor, the changes are transmitted to the host editor. This video briefly demonstrates how embedded TEF editors look like:

Click the image, or simply download the video as Xvid-encoded AVI.

Examples

A Small Language for Mathematical Expressions

This is our standard toy example, used to exemplify all the TEF features. For this language, editors can be easily created from TEF provided plug-in extension templates. You can can choose from a text, model, or embedded editor.

A Textual Editor for Ecore

This editor uses a Java like notation to represent Ecore models. This editor is especially useful when used as an embedded editor. The editor can be used from within the standard EMF tree-based Ecore editor and the GMF Ecore example editor.

An Comprehensive Editor for OCL

As an extensive example and to reason that TEF can work for a real world language with specific meta-model and textual syntax, we are developing an editor for the Object Constraint Language. Besides having a good example for TEF, we think that the OCL editor is a worthwhile contribution to the modelling community, which is, for unknown reasons, missing a comprehensive OCL editor. The video in explaining the background parsing strategy, also shows this editor in action.

Features and Architecture

Basics

TEF is based on the eclipse platform. Each TEF editor is an eclipse plug-in. TEF is designed to be used with the Eclipse Modelling Framework (EMF), but it accesses EMF models through an abstract interface. It is easy to provide an adaptor for your own modelling technology.

The TEF project is still in an experimental phase: we try new features every day. But there is a core set of features emerging that will definitely be part of TEF:

  • TEF is based on a syntax defintion language called TSL. This language provides you with the concepts needed to create a textual notation based on a meta-model. This language is based on (E)BNF-grammars. The rules and symbols in an TSL description can be related to according meta-model elements.
  • Syntax highlighting. You decide which element is displayed in which colour, font, or style. We are distinguishing between syntactic highlighting and semantic highlighting. Syntactic highlighting is based on an elements position or role within a template. For example to put keywords in a specific colour is syntactic highlighting. Semantic highlighting is when items which are equal by there syntactical role or position are highlighted differently. In eclipse' JDT variables are highlighted differently when they are local or member variables.
  • Context sensitive code assists. Also referred to as "code completion". TEF automatically provides simply code completions for keywords and references. You can also alter the automatic behaviour based on hand-written Java code.
  • Occurrences and navigation. The same model element often occurs at different places in a textual model representation. TEF editors mark these occurrences and (will soon) allows navigation between them.
  • Context information. These are little hovers that provide additional information for a model element.
  • Error annotations. You can add constraints to your model and TEF will generate error annotations where these constraints are violated. This is based on the EMF validation framework and function fully automatic.

Related Work

Textual notations for models are rooted in programming languages, which are defined based on a textual syntax and have no other notation. There are a series of projects which try to fill the gap between grammar-ware and model-ware.

Many workbenches for domain specific languages support textual notations. These are for examples Mosaic from Xactium, kermeta, TCS, MontiCore, Eclipse IMP, or Open Architecture Wares XText. All these products are based on a text-parser-model paradigm and basically implement the background parsing strategy. The Meta Programming System implements the model view controller strategy. Its a system that allows to create notations that feel like tabulars or forms.

TEF-editors work more like a graphical editor. The textual model representation is separated into smaller sub-views for each model element. Changes to a view result in immediate model changes, the view only reacts to these model changes. This model view controller pattern is realised in graphical editing frameworks, like eclipse' Graphical Editor Framework (GEF). The next step towards an even simpler editor construction is to replace editor programming, with a model based editor description. This is done in eclipse' Graphical Modelling Framework.

Plan

TEF is stable. Ok, TEF is an academic project, so lets say its relativ stable. You can download the current sources at berliOS. The first major release is now also availble as an Eclipse update-site: http://tef.berlios.de/updatesite