Xtext to Langium

Web applications have become firmly established in many areas of software development. This is also true for domain-specific languages, where many companies and hobby developers want to bring their existing DSL-driven applications to the web.

One of the most popular DSL frameworks in the Java world is Xtext. For a long time, this framework has been convincing with its flexibility and the good Eclipse integration. Thus it has secured a firm place in many open source, but also commercial projects.

The successor of Xtext in the web is Langium. Langium is a language engineering tool with built-in support for the Language Server Protocol - LSP. Langium is implemented in TypeScript, runs in NodeJS and has a simple and direct integration with VS Code. The grammar declaration language used in Langium is similar to Xtext but not exactly the same.

This blogpost is intended for those who have decided to port their DSL from Xtext to Langium. Or for those who just want to try out how their Xtext DSL works in a web based editor.

Xtext2Langium Generator Fragment

As already mentioned, a Langium grammar declaration language is similar to Xtext, but has some special features and in some places a slightly different syntax. Of course you can migrate a small Xtext grammar by yourself, but if your DSL is out of the proof-of-concept phase and has gained some weight, the Xtext2Langium MWE2 generator fragment could make the task much easier.

The job of Xtext2Langium MWE2 Generator Fragment is to read an Xtext grammar and then convert it according to the Langium syntax and semantics. If your Xtext DSL uses a non-generated metamodel (Genmodel, Xcore) this will also be translated to Langium, but then as a Langium type definitions file. In version v0.3.0 Xtext2Langium also provides multi language support, but this is still experimental.

Precondition

To use Xtext2Langium you have to get it first. When I wrote this blogpost the latest version was v0.3.0. In the meantime there might be newer versions available but in the examples 0.3.0 is used. The current latest version can be found here.

Tycho

For projects using Eclipse and Tycho a p2 repository is the preferred data source. Xtext2Langium p2 repository is provided by GitHub Pages and can be accessed with the following URL: https://typefox.github.io/xtext2langium/download/updates/v0.3.0/

Maven/Gradle

For projects using maven or gradle, a version can be obtained from the maven central repository.

Maven:

<dependency>
    <groupId>io.typefox.xtext2langium</groupId>
    <artifactId>io.typefox.xtext2langium</artifactId>
    <version>0.3.0</version>
</dependency>

Gradle:

dependencies {
	implementation "io.typefox.xtext2langium:io.typefox.xtext2langium:0.3.0"
}

How to use

Once Xtext2Langium is on the development Java classpath, you can include the generator fragment accordingly in the mwe2 workflow. To do so, add a new Xtext2Langium fragment to the language configuration.

language = StandardLanguage {
    name = "io.typefox.Example"
    // ... //

    fragment = io.typefox.xtext2langium.Xtext2LangiumFragment {
        outputPath = './langium'
    }
}

The outputPath option is mandatory and specifies where the converted Langium files will be stored.

If you start the workflow now, the fragment will generate one or more files into the output directory. These can then be opened with a Langium editor in VS Code and edited if necessary. For a quick test, if you have only one Langium file in the output, you can also use the Langium Playground.

Configuration options

Besides outputPath there are other options that can be used to control the output of the fragment.

prefixEnumLiterals:

Langium currently does not offer the possibility to create an enum rule, so the enum rules are derived automatically. Since it can happen that an enum literal can occur several times, the names of the enum literals are qualified with the name of the enum rule.

Xtext:

enum Color: UNDEFINED | RED | GREEN;

Langium:

type Color = 'UNDEFINED' | 'RED' | 'GREEN';
Color returns Color:
	Color_UNDEFINED | Color_RED | Color_GREEN
;
Color_UNDEFINED returns string: 'UNDEFINED';
Color_RED returns string: 'RED';
Color_GREEN returns string: 'GREEN';

With the option prefixEnumLiterals = false you can disable the qualification of the enum literals, so that the output changes as follows:

type Color = 'UNDEFINED' | 'RED' | 'GREEN';
Color returns Color:
    UNDEFINED | RED | GREEN
;
UNDEFINED returns string: 'UNDEFINED';
RED returns string: 'RED';
GREEN returns string: 'GREEN';

useStringAsEnumRuleType:

As you can see from the example above, an enum type is also generated for enum rules. If you want to use the string type instead of the enum type you can use the option useStringAsEnumRuleType = true. The output then looks like this:

Color returns string:
    Color_UNDEFINED | Color_RED | Color_GREEN
;
Color_UNDEFINED returns string: 'UNDEFINED';
Color_RED returns string: 'RED';
Color_GREEN returns string: 'GREEN';

In Xtext data types from the Ecore metamodel are often used, the most common ones like String, Integer, Boolean and Date can be directly converted to primitive Langium data types. Here is an example:

Xtext:

terminal ID returns ecore::EString:
	'^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;
terminal INT returns ecore::EInt:
	('0'..'9')+;

Langium:

terminal ID returns string:
    '^'? ('a'..'z' | 'A'..'Z' | '_')('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;
terminal INT returns number: '0'..'9' +;

In Ecore there are many other data types like EBigDecimal, EEChar, EELIst. If these types are used in your language and you don’t want to lose the information about them you can force the generation of the Ecore types file with the option generateEcoreTypes = true. The output would then look like this:

import 'Ecore-types'

terminal ID returns EString:
    '^'? ('a'..'z' | 'A'..'Z' | '_')('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;
terminal INT returns EInt:'0' ..'9' +;

And the corresponding Ecore-types file:

type EString = string;
type EInt = number;

What is next?

Of course the grammar alone does not make a DSL, existing components like linking, scoping, validation, generator and others have to be migrated too. If you need help with this please contact the nice Langium Community or our Langium Support.

I hope the conversion of your Xtext grammar worked well. If not, you can open an issue in GitHub and I will take care of it.