Xtext's new Generator

Xtext 2.9 adds support for two additional editor platforms: Web-editors and IDEA. It also adds generic build system integration for Maven and for Gradle. As a result the number of generator options has grown a lot. So we took the opportunity to re-implement Xtext’s code generator – the one that creates the language infrastructure from the grammar. The new generator no longer relies on Xpand, uses dependency injection to wire up the components and is a lot easier to configure for the user. It is still run by means of an MWE2 workflow. Don’t worry: The old generator is still there so you don’t have to migrate immediately. But new languages will automatically make use of the new generator.

The New Xtext Project Wizard

To understand the new code generator let us first have a look at the new New Xtext Project Wizard.

The first three facets stand for the three supported editor front-ends. The new Generic IDE Support is needed by all of them. It encapsulates platform-neutral editor functionality such as syntax coloring and parser based code completion. Note that you can skip the generation of an editor entirely by not checking any of the first four facets. In addition to Testing Support, you can choose a preferred build system. If either Maven or Gradle is checked, the wizard will generate all the projects within a parent project to comply to the standard hierarchical file layout of a Maven/Gradle application. Selecting Maven/Gradle as Source Layout will create the common Maven file structure with source folders as src/main/java, src/test/java etc. Note that not all combinations make sense, and you will be warned or even corrected by the wizard if your config is irregular.

The project layout

The resulting project layout for the above configuration will look like this:

• mydsl: The base runtime infrastructure of the language, like the parser, validation etc. Always generated.
• mydsl.ide: The common editor functionality for all front-ends. Generated when Generic IDE Support is checked.
• mydsl.idea: The Intellij IDEA plug-in.
• mydsl.parent: The parent project containing all others. Generated if a Preferred Build System is chosen.
• mydsl.target: Contains the target platform definition, i.e. the definition of the Eclipse plug-ins and features the code is compiled against. Needed when using Maven or Gradle as build system and creating an Eclipse editor.
• mydsl.tests: The tests that only depend on the base infrastructure, i.e. not on any editor, should go here as plain JUnit tests. Generated when Testing Support is enabled.
• mydsl.ui: The Eclipse editor plug-in, generated by the Eclipse plug-in facet.
• mydsl.ui.tests: JUnit plug-in tests for the Eclipse editor plug-in go in here. Generated if Eclipse plug-in and Testing Support are checked.

The workflow

The MWE2 workflow generated from the above wizard settings is shown below. The new generator workflow uses a single new component of type XtextGenerator. If you are missing the former DirectoryCleaner and StandaloneSetup: They are now fields of the generator with good defaults. The generator is configured with a DefaultGeneratorModule. It is a real Guice module, so if you need additional services in custom code generator fragments, you can use Xtext’s module API to bind them in a subclass and use that one instead. All other components can just @Inject any service. The DefaultGeneratorModule also binds an IXtextProjectConfig that defines what kind of Eclipse projects should be created and how they are named, and a CodeConfig holding common code generation parameters (see the source code for substituted variables in the file header). The XtextGenerator also defines any number of IXtextGeneratorLanguages, each with a language name and a set of file extensions. The StandardLanguage defines a common set of generator fragments as fields, which can be customized individually as in the example. If you want to skip/add fragments, the easiest way is to copy StandardLanguage and adapt it.

module org.xtext.example.mydsl.GenerateMyDsl

import ....

Workflow {
  component = XtextGenerator {
    configuration = /* DefaultGeneratorModule */{
      project = StandardProjectConfig {
        baseName = "org.xtext.example.mydsl"
        rootPath = ".."
        runtimeTest = {
          enabled = true
        }
        eclipsePlugin = {
          enabled = true
        }
        eclipsePluginTest = {
          enabled = true
        }
        ideaPlugin = {
          enabled = true
        }
        web = {
          enabled = true
        }
        createEclipseMetaData = true
      }
      code = /* CodeConfig */ {
        encoding = "UTF-8"
        fileHeader = "/*n * generated by Xtext ${version}n */"
      }
    }
    language = StandardLanguage {
      name = "org.xtext.example.mydsl.MyDsl"
      fileExtensions = "mydsl"

      serializer = {
        generateStub = false
      }
      validator = {
        // composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
      }
    }
  }
}