Xtext's new Generator: migration

Xtext 2.9 ships with a new generator architecture, which is described in the previous post. Even though there is no immediate urge to migrate an existing language to the new generator infrastructure, here is how to do it. Make sure to keep a copy of your old code in order to safely roll back in case something goes wrong. If you customized a lot or if you want additional functionality such as IDEA, Web or Maven support, it may be easier to create new plug-ins using the New Xtext Project wizard and then copy existing files over.

Prepare the ide plug-in

Core UI functionality that could be used for Eclipse, IDEA and Web editors, is extracted to a new plug-in <myLang>.ide. As it is usually created by the New Xtext Project wizard, you have to create it manually when migrating.

1. Create the plug-in using File > New > Project… > Plug-in Project. Make sure it is physically located in the same directory as the other plug-in projects of your language (e.g. eGit links directories from different locations into the workspace)
2. Add a new source folder src-gen and don’t forget to add it in the source… section of the build.properties.
3. In the MANIFEST.MF, add a plug-in dependency to
   • <myLang>
   • org.eclipse.xtext.ide
   • org.eclipse.xtext.xbase.ide (if your language uses Xbase)
4. You may want to add the new plug-in to the automatically refresh list in the workflow’s launch configuration: Run as… > Run configurations… > MWE launch > Generate language infrastructure (<myLang>) > Refresh > Specific Resources > Specify Resources…

Optional: Prepare a test.ui plug-in

This is only necessary when you have JUnit plug-in tests. These should be moved into a separate plug-in <myLang>.ui.tests now. Create the plug-in analogously to the ide plug-in above, and add a plug-in dependency to <myLang>.ui.

Convert modules and standalone setup to Xtend

The new generator will by default generate <myLang>RuntimeModule, <myLang>StandaloneSetup and <myLang>UiModule in Xtend. To avoid duplicate classes, convert the existing ones to Xtend by using Convert to Xtend from the context menu of the respective Java files.

Change the workflow

If you have not changed anything in the initially generated workflow, you can replace it by something like this

// adapt name
module <myPlugin>.Generate`<MyLang>`

import org.eclipse.xtext.xtext.generator.*
import org.eclipse.xtext.xtext.generator.model.project.*

var rootPath = ".."

Workflow {
  component = XtextGenerator {
    configuration = {
      project = StandardProjectConfig {
        // replace with your base plug-in's name 
        baseName = "<myPlugin>" 
        rootPath = rootPath
        runtimeTest = {
          enabled = true
        }
        genericIde = {
          enabled = true
        }
        eclipsePlugin = {
          enabled = true
        }
        // only needed if you have plug-in tests
        eclipsePluginTest = {
          enabled = true
        }
        createEclipseMetaData = true
      }
      code = {
        encoding = "UTF-8"
        fileHeader = "/*n * generated by Xtext ${version}n */"
        // preferXtendStubs = false
      }
    }
    language = StandardLanguage {
      // replace with your language’s qualified name
      name = "`<myLang>`"
      // needed when you import an Ecore model in the grammar
      referencedResource = "platform:/resource/path/to/genmodel"
      // needed when you import an Xcore model in the grammar
      referencedResource = "platform:/resource/path/to/Xcore/model"
      // replace file extensions here
      fileExtensions = "myLang"

      serializer = {
        generateStub = false
      }
    }
  }
}

1. Note that we no longer use auto-inject, so if you want to generate only Java stubs, you have to do that in the code section.
2. Use code completion for a list of available fragments and their properties if you want to customize them.
3. If your grammar uses imported Ecore models, load the genmodels as referencedResource in the StandardLanguage section.
4. If your grammar uses imported Xcore models, load them as referencedResource in the StandardLanguage section.

Run the workflow and fix remaining compile errors

The MWE2 workflow should run without errors. After that some plug-ins may have compile errors.

<myLang>.ui

• In the MANIFEST.MF, add <myLang>.ide as a dependency and remove the Antlr contentassist packages from package exports.
• To better serve the use case of multiple languages per plug-in, the Activator is now named the same as the plug-in. This may yield a different case if your languages name uses camel case, e.g. the former MyCamelCaseLanguageActivator will now become MycamelcaseActivator. If that is the case, the manifest shows a warning that you have to fix manually. Some repositories have problems when filenames are changed to a different case, e.g. in Git you have to delete the file, stage the deletion, restore the file from history and stage the new one to make it work.

<myLang>.tests:

• The runtime injection provider has moved to <myLang>.tests. Fix compile errors with Organize Imports and remove the stale package import from the manifest.
• You can remove the dependency to the <myLang>.ui plug-in.

<myLang>.ui.tests:

• The UI injection provider has moved to the <myLang>.ui.tests package.

Adapt plugin.xml files

Compare the plugin.xml and plugin.xml_gen in the runtime and the UI project using the compare editor. We have harmonized the whitespaces in generated files, so the button Ignore Whitespace may help you track down the real differences you have to merge manually.

Use new APIs

1. (Semantic) highlighting has moved to the <myLang>.ide plug-in. If you see deprecation warnings in classes you customized, just redirect the deprecated imports from org.eclipse.xtext.ui.editor.syntaxcoloring.X to org.eclipse.xtext.ide.editor.syntaxcoloring.XXX. For the binding in the module, you might have to override bindIdeISemanticHighlightingCalculator instead.
2. Same holds for bracket matching and parser based content assist.

That is it, I hope you were successful. For further assist, ask us in the Xtext forum.