Extending a language server with sprotty diagrams

In my last post, I described how we connected an Xtext-based language server for the YANG language into various editor clients. Today, I will show how we added graphical diagrams using the Sprotty framework. These diagrams show a graphical overview on your YANG documents. They are automatically layouted and updated on changes. They allow to navigate between diagram and text, to expand and collapse referenced modules, to export them as an SVG file and more.

Sprotty client-server communication

Sprotty is a new graphics framework written in TypeScript. In addition to a modern look and feel, built-in animations, a fast virtual DOM architecture, and high-customizability by use of dependency injection, one of its key features is a lean, JSON-based protocol. This protocol allows to separate a server holding the data to be visualized from a client that renders a diagram model and that allows the user to interact with the diagram. The communication messages in Sprotty are called Actions. An Action is a behaviour-less data structure that usually describes a coarse grained semantic operation, like setting a new diagram model or applying the diagram layout. Actions are usually triggered by the user, by the client or by the server. On the client, they are usually translated into Commands which modify the diagrams state and trigger rendering updates. Actions can be sent in both directions.

On the client, the ActionDispatcher decides which Actions are executed locally and which ones are sent to and processed by the server. This allows to delegate expensive calculations to the server. E.g. in the YANG tools, we let the server perform the auto-layout calculation using the Eclipse Layout Kernel. The ActionDispatcher also acts as a gateway to the client’s event cycle for Actions coming in from the server.

Extending the LSP

As the Language Server Protocol (LSP) has the same separation of client and server, the idea is to extend it such that it supports tunneling Sprotty’s Actions as notifications. LSP is based on JSON-RPC, so we have to extend the interfaces for the remote procedure calls. This is what the interface DiagramEndpoint is doing on the server side: It uses annotations from lsp4j to add a JSON-RPC method diagram/accept that takes an ActionMessage as parameter to both endpoints of the communication.

@JsonSegment('diagram')
public interface DiagramEndpoint extends Consumer {
   @JsonNotification
   void accept(ActionMessage actionMessage);
}

In LSP you usually define a separate method for each functionality. For Sprotty, we use the more generic approach with a single method accept taking a polymorphic ActionMessage parameter. This allows users to add any kind of custom actions they want without having to fiddle with the details of JSON-RPC. Xtext provides the interface ILanguageServerExtension to define language specific extensions to the LSP. By creating and binding a YangLanguageServerExtension that implements both ILanguageServerExtension and DiagramEndpoint, the YANG language server is capable of receiving Actions from a Sprotty diagram client. By adding the same interface to the client endpoint, the server can also send Actions to the client.

public class YangLanguageServerExtension 
    implements DiagramEndpoint, ILanguageServerExtension {
...
}

At least for us, adding diagrams to Xtext-based language servers is going to be a common task, so we’ve added a lot of infrastructure around this. The code is mostly contained in the project xtext-diagram within Sprotty. It also contains the mechanics needed for updating diagram models when documents change, synchronizing selection, tracing etc. For YANG, we have implemented the IDiagramGenerator interface. This is called within the Xtext language server for all changed files. Its task is to transform the EMF model of the modified document to a Sprotty diagram model (SModel) that describes the associated diagram. As part of the YANG language server, this generator is reused by all our YANG tools.

Sprotty diagrams in Theia

The npm package theia-sprotty contains the glue code to have – you guessed it – Sprotty diagrams in the Theia IDE. The code is the same for Theia’s browser and Electron mode. It defines things like

• a diagram widget that is backed by a Sprotty diagram server,
• communication with a Sprotty-extended language server backend,
• alignment of the different DI containers (one container per app in Theia, one per diagram instance in Sprotty), and
• synchonization of editor and diagram selection.

The code is generic and can be reused directly as an npm dependency. As a Theia diagram implementor, you have to make the Theia extension of your language diagram-aware. This involves

• implementing a DiagramManager, a DiagramConfiguration, and a Sprotty DI configuration for each diagram type,
• adding DI bindings for these in the frontend module,
• create a Sprotty DI module for each diagram type, and
• implement the Sprotty diagram client (diagram model, model factory, views, commands etc)

Have a look at the package theia-yang-extension in the project yangster to see how we did that for YANG. The last point has been extracted to a separate repo yang-sprotty to be reusable in the Eclipse IDE.

Sprotty diagrams in Eclipse

For Eclipse, we use SWT’s browser widget to render Sprotty diagrams. In yangster-eclipse we spawn a Jetty server to host JavaScript and CSS files for the client and to parse request parameters, e.g. for the YANG file or for the color theme in use. Each Eclipse diagram view opens a web socket to this Jetty server, which is used to relay Sprotty Actions between the YANG language server and the diagram. Some of these Actions are intercepted, e.g. to open Eclipse’s file selection dialog when exporting a diagram as an SVG image. You will find the corresponding code in the plug-in io.typefox.yang.eclipse.diagram of the yangster-eclipse feature.