Dedicated docs website for Langium AI

We’re happy to announce we now have dedicated documentation for Langium AI at ai.langium.org!

In addition, we have a new major release of langium-ai-tools and a bump of langium-ai, both of which were motivated by findings while building up the documentation website.

If you’re not familiar with Langium AI, you can read our initial post Langium AI: The fusion of DSLs and LLMs. Langium AI is a suite of tools to make it simpler to integrate AI applications with Langium DSLs, particularly for evaluating agentic workflows or harnesses that interact with DSL programs.

Since we first set up Langium AI, we’ve been writing up documentation within the project repo itself. This worked for a time, but we recently pulled a lot of that down and moved everything over to the new documentation website. We had ideas for additional guides and information, but we really wanted to put these into a dedicated location for documentation that was more presentable than the GitHub repo. By having such a site, it serves as the authoritative source for reading about Langium AI. In addition, we added typechecking to verify embedded code listings in the documentation pages themselves.

What was quite interesting is that the process of typechecking code in our documentation revealed some bugs that we had missed for some time. Most of these were related to older features in Langium AI that had been in place since 2024, but didn’t align with some of the newer features. This motivated the v6.0.0 release, which we’ll go into more detail below.

New documentation website

All the existing documentation that we have about langium-ai-tools, langium-ai (the CLI), and skills is now hosted over at ai.langium.org. This covers most of the existing details you could pick up from the README (such as getting started), as well as really diving into each of these in much more detail. We’ve also taken the time to expand on this documentation with new guides to demonstrate usage in practice. Going forward, we’ll be keeping all documentation on the site rather than the repo (with the exception of the READMEs).

Regarding the examples, we’ll also be keeping these up to date on the docs website via twoslash. If you haven’t heard of it, twoslash is inspired by TypeScript’s fourslash testing framework (which we’ve also used in practice).

What’s really neat is that with twoslash we can typecheck our code listings against the actual packages they’re consuming. This means we get documentation that’s in line with the currently published version of Langium AI. As that spec changes in the future, so will the docs by necessity.

This also extends to the lai examples that demonstrate the CLI usage, which are backed by similar tests. Although we check them, we don’t programmatically run over the actual code listings themselves. At least not yet, but it’s likely we’ll get that in there too to cover our bases.

So, overall, the documentation website not only serves as a source of information on Langium AI, but also as a form of integration test on the packages. This helps to put increased confidence in the correctness of the code within the documentation.

New langium-ai-tools release

As noted above, we’ve released a major version of langium-ai-tools v6.0.0 as well. Overall, this release harmonizes parts of Langium AI that were starting to drift a bit API-wise. This was particularly relevant for parts that were introduced early on in 2024, versus what we’ve been adding in 2026. So, naturally, we needed to make a pass to shore things up. Those changes are breaking, which you can read more about in the ‘in more detail’ section below.

The impact is limited to existing consumers of the older API segment from 2024, but the changes are quite minor and easy to account for. Generally, most downstream consumers shouldn’t be affected.

If you want to see more details, you can read over the following section.

In more detail

For those that aren’t familiar, langium-ai-tools exposes a base Evaluator that can be customized to evaluate model output, pretty self-explanatory. The extent of the changes in v6.0.0 that are breaking are limited to this Evaluator implementation. The twoslash checks we mentioned above were also the reason why we found these issues in the first place.

What we found was an API incompatibility between the legacy Evaluator class and the new CLI-motivated approach. As noted above, this was the result of API drift between the logic we had in early 2024, and the latest changes we’ve been putting in place. It was a subtle issue, but this led to typechecking issues once we put examples in place for the Langium AI website.

The main change is related to the return type of evaluations. Generally, these should be EvaluatorResult, which contains EvaluatorResultData. What we found is that some of our older API spec from 2024 still had evaluators returning EvaluatorResultData, which was incompatible. The fix was to ensure all evaluators consistently return EvaluatorResult across the board. The adjustment itself is quite small, but it ensures that the returned value wraps the data in all cases, instead of returning it directly.

Also, Evaluator.evaluate now marks its second argument as an optional one, since not all evaluators need the expected_response field. Previously there were cases where we were forced to pass in an empty string just to satisfy the signature, which didn’t really match up with most evaluator use cases that skip it.

Lastly, there was an extra chart export that was used for demos back in 2024. This was always intended to be internal and part of the examples, but it managed to sneak into the package export. That has since been removed too.

For more details, you can read the v6.0.0 CHANGELOG entry.

Wrapping up

Going forward, the ai.langium.org site will be the official source of documentation and the intended landing page for new and existing users of Langium AI.

It’s also an extra assurance that we’re showing documentation that’s typechecked directly against Langium AI itself. Overall, we hope you find the documentation just as helpful as we do!