[WIP] Scriban-based code generation

[jods4]

A long time ago I floated the idea of using text templates to generate code instead of CodeDom.

I finally got to do a prototype. This is not ready to merge but here's what already committed:

1. I included Scriban library in the project and quickly hacked it into the code to generate the files instead of CodeDom.
2. I created a Scriban template to generate proof-of-concept C#. More about what it supports below.
3. The model that drives the template is still the same CodeDom, I haven't changed the C# side of things yet. This is a temporary hack of course. The end goal is to replace that code by a clean POCO "data" model of the schema to generate and getting rid of CodeDom completely.
4. I have created a few temporary helpers methods in ScribanGlobals. Those methods are used inside the template to "extract" from CodeDom information about the model being generated.

Please bear in mind the template is a proof of concept but it's not final. It could be better split in smaller parts, the model could have better names, we can add some comments, etc.

It's already somewhat capable, though. I have experimented with schema Microsoft.Search.Query.xsd which is reasonably large and makes use of:

• Elements and attributes, both optional and required
• Elements with simple content
• Default values
• Enums
• Restrictions on atom types such as string

If you run the code on this branch, the generated code is exactly the same as the current generator if you ignore whitespace differences. I was not masochist enough to match whitespace 1:1. Generating exactly the same code helps ensure there is no regression.

Tip

There is a VS Code extension to highlight the Scriban syntax

What would be the next steps?

I was surprised how quickly I was able to put this together, which speaks in favor of templates.
I am also extremely tempted to modify the templates to improve generated code, something I never felt empowered to do in CodeDom. But I won't do that before a 1:1 match of every generated file is possible and merged. Improvements are planned but will come afterwards.

The big amount of work right now is to (1) remove CodeDom and generate a data-only equivalent model; (2) support all features, this prototype is far from complete.

Things that could be possible in the future but I don't envision in this PR:

• Having multiple sets of templates, maybe to generate code in other languages like VB or F#... or even non-.NET languages!
• Support bring-your-own template where users could fix/extend/modify templates. I mean... they can easily do that locally but I don't plan on having a system to load a modified template stored elsewhere.

@mamift That's a drastic change but I believe it's for the better. What do you think? Should I continue working on this?

[mamift]

I am open to the general idea of what you're trying to achieve.

I find the existing code for code gen in LinqToXsdCore is on the verge of becoming literally unmaintainable, so a clean break would be good.

I encourage you to continue work on this.

[jods4]

@mamift ok I'll pick it up.
I was waiting until the large work on .net 10 PR was complete.
Now that it's merged, are you planning on making other large changes to the code base?
I'd like to know because once I'm starting to remove the C# CodeDom code and introduce a plain object mode,l it'll create massive changes that will be a headache to merge if there are other large PR going on at the same time.

[jods4]

Forced push a rebase on current master (.net sdk upgrade from February)

[jods4]

@mamift My strategy for validating this PR is still to generate all schemas identically (besides whitespace changes) but I decided to take a minor change under certain settings, for the sake of simplicity.

When rendering with public visibility (default) then Type manager looks like this:

public class TypeManager
{
  public static Type GetRootType { .. }
  public static TypeManager Instance { .. }
}

I match this 1:1.

When rendering with internal visibility I render this:

internal class TypeManager
{
  public static Type GetRootType { .. }
  public static TypeManager Instance { .. }
}

CodeDom implementation actually generates internal type for those two properties (and not any other), but it makes no difference in practice. To not add needless templates complexity I always generate public properties (this is certainly going to come back many times in other classes.

I think that's fine because the generation can still easily be verified with the default public setting.

BTW I did some progress on this, the TypeManager is now fully templated and CodeDom for it was removed.
It's painstaking to go through all the CodeDom stuff to ensure everything is ported to the template and I can't express how good it felt when I could remove the 200 lines function that created TypeManager before...
For comparison the scriban template is half as long, 100 lines, which just look like C# code that is easy to modify.

Obviously there's still a long way to go, but I'm slowly working my way through this.