Oh, the macro is a good idea! In Conjure I use quasi-quoted Essence fragments both as patterns and as values: [search for essence| in the repo

Not sure if we can go that far in Rust, or with how much effort, though I must say it is a nice feature.

Not quite a macro yet, but I've got a function to parse arbitrary Essence expressions working:

let src = "x >= 5, y = a / 2";
let exprs = parse_expressions(src).unwrap();
assert_eq!(exprs.len(), 2);
assert_eq!(
    exprs[0],
    Expression::Geq(
        Metadata::new(),
        Box::new(Expression::Atomic(Metadata::new(), Atom::new_uref("x"))),
        Box::new(Expression::Atomic(Metadata::new(), 5.into()))
    )
);
assert_eq!(
    exprs[1],
    Expression::Eq(
        Metadata::new(),
        Box::new(Expression::Atomic(Metadata::new(), Atom::new_uref("y"))),
        Box::new(Expression::UnsafeDiv(
            Metadata::new(),
            Box::new(Expression::Atomic(Metadata::new(), Atom::new_uref("a"))),
            Box::new(Expression::Atomic(Metadata::new(), 2.into()))
        ))
    )
);

The next step is making it a macro with some basic arguments (similar to the {} in print!) and doing the same for a full model, and for lettings etc

Oh, the macro is a good idea! In Conjure I use quasi-quoted Essence fragments both as patterns and as values: [search for essence| in the repo

Not sure if we can go that far in Rust, or with how much effort, though I must say it is a nice feature.

I'll have a look!

At the very least, "plugging" any literals into our essence code should not be a problem.
In a declarative macro, we can capture and parse arbitrary token trees:

($query:expr, $($args:tt)*) => { ... }

And if we're not doing compile time type checking, it should be enough to just coerce the value into a string.
(Nik said that Expression displays into valid Essence now, so we should even be able to pass arbitrary expressions as arguments to the macro as well as literals)

The parser will catch syntax errors anyway, and type checking is a whole separate thing

I can show you how I do this in Haskell, the values do not need to go via string there. In case you want to read some extremely beautiful code in the meantime, have a look at this module: https://github.com/conjure-cp/conjure/blob/e1bba435a3b8217b370e1fb3fd418b5831f07806/src/Conjure/Language/TH.hs

Question: should we not call the crate tree-sitter-essence instead of conjure_essence_parser? tree-sitter-LANGUAGENAME seems to be the convention.

I've kept the tree-sitter-essence crate as just the grammar + tree-sitter bindings, and the new crate only has the "client" code for that (i.e. actually converting the treesitter AST into conjure types)

I can merge them if that makes more sense!

I've kept the tree-sitter-essence crate as just the grammar + tree-sitter bindings, and the new crate only has the "client" code for that (i.e. actually converting the treesitter AST into conjure types)

I think we need a diagram of the crates and their dependencies in the wiki at this point.

I can show you how I do this in Haskell, the values do not need to go via string there.

It looks like a combination of call_site and the syn crate might do this. Something like let ident = syn::Ident::new(var_name, Span::call_site()); should work for an expression from my understanding.

Is this ready to merge as far as you are concerned @gskorokhod? Can you mark it as not-draft if/when it's ready please?

Closes: #562

We have a proof of concept going!🚀

Metavars not supported yet but will be asap
The hardest bit (dealing with ToTokens) seems to be working reasonably well!

commit hygiene on this PR has deteriorated along with my sanity so we should probably squash :)

when you search for "define: scope creep" in google this PR will come up

well done, I'll review asap :)

you should present to the group next week!

undo the moving of the parser to its own crate so we have a somewhat clean diff for review? don't worry about rewriting history, we can squash.

undo the moving of the parser to its own crate so we have a somewhat clean diff for review? don't worry about rewriting history, we can squash.

Sure! I'll try and split this into separate PRs by tomorrow so we can merge the parser refactor and the macros separately

I was thinking about this in the shower just now and i realised i was wrong yet again...

What I'm trying to do is:

string -> treesitter AST -> Expression instance -> Generated Rust code

That last bit being the hardest in Rust due to the need to implement / derive ToTokens

But the treesitter parser already has a step of "go through the treesitter nodes recursively and build up Expreession variants".

There's no reason why, instead of actually constructing them, it couldn't just quote! the constructor call it would make and append that to a TokenStream

This way, we can go:

string -> treesitter AST -> Generated Rust code

Directly, eliminating all the brittleness and complexity of having to implement ToTokens

The final expansion of the macro would be the same, just with fewer extra steps!

The only tradeoff would be not being able to do any "smart" transformations on the Expressions we generate (metavars are still fine, as we just replace them with an Ident without having to go up or down) but i think it's worth it

It does mean that about half of this 2000LOC PR can just go in the bin :)

OK, maybe don't work on this PR so we can compare the two approaches?

Implemented the alternative approach in #785

merged #785 instead