From eb65769fa1b5cea2044a3dee340ec70928a235fb Mon Sep 17 00:00:00 2001 From: Yutaka Nishimura Date: Sun, 25 Oct 2026 19:27:31 +0900 Subject: [PATCH] feat: generate a typed async client for a Teal module from `[rust] exports` (#538) `htl.toml` `[rust] exports = [""]` makes `htl rs` and `teal_types!()` write, for each listed module, a typed async client over `htl::run`. This is the second half of #538; the first half, `htl::run`, is the runner this client sits on. Both entry points build their spec the same way, `Spec::from(&RustConfig)`, and call the same `rsgen::generate`, so the CLI and the macro cannot disagree about what a module exports. Shape of the client: - The struct is named after the record the module returns: `app` gives `App`, and `M` stays `M`. - `load(runner, extra_paths)` requires the module on a runner and `instance()` gives the loaded instance back. Each declared function becomes one async method. - A method is made for each function-typed field declared in the record body. Parameters and results go through the same type table as the data types. Two or more results come back as a tuple. - A parameter is `T`, and `Option` only when the Teal type is `T | nil`. The type index keeps a count of required parameters, not which ones, so that count is not used: an optional `?` parameter is required in Rust. Per-parameter marks belong with #557. - Arguments are named `a1, a2, ...` until #557 gives them names. - The generated struct's doc says it needs htl's `async` feature. What `generate` refuses, as a problem reported at the Teal line of the field or the module: - a method, a generic function, a variadic parameter or result, an overload - `any`, and any record that reaches `any` transitively, computed from the lowered records - more than 16 parameters or results, which is mlua's tuple limit - the names `load`, `instance`, `clone` and `clone_from`, which the client's own items use - a name that is no Rust name, and two names that clash - a module that returns no record Two cases are notes and not failures: a record that declares no function, and a function defined outside the record body (`function M.f`), for which the index keeps no type. The rules are in the Exports section of the `rsgen` module doc. The client lowering lives in `rsgen/client.rs`, so `rsgen.rs` only gains the part that decides which modules to lower and where the problems go. `config.rs` gains the `exports` key. The README gains a section on the client. Tests: - `rsgen` unit tests: a golden for the client, one case per problem above, the order and duplicates of `exports`, and the regressions found in review. - A shared fixture and golden under `crates/htl-core/tests/fixtures/client`, compared byte for byte by `htl-macros` (the expansion) and by `htl-cli` (`htl rs`), so the two entry points are held to one output. - `crates/htl/tests/run_client.rs`: the golden is compiled and called on a real runner. - `e2e/tests/teal_exports_client.rs`: a correct call builds and runs; a call with a wrong argument type makes `cargo check --message-format=json` report error code E0308. Refs #538 --- README.md | 47 +- crates/htl-cli/src/lib.rs | 24 +- crates/htl-cli/tests/rs_cli.rs | 52 +- crates/htl-core/src/config.rs | 30 +- crates/htl-core/src/rsgen.rs | 900 ++++++++++++++++-- crates/htl-core/src/rsgen/client.rs | 371 ++++++++ .../htl-core/tests/fixtures/client/htl.toml | 3 + .../tests/fixtures/client/src/shop/app.tl | 17 + .../tests/fixtures/client/src/shop/types.tl | 7 + .../tests/fixtures/client/teal_types.rs | 55 ++ crates/htl-macros/src/lib.rs | 61 +- crates/htl/tests/run_client.rs | 61 ++ e2e/tests/teal_exports_client.rs | 171 ++++ 13 files changed, 1697 insertions(+), 102 deletions(-) create mode 100644 crates/htl-core/src/rsgen/client.rs create mode 100644 crates/htl-core/tests/fixtures/client/htl.toml create mode 100644 crates/htl-core/tests/fixtures/client/src/shop/app.tl create mode 100644 crates/htl-core/tests/fixtures/client/src/shop/types.tl create mode 100644 crates/htl-core/tests/fixtures/client/teal_types.rs create mode 100644 crates/htl/tests/run_client.rs create mode 100644 e2e/tests/teal_exports_client.rs diff --git a/README.md b/README.md index 8432e64..f5865f7 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ Each command's arguments, flags and examples are in `htl --help` (and | `htl cache` | report what the `.htl/cache` store holds, or empty it (`status`, `clear`; see Caching) | | `htl dts` | write the `.d.tl` files this project declares from Rust (`#[host_module]`, `#[derive(TealRecord)]`), Teal (`---@contract`) and crate dependencies ([`htl::dep_dts`](https://docs.rs/htl/latest/htl/dep_dts/index.html)) | | `htl shapes` | write an `lshape` run-time schema and typed `Schema` `.d.tl` for each record and enum the project declares ([`htl::shapes`](https://docs.rs/htl/latest/htl/shapes/index.html)) | -| `htl rs` | write one Rust file deriving `TealRecord` for each record and enum the modules in `[rust] modules` declare ([`htl::rsgen`](https://docs.rs/htl/latest/htl/rsgen/index.html)) | +| `htl rs` | write one Rust file: a `TealRecord` type for each record and enum of the modules in `[rust] modules`, a client for each module in `[rust] exports` ([`htl::rsgen`](https://docs.rs/htl/latest/htl/rsgen/index.html)) | | `htl lsp` | run the language server on stdio, whose diagnostics are `htl check`'s (see Editors) | Exit code 1 is a verdict (an error in a file, a finding at `deny`, a failing test, a name @@ -221,6 +221,51 @@ and the pages behind it are nothing is embedded and `require` goes through mlua-pkg at run time. Its README says what each one prints and which line of the output is the point. +### Calling Teal from async Rust (`htl::run`, `[rust] exports`) + +A host whose logic is in Teal starts one runner and calls the functions of a module on it. +With `htl = { features = ["async"] }`, `[rust] exports = ["app"]` in `htl.toml`, and +`htl::teal_types!();` at the crate root (or `htl rs`, which writes the same code to a file), +the record a module returns becomes a typed client; an argument or a result that disagrees +with the Teal declaration fails `cargo build`: + +```teal +local record app + add: function(a: integer, b: integer): integer -- a function the body declares is a method +end + +function app.add(a: integer, b: integer): integer + return a + b +end + +return app +``` + +```rust +htl::teal_types!(); // at the crate root: reads `[rust]` of htl.toml + +async fn run() -> anyhow::Result<()> { + let (runner, driver) = htl::run::start(htl::run::RunnerConfig { + search_paths: vec!["src".into()], + deadline: std::time::Duration::from_secs(5), + queue: 64, + }) + .await?; + let app = app::App::load(&runner, &[]).await?; // `require("app")`, in a state of its own + let n: i64 = app.add(2, 3).await?; // Result + driver.shutdown().await?; + Ok(()) +} +``` + +Every call runs under the runner's `deadline` and is cut off past it. Each `load` is an +instance with its own program state, so two instances of one module share nothing of the +module's own. A client is named for the record its module returns, so the clients of +`local record M` modules are all `M`, told apart by their module path (`shop::app::M`). +What a runner promises is [`htl::run`](https://docs.rs/htl/latest/htl/run/index.html); +which functions get a method, how each type is written, and what is refused is the +Exports section of [`htl::rsgen`](https://docs.rs/htl/latest/htl/rsgen/index.html). + ## Lints (`htl check`, `include_tl!`) Every rule has a level — `allow`, `warn` or `deny` — and a project sets it by name: diff --git a/crates/htl-cli/src/lib.rs b/crates/htl-cli/src/lib.rs index 6c3fb75..7a4fc31 100644 --- a/crates/htl-cli/src/lib.rs +++ b/crates/htl-cli/src/lib.rs @@ -423,7 +423,7 @@ Examples: dir: Option, }, /// Write a Rust struct for each record and a Rust enum for each enum that the modules - /// listed in `[rust] modules` declare + /// listed in `[rust] modules` declare, and a client for each module in `[rust] exports` /// /// Reads the project's own `.tl` files, as `htl shapes` does, and writes one file, /// `/teal_types.rs` unless `[rust] out` says otherwise: one `pub mod` per module @@ -432,7 +432,9 @@ Examples: /// back. The record is the source: nothing generated is to be edited. A field is /// `Option` unless its record is marked `---@struct` and the field is neither /// `---@optional` nor `T | nil`; `[rust.types] "ev.Event.at" = "u32"` gives one field - /// the Rust type written there. What is lowered, the mapping, and every case that is + /// the Rust type written there. A module listed in `[rust] exports` gets a client for + /// its functions, called on an `htl::run` instance (the host builds `htl` with its + /// `async` feature). What is lowered, the mapping, and every case that is /// refused (a function field, a generic, a union, a tuple, `---@extensible`, a type of /// a module that is not listed, a name Rust cannot have, ...) is the module doc of /// `htl::rsgen`; a run with any such problem names each at `file:line:col` and writes @@ -441,13 +443,14 @@ Examples: /// Reports `htl rs: wrote ` or `htl rs: is current` on stderr; a file /// whose text would not change is not touched. A target that exists and does not begin /// with the line `htl rs` writes is not one of its own and is never overwritten or - /// removed: it exits non-zero, saying so. With `[rust] modules` empty (the default) it - /// writes nothing, and removes a file an earlier run wrote. A type error in any own + /// removed: it exits non-zero, saying so. With `[rust] modules` and `exports` empty (the + /// default) it writes nothing, and removes a file an earlier run wrote. A type error in any own /// file, listed or not, is reported and nothing is written. /// /// `--check` writes nothing: it compares what would be written with the file, and /// exits non-zero with `htl rs: is stale; run htl rs` when they differ, the file - /// is missing, or `[rust] modules` is empty and a file `htl rs` wrote is still there. + /// is missing, or `[rust] modules` and `exports` are empty and a file `htl rs` wrote is + /// still there. /// The file is committed, and this is how CI says it has fallen behind the Teal. Rs { /// Project root or any path inside it (default: current directory) @@ -1623,15 +1626,12 @@ fn cmd_rs(dir: Option<&Path>, check: bool) -> Result { None => source.join("teal_types.rs"), }; let shown = out.strip_prefix(&root).unwrap_or(&out).to_path_buf(); - let spec = htl::rsgen::Spec { - modules: cfg.rust.modules.clone(), - types: cfg.rust.types.clone(), - }; + let spec = htl::rsgen::Spec::from(&cfg.rust); // Nothing is asked for: nothing is checked, and a file an earlier run wrote is no longer // wanted. One that is somebody's is none of this command's business. - if spec.modules.is_empty() { - eprintln!("htl rs: nothing to write ([rust] modules is empty)"); + if spec.is_empty() { + eprintln!("htl rs: nothing to write ([rust] modules and exports are empty)"); if !written_by(&out, Writer::Rs)? { return Ok(ExitCode::SUCCESS); } @@ -1683,7 +1683,7 @@ fn cmd_rs(dir: Option<&Path>, check: bool) -> Result { eprintln!("htl rs: {} not written", shown.display()); return Ok(ExitCode::FAILURE); } - // `modules` was not empty above. + // `modules` and `exports` were not both empty above. htl::rsgen::Rendered::Nothing => return Ok(ExitCode::SUCCESS), }; diff --git a/crates/htl-cli/tests/rs_cli.rs b/crates/htl-cli/tests/rs_cli.rs index 13e339a..468327b 100644 --- a/crates/htl-cli/tests/rs_cli.rs +++ b/crates/htl-cli/tests/rs_cli.rs @@ -103,6 +103,56 @@ fn a_record_and_an_enum_are_written_as_a_struct_and_an_enum() { ); } +/// The project of `htl-core/tests/fixtures/client`: `[rust] exports` next to `modules`, and +/// the file `htl rs` writes for it, which `teal_types!()` expands to as well (the test of +/// `htl-macros`) and which the `htl` crate compiles against `htl::run`. +#[test] +fn an_exported_module_is_written_as_the_client_the_macro_expands_to() { + let root = common::tempdir("htl-cli-rs", "client"); + write( + &root.join("htl.toml"), + include_str!("../../htl-core/tests/fixtures/client/htl.toml"), + ); + write( + &root.join("src/shop/types.tl"), + include_str!("../../htl-core/tests/fixtures/client/src/shop/types.tl"), + ); + write( + &root.join("src/shop/app.tl"), + include_str!("../../htl-core/tests/fixtures/client/src/shop/app.tl"), + ); + let out = htl(&root, &["rs"]); + assert!(out.status.success(), "{}", stderr(&out)); + assert_eq!(stderr(&out), "htl rs: wrote src/teal_types.rs\n"); + assert_eq!( + read(&root.join("src/teal_types.rs")), + include_str!("../../htl-core/tests/fixtures/client/teal_types.rs") + ); +} + +/// `exports` without `modules` is something to write, and a note about a function the +/// client does not have is said without failing the run. +#[test] +fn exports_alone_are_written_and_a_function_outside_the_body_is_a_note() { + let app = "local record app\n add: function(integer, integer): integer\nend\n\ + function app.add(a: integer, b: integer): integer\n return a + b\nend\n\ + function app.helper()\nend\nreturn app\n"; + let root = project_with("exports-alone", "[rust]\nexports = [\"app\"]\n", EV); + write(&root.join("src/app.tl"), app); + let out = htl(&root, &["rs"]); + assert!(out.status.success(), "{}", stderr(&out)); + assert!( + stderr(&out).contains( + "src/app.tl:7:1: `app.helper` is defined outside the body of the record, so it is not in the client; declare `helper: function(..)` in the body to add it" + ), + "{}", + stderr(&out) + ); + let file = read(&root.join("src/teal_types.rs")); + assert!(file.contains("pub struct App {"), "{file}"); + assert!(!file.contains("helper"), "{file}"); +} + /// Acceptance 2: `---@struct` makes a field required unless it is `---@optional`. #[test] fn a_struct_record_requires_the_fields_that_are_not_optional() { @@ -204,7 +254,7 @@ fn an_empty_modules_removes_the_file_it_wrote() { assert!(out.status.success(), "{}", stderr(&out)); assert_eq!( stderr(&out), - "htl rs: nothing to write ([rust] modules is empty)\n\ + "htl rs: nothing to write ([rust] modules and exports are empty)\n\ htl rs: removed src/teal_types.rs\n" ); assert!(!root.join("src/teal_types.rs").exists()); diff --git a/crates/htl-core/src/config.rs b/crates/htl-core/src/config.rs index 0dd8a9e..95a5f19 100644 --- a/crates/htl-core/src/config.rs +++ b/crates/htl-core/src/config.rs @@ -53,8 +53,10 @@ //! async = true # async / await are keywords (local async function f, async local //! # x = e, await x, await f()); off, they are ordinary names //! -//! [rust] # htl rs: Rust types written from Teal records and enums -//! modules = ["ev"] # the modules written; absent or empty, htl rs writes nothing +//! [rust] # htl rs: Rust types from Teal records and enums, clients of modules +//! modules = ["ev"] # the modules whose types are written +//! # exports = ["app"] # the modules whose functions get a client (needs htl's `async`); +//! # no module and no export: htl rs writes nothing //! # out = "src/teal_types.rs" # where; default "/teal_types.rs" //! //! [rust.types] @@ -119,8 +121,8 @@ pub struct HtlConfig { /// `await`, off unless the project says so. #[serde(default)] pub lang: LangConfig, - /// `[rust]` — which modules `htl rs` writes Rust types for, where, and how a field's - /// Rust type is overridden. + /// `[rust]` — which modules `htl rs` writes Rust types and clients for, where, and how + /// a field's Rust type is overridden. #[serde(default)] pub rust: RustConfig, /// Static counterpart of `TealResolver::expect_type` / `require_fields`: files @@ -547,11 +549,13 @@ impl LangConfig { } } -/// `[rust]`: the Rust types `htl rs` writes from this project's Teal records and enums. +/// `[rust]`: the Rust types and clients `htl rs` writes from this project's Teal records, +/// enums and module functions. /// /// ```toml /// [rust] /// modules = ["ev", "geom.types"] # the modules whose records and enums are written +/// exports = ["app"] # the modules whose functions get a client /// out = "src/teal_types.rs" # where; default "/teal_types.rs" /// /// [rust.types] @@ -564,12 +568,20 @@ impl LangConfig { #[serde(deny_unknown_fields)] pub struct RustConfig { /// The modules (`ev`, `geom.types`: the names `require` takes) whose records and enums - /// `htl rs` writes. Absent or empty: `htl rs` writes nothing, and removes the file an - /// earlier run wrote. The opt-in is per module, because a module of shared types is the - /// unit a Rust host imports. A name that is no module of the project's own sources is a - /// problem, not a no-op. + /// `htl rs` writes. Absent or empty, and `exports` too: `htl rs` writes nothing, and + /// removes the file an earlier run wrote. The opt-in is per module, because a module of + /// shared types is the unit a Rust host imports. A name that is no module of the + /// project's own sources is a problem, not a no-op. #[serde(default)] pub modules: Vec, + /// The modules (`app`: the name `require` takes) whose returned record `htl rs` writes + /// as a client: a struct with one async method per function the record's body declares, + /// each a call on an `htl::run::Instance`. What a client is made of, and what refuses + /// one, is the Exports section of the module doc of [`crate::rsgen`]. The generated + /// code names `htl::run`, so the host builds `htl` with its `async` feature. A name that + /// is no module of the project's own sources is a problem, as in `modules`. + #[serde(default)] + pub exports: Vec, /// The file `htl rs` writes, relative to `htl.toml`. Absent: `teal_types.rs` in the /// source directory (`[layout] source`). A file there that `htl rs` did not write is /// never overwritten. Only the command reads this; the lowering does not. diff --git a/crates/htl-core/src/rsgen.rs b/crates/htl-core/src/rsgen.rs index cbff115..0acc6e5 100644 --- a/crates/htl-core/src/rsgen.rs +++ b/crates/htl-core/src/rsgen.rs @@ -14,7 +14,8 @@ //! //! # What is written //! -//! One file. The modules named by `[rust] modules` ([`Spec::modules`]) are merged into one +//! One file. The modules named by `[rust] modules` ([`Spec::modules`]), and the clients of +//! `[rust] exports` (Exports, below), are merged into one //! tree by their path segments (`geom.a` and `geom.b` both sit in one `pub mod geom`), and //! each record becomes a `struct` and each enum a unit-only `enum`, deriving //! `::htl::TealRecord` so that the host converts it to and from a Lua table: @@ -139,6 +140,7 @@ //! | a type with the name of a child module (`geom.types` is a module and `geom` has a type `types`) | one namespace in Rust | //! | `[rust] modules` names a module that is no own source, or one whose check kept no index | the name is wrong or the file did not parse | //! | a `[rust.types]` key that matches no field, one that matches two, a value that is not a Rust type | see below | +//! | the client of an export | see Exports | //! //! A project whose check fails on *any* own file (a listed module or not) is stopped before //! this runs, by the subcommand: an unlisted module with a type error stops `htl rs` too, @@ -149,16 +151,19 @@ //! //! ```toml //! [rust] -//! modules = ["ev", "geom.types"] # the module names `require` takes; empty writes nothing +//! modules = ["ev", "geom.types"] # the module names `require` takes +//! exports = ["app"] # the modules a client is written for (Exports) //! out = "src/teal_types.rs" # the command's: where the file is written //! //! [rust.types] //! "ev.Event.at" = "u32" //! ``` //! -//! The lowering takes [`Spec`], which is `modules` and `types`; `out` is where the caller -//! writes. An empty `modules` is [`Rendered::Nothing`]: the opt-in is per module, because a -//! module of shared types is what a Rust host imports. +//! The lowering takes [`Spec`], which is `modules`, `exports` and `types` (`Spec::from` a +//! [`RustConfig`](crate::config::RustConfig), the one construction `htl rs` and +//! `htl::teal_types!()` share); `out` is where the caller writes. Empty `modules` and +//! `exports` are [`Rendered::Nothing`]: the opt-in is per module, because a module of shared +//! types is what a Rust host imports. //! //! A key of `types` is `..`, where `` is the declaration's path as //! its name is made (`Event`, `Outer.Inner`), dotted. The key is read against every listed @@ -175,6 +180,74 @@ //! side. A field with an entry is not lowered, so its Teal type is free to be one the mapping //! has no row for. //! +//! # Exports +//! +//! The types above carry data; a project that keeps its logic in Teal also has Rust *call* +//! it. `[rust] exports` names the modules whose functions are called from Rust, and each gets +//! a client in the module's own `pub mod` (after its types, when it is listed in `modules` +//! too): a struct over an `htl::run::Instance`, with one async method per function **the +//! record's body declares** (`check: function(string): types.Output`). The struct is named +//! for the record the module returns, in PascalCase: `local record app` is `App`, and +//! `local record M` is `M`, so the clients of the `M`-style modules of a project are all `M` +//! and are told apart by their module path. The generated code names `htl::run`, so the host +//! builds `htl` with its `async` feature; without it rustc says `run` is not in `htl`. +//! +//! ```text +//! pub mod app { +//! /// The functions of the Teal module `app`, called on a runner instance; .. +//! #[derive(Debug, Clone)] +//! pub struct App { +//! inst: ::htl::run::Instance, +//! } +//! +//! impl App { +//! /// `require("app")` in a fresh instance of `runner`. +//! pub async fn load(runner: &::htl::run::Runner, extra_paths: &[::std::path::PathBuf]) +//! -> ::core::result::Result { .. } +//! /// The instance the functions are called on. +//! pub fn instance(&self) -> &::htl::run::Instance { .. } +//! /// `app.check`, typed `function(string): types.Output` in Teal. +//! pub async fn check(&self, a1: ::std::string::String) +//! -> ::core::result::Result { +//! self.inst.call("check", (a1,)).await +//! } +//! } +//! } +//! ``` +//! +//! A method is the function's field name (`r#` before a keyword), called with `.`; its +//! arguments are `a1`, `a2`, ... (the index keeps no parameter names, #557). Each argument +//! and result is typed by the mapping above, with the nilability of a `T | nil` and nothing +//! else: a parameter is `T` unless its Teal type is `T | nil`, then `Option`. An optional +//! parameter (`title?: string`) is `T` too: the index says how many arguments a call has to +//! pass ([`FnSig::required`](crate::typeindex::FnSig::required)) and not which ones, so the +//! host always passes a value. No result is `()`, one is its type, several are a tuple. The +//! methods are in position order. +//! +//! A function the client cannot call is a problem, at the line of the function: +//! +//! | | problem | +//! |---|---| +//! | a method (`function(self: app)`, called with `:`) | a client calls with `.` | +//! | a generic function (`function`) | a type variable has no Rust type | +//! | a variadic argument or result (`...: T`, `: T...`) | no Rust type | +//! | more than 16 arguments, or more than 16 results | they cross as one tuple each way, and mlua converts at most 16 | +//! | an overload (two function types for one field) | no one Rust signature | +//! | an argument or result the mapping has no row for | as for a field | +//! | `any`, also inside a `{T}`, a `{K : V}` or a `T \| nil`; a record that holds `any` in a field, directly or through other records | a Lua value does not leave the VM thread `htl::run` keeps it on | +//! | a function named `load` or `instance`, or one whose name is no Rust name | the client writes those two itself | +//! | a function named `clone` or `clone_from` | the client derives `Clone`, and an inherent method of that name would win over the trait's in `app.clone()` | +//! | a module that returns no record; a record whose PascalCase name is no Rust name or is the name of a type or module beside it | there is no client to name | +//! +//! A field a `[rust.types]` entry types is the author's declaration, not read for `any`: a +//! `mlua::Value` the entry names is stopped by rustc and not by this module. +//! +//! A note, which does not fail the run, is said for a record that declares no function, and +//! for each function the module defines outside the record's body (`function app.helper()` +//! with no `helper: function(..)` in the body): the type index keeps no type for it, so it is +//! not in the client. Declaring `helper: function(..)` in the body adds it. A field that is +//! not a function is left out without a word. +//! //! # Drift //! //! The file is committed, and the Teal moves. `htl rs --check` runs the same function with @@ -193,6 +266,8 @@ use crate::typeindex::{Decl, Place, Prim, TypeExpr, TypeInfo, Where}; use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::path::{Path, PathBuf}; +mod client; + /// What starts the first line of the file `htl rs` writes. A caller that is about to /// overwrite a file checks that its first line starts with this, and leaves it alone /// otherwise. @@ -210,6 +285,29 @@ pub struct Spec { pub modules: Vec, /// `..` to the Rust type of that field; see the module doc. pub types: BTreeMap, + /// The modules whose returned record is written as a client over `htl::run::Instance`, + /// by the name `require` takes; see the module doc's Exports. Empty is no client. + pub exports: Vec, +} + +impl Spec { + /// Whether there is nothing to write: what [`generate`] answers with + /// [`Rendered::Nothing`], before it looks at a source. + pub fn is_empty(&self) -> bool { + self.modules.is_empty() && self.exports.is_empty() + } +} + +impl From<&crate::config::RustConfig> for Spec { + /// `[rust]` without `out`: the one construction `htl rs` and `htl::teal_types!()` lower + /// from, so that the two cannot ask different things of [`generate`]. + fn from(cfg: &crate::config::RustConfig) -> Self { + Spec { + modules: cfg.modules.clone(), + types: cfg.types.clone(), + exports: cfg.exports.clone(), + } + } } /// One of three outcomes, so "written with problems" is not a state. @@ -230,11 +328,12 @@ pub enum Rendered { /// What is said and does not fail. notes: Vec, }, - /// `modules` is empty: there is nothing to write. + /// `modules` and `exports` are empty: there is nothing to write. Nothing, } -/// Lower the records and enums of the listed modules of `sources`. +/// Lower the records and enums of the listed modules of `sources`, and the clients of the +/// exported ones. /// /// `sources` is every own module of the project, listed or not, each checked with the index /// kept: a field of a listed module may name a type of an unlisted one, and has to be told @@ -242,10 +341,11 @@ pub enum Rendered { /// a place, as [`RefTable::new`] takes it; this module reads no file, so the caller brings /// that. pub fn generate(sources: &[Source], spec: &Spec, canon: fn(&Path) -> PathBuf) -> Rendered { - if spec.modules.is_empty() { + if spec.is_empty() { return Rendered::Nothing; } let listed: BTreeSet = spec.modules.iter().cloned().collect(); + let exported: BTreeSet = spec.exports.iter().cloned().collect(); let refs = Refs { table: shapes::ref_table(sources, canon), listed: &listed, @@ -254,65 +354,74 @@ pub fn generate(sources: &[Source], spec: &Spec, canon: fn(&Path) -> PathBuf) -> let mut problems = Vec::new(); let mut notes = Vec::new(); - // The modules to lower, in bytewise order of their names. + // The modules to lower, and the ones to export, each in bytewise order of their names. let mut units: Vec = Vec::new(); for module in &listed { - let segs: Vec<&str> = module.split('.').collect(); - if let Some(bad) = segs.iter().find(|s| !is_module_segment(s)) { - config_problems.push(config_problem(format!( - "[rust] modules: `{module}`: `{bad}` cannot be the name of a Rust module" - ))); - continue; - } - let Some(src) = sources.iter().find(|s| s.module == module.as_str()) else { - config_problems.push(config_problem(format!( - "[rust] modules lists `{module}`, which is no module of this project's own sources" - ))); + units.extend(unit_of( + "modules", + module, + sources, + &refs, + &mut config_problems, + &mut problems, + )); + } + let mut exports: Vec = Vec::new(); + for module in &exported { + let Some(unit) = unit_of( + "exports", + module, + sources, + &refs, + &mut config_problems, + &mut problems, + ) else { continue; }; - let rel = rel_display(src.rel_path); - let Some(decls) = src.info.types.declarations() else { - problems.push(Diagnostic::new( + match client::Export::new(unit) { + Ok(export) => exports.push(export), + Err(unit) => problems.push(Diagnostic::new( Severity::Error, - rel, + unit.rel, 1, 1, - format!("`{module}` kept no type index (the file did not parse); nothing to lower"), + format!( + "`{module}` is listed in [rust] exports and returns no record; a client is made of the functions of the record a module returns" + ), None, - )); - continue; - }; - let cands = shapes::candidates(decls); - let mut aliases = Vec::new(); - shapes::collect_aliases(decls, &mut aliases, &mut None, &refs.table); - units.push(Unit { - src, - rel, - segs, - cands, - aliases, - }); + )), + } } + // Every module that has a `pub mod`: a type may not take the name of one below it. + let tree: Vec<&[&str]> = units + .iter() + .map(|u| u.segs.as_slice()) + .chain(exports.iter().map(|e| e.unit.segs.as_slice())) + .collect(); let overrides = resolve_types(spec, &units, &mut config_problems); // Lowered one module at a time; what each one says of itself stays with it. let mut blocks: Vec> = Vec::new(); let mut edges = Vec::new(); + let mut holds = Vec::new(); for unit in &units { let mut l = Lowering { unit, units: &units, refs: &refs, overrides: &overrides, - children: children_of(unit, &units), + children: children_of(&unit.segs, &tree), cur: String::new(), problems: Vec::new(), edges: Vec::new(), + reached: Vec::new(), + holds: Vec::new(), }; blocks.push(l.run()); problems.append(&mut l.problems); edges.append(&mut l.edges); + holds.append(&mut l.holds); if unit.cands.is_empty() { notes.push(Diagnostic::new( Severity::Warning, @@ -327,6 +436,16 @@ pub fn generate(sources: &[Source], spec: &Spec, canon: fn(&Path) -> PathBuf) -> )); } } + // The clients come after every type is lowered: what a record holds is what a method may + // take or return. + let lua = client::holding_lua_values(&holds); + let mut clients: Vec = Vec::new(); + for export in &exports { + let made = export.lower(&units, &refs, &overrides, &tree, &lua); + clients.push(made.text); + problems.extend(made.problems); + notes.extend(made.notes); + } problems.extend(cycles(&edges)); if !config_problems.is_empty() || !problems.is_empty() { problems.sort_by(|a, b| (&a.file, a.line, a.col).cmp(&(&b.file, b.line, b.col))); @@ -340,21 +459,19 @@ pub fn generate(sources: &[Source], spec: &Spec, canon: fn(&Path) -> PathBuf) -> // The tree, and the text. let mut root = Node::default(); for (i, u) in units.iter().enumerate() { - let mut node = &mut root; - for seg in &u.segs { - node = node.children.entry((*seg).to_string()).or_default(); - } - node.unit = Some(i); + root.at(&u.segs).unit = Some(i); + } + for (i, e) in exports.iter().enumerate() { + root.at(&e.unit.segs).client = Some(i); } - let mut order = Vec::new(); - root.sources(&mut order); - let sources_line: Vec<&str> = order.iter().map(|&i| units[i].rel.as_str()).collect(); + let mut sources_line: Vec<&str> = Vec::new(); + root.sources(&units, &exports, &mut sources_line); let mut text = format!( "{HEADER}. Do not edit: rerun `htl rs`.\n// Sources: {}\n", sources_line.join(", ") ); for (name, node) in &root.children { - if let Some(m) = node.render(name, 0, &blocks) { + if let Some(m) = node.render(name, 0, &blocks, &clients) { text.push('\n'); text.push_str(&m); text.push('\n'); @@ -408,13 +525,59 @@ struct Unit<'a> { aliases: Vec<&'a Decl>, } -/// The names of the modules directly below `unit` that are listed. -fn children_of<'a>(unit: &Unit, units: &[Unit<'a>]) -> BTreeSet<&'a str> { - let n = unit.segs.len(); - units - .iter() - .filter(|u| u.segs.len() > n && u.segs[..n] == unit.segs[..]) - .map(|u| u.segs[n]) +/// The module `module` names, as `[rust] ` lists it: `None` once a problem has been said +/// (a segment that is no Rust name, no own module of that name, a file that kept no index). +fn unit_of<'a>( + key: &str, + module: &'a str, + sources: &'a [Source<'a>], + refs: &Refs, + config_problems: &mut Vec, + problems: &mut Vec, +) -> Option> { + let segs: Vec<&str> = module.split('.').collect(); + if let Some(bad) = segs.iter().find(|s| !is_module_segment(s)) { + config_problems.push(config_problem(format!( + "[rust] {key}: `{module}`: `{bad}` cannot be the name of a Rust module" + ))); + return None; + } + let Some(src) = sources.iter().find(|s| s.module == module) else { + config_problems.push(config_problem(format!( + "[rust] {key} lists `{module}`, which is no module of this project's own sources" + ))); + return None; + }; + let rel = rel_display(src.rel_path); + let Some(decls) = src.info.types.declarations() else { + problems.push(Diagnostic::new( + Severity::Error, + rel, + 1, + 1, + format!("`{module}` kept no type index (the file did not parse); nothing to lower"), + None, + )); + return None; + }; + let cands = shapes::candidates(decls); + let mut aliases = Vec::new(); + shapes::collect_aliases(decls, &mut aliases, &mut None, &refs.table); + Some(Unit { + src, + rel, + segs, + cands, + aliases, + }) +} + +/// The names of the modules of `tree` directly below the module `segs`. +fn children_of<'a>(segs: &[&str], tree: &[&[&'a str]]) -> BTreeSet<&'a str> { + let n = segs.len(); + tree.iter() + .filter(|t| t.len() > n && t[..n] == *segs) + .map(|t| t[n]) .collect() } @@ -596,28 +759,59 @@ fn cycles(edges: &[Edge]) -> Vec { struct Node { /// The listed module that is this node, an index into the units. unit: Option, + /// The exported module that is this node, an index into the exports and their clients. + client: Option, children: BTreeMap, } impl Node { - /// The listed modules at and below this node, the node's own first, then its children in - /// order: the order the second line of the file names their sources in. - fn sources(&self, out: &mut Vec) { - out.extend(self.unit); + /// The node at the module path `segs` below this one, made on the way. + fn at(&mut self, segs: &[&str]) -> &mut Node { + let mut node = self; + for seg in segs { + node = node.children.entry((*seg).to_string()).or_default(); + } + node + } + + /// The files of the modules at and below this node, the node's own first (once, when it + /// is listed and exported both), then its children in order: the order the second line + /// of the file names their sources in. + fn sources<'u>( + &self, + units: &'u [Unit], + exports: &'u [client::Export], + out: &mut Vec<&'u str>, + ) { + match (self.unit, self.client) { + (Some(u), _) => out.push(&units[u].rel), + (None, Some(e)) => out.push(&exports[e].unit.rel), + (None, None) => {} + } for c in self.children.values() { - c.sources(out); + c.sources(units, exports, out); } } /// The `pub mod` of this node at `depth` (four spaces each), or `None` when nothing is - /// in it: a listed module that declares nothing has no module of its own. - fn render(&self, name: &str, depth: usize, blocks: &[Vec]) -> Option { + /// in it: a listed module that declares nothing has no module of its own. A module's + /// types come first, then its client, then the modules below it. + fn render( + &self, + name: &str, + depth: usize, + blocks: &[Vec], + clients: &[String], + ) -> Option { let mut items: Vec = Vec::new(); if let Some(u) = self.unit { items.extend(blocks[u].iter().map(|b| indent(b, depth + 1))); } + if let Some(e) = self.client { + items.push(indent(&clients[e], depth + 1)); + } for (n, c) in &self.children { - items.extend(c.render(n, depth + 1, blocks)); + items.extend(c.render(n, depth + 1, blocks, clients)); } if items.is_empty() { return None; @@ -721,12 +915,37 @@ struct Lowering<'a, 'r> { units: &'r [Unit<'a>], refs: &'r Refs<'r>, overrides: &'r HashMap<(String, String), String>, - /// The names of the listed modules directly below this one. + /// The names of the modules directly below this one that have a `pub mod` (listed or + /// exported). children: BTreeSet<&'a str>, /// The name of the type being lowered: where an edge starts. cur: String, problems: Vec, edges: Vec, + /// What the types lowered since it was last emptied reached: the caller empties it, + /// lowers, and reads it ([`Reached`]). + reached: Vec, + /// What each record of the module reached through its fields. + holds: Vec, +} + +/// What a lowered type is made of that the client of an export asks about: a Lua value, or +/// a record (which may hold one). +enum Reached { + Any, + Record { + /// The module and name of the record, as an [`Edge`] names it. + key: (String, String), + /// The name as the Teal source wrote it. + written: String, + }, +} + +/// The record `from` and what its fields reach, through a `{T}`, a `{K : V}`, a `T | nil` or +/// directly. A field a `[rust.types]` entry types is not lowered and reaches nothing. +struct Hold { + from: (String, String), + reached: Vec, } impl Lowering<'_, '_> { @@ -785,8 +1004,14 @@ impl Lowering<'_, '_> { by_name.insert(plain.clone(), i); } if self.children.contains(plain.as_str()) { + // A child is a `pub mod` because it is listed, or only because it is exported. + let n = unit.segs.len(); + let listed = self.units.iter().any(|u| { + u.segs.len() == n + 1 && u.segs[..n] == unit.segs[..] && u.segs[n] == plain + }); + let how = if listed { "listed" } else { "exported" }; let message = format!( - "`{}` has the name of the module `{}.{plain}`, which is listed; a type and a module share one namespace in Rust", + "`{}` has the name of the module `{}.{plain}`, which is {how}; a type and a module share one namespace in Rust", c.qual, unit.src.module ); self.problem(line, col, message); @@ -809,7 +1034,12 @@ impl Lowering<'_, '_> { match c.kind { CandKind::Enum { members } => self.enumeration(c, name, teal, members), CandKind::Record => { + self.reached.clear(); let fields = self.record(c); + self.holds.push(Hold { + from: (unit.src.module.to_string(), c.name.clone()), + reached: std::mem::take(&mut self.reached), + }); let mut s = format!("{STRUCT_DERIVE}\n#[teal(name = \"{teal}\")]\npub struct {name} {{"); if !fields.is_empty() { @@ -952,7 +1182,10 @@ impl Lowering<'_, '_> { Prim::Number => prim("f64", Class::Number), Prim::Integer => prim("i64", Class::Integer), Prim::Boolean => prim("bool", Class::Boolean), - Prim::Any => prim("::htl::mlua::Value", Class::Any), + Prim::Any => { + self.reached.push(Reached::Any); + prim("::htl::mlua::Value", Class::Any) + } Prim::Nil => self.no_type(at, site, "a type that is only `nil`"), Prim::Thread => self.no_type(at, site, "`thread`"), Prim::Userdata => self.no_type(at, site, "`userdata`"), @@ -1050,6 +1283,12 @@ impl Lowering<'_, '_> { match refs.entry(p) { Some(Entry::Type(r)) => { let module: Vec<&str> = r.module.split('.').collect(); + if r.record { + self.reached.push(Reached::Record { + key: (r.module.clone(), r.name.clone()), + written: written.clone(), + }); + } if r.record && !indirect { self.edges.push(Edge { from: (unit.src.module.to_string(), self.cur.clone()), @@ -1088,9 +1327,10 @@ impl Lowering<'_, '_> { } None => {} } - // The alias may be declared in any listed module, not only the one being lowered: - // `alias_in` matches only a place in the file of the unit it is handed. - let found = self.units.iter().find_map(|u| { + // The alias may be declared in any listed module, not only the one being lowered + // (which, for a client, need not be listed): `alias_in` matches only a place in + // the file of the unit it is handed. + let found = self.units.iter().chain([unit]).find_map(|u| { shapes::alias_in(&u.aliases, u.src.info.types.file(), &refs.table, names, p) .map(|a| (format!("{}.{}", u.src.module, a.name), a)) }); @@ -1139,8 +1379,10 @@ mod tests { info: CheckInfo, } - /// The module a file answers to: `geom/types.tl` is `geom.types`, `ev/init.tl` is `ev`. + /// The module a file answers to: `geom/types.tl` is `geom.types`, `ev/init.tl` is `ev`, + /// and `src/shop/app.tl` is `shop.app` (a project laid out as `htl new` does). fn module_of(rel: &str) -> String { + let rel = rel.strip_prefix("src/").unwrap_or(rel); let stem = rel.strip_suffix(".tl").unwrap(); stem.strip_suffix("/init").unwrap_or(stem).replace('/', ".") } @@ -1155,6 +1397,9 @@ mod tests { } let h = Htl::new().unwrap(); h.add_path(&root).unwrap(); + if root.join("src").is_dir() { + h.add_path(&root.join("src")).unwrap(); + } let checked = files .iter() .map(|(rel, _)| { @@ -1177,6 +1422,7 @@ mod tests { .iter() .map(|(k, v)| (k.to_string(), v.to_string())) .collect(), + exports: Vec::new(), } } @@ -1996,6 +2242,520 @@ return M ); } + /// `generate` with `modules`, `exports` and the `[rust.types]` entries. + fn exported_with( + name: &str, + files: &[(&str, &str)], + modules: &[&str], + exports: &[&str], + types: &[(&str, &str)], + ) -> Rendered { + let (_root, checked) = project(name, files); + let spec = Spec { + exports: exports.iter().map(|m| m.to_string()).collect(), + ..spec(modules, types) + }; + lower(&checked, &spec) + } + + fn exported( + name: &str, + files: &[(&str, &str)], + modules: &[&str], + exports: &[&str], + ) -> Rendered { + exported_with(name, files, modules, exports, &[]) + } + + fn problems_of(r: Rendered) -> Vec { + match r { + Rendered::Refused { problems, .. } => problems.iter().map(|p| p.to_string()).collect(), + other => panic!("not refused: {other:#?}"), + } + } + + /// The text and the notes of a run that wrote. + fn written_of(r: Rendered) -> (String, Vec) { + match r { + Rendered::Text { text, notes } => (text, notes.iter().map(|n| n.to_string()).collect()), + other => panic!("not written: {other:#?}"), + } + } + + /// The project `tests/fixtures/client` is: the golden file is what `htl rs` writes for + /// it, and the same file is compared by the CLI and the macro and compiled against + /// `htl::run` by the `htl` crate's tests. + const SHOP: [(&str, &str); 2] = [ + ( + "src/shop/types.tl", + include_str!("../tests/fixtures/client/src/shop/types.tl"), + ), + ( + "src/shop/app.tl", + include_str!("../tests/fixtures/client/src/shop/app.tl"), + ), + ]; + const SHOP_GOLDEN: &str = include_str!("../tests/fixtures/client/teal_types.rs"); + + /// An exported module is a client in the `pub mod` of its path, before the types of the + /// modules beside it; a type of a listed module is a `super::` path, a field that is no + /// function is left out, and a result of two values is a tuple. + #[test] + fn an_exported_module_is_a_client_of_its_functions() { + let (text, notes) = written_of(exported("client", &SHOP, &["shop.types"], &["shop.app"])); + assert_eq!(notes, Vec::::new()); + assert_eq!(text, SHOP_GOLDEN); + } + + /// The order `exports` and `modules` are written in, and a module named twice, change + /// nothing. + #[test] + fn the_order_and_repeats_of_exports_do_not_change_the_output() { + let a = "local record a\n one: function(): integer\nend\nreturn a\n"; + let b = "local record b\n two: function(): string\nend\nreturn b\n"; + let files = [("a.tl", a), ("b.tl", b)]; + let (first, _) = written_of(exported("order-1", &files, &[], &["a", "b"])); + assert!(first.contains("pub struct A {"), "{first}"); + assert!(first.contains("pub struct B {"), "{first}"); + for (name, exports) in [ + ("order-2", &["b", "a"][..]), + ("order-3", &["a", "b", "a", "b"][..]), + ] { + let (again, _) = written_of(exported(name, &files, &[], exports)); + assert_eq!(again, first, "{exports:?}"); + } + } + + /// What a client is refused for, in the file of the exported module. One module per + /// problem, so that each is said alone. + fn export_problems(name: &str, body: &str) -> Vec { + let src = format!("local record app\n{body}end\nreturn app\n"); + problems_of(exported(name, &[("app.tl", &src)], &[], &["app"])) + } + + #[test] + fn an_exported_method_is_a_problem() { + assert_eq!( + export_problems( + "export-method", + " m: function(self: app, n: integer): integer\n" + ), + ["app.tl:2:4: `app.m` is a method, called with `:`; a client calls functions with `.`"] + ); + } + + #[test] + fn an_exported_generic_is_a_problem() { + assert_eq!( + export_problems("export-generic", " id: function(x: T): T\n"), + ["app.tl:2:4: `app.id` is generic (``); a type variable has no Rust type"] + ); + } + + #[test] + fn an_exported_variadic_argument_is_a_problem() { + assert_eq!( + export_problems("export-rest", " sum: function(...: integer): integer\n"), + ["app.tl:2:4: `app.sum` takes a variadic `...`, which has no Rust type"] + ); + } + + #[test] + fn an_exported_variadic_result_is_a_problem() { + assert_eq!( + export_problems("export-rest-result", " many: function(): integer...\n"), + ["app.tl:2:4: `app.many` returns a variadic `...`, which has no Rust type"] + ); + } + + #[test] + fn an_exported_type_with_no_row_is_a_problem() { + assert_eq!( + export_problems( + "export-no-row", + " th: function(t: thread): {string, integer}\n loose: function(a: any): {any}\n deep: function(a: {string : any | nil}): integer\n" + ), + [ + "app.tl:2:4: `app.th, argument 1`: `thread` has no Rust type", + "app.tl:2:4: `app.th, result 1`: a tuple has no Rust type", + "app.tl:3:4: `app.loose, argument 1`: `any` is a Lua value, which does not leave the VM thread; give it a type", + "app.tl:3:4: `app.loose, result 1`: `any` is a Lua value, which does not leave the VM thread; give it a type", + "app.tl:4:4: `app.deep, argument 1`: `any` is a Lua value, which does not leave the VM thread; give it a type", + ] + ); + } + + #[test] + fn an_exported_overload_is_a_problem() { + let problems = export_problems( + "export-poly", + " f: function(integer): integer\n f: function(string): string\n", + ); + assert_eq!( + problems, + [ + "app.tl:3:4: `app.f`: `polymorphic function (with types function(integer): integer and function(string): string)` is not one function type (an overload, or an alias of one); it has no Rust signature" + ] + ); + } + + #[test] + fn an_exported_function_named_like_a_method_of_the_client_is_a_problem() { + assert_eq!( + export_problems("export-load", " load: function()\n"), + [ + "app.tl:2:4: `app.load`: `load` is the name of a method the client writes itself (`load`, `instance`); rename the function" + ] + ); + } + + #[test] + fn an_exported_function_named_like_a_clone_method_is_a_problem() { + assert_eq!( + export_problems("export-clone", " clone: function(): integer\n"), + [ + "app.tl:2:4: `app.clone`: `clone` would hide the client's `Clone::clone`; rename the function" + ] + ); + } + + #[test] + fn an_exported_function_named_like_no_rust_name_is_a_problem() { + assert_eq!( + export_problems("export-self", " self: function()\n"), + ["app.tl:2:4: `app.self`: `self` cannot be the name of a Rust method"] + ); + } + + #[test] + fn an_exported_function_of_more_than_sixteen_values_is_a_problem() { + let args = vec!["integer"; 17].join(", "); + assert_eq!( + export_problems("export-wide", &format!(" wide: function({args})\n")), + [ + "app.tl:2:4: `app.wide` takes 17 values; a call crosses with at most 16 each way, the most a Rust tuple mlua converts holds" + ] + ); + let sixteen = vec!["integer"; 16].join(", "); + let src = + format!("local record app\n ok: function({sixteen}): {sixteen}\nend\nreturn app\n"); + let (text, _) = written_of(exported( + "export-sixteen", + &[("app.tl", &src)], + &[], + &["app"], + )); + assert!(text.contains("a16: i64,"), "{text}"); + } + + /// A body whose functions are all refused is not a body that declares none: the note is + /// not said beside the problems. + #[test] + fn a_refused_function_is_not_reported_as_no_function() { + let src = "local record app\n m: function(self: app)\nend\nreturn app\n"; + match exported("export-refused-note", &[("app.tl", src)], &[], &["app"]) { + Rendered::Refused { problems, notes } => { + assert_eq!(problems.len(), 1, "{problems:?}"); + assert!(notes.is_empty(), "{notes:?}"); + } + other => panic!("not refused: {other:#?}"), + } + } + + /// A type of a listed module that takes the name of an exported (not listed) module below + /// it says which of the two the module is. + #[test] + fn a_type_named_like_an_exported_child_module_says_exported() { + let geom = "local record M\n record app\n x: integer\n end\nend\nreturn M\n"; + let app = "local record app\n one: function()\nend\nreturn app\n"; + assert_eq!( + problems_of(exported( + "export-child", + &[("geom/init.tl", geom), ("geom/app.tl", app)], + &["geom"], + &["geom.app"], + )), + [ + "geom/init.tl:2:11: `M.app` has the name of the module `geom.app`, which is exported; a type and a module share one namespace in Rust" + ] + ); + } + + #[test] + fn an_exported_module_must_return_a_record() { + let src = "local function f(): integer\n return 1\nend\nreturn { f = f }\n"; + assert_eq!( + problems_of(exported( + "export-none", + &[("app.tl", src)], + &[], + &["app", "nope"] + )), + [ + "[rust] exports lists `nope`, which is no module of this project's own sources", + "app.tl:1:1: `app` is listed in [rust] exports and returns no record; a client is made of the functions of the record a module returns", + ] + ); + } + + #[test] + fn a_client_may_not_take_the_name_of_a_type_or_a_module_beside_it() { + // `App` is the client of `app` and a record `app` declares, once `app` is listed too. + let app = "local record app\n one: function()\n record App\n end\nend\nreturn app\n"; + assert_eq!( + problems_of(exported( + "export-name-type", + &[("app.tl", app)], + &["app"], + &["app"] + )), + [ + "app.tl:1:14: `app` makes the client `App`, which is the name of a type or a module of `app` too; rename one" + ] + ); + // And a listed module `app.App` is a `pub mod App` below it. + let sub = "local record M\n record X\n end\nend\nreturn M\n"; + let plain = "local record app\n one: function()\nend\nreturn app\n"; + assert_eq!( + problems_of(exported( + "export-name-module", + &[("app.tl", plain), ("app/App.tl", sub)], + &["app.App"], + &["app"], + )), + [ + "app.tl:1:14: `app` makes the client `App`, which is the name of a type or a module of `app` too; rename one" + ] + ); + } + + /// The client is named for the record the module returns: `M` stays `M`, a snake case + /// name is PascalCased. + #[test] + fn the_client_is_named_for_the_returned_record() { + let m = "local record M\n one: function(): integer\nend\nreturn M\n"; + let snake = "local record my_app\n one: function(): integer\nend\nreturn my_app\n"; + let (text, _) = written_of(exported( + "export-names", + &[("m.tl", m), ("snake.tl", snake)], + &[], + &["m", "snake"], + )); + assert!(text.contains("pub struct M {"), "{text}"); + assert!(text.contains("impl M {"), "{text}"); + assert!(text.contains("pub struct MyApp {"), "{text}"); + } + + /// Only a Teal type of `T | nil` is an `Option`. The number of arguments a call has to + /// pass is not read as "the rest are optional": the index does not say which one `?` + /// stands for, and `Option` on the wrong one would send a nil where Teal wants a value. + #[test] + fn only_a_nilable_type_is_an_option_and_an_optional_marker_is_not_read() { + let src = "local record app + both: function(a?: string, b: integer): integer + nilable: function(a: string, b: string | nil, c?: integer): {string : integer | nil} + later: function(a: string, b?: string): string | nil +end +return app +"; + let (text, _) = written_of(exported("export-option", &[("app.tl", src)], &[], &["app"])); + let string = "::std::string::String"; + let opt = |t: &str| format!("::core::option::Option<{t}>"); + let method = |name: &str| { + text.split("pub async fn ") + .find(|m| m.starts_with(&format!("{name}("))) + .unwrap_or_else(|| panic!("no method {name}:\n{text}")) + .to_string() + }; + // `(a?: string, b: integer)`: both are `T`, whichever the `?` stands for. + let both = method("both"); + assert!(both.contains(&format!("a1: {string},")), "{both}"); + assert!(both.contains("a2: i64,"), "{both}"); + // Only `string | nil` is an `Option`; a `?` on the third changes nothing. + let nilable = method("nilable"); + assert!( + nilable.contains(&format!("a2: {},", opt(string))), + "{nilable}" + ); + assert!(nilable.contains("a3: i64,"), "{nilable}"); + assert!( + nilable.contains(&format!( + "HashMap<{string}, {}>, ::htl::run::CallError>", + opt("i64") + )), + "{nilable}" + ); + // The last argument may be left out in Teal and is a `String` here; a nilable result + // is an `Option`. + let later = method("later"); + assert!(later.contains(&format!("a2: {string},")), "{later}"); + assert!( + later.contains(&format!("Result<{}, ::htl::run::CallError>", opt(string))), + "{later}" + ); + } + + const HOLDING_TYPES: &str = "local record types + record Inner + v: any + end + record Outer + inner: Inner + end + record Far + outers: {string : Outer} + end + record Safe + n: integer + end +end +return types +"; + + fn holding(name: &str, function: &str, types: &[(&str, &str)]) -> Rendered { + let app = format!( + "local types = require(\"types\")\nlocal record app\n {function}\nend\nreturn app\n" + ); + exported_with( + name, + &[("types.tl", HOLDING_TYPES), ("app.tl", &app)], + &["types"], + &["app"], + types, + ) + } + + /// A record that holds an `any` (a field of its own, or through the records it holds, also + /// behind a `{T}` or a `{K : V}`) is no more a type a method crosses with than `any`. + #[test] + fn a_record_that_reaches_any_is_a_problem_as_an_argument_and_as_a_result() { + let said = |site: &str, written: &str| { + format!( + "app.tl:3:4: `{site}`: `{written}` holds an `any`, directly or through other records, which is a Lua value that does not leave the VM thread; give that field a type, or a Rust one under [rust.types]" + ) + }; + for (n, (function, expected)) in [ + ( + "take: function(i: types.Inner)", + said("app.take, argument 1", "types.Inner"), + ), + ( + "take: function(o: types.Outer)", + said("app.take, argument 1", "types.Outer"), + ), + ( + "give: function(): types.Outer", + said("app.give, result 1", "types.Outer"), + ), + ( + "give: function(): {types.Far}", + said("app.give, result 1", "types.Far"), + ), + ( + "give: function(): {string : types.Far | nil}", + said("app.give, result 1", "types.Far"), + ), + ] + .into_iter() + .enumerate() + { + assert_eq!( + problems_of(holding(&format!("holding-{n}"), function, &[])), + [expected], + "{function}" + ); + } + // A record that holds no `any` is a type like another. + let (text, _) = written_of(holding( + "holding-safe", + "ok: function(s: types.Safe): types.Safe", + &[], + )); + assert!(text.contains("a1: super::types::Safe,"), "{text}"); + } + + /// A field a `[rust.types]` entry types is the author's: it is not lowered, so it is not + /// read for an `any`, and the record is no longer one that holds a Lua value. + #[test] + fn a_field_typed_under_rust_types_does_not_make_its_record_hold_any() { + let (text, _) = written_of(holding( + "holding-override", + "take: function(o: types.Outer): types.Far", + &[("types.Inner.v", "u32")], + )); + assert!(text.contains("a1: super::types::Outer,"), "{text}"); + assert!(text.contains("pub v: u32,"), "{text}"); + } + + /// A function written below the body of the record has no type in the index and is not in + /// the client; the note says where it is and what to write. A function the body declares + /// is in it, whether or not the module defines it below. + #[test] + fn a_function_defined_outside_the_body_is_a_note() { + let src = "local record app + declared: function(n: integer): integer +end +function app.declared(n: integer): integer + return n +end +function app.helper(n: integer): integer + return n + 1 +end +function app:method(n: integer): integer + return n +end +return app +"; + let (text, notes) = written_of(exported( + "export-outside", + &[("app.tl", src)], + &[], + &["app"], + )); + assert!(text.contains("pub async fn declared("), "{text}"); + assert!(!text.contains("helper"), "{text}"); + assert_eq!( + notes, + [ + "app.tl:7:1: `app.helper` is defined outside the body of the record, so it is not in the client; declare `helper: function(..)` in the body to add it [htl rust]" + ] + ); + } + + /// A record that declares no function has a client with none, and says so. + #[test] + fn an_export_without_a_function_is_a_note() { + let src = "local record app\n limit: integer\nend\nreturn app\n"; + let (text, notes) = written_of(exported("export-empty", &[("app.tl", src)], &[], &["app"])); + assert!(text.contains("pub struct App {"), "{text}"); + assert_eq!( + notes, + [ + "app.tl:1:14: `app` is listed in [rust] exports and its record declares no function in its body [htl rust]" + ] + ); + } + + /// Only `exports`: something to write. Neither: nothing. + #[test] + fn exports_alone_are_something_to_write_and_none_is_nothing() { + let only = Spec { + exports: vec!["app".into()], + ..Spec::default() + }; + assert!(!only.is_empty()); + assert!(Spec::default().is_empty()); + let from = Spec::from(&crate::config::RustConfig { + modules: vec!["ev".into()], + exports: vec!["app".into()], + types: [("ev.Event.at".to_string(), "u32".to_string())].into(), + out: Some("ignored.rs".into()), + }); + assert_eq!(from.modules, ["ev"]); + assert_eq!(from.exports, ["app"]); + assert_eq!(from.types["ev.Event.at"], "u32"); + } + #[test] fn the_types_table_is_held_to_what_it_says() { let geom = "local record M diff --git a/crates/htl-core/src/rsgen/client.rs b/crates/htl-core/src/rsgen/client.rs new file mode 100644 index 0000000..aca799d --- /dev/null +++ b/crates/htl-core/src/rsgen/client.rs @@ -0,0 +1,371 @@ +//! The client of an exported module (`[rust] exports`): the struct over an +//! `htl::run::Instance`, one async method per function the module's record declares. +//! +//! What a client is, what it is made of and what refuses one is the module doc of +//! [`crate::rsgen`] (Exports); the types of its arguments and results are the mapping's, so +//! this is the other side of the same lowering and not a second one: a client is built on the +//! [`Lowering`] of its module, after every type of the project is lowered. + +use super::{ + Diagnostic, Hold, Lowering, Out, RULE, Reached, Refs, Severity, Unit, children_of, element, + pascal, rust_ident, +}; +use crate::shapes::record_fields; +use crate::typeindex::{Decl, DeclKind, TypeExpr, TypeInfo}; +use std::collections::{HashMap, HashSet}; + +/// The most values mlua converts as one tuple (`IntoLuaMulti` / `FromLuaMulti`): the +/// arguments of a call, and its results, are each one tuple. +const MAX_TUPLE: usize = 16; + +/// An exported module: the module, and the record it returns. +pub(super) struct Export<'a> { + pub(super) unit: Unit<'a>, + record: &'a Decl, +} + +/// What lowering a client made. The text is discarded when a problem was said. +pub(super) struct Made { + pub(super) text: String, + pub(super) problems: Vec, + pub(super) notes: Vec, +} + +/// The records that hold an `any`, directly or through other records: the ones a method +/// may neither take nor return, because a Lua value does not leave the VM thread. +pub(super) fn holding_lua_values(holds: &[Hold]) -> HashSet<(String, String)> { + let mut set: HashSet<(String, String)> = holds + .iter() + .filter(|h| h.reached.iter().any(|r| matches!(r, Reached::Any))) + .map(|h| h.from.clone()) + .collect(); + loop { + let before = set.len(); + for h in holds { + let through = h + .reached + .iter() + .any(|r| matches!(r, Reached::Record { key, .. } if set.contains(key))); + if through { + set.insert(h.from.clone()); + } + } + if set.len() == before { + return set; + } + } +} + +impl<'a> Export<'a> { + /// The export of `unit`, or the unit back when its module returns no record. + pub(super) fn new(unit: Unit<'a>) -> Result> { + let info = unit.src.info; + let record = info.types.declarations().and_then(|decls| { + decls + .iter() + .find(|d| matches!(d.info, Some(TypeInfo::Record { returned: true, .. }))) + }); + match record { + Some(record) => Ok(Export { unit, record }), + None => Err(unit), + } + } + + /// The client, and what is said of it. `tree` is every module that has a `pub mod`, `lua` + /// the records [`holding_lua_values`] found. + pub(super) fn lower( + &self, + units: &[Unit<'a>], + refs: &Refs, + overrides: &HashMap<(String, String), String>, + tree: &[&[&'a str]], + lua: &HashSet<(String, String)>, + ) -> Made { + let unit = &self.unit; + let record = self.record; + let mut l = Lowering { + unit, + units, + refs, + overrides, + children: children_of(&unit.segs, tree), + cur: String::new(), + problems: Vec::new(), + edges: Vec::new(), + reached: Vec::new(), + holds: Vec::new(), + }; + // The types the module's `pub mod` holds when it is listed too. + let taken: Vec<&str> = units + .iter() + .filter(|u| u.src.module == unit.src.module) + .flat_map(|u| u.cands.iter().map(|c| c.name.as_str())) + .collect(); + let (text, functions) = l.client(record, &taken, lua); + let mut notes = Vec::new(); + if functions == 0 { + notes.push(Diagnostic::new( + Severity::Warning, + unit.rel.clone(), + record.name_line, + record.name_col, + format!( + "`{}` is listed in [rust] exports and its record declares no function in its body", + unit.src.module + ), + Some(RULE), + )); + } + for f in &record.children { + // A function written below the body, with no `f: function(..)` in it: the index + // has the definition and no type for it. + if f.kind == DeclKind::Function && !matches!(f.info, Some(TypeInfo::Field { .. })) { + let short = f.name.rsplit('.').next().unwrap_or(&f.name); + notes.push(Diagnostic::new( + Severity::Warning, + unit.rel.clone(), + f.line, + f.col, + format!( + "`{}.{short}` is defined outside the body of the record, so it is not in the client; declare `{short}: function(..)` in the body to add it", + record.name + ), + Some(RULE), + )); + } + } + Made { + text, + problems: l.problems, + notes, + } + } +} + +impl Lowering<'_, '_> { + /// The client of an exported module whose returned record is `record`, and how many + /// functions the record's body declares (a refused one counted: the client has no method + /// for it, but the body is not empty). `taken` is the names of the types the module's + /// `pub mod` holds besides. + fn client( + &mut self, + record: &Decl, + taken: &[&str], + lua: &HashSet<(String, String)>, + ) -> (String, usize) { + let module = self.unit.src.module; + let name = pascal(&record.name); + let (line, col) = (record.name_line, record.name_col); + if rust_ident(&name).is_none_or(|n| n != name) { + let message = format!( + "`{}` cannot be PascalCased into the name of a Rust type for its client", + record.name + ); + self.problem(line, col, message); + } else if taken.contains(&name.as_str()) || self.children.contains(name.as_str()) { + let message = format!( + "`{}` makes the client `{name}`, which is the name of a type or a module of `{module}` too; rename one", + record.name + ); + self.problem(line, col, message); + } + let mut methods = Vec::new(); + let mut declared = 0; + for (f, ty) in record_fields(record) { + // An overload (two function types for one field) is a field whose type is the + // text Teal prints for it. + let function = matches!(f.kind, DeclKind::Function | DeclKind::Method) + || matches!(ty, TypeExpr::Fn(_)) + || matches!(ty, TypeExpr::Other(t) if t.starts_with("polymorphic function")); + if function { + declared += 1; + let site = format!("{}.{}", record.name, f.name); + methods.extend(self.method(f, ty, &site, lua)); + } + } + let mut s = format!( + "/// The functions of the Teal module `{module}`, called on a runner instance; it uses `htl::run`, so the host builds `htl` with its `async` feature. +#[derive(Debug, Clone)] +pub struct {name} {{ + inst: ::htl::run::Instance, +}} + +impl {name} {{ + /// `require(\"{module}\")` in a fresh instance of `runner`. + pub async fn load( + runner: &::htl::run::Runner, + extra_paths: &[::std::path::PathBuf], + ) -> ::core::result::Result {{ + let inst = runner.instance(\"{module}\", extra_paths).await?; + ::core::result::Result::Ok(Self {{ inst }}) + }} + + /// The instance the functions are called on. + pub fn instance(&self) -> &::htl::run::Instance {{ + &self.inst + }} +" + ); + for m in &methods { + s.push('\n'); + s.push_str(&super::indent(m, 1)); + s.push('\n'); + } + s.push('}'); + (s, declared) + } + + /// The method of the client that calls the function `f` of type `ty`; `None` once a + /// problem has been said. Every problem is said at `f`. + fn method( + &mut self, + f: &Decl, + ty: &TypeExpr, + site: &str, + lua: &HashSet<(String, String)>, + ) -> Option { + let (line, col) = (f.line, f.col); + let TypeExpr::Fn(sig) = ty else { + let written = f.detail.as_deref().unwrap_or("this type"); + let message = format!( + "`{site}`: `{written}` is not one function type (an overload, or an alias of one); it has no Rust signature" + ); + self.problem(line, col, message); + return None; + }; + let mut said = Vec::new(); + if sig.method { + said.push(format!( + "`{site}` is a method, called with `:`; a client calls functions with `.`" + )); + } + if !sig.type_params.is_empty() { + said.push(format!( + "`{site}` is generic (`<{}>`); a type variable has no Rust type", + sig.type_params.join(", ") + )); + } + if sig.params.rest.is_some() { + said.push(format!( + "`{site}` takes a variadic `...`, which has no Rust type" + )); + } + if sig.results.rest.is_some() { + said.push(format!( + "`{site}` returns a variadic `...`, which has no Rust type" + )); + } + // The arguments cross as one tuple and the results come back as one, and mlua + // converts a tuple of at most `MAX_TUPLE` values. + for (what, n) in [ + ("takes", sig.params.items.len()), + ("returns", sig.results.items.len()), + ] { + if n > MAX_TUPLE { + said.push(format!( + "`{site}` {what} {n} values; a call crosses with at most {MAX_TUPLE} each way, the most a Rust tuple mlua converts holds" + )); + } + } + let name = rust_ident(&f.name); + match name.as_deref() { + None => said.push(format!( + "`{site}`: `{}` cannot be the name of a Rust method", + f.name + )), + Some(n @ ("load" | "instance")) => said.push(format!( + "`{site}`: `{n}` is the name of a method the client writes itself (`load`, `instance`); rename the function" + )), + // The client derives `Clone`: an inherent method of either name would win over + // the trait's in method-call syntax, and `app.clone()` would call Teal. + Some(n @ ("clone" | "clone_from")) => said.push(format!( + "`{site}`: `{n}` would hide the client's `Clone::{n}`; rename the function" + )), + Some(_) => {} + } + let failed = !said.is_empty(); + for message in said { + self.problem(line, col, message); + } + // A method's `self` and a generic's type variables would only repeat the problem. + if sig.method || !sig.type_params.is_empty() { + return None; + } + // A parameter is `T`, an `Option` only when Teal types it `T | nil`: the index + // says how many arguments a call has to pass and not which ones, so an optional + // parameter (`x?: T`) is not told from a required one, and the host passes a value. + let mut params = Vec::new(); + for (i, p) in sig.params.items.iter().enumerate() { + let at = format!("{site}, argument {}", i + 1); + params.push(self.crossing(p, f, &at, lua).map(element)); + } + let mut results = Vec::new(); + for (i, r) in sig.results.items.iter().enumerate() { + let at = format!("{site}, result {}", i + 1); + results.push(self.crossing(r, f, &at, lua).map(element)); + } + let (Some(name), false) = (name, failed) else { + return None; + }; + let params: Vec = params.into_iter().collect::>()?; + let mut results: Vec = results.into_iter().collect::>()?; + let ret = match results.len() { + 0 => "()".to_string(), + 1 => results.remove(0), + _ => format!("({})", results.join(", ")), + }; + let names: Vec = (1..=params.len()).map(|i| format!("a{i}")).collect(); + let args = match names.as_slice() { + [] => "()".to_string(), + [one] => format!("({one},)"), + many => format!("({})", many.join(", ")), + }; + let doc = match &f.detail { + Some(d) => format!("/// `{site}`, typed `{d}` in Teal.\n"), + None => format!("/// `{site}`.\n"), + }; + let head = if params.is_empty() { + format!("pub async fn {name}(&self)") + } else { + let mut h = format!("pub async fn {name}(\n &self,\n"); + for (n, t) in names.iter().zip(¶ms) { + h.push_str(&format!(" {n}: {t},\n")); + } + h.push(')'); + h + }; + Some(format!( + "{doc}{head} -> ::core::result::Result<{ret}, ::htl::run::CallError> {{\n self.inst.call(\"{}\", {args}).await\n}}", + f.name + )) + } + + /// The Rust type of an argument or a result of a client's method: the mapping's, with + /// one rule more. A Lua value does not leave the VM thread, where `htl::run` keeps every + /// Lua value, so a type that is or holds an `any` (also inside a `{T}`, a `{K : V}`, or + /// through a record of `lua`) is a problem. + fn crossing( + &mut self, + ty: &TypeExpr, + at: &Decl, + site: &str, + lua: &HashSet<(String, String)>, + ) -> Option { + self.reached.clear(); + let out = self.lower_ty(ty, at, site, &mut Vec::new(), true)?; + let held = self.reached.iter().find_map(|r| match r { + Reached::Any => Some(format!( + "`{site}`: `any` is a Lua value, which does not leave the VM thread; give it a type" + )), + Reached::Record { key, written } if lua.contains(key) => Some(format!( + "`{site}`: `{written}` holds an `any`, directly or through other records, which is a Lua value that does not leave the VM thread; give that field a type, or a Rust one under [rust.types]" + )), + Reached::Record { .. } => None, + }); + if let Some(message) = held { + self.problem(at.line, at.col, message); + return None; + } + Some(out) + } +} diff --git a/crates/htl-core/tests/fixtures/client/htl.toml b/crates/htl-core/tests/fixtures/client/htl.toml new file mode 100644 index 0000000..5e21751 --- /dev/null +++ b/crates/htl-core/tests/fixtures/client/htl.toml @@ -0,0 +1,3 @@ +[rust] +modules = ["shop.types"] +exports = ["shop.app"] diff --git a/crates/htl-core/tests/fixtures/client/src/shop/app.tl b/crates/htl-core/tests/fixtures/client/src/shop/app.tl new file mode 100644 index 0000000..7b30af7 --- /dev/null +++ b/crates/htl-core/tests/fixtures/client/src/shop/app.tl @@ -0,0 +1,17 @@ +local types = require("shop.types") + +local record app + split: function(s: string): string, integer + greet: function(name: string, title: string | nil): types.Output + limit: integer +end + +function app.split(s: string): string, integer + return s, #s +end + +function app.greet(name: string, title: string | nil): types.Output + return { ok = true, text = (title or "") .. name } +end + +return app diff --git a/crates/htl-core/tests/fixtures/client/src/shop/types.tl b/crates/htl-core/tests/fixtures/client/src/shop/types.tl new file mode 100644 index 0000000..9d2d989 --- /dev/null +++ b/crates/htl-core/tests/fixtures/client/src/shop/types.tl @@ -0,0 +1,7 @@ +local record types + record Output ---@struct + ok: boolean + text: string + end +end +return types diff --git a/crates/htl-core/tests/fixtures/client/teal_types.rs b/crates/htl-core/tests/fixtures/client/teal_types.rs new file mode 100644 index 0000000..4b86d7f --- /dev/null +++ b/crates/htl-core/tests/fixtures/client/teal_types.rs @@ -0,0 +1,55 @@ +// Generated by htl rs. Do not edit: rerun `htl rs`. +// Sources: src/shop/app.tl, src/shop/types.tl + +#[allow(non_snake_case, non_camel_case_types, dead_code, clippy::all)] +pub mod shop { + pub mod app { + /// The functions of the Teal module `shop.app`, called on a runner instance; it uses `htl::run`, so the host builds `htl` with its `async` feature. + #[derive(Debug, Clone)] + pub struct App { + inst: ::htl::run::Instance, + } + + impl App { + /// `require("shop.app")` in a fresh instance of `runner`. + pub async fn load( + runner: &::htl::run::Runner, + extra_paths: &[::std::path::PathBuf], + ) -> ::core::result::Result { + let inst = runner.instance("shop.app", extra_paths).await?; + ::core::result::Result::Ok(Self { inst }) + } + + /// The instance the functions are called on. + pub fn instance(&self) -> &::htl::run::Instance { + &self.inst + } + + /// `app.split`, typed `function(string): (string, integer)` in Teal. + pub async fn split( + &self, + a1: ::std::string::String, + ) -> ::core::result::Result<(::std::string::String, i64), ::htl::run::CallError> { + self.inst.call("split", (a1,)).await + } + + /// `app.greet`, typed `function(string, string | nil): types.Output` in Teal. + pub async fn greet( + &self, + a1: ::std::string::String, + a2: ::core::option::Option<::std::string::String>, + ) -> ::core::result::Result { + self.inst.call("greet", (a1, a2)).await + } + } + } + + pub mod types { + #[derive(Debug, Clone, PartialEq, ::htl::TealRecord)] + #[teal(name = "Output")] + pub struct Output { + pub ok: bool, + pub text: ::std::string::String, + } + } +} diff --git a/crates/htl-macros/src/lib.rs b/crates/htl-macros/src/lib.rs index fc3fae8..9a8a04f 100644 --- a/crates/htl-macros/src/lib.rs +++ b/crates/htl-macros/src/lib.rs @@ -679,7 +679,7 @@ fn expand_include(rel: &str, bytes: bool) -> Result { // ------------------------------------------------------------------ teal_types! /// `teal_types!()` -> the `pub mod ... { ... }` items `htl rs` writes, generated at -/// `cargo build` from `htl.toml` (`[rust] modules` and `[rust.types]`) through +/// `cargo build` from `htl.toml` (`[rust] modules`, `exports` and `[rust.types]`) through /// [`htl_core::rsgen::generate`], for a project that does not commit generated Rust. /// /// Every `.tl` source file checked (and every `.d.tl` or `.tl` it resolved) and @@ -703,8 +703,8 @@ pub fn teal_types(input: TokenStream) -> TokenStream { /// unit-tested and compared directly with [`htl_core::rsgen::generate`]. #[derive(Debug)] struct TealTypesOut { - /// The Rust source text `rsgen::generate` produced, or empty when `[rust] modules` is - /// empty. + /// The Rust source text `rsgen::generate` produced, or empty when `[rust] modules` and + /// `exports` are empty. code: String, /// Every `.tl` file checked and every dependency it resolved (`include_str!` inputs), /// canonical and absolute. @@ -734,12 +734,9 @@ fn resolve_teal_types(manifest_dir: &Path) -> Result { manifest_dir.display() )); }; - let spec = htl_core::rsgen::Spec { - modules: cfg.rust.modules.clone(), - types: cfg.rust.types.clone(), - }; + let spec = htl_core::rsgen::Spec::from(&cfg.rust); let config = config_input(&ck); - if spec.modules.is_empty() { + if spec.is_empty() { return Ok(TealTypesOut { code: String::new(), inputs: Vec::new(), @@ -3312,7 +3309,7 @@ return M .collect(); let spec = htl_core::rsgen::Spec { modules: vec!["ev".to_string()], - types: Default::default(), + ..Default::default() }; let htl_core::rsgen::Rendered::Text { text: rsgen_text, .. @@ -3348,6 +3345,52 @@ return M assert_eq!(expanded_mods, rsgen_mods); } + /// `[rust] exports` alone is something to write: the client of the module, as items + /// that parse. + #[test] + fn teal_types_writes_the_client_of_an_exported_module() { + let root = tempdir("teal-types-exports"); + write(&root.join("htl.toml"), "[rust]\nexports = [\"app\"]\n"); + write( + &root.join("src/app.tl"), + "local record app\n add: function(integer, integer): integer\nend\nfunction app.add(a: integer, b: integer): integer\n return a + b\nend\nreturn app\n", + ); + let out = resolve_teal_types(&root).expect("teal_types! resolves"); + assert!(out.code.contains("pub struct App {"), "{}", out.code); + assert!( + out.code.contains("self.inst.call(\"add\", (a1, a2)).await"), + "{}", + out.code + ); + expand_teal_types_from(&out).expect("parses as Rust items"); + } + + /// The expansion and `htl rs` write the same text: the project of + /// `htl-core/tests/fixtures/client` expands to the golden file there, which the + /// `htl-cli` test of `htl rs` compares the command's file with as well. + #[test] + fn teal_types_expands_to_the_text_htl_rs_writes() { + let root = tempdir("teal-types-client"); + write( + &root.join("htl.toml"), + include_str!("../../htl-core/tests/fixtures/client/htl.toml"), + ); + write( + &root.join("src/shop/types.tl"), + include_str!("../../htl-core/tests/fixtures/client/src/shop/types.tl"), + ); + write( + &root.join("src/shop/app.tl"), + include_str!("../../htl-core/tests/fixtures/client/src/shop/app.tl"), + ); + let out = resolve_teal_types(&root).expect("teal_types! resolves"); + assert_eq!( + out.code, + include_str!("../../htl-core/tests/fixtures/client/teal_types.rs") + ); + expand_teal_types_from(&out).expect("parses as Rust items"); + } + /// #539 (Acceptance 2): a type error in `src/ev/init.tl` fails `resolve_teal_types` /// naming the Teal file and line; a shape `rsgen::generate` refuses or a missing /// `htl.toml` is refused as well. diff --git a/crates/htl/tests/run_client.rs b/crates/htl/tests/run_client.rs new file mode 100644 index 0000000..c3af002 --- /dev/null +++ b/crates/htl/tests/run_client.rs @@ -0,0 +1,61 @@ +//! The client `htl rs` writes for an exported module (`[rust] exports`), compiled against +//! `htl::run` and called on a runner. +//! +//! The file is `htl-core/tests/fixtures/client/teal_types.rs`, the golden that `htl rs` and +//! `teal_types!()` are held to by the tests of `htl-cli` and `htl-macros`, brought in whole +//! by `include!`: what is compiled here is what is written. Its Teal is the `src/` beside it. +#![cfg(feature = "async")] + +mod common; + +use std::time::Duration; + +include!("../../htl-core/tests/fixtures/client/teal_types.rs"); + +const TYPES_TL: &str = include_str!("../../htl-core/tests/fixtures/client/src/shop/types.tl"); +const APP_TL: &str = include_str!("../../htl-core/tests/fixtures/client/src/shop/app.tl"); + +fn assert_send_sync() {} + +#[tokio::test] +async fn the_generated_client_calls_the_teal_module() { + assert_send_sync::(); + let root = common::tempdir("htl-run-client", "shop"); + std::fs::create_dir_all(root.join("shop")).unwrap(); + std::fs::write(root.join("shop/types.tl"), TYPES_TL).unwrap(); + std::fs::write(root.join("shop/app.tl"), APP_TL).unwrap(); + + let (runner, driver) = htl::run::start(htl::run::RunnerConfig { + search_paths: vec![root.to_path_buf()], + deadline: Duration::from_secs(30), + queue: 8, + }) + .await + .unwrap(); + let app = shop::app::App::load(&runner, &[]).await.unwrap(); + + // A task of the host's runtime holds the client, and two results come back as a tuple. + let task = { + let app = app.clone(); + tokio::spawn(async move { app.split("abc".into()).await }) + }; + assert_eq!(task.await.unwrap().unwrap(), ("abc".to_string(), 3)); + + // A record comes back as the struct the file generates, and `string | nil` takes `Some` + // and `None`. + let greeted = app.greet("Ada".into(), Some("Dr. ".into())).await.unwrap(); + assert_eq!( + greeted, + shop::types::Output { + ok: true, + text: "Dr. Ada".into() + } + ); + assert_eq!( + app.greet("Ada".into(), None).await.unwrap().text, + "Ada".to_string() + ); + + app.instance().clone().close().await.unwrap(); + driver.shutdown().await.unwrap(); +} diff --git a/e2e/tests/teal_exports_client.rs b/e2e/tests/teal_exports_client.rs new file mode 100644 index 0000000..1327ca4 --- /dev/null +++ b/e2e/tests/teal_exports_client.rs @@ -0,0 +1,171 @@ +//! The acceptance of #538 that is about a *host crate*: `[rust] exports` in `htl.toml` and +//! `htl::teal_types!();` at the crate root give it a typed client of a Teal module. +//! +//! 1. A call with the arguments Teal declares builds, and run, returns what the Teal +//! function computed. +//! 2. A call with an argument of the wrong type fails `cargo check` with `E0308`. The +//! error is read from the compiler's JSON, by its code: rustc's wording differs between +//! the stable release and the MSRV, and a snapshot of it would break with the version. +//! +//! The project is `crates/htl-core/tests/fixtures/client`, the one whose generated file the +//! tests of `htl rs`, `teal_types!()` and `htl::run` are held to. + +use std::ffi::OsString; +use std::fs; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output, Stdio}; +use std::sync::OnceLock; + +mod common; + +fn workspace_root() -> &'static Path { + Path::new(env!("CARGO_MANIFEST_DIR")).parent().unwrap() +} + +fn cargo() -> Command { + let mut cmd = + Command::new(std::env::var_os("CARGO").unwrap_or_else(|| OsString::from("cargo"))); + for (key, _) in std::env::vars_os() { + let name = key.to_string_lossy(); + let keep = + name == "CARGO_HOME" || name == "CARGO_TARGET_DIR" || name == "CARGO_NET_OFFLINE"; + if name.starts_with("CARGO_") && !keep { + cmd.env_remove(&key); + } + } + cmd +} + +fn cargo_target_dir() -> &'static Path { + static TARGET: OnceLock = OnceLock::new(); + TARGET.get_or_init(|| { + if let Some(given) = std::env::var_os("HTL_E2E_TARGET") { + return PathBuf::from(given); + } + let out = cargo() + .args(["metadata", "--no-deps", "--format-version", "1"]) + .current_dir(workspace_root()) + .stderr(Stdio::inherit()) + .output() + .expect("cargo metadata could not be started"); + assert!(out.status.success(), "cargo metadata: {}", out.status); + let meta: serde_json::Value = + serde_json::from_slice(&out.stdout).expect("cargo metadata did not answer in JSON"); + let dir = meta["target_directory"] + .as_str() + .expect("cargo metadata named no target_directory"); + PathBuf::from(dir).join("e2e-scaffold") + }) +} + +fn run_cargo(project: &Path, args: &[&str]) -> Output { + cargo() + .args(args) + .arg("--target-dir") + .arg(cargo_target_dir()) + .current_dir(project) + .output() + .unwrap_or_else(|e| panic!("cargo {args:?}: could not be started: {e}")) +} + +fn text_of(out: &Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +/// `main.rs` of the host: loads `shop.app` on a runner, calls both functions and prints what +/// they return. `call` is the line that is wrong in the second case. +fn main_rs(call: &str) -> String { + format!( + "htl::teal_types!(); + +#[tokio::main(flavor = \"current_thread\")] +async fn main() {{ + let src = std::path::PathBuf::from(env!(\"CARGO_MANIFEST_DIR\")).join(\"src\"); + let (runner, driver) = htl::run::start(htl::run::RunnerConfig {{ + search_paths: vec![src], + deadline: std::time::Duration::from_secs(30), + queue: 8, + }}) + .await + .unwrap(); + let app = shop::app::App::load(&runner, &[]).await.unwrap(); + {call} + let greeted = app.greet(\"Ada\".to_string(), Some(\"Dr. \".to_string())).await.unwrap(); + println!(\"{{}} {{}}\", greeted.ok, greeted.text); + driver.shutdown().await.unwrap(); +}} +" + ) +} + +#[test] +fn a_host_calls_a_teal_module_through_the_generated_client_and_a_wrong_argument_fails_to_build() { + let dir = common::tempdir("htl-e2e", "exports-client"); + let project = dir.join("host"); + let fixture = workspace_root().join("crates/htl-core/tests/fixtures/client"); + fs::create_dir_all(project.join("src/shop")).unwrap(); + for rel in ["htl.toml", "src/shop/types.tl", "src/shop/app.tl"] { + fs::copy(fixture.join(rel), project.join(rel)).unwrap(); + } + fs::write( + project.join("Cargo.toml"), + format!( + "[package]\n\ + name = \"host\"\n\ + version = \"0.1.0\"\n\ + edition = \"2024\"\n\ + publish = false\n\ + \n\ + [dependencies]\n\ + htl = {{ path = \"{}\", features = [\"async\"] }}\n\ + tokio = {{ version = \"1\", features = [\"rt\", \"macros\"] }}\n", + workspace_root().join("crates/htl").display() + ), + ) + .unwrap(); + fs::copy( + workspace_root().join("Cargo.lock"), + project.join("Cargo.lock"), + ) + .unwrap(); + + // 1. The call Teal declares builds, and returns what the function computed. + fs::write( + project.join("src/main.rs"), + main_rs("let (s, n) = app.split(\"abc\".to_string()).await.unwrap();\n println!(\"{s} {n}\");"), + ) + .unwrap(); + let out = run_cargo(&project, &["run", "-q"]); + assert!(out.status.success(), "cargo run:\n{}", text_of(&out)); + assert_eq!( + String::from_utf8_lossy(&out.stdout), + "abc 3\ntrue Dr. Ada\n", + "{}", + text_of(&out) + ); + + // 2. An integer where Teal says `string` is a type error of the host, not a run-time + // error of the call: rustc's `E0308`, mismatched types. + fs::write( + project.join("src/main.rs"), + main_rs("let (s, n) = app.split(42).await.unwrap();\n println!(\"{s} {n}\");"), + ) + .unwrap(); + let out = run_cargo(&project, &["check", "--message-format=json"]); + assert!( + !out.status.success(), + "expected cargo check to fail:\n{}", + text_of(&out) + ); + let codes: Vec = String::from_utf8_lossy(&out.stdout) + .lines() + .filter_map(|l| serde_json::from_str::(l).ok()) + .filter(|m| m["reason"] == "compiler-message" && m["message"]["level"] == "error") + .filter_map(|m| m["message"]["code"]["code"].as_str().map(str::to_string)) + .collect(); + assert_eq!(codes, ["E0308"], "{}", text_of(&out)); +}