Skip to content

dsl: a {{!input.X}} bang ref in a prompt compiles, and the model reads the literal placeholder #1775

Description

@devthejo

Surfaced by the adversarial round on #1740's guard. Latent: no catalogue prompt uses the form today (grep -rn '{{!' bots/ → 0).

The mismatch

pkg/dsl/ir/ref.go:52-58 documents the bang prefix as shell-only:

The flag has no effect on non-shell template contexts (prompts, edge data mappings) — they always render values via formatValue.

The prompt renderer does not do that. TemplateResolver.Resolve hands !input.a to ResolveValue, which splits it into namespace !input, finds nothing, and the placeholder stays in the text verbatim. Probed against the real resolver:

"x={{input.a}}"     -> "x=VALUE_OF_A"
"x={{!input.a}}"    -> "x={{!input.a}}"
"x={{ !input.a }}"  -> "x={{ !input.a }}"

So a prompt containing {{!input.X}} compiles without a diagnostic — ParseRefs strips the bang and returns a normal RefInput — and the model reads the literal placeholder instead of the value.

Why it matters beyond the render

Anything that reasons about a prompt from its compiled TemplateRefs sees input.X as rendered, because the IR and the renderer disagree. The first version of #1740's guard did exactly that and would have passed a review_input field rendered as {{!input.build_skipped}} — the #1598 shape, green. The guard now ignores unquoted refs; the disagreement it works around is still here.

What would close it — one of

  • render it: strip the bang in the prompt/mapping resolver, so the code matches the comment; or
  • refuse it where it cannot mean anything: a C1xx warning when a {{!…}} ref sits in a prompt body or an edge mapping, naming it shell-only.

Either way, the IR and the renderer must agree on what a prompt shows. Whichever lands, bots/llm_input_reaches_its_prompt_test.go should follow: today it treats an unquoted ref as NOT consumed, which is true only while the renderer leaves it verbatim.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions