chore(poc): explore intents #833
Conversation
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
ByronDWall
left a comment
There was a problem hiding this comment.
A super interesting pattern. Really similar to my hacky event listener implementation but also with ai.
We should definitely explore this and other client->server->model data flows more
4b85076 to
06dde3a
Compare
06dde3a to
653293c
Compare
| type?: "button" | "submit" | "reset"; | ||
| ariaLabel?: string; | ||
| /** Optional intent to emit when button is pressed */ | ||
| intent?: Intent; |
There was a problem hiding this comment.
@ByronDWall It is powerful to give the agent complete control over the intent.
One of the issues I've been grappling with lately is how much control we exert over these UI templates when it comes to statefulness, that is, what priority data do we attach to intent handlers (product name, id, sku, etc)? I don't know that we can make this call in all cases, when we as developers won't have full conversation context. So, I chose to experiment by leaning into the inverse of this to give agents more control (continued below)
| buttonLabel: z | ||
| .string() | ||
| .describe( | ||
| "Label text for the action button. Choose based on the intent type and user context." | ||
| ), | ||
| buttonIntent: z | ||
| .object({ | ||
| type: z | ||
| .string() | ||
| .describe( | ||
| "Intent type identifier in underscore_case. Choose based on what makes sense for the user's query context." | ||
| ), | ||
| description: z | ||
| .string() | ||
| .describe( | ||
| "Human-readable description of what the user wants to do when clicking this button. Consider the current conversation context and what logical next step." | ||
| ), | ||
| payload: z | ||
| .record(z.any()) | ||
| .describe( | ||
| "Structured data payload for this intent. Include all relevant entity information." | ||
| ), | ||
| }) | ||
| .describe( | ||
| "Intent configuration for the product card's action button. You decide what this button should do based on the user's query context and next logical steps." | ||
| ), |
There was a problem hiding this comment.
@ByronDWall Here, the agent has complete control over the type of the intent, description, and payload. This has a few benefits:
- The agent already has user query and conversation context available to it, so it's more informed than we are
- The agent can suggest next steps for a flow, which is a powerful way to steer users in novel directions
- Reduced client/server coupling to predefined intents, reducing our maintenance burden
This also has a few drawbacks:
- Our templating becomes less deterministic
- Few guards against hallucinations (but I remain positive here, as I believe the agent should be able to recover with sufficient type, description, and payload content)
As with most things, we can use the best of both worlds. With this strategy, we can offer a dynamic avenue for agent-steered suggestion. For templates that require a high level of determinism, we can create hardcoded intents with stricter data boundaries.
| buildButtonElement({ | ||
| label: "Add to Cart", | ||
| variant: "solid", | ||
| label: buttonLabel, |
There was a problem hiding this comment.
Along with the agent steering the intent, the agent also chooses the button text
| console.log("🎯 Intent received:", intent); | ||
|
|
||
| // Combine description + structured payload for Claude | ||
| const message = `${intent.description} |
There was a problem hiding this comment.
We don't have to show a message to the user. It's just here for DX
|
|
||
| // Intercept form submissions globally | ||
| useEffect(() => { | ||
| const handleFormSubmit = (event: SubmitEvent) => { |
There was a problem hiding this comment.
This was competing with the intents, so it was removed
| const formElement = document.querySelector("form"); | ||
| if (formElement) { | ||
| console.log("📝 Button is in a form, capturing form data..."); | ||
|
|
||
| const formData = new FormData(formElement); | ||
| const data: Record<string, unknown> = {}; | ||
|
|
||
| // Extract all form fields | ||
| formData.forEach((value, key) => { | ||
| // Handle file inputs specially (can't serialize File objects) | ||
| if (value instanceof File) { | ||
| data[key] = { | ||
| fileName: value.name, | ||
| fileSize: value.size, | ||
| fileType: value.type, | ||
| }; | ||
| } else { | ||
| data[key] = value; | ||
| } | ||
| }); | ||
|
|
||
| // Enhance intent with captured form data | ||
| const enhancedIntent = { | ||
| ...intent, | ||
| payload: { | ||
| ...intent.payload, | ||
| formData: data, | ||
| }, | ||
| }; |
There was a problem hiding this comment.
If we have a form, capture the uncontrolled form data, and attach it to the intent. This then gets sent to Claude so he can do with it what he should
| ? `IMPORTANT PERSONA: You are an administrator and analyst for internal merchant tooling, NOT a shopper or consumer. You are a business-user tooling power user of the highest order. | ||
| ALWAYS provide UI components that relate to administration, modification, analysis, or similar, for requested entities. |
There was a problem hiding this comment.
This reliably modified the chat to be more MC-like, that is, focused on the administrator experience rather than the shopper.
@ByronDWall Something I'd like to do is experiment with toggling personas off and on in the chat window to see how it affects the experience, would be neat!
| 🔥 CRITICAL: Form Data Capture | ||
| When creating buttons inside forms: | ||
| - Form data is AUTOMATICALLY captured when any button with an intent is clicked inside a form | ||
| - You don't need to set type="submit" - type="button" works fine | ||
| - The intent will receive a formData object with all field name/value pairs in the payload | ||
| - Example: createButton({ label: "Update Product", intent: { type: "update_product", description: "...", payload: { productId: "123" } } }) | ||
| - When clicked inside a form, Claude receives: { productId: "123", formData: { productName: "New Name", ... } } | ||
There was a problem hiding this comment.
I'm not convinced that we need stuff like this if we solve it in the application itself. Might remove this
| name: args.name, | ||
| placeholder: args.placeholder, | ||
| value: args.defaultValue, | ||
| defaultValue: args.defaultValue, |
There was a problem hiding this comment.
We need to use uncontrolled values.
| .optional() | ||
| .describe( | ||
| "Button type for HTML forms (default: 'button'). Use 'submit' for form submission buttons." | ||
| "Button type for HTML forms (default: 'button'). Note: Form data capture is automatic - any button with an intent inside a form will automatically capture form field values, regardless of type." |
There was a problem hiding this comment.
I'm also not convinced we need application implementation details described at the schema level, but it's hard to reliably test how this affects the chat because it's so nondeterministic
This is rad
See this article which is the only place I could find an example of the intents system.
This is a powerful paradigm which allows us to cut scope on creating event handlers while also leveraging agents for stateful interactions.