feat(tools): add surveys tools #117
Conversation
joshsny
left a comment
There was a problem hiding this comment.
This is looking great! Will give it a test run now locally - left some comments, main question is whether the LLM handles the input complexity well enough
joshsny
left a comment
There was a problem hiding this comment.
This is looking great! Will give it a test run now locally - left some comments, main question is whether the LLM handles the input complexity well enough
|
Did some testing locally, a few thoughts:
Let me know if you want any more testing :) |
ioannisj
left a comment
There was a problem hiding this comment.
Hey @joshsny, thanks for reviewing this. I tried to address all the comments here.
Regarding the following
Did some testing locally, a few thoughts:
- The stats tools were not returning results for me, just the survey name, id and some other details, not sure if I was doing this wrong?
This is based on responses, so for a fresh survey with no responses I guess it's because the response counts were zero? Tried running it for a survey with some responses and got some stats back
- Surveys list works great
- I got 'Permission Denied' whilst trying to use the create survey tool, I think it might just be struggling to get the input right as it's quite complex, I think we might need to simplify inputs here
- Same for update, we probably need to simplify the input schema the same as the create one
When I was testing, Permission Denied was usually trying to create or update a survey to a project that was not in my Personal API key. I'm attaching some screenshots below with some prompts I tried. Looks like it's handing it okay but tbh I have no idea how to test this thoroughly.
This prompt here created the survey correctly with
- Reasonable title and theme
- Conditional logic on detractors with a reasonable follow up question
- Correct display conditions
Then the follow-up prompt to link this to a feature flag seemed to work okay as well
Not sure if my prompts are just a bit too specific and giving it too much guidance though.
|
Awesome - thanks again for this @ioannisj - looks great! Since you are on paternity from Monday, I can merge this in next week - have a great time! |
|
@joshsny when can we get this merged so i can test some changes/fixes on top of this? |
|
@marandaneto I'll be looking at this today, once I am finished reviewing the experiments implementation - sorry for the delay! |
joshsny
left a comment
There was a problem hiding this comment.
This should be ready to go now
There was a problem hiding this comment.
Pull Request Overview
This PR adds a comprehensive set of survey tools to the MCP toolkit, enabling CRUD operations and analytics for PostHog surveys. The implementation includes proper TypeScript schemas, API client integration, and extensive test coverage.
- Creates 7 new survey tools for complete lifecycle management (create, read, update, delete, stats)
- Adds comprehensive survey schema definitions with support for various question types, branching logic, and targeting
- Implements proper test coverage with integration tests covering all survey workflows
Reviewed Changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| typescript/src/tools/surveys/*.ts | Survey tool implementations for CRUD operations and statistics |
| typescript/src/schema/surveys.ts | Comprehensive survey schema definitions with question types and validation |
| typescript/src/api/client.ts | API client methods for survey operations |
| typescript/tests/tools/surveys.integration.test.ts | Extensive integration tests for survey tools |
| typescript/tests/shared/test-utils.ts | Test utilities updated to support survey cleanup |
| schema/tool-definitions.json | Tool definitions for survey operations |
| schema/tool-inputs.json | JSON schema definitions for survey inputs |
| python/schema/tool_inputs.py | Python schema definitions for survey inputs |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Description
Towards #87
Adds a set of survey-related tools:
survey-create: Creates a new survey in the project.survey-get: Get a specific survey by ID.surveys-get-all: Get all surveys in the project with optional filtering.survey-update: Update an existing survey by ID.survey-delete: Delete a survey by ID (soft delete - marks as archived.Plus:
surveys-global-stats: Get aggregated response statistics across all surveys in the project.survey-stats: Get response statistics for a specific survey.This is just CRUD operations mostly. Much more valuable response analysis tools will soon come from @marandaneto
Tried to follow tools documentation. Fair warning: this is my first time touching these tools and language, so excuse any naive bits of code
Tests?