Tags: txn2/mcp-datahub
Tags
ci: roll up dependabot action bumps (#175, #176, #177, #178) (#183) * ci: roll up dependabot action bumps (#175, #176, #177, #178) - github/codeql-action/autobuild 4.36.2 -> 4.37.0 (#175) - docker/setup-buildx-action 4.1.0 -> 4.2.0 (#176) - github/codeql-action/upload-sarif 4.36.2 -> 4.37.0 (#177) - github/codeql-action/analyze 4.36.2 -> 4.37.0 (#178) * ci: bump codeql-action/init to 4.37.0 to match autobuild/analyze All codeql-action sub-actions (init, autobuild, analyze) must run the same version; the init line was left at 4.36.2, causing autobuild to fail with 'Loaded a configuration file for version 4.36.2, but running version 4.37.0'.
feat(client): tag descriptions and custom properties (#173) (#174) * fix(client): correct v3 UPSERT and v1 aspect-read envelope Two defects surfaced by live testing against DataHub v1.6.0: - postAspectV3 wrote with default CREATE semantics, so every OpenAPI v3 read-modify-write (UpdateDescription, AddTag, etc.) returned HTTP 400 ('Cannot perform CREATE since the aspect already exists') whenever the target aspect already existed. Append createIfNotExists=false to force UPSERT, which both creates and updates. - getAspect parsed only the v3 {"value":...} envelope. The legacy v1 Rest.li GET returns {"aspect":{"<FQCN>":...}}, so v1 read-modify-write started from an empty aspect and silently dropped existing tags, terms, links, and required fields on write-back. Parse the Rest.li envelope (tolerating a normalized flat body), covered by a real-envelope test. Validated on DataHub v1.6.0 across both v1 and v3 paths. * feat(client): tag descriptions and custom properties (#173) Add the two capabilities the downstream curation flow needs: - UpdateDescription now accepts tag URNs, routing them through the GraphQL updateDescription mutation (DataHub's UpdateDescriptionResolver handles TAG -> tagProperties.description). Previously tags returned ErrUnsupportedEntityType. - New SetCustomProperties / RemoveCustomProperties client methods write the legacy customProperties map via REST read-modify-write, preserving all other aspect fields. Aspect names verified against upstream PDL; tag is excluded because tagProperties has no customProperties field. Exposed through the datahub_update tool as what=custom_properties (set/remove), keeping the tool count unchanged. Validated end-to-end on DataHub v1.6.0 (v1 and v3 paths).
fix(client): read tags/terms for GraphQL-only entity types (#168) (#169) domain, glossaryTerm, and glossaryNode support writing globalTags and glossaryTerms via the addTag/addTerm GraphQL mutations, but their associations could not be read back: these types do not expose typed tags/glossaryTerms GraphQL fields, and the REST aspect API does not register those aspects for them. GetEntity therefore returned empty tags/terms, breaking post-write verification and rollback before-images for downstream consumers. Populate Tags/GlossaryTerms (by URN) for these three types by reading the globalTags/glossaryTerms raw aspects through the experimental aspects API. The read runs as a separate GetEntityAspectsQuery and degrades gracefully: any failure (older DataHub, unauthorized token) leaves the associations empty and logs at debug level rather than failing the whole GetEntity call, matching the existing graceful fallback pattern used by GetIncidents/GetStructuredProperties. Note: raw aspect payloads carry only URNs, so Name/Description are not populated for these types.
fix(client): return full document projection from GetRelatedDocuments (… …#166) (#167) GetRelatedDocumentsQuery requested a hand-written thin subset (urn, subType, title, contents, status) in each per-entity ... on Document block, omitting settings{showInGlobalContext} and relatedAssets even though the relatedDocumentsResult parse struct and mapping already support them. As a result every related document parsed with Settings == nil, so a visibility-gated consumer treating absent settings as visible (DataHub's documented default) would surface a document a steward explicitly hid via showInGlobalContext=false. The same document fetched via GetDocument/SearchDocuments carried the flag and was correctly suppressed, so documents leaked through this path only. Reuse the shared documentSelectionFields fragment (introduced in #165 for GetDocument/SearchDocuments) in each relatedDocuments block, so related documents carry the identical projection: settings, relatedAssets, status, subType, owners, tags, glossary terms, domain. Add a httptest-mock test asserting a related document returned with showInGlobalContext=false and a relatedAssets entry parses with Settings.ShowInGlobalContext == false and the related-asset URN set.
feat(client): add SearchDocuments for document discovery (#164) (#165) Add Client.SearchDocuments(ctx, query, opts...) to discover context documents by relevance without a known URN, scoped to the DOCUMENT entity type via searchAcrossEntities. A "*" query lists all documents; a text query ranks by relevance. Each result carries the same metadata as GetDocument (URN, title, sub-type, related-asset URNs, showInGlobalContext, ownership, tags, glossary terms, domain). This is a library method, not a new MCP tool: it unblocks txn2/mcp-data-platform#692, which can add a DataHub-documents source to its unified search so context-document knowledge becomes discoverable. - Extract shared documentSelectionFields GraphQL fragment reused by GetDocumentQuery and SearchDocumentsQuery so single-document reads and document search return identical metadata from one source of truth. - Extract Client.buildBaseSearchInput helper shared by doSearchAcrossEntities and SearchDocuments. - Add EntityTypeDocument constant. - Register SearchDocumentsQuery in schema validation. - Cover with GraphQL httptest mock tests (request scoping, result fields incl. governance metadata, wildcard listing, limit capping, error handling).
feat(dataproducts): return constituent datasets from get_data_product (… …#156) datahub_get_data_product promised "constituent datasets" and "member datasets" in its description but never returned them: the getDataProduct query did not request members and the Assets field was left empty. Fetch members via the DataProduct.entities GraphQL resolver in a separate best-effort query (GetDataProductEntitiesQuery). Issuing it separately means a DataHub instance whose schema lacks the resolver returns the product without members rather than failing the whole lookup. The entities argument is SearchAcrossEntitiesInput, whose query field is required, so "*" matches all members; the new query is covered by the schema-validation test against the vendored DataHub schema. Upgrade DataProduct.Assets from []string to []Entity (urn, name, type) to match the already-documented output contract (tools-api.md declared []Entity; the tools.md example showed objects) and update the get_data_product output JSON Schema accordingly. Also pin toolchain go1.26.4 to pick up the patched standard library (clears reachable govulncheck GO-2026-5039 and GO-2026-5037), and handle the strings.Builder write returns in rest.go that the linter flagged.
fix: return empty entities array instead of null on zero search resul… …ts (#131) * fix: return empty entities array instead of null on zero search results Initialize SearchResult.Entities with make([]SearchEntity, 0, ...) so json.Marshal produces "entities": [] instead of "entities": null when there are no results. The OutputSchema declares entities as "type": "array" (no null allowed), so nil slices caused validation errors. Fixed in both doSearchAcrossEntities (keyword + semantic) and Search (legacy client method). Added regression test asserting [] not null. * test: add client-level regression test for nil entities fix Add TestSearchAcrossEntities_ZeroResults_EntitiesNotNil that hits a real httptest server returning zero results and asserts Entities is non-nil and marshals to [] not null. This exercises the actual make() fix in doSearchAcrossEntities rather than relying on the mock to pre-initialize the slice.
feat: upgrade datahub_search to use searchAcrossEntities with filters (… …#130) * feat: upgrade datahub_search to use searchAcrossEntities with filters (#129) Replace the type-scoped `search` GraphQL query with `searchAcrossEntities` for keyword mode, enabling advanced field-level filtering and multi-type search while maintaining full backward compatibility. New SearchInput fields: - `types`: search across multiple entity types (overrides entity_type) - `filters`: advanced field-level filters (fieldPaths, fieldTags, platform, owners, domains, glossaryTerms, etc.) that are AND'd together New client API: - `SearchFilter` type with Field, Values, Condition, Negated - `WithTypes()` and `WithOrFilters()` search options - `SearchAcrossEntities()` client method with full entity fragments * fix: address review findings for searchAcrossEntities PR - SearchAcrossEntities now falls back to entityType when types is empty - SemanticSearch now supports WithTypes and WithSearchFilters options - SemanticSearchQuery gains DataFlow, Tag, Document entity fragments (parity with SearchAcrossEntitiesQuery and SearchQuery) - Rename WithOrFilters to WithSearchFilters (clearer: filters are AND'd) - convertFilters merges both Value and Values instead of dropping Value - Add validateFilters rejecting empty Field or empty Values - Add tests for all fixes: entityType fallback, types override, semantic+filters, Value+Values merge, validation errors * refactor: extract shared search helper, eliminate query duplication - Extract doSearchAcrossEntities shared helper; SearchAcrossEntities and SemanticSearch are now one-liner delegates (only difference: fulltext flag) - SemanticSearchQuery is now an alias for SearchAcrossEntitiesQuery — single GraphQL query constant with all entity fragments - Add DefaultEntityType constant to satisfy goconst lint - Deduplicate values in convertFilters when Value is already in Values - Add test for client no-types path (all-type search sends no types key) - Add test for Value+Values deduplication - Update CLAUDE.md datahub_search description for new capabilities
fix: default context documents to visible and published (#122) (#123) Documents created via UpsertContextDocument and datahub_create were invisible in the DataHub UI because GlobalContext defaulted to false (Go zero value) and Status was empty (server default: UNPUBLISHED). Changes: - context_documents.go: set GlobalContext: true in createContextDocument - write_create.go: default Status to "PUBLISHED" when not provided, default GlobalContext to true (use *bool to allow explicit false override) Closes #122
fix: align GraphQL queries with upstream DataHub schema (#121) Validated all 59 GraphQL query/mutation constants against the upstream DataHub schema files (datahub-project/datahub v1.5.0.1) and fixed field paths that did not match the actual API: - documents.go: relatedAssets/relatedDocuments/parentDocument use wrapper objects (asset/document) rather than direct urn fields - structured_properties.go: fragment target type is StructuredProperties - data_contracts.go: DataContract uses properties/status, not result() - semantic_search.go: use searchAcrossEntities with SearchAcrossEntitiesInput Adds schema validation infrastructure to prevent future drift: - testdata/datahub-schema/: 31 .graphql files from upstream v1.5.0.1 - testdata/datahub-schema/sync.sh: downloads schema for any tagged version - schema_validation_test.go: validates all queries against schema files - make schema-sync / make schema-check (included in make verify) Updates CLAUDE.md with version compatibility matrix and schema workflow.
PreviousNext