Skip to content

peterswimm/msr_connectors

Repository files navigation

MSR Connectors

CI License: MIT

Overview

A monorepo of Microsoft 365 Copilot connectors that bring MSR Event Agent content into the Microsoft 365 ecosystem. The repository contains two complementary connectors that share a common set of packages:

  • Search Connector — A Microsoft Graph external connector that indexes MSR content (events, sessions, people, publications, projects, and videos) into Microsoft Graph for offline search and Copilot reasoning via Microsoft Search.
  • Knowledge Connector — A Copilot declarative agent plugin that proxies live API queries against the MSR Event Agent API, providing real-time grounding for Microsoft 365 Copilot without pre-indexing.

Both connectors consume the MSR Event Agent API as their data source.

Architecture

                        ┌────────────────────────┐
                        │    MSR Event Agent API  │
                        │  (events, sessions,     │
                        │  people, publications,  │
                        │  projects, videos)      │
                        └───────────┬────────────┘
                                    │
              ┌─────────────────────┼─────────────────────┐
              │                                           │
              ▼                                           ▼
┌──────────────────────────┐              ┌──────────────────────────┐
│   @msr-connectors/       │              │   @msr-connectors/       │
│   agent-api-client       │              │   agent-api-client       │
│   (typed HTTP client)    │              │   (typed HTTP client)    │
└────────────┬─────────────┘              └────────────┬─────────────┘
             │                                         │
             ▼                                         ▼
┌──────────────────────────┐              ┌──────────────────────────┐
│   search-connector       │              │   knowledge-connector    │
│   + graph-connector-core │              │   (declarative agent /   │
│   (batch indexing)       │              │    API plugin)           │
└────────────┬─────────────┘              └────────────┬─────────────┘
             │                                         │
             ▼                                         ▼
┌──────────────────────────┐              ┌──────────────────────────┐
│   Microsoft Graph        │              │   Microsoft 365 Copilot  │
│   External Connections   │              │   (live query grounding) │
└────────────┬─────────────┘              └──────────────────────────┘
             │
             ▼
┌──────────────────────────┐
│   Microsoft Search /     │
│   Microsoft 365 Copilot  │
│   (indexed content)      │
└──────────────────────────┘

Repository Structure

msr-connectors/
├── apps/
│   ├── search-connector/              # Graph external connector (indexing + schema)
│   │   ├── src/
│   │   │   ├── schema/
│   │   │   │   ├── msr-schema.ts      # 19-property unified schema definition
│   │   │   │   ├── create-connection.ts
│   │   │   │   └── delete-connection.ts
│   │   │   ├── ingestion/
│   │   │   │   ├── run-ingestion.ts   # Full/incremental content ingestion
│   │   │   │   └── content-builder.ts # Maps API models → Graph external items
│   │   │   ├── dev-server.ts          # Express dev server with Swagger UI
│   │   │   └── openapi-spec.ts
│   │   └── .env.example
│   └── knowledge-connector/           # Copilot declarative agent plugin
│       ├── src/
│       │   └── api/
│       │       └── server.ts          # Express API server (proxy to Agent API)
│       ├── appPackage/
│       │   ├── declarativeAgent.json  # Agent manifest (v1.3)
│       │   ├── api-plugin.json        # Plugin definition (v2.2)
│       │   └── openapi.yaml           # OpenAPI spec for Copilot
│       └── .env.example
├── packages/
│   ├── graph-connector-core/          # Shared Microsoft Graph utilities
│   │   └── src/
│   │       ├── graph-client.ts        # Authenticated Graph client factory
│   │       └── throttle.ts            # Retry/throttle, semaphore, payload checks
│   └── agent-api-client/              # Typed MSR Event Agent API client
│       └── src/
│           ├── client.ts              # AgentApiClient class
│           └── types.ts               # MsrEvent, MsrSession, MsrPerson, etc.
├── scripts/
│   └── validate-connectors.ts         # 18 pre-flight validation checks
├── docs/
│   └── deployment.md                  # ALM-style deployment guide
├── .github/
│   └── workflows/
│       ├── ci.yml                     # Build, test, lint, validate on every PR
│       ├── validate-pr.yml            # Schema change detection, manifest checks
│       └── deploy.yml                 # Environment-gated deployment
├── package.json                       # npm workspace root
├── tsconfig.base.json                 # Shared TypeScript configuration
└── vitest.workspace.ts                # Vitest workspace test configuration

Prerequisites

Requirement Details
Node.js >= 20.0.0
npm >= 9 (ships with Node 20+)
Azure AD app registration With the following Microsoft Graph application permissions: ExternalConnection.ReadWrite.OwnedBy, ExternalItem.ReadWrite.OwnedBy
Admin consent A tenant administrator must grant consent for the Graph permissions above
MSR Event Agent API Access to the MSR Event Agent data service (base URL and API key)

Getting Started

1. Clone and install

git clone https://github.com/microsoft/msr-connectors.git
cd msr-connectors
npm install

2. Configure environment variables

cp apps/search-connector/.env.example apps/search-connector/.env
cp apps/knowledge-connector/.env.example apps/knowledge-connector/.env

Edit each .env file with your Azure AD credentials and Agent API details. See Environment Variables for the full reference.

3. Build and validate

npm run build
npm run typecheck
npm run test
npm run validate

4. Run dev servers

# Search Connector — http://localhost:8081
cd apps/search-connector && npm run dev

# Knowledge Connector — http://localhost:8080
cd apps/knowledge-connector && npm run dev

Both connectors serve a Swagger UI at /docs for interactive exploration:

5. Create a Graph connection

cd apps/search-connector
npm run schema:create

This registers the external connection and 19-property schema with Microsoft Graph.

6. Run content ingestion

cd apps/search-connector
npm run ingest

This fetches content from the MSR Event Agent API and pushes it to Microsoft Graph as external items.

Packages

@msr-connectors/graph-connector-core

Shared library for Microsoft Graph external connector operations. Wraps the @microsoft/microsoft-graph-client and @azure/identity SDKs with connector-specific utilities.

Export Description
createGraphClient(options?) Creates an authenticated Microsoft Graph client using client credentials (tenant ID, client ID, client secret)
throttledRequest<T>(fn) Executes a Graph API request with automatic retry on HTTP 429 (throttle) and 503 (service unavailable) responses
createSemaphore(concurrency) Creates a concurrency-limiting semaphore for controlling parallel Graph API requests during ingestion
checkPayloadSize(payload) Validates that an external item payload does not exceed the Graph API size limit before submission

@msr-connectors/agent-api-client

Typed HTTP client for the MSR Event Agent API. Provides strongly typed request/response models, pagination support, and error mapping.

Export Description
AgentApiClient Main client class — methods for querying events, sessions, people, publications, projects, and videos
MsrEvent Event data model (name, dates, location, tags, research areas)
MsrSession Session data model (speakers, start/end time, room, topic)
MsrPerson Person data model (name, title, bio, research areas, lab)
MsrPublication Publication data model (title, authors, DOI, venue, citation count)
MsrProject Project data model (title, status, research areas, impact themes)
MsrVideo Video data model (title, URLs, research areas, video types)
ApiListResponse<T> Paginated response envelope (items, totalCount, nextLink)
ContentDomain Union type: "events" | "sessions" | "people" | "publications" | "projects" | "videos"
QueryParams Query parameter interface (fields, top, skip, filter, q, limit)

Apps

search-connector

Creates a Microsoft Graph external connection, registers a unified schema, and ingests MSR Event Agent content as external items. Indexed content surfaces in Microsoft Search results and Microsoft 365 Copilot reasoning.

The schema covers 6 content types with 19 searchable/retrievable properties, enabling rich filtering and refinement across all MSR content domains.

Available scripts:

Script Command Description
dev npm run dev Start Express dev server with hot reload (port 8081)
build npm run build Compile TypeScript to dist/
ingest npm run ingest Run full content ingestion from the Agent API
schema:create npm run schema:create Create the Graph external connection and register the schema
schema:delete npm run schema:delete Delete the Graph external connection
test npm run test Run unit tests

Dev server endpoints:

Endpoint Description
GET /docs Swagger UI
GET /api/openapi.json OpenAPI specification
GET /health Health check
GET /admin/status Connection config and environment variable status
GET /admin/schema Current schema definition and property count

knowledge-connector

A Microsoft 365 Copilot plugin implemented as a declarative agent (v1.3 manifest). Proxies natural-language queries from Copilot to the MSR Event Agent API in real time, enabling Copilot to ground responses with live data without requiring content to be pre-indexed.

The agent is configured via the appPackage/ directory:

File Description
declarativeAgent.json Agent manifest — defines the "MSR Event Agent Assistant" with Graph connector capabilities and plugin actions
api-plugin.json Plugin definition (v2.2) — maps Copilot functions (search_events, get_people, get_publications, etc.) to API operations
openapi.yaml OpenAPI specification consumed by the Copilot runtime

Available scripts:

Script Command Description
dev npm run dev Start Express API server with hot reload (port 8080)
build npm run build Compile TypeScript to dist/
start npm run start Start the production server
test npm run test Run unit tests

Local Development

Environment Variables

search-connector

Variable Description Default
AZURE_TENANT_ID Azure AD tenant ID (required)
AZURE_CLIENT_ID Azure AD app registration client ID (required)
AZURE_CLIENT_SECRET Azure AD app registration client secret (required)
CONNECTION_ID Graph external connection identifier msrApiEventContent
CONNECTION_NAME Human-readable connection name MSR Event Agent Content
CONNECTION_DESCRIPTION Connection description shown in Microsoft Search (auto-generated)
DATA_API_URL MSR Event Agent API base URL https://msr-data-service-api-dev.azurewebsites.net
INGESTION_CONCURRENCY Simultaneous Graph API requests during ingestion 4

knowledge-connector

Variable Description Default
PORT API server listen port 8080
DATA_API_URL MSR Event Agent API base URL https://msr-data-service-api-dev.azurewebsites.net
AZURE_TENANT_ID Azure AD tenant ID (for request validation) (required)
AZURE_CLIENT_ID Azure AD app registration client ID (required)
COPILOT_ALLOWED_AUDIENCES Allowed token audiences for incoming Copilot requests (required)

Running the Connectors

# Terminal 1 — Search Connector (http://localhost:8081)
cd apps/search-connector
npm run dev

# Terminal 2 — Knowledge Connector (http://localhost:8080)
cd apps/knowledge-connector
npm run dev

Validation

# Run all 18 pre-flight checks
npm run validate

# CI mode — treats warnings as failures
npm run validate:ci

Testing

# Run the full test suite
npm run test

# Watch mode
npm run test:watch

Tests use Vitest and are co-located with source files.

Type Checking

npm run typecheck

Connector Validation

The validate-connectors.ts script runs 18 pre-flight checks across three categories before deployment:

Search Connector — Schema integrity checks:

  • All searchable properties have descriptions
  • All properties have descriptions
  • Recommended semantic labels are applied (title, url, iconUrl, lastModifiedDateTime, createdBy, authors)
  • No duplicate property names
  • All property types are valid Graph types (String, DateTime, Boolean, Int64, Double, StringCollection)
  • Connection ID matches the required pattern (^[a-zA-Z][a-zA-Z0-9]{2,31}$)
  • Content builder exports cover all 6 content types

Knowledge Connector — Manifest and API checks:

  • Declarative agent manifest validates against the v1.3 schema
  • GraphConnectors capability references the correct connection ID
  • Actions reference points to a valid api-plugin.json
  • API plugin has a valid runtime configuration
  • Plugin function names match runtime definitions
  • OpenAPI operation IDs are unique
  • Required parameters are documented
  • OpenAPI paths match Express server routes

Cross-Connector — Consistency checks:

  • Connection ID in declarativeAgent.json matches the search connector schema
  • search-connector/.env.example documents all required variables
  • knowledge-connector/.env.example documents all required variables

Run in CI mode (npm run validate:ci) to treat warnings as failures.

CI/CD

The repository includes three GitHub Actions workflows:

Workflow Trigger Purpose
CI (ci.yml) Push / PR to main Builds, typechecks, lints, tests, and validates across Node.js 20.x and 22.x. Includes CodeQL security analysis.
Validate PR (validate-pr.yml) PR to main Runs connector validation, detects schema file changes (with automatic PR comment), and validates JSON/YAML manifests in appPackage/.
Deploy Connectors (deploy.yml) Manual (workflow_dispatch) Environment-gated deployment (dev / staging / production) with actions: deploy-all, schema-only, ingest-only, knowledge-connector-only, validate-only. Uses Azure federated identity (OIDC).

See docs/deployment.md for the full deployment guide, including Azure federated identity setup, environment configuration, and rollback procedures.

Schema Reference

The search connector registers a unified schema with 19 properties across all content types:

Common Properties

Property Type Searchable Retrievable Queryable Refinable
contentType String Yes Yes Yes
title String Yes Yes Yes
description String Yes
url String Yes
iconUrl String Yes
lastModifiedDateTime DateTime Yes Yes Yes
createdBy String Yes Yes Yes

Event Properties

Property Type Searchable Retrievable Queryable Refinable
startDate DateTime Yes Yes Yes
endDate DateTime Yes Yes
location String Yes Yes Yes

Session Properties

Property Type Searchable Retrievable Queryable Refinable
speakers StringCollection Yes Yes Yes
startTime DateTime Yes Yes Yes

Person Properties

Property Type Searchable Retrievable Queryable Refinable
personName String Yes Yes Yes
role String Yes Yes Yes
researchAreas StringCollection Yes Yes Yes

Publication Properties

Property Type Searchable Retrievable Queryable Refinable
authors StringCollection Yes Yes Yes
publicationDate DateTime Yes Yes Yes

Project and Video Properties

Property Type Searchable Retrievable Queryable Refinable
tags StringCollection Yes Yes Yes
status String Yes Yes Yes

Content Types

The MSR Event Agent API exposes six content domains, each mapped to a contentType value in the unified schema:

Content Type Description
event Research conferences, workshops, symposiums, and other organized events
session Individual talks, panels, keynotes, and presentations within events
person Researchers, speakers, and organizers with profiles and research areas
publication Academic papers, technical reports, and other research publications
project Research projects with status tracking, areas, and impact themes
video Recorded talks, demos, and other video content from events

Related Projects

Contributing

We welcome contributions! Please review the following before submitting a pull request:

  1. Read and follow the Code of Conduct
  2. Fork the repository and create a feature branch from main
  3. Use descriptive branch names: feature/add-incremental-sync, fix/retry-on-throttle
  4. Write or update tests for your changes
  5. Ensure all checks pass:
    npm run build && npm run typecheck && npm run test && npm run validate
  6. Open a pull request with a clear description of the changes

Security

For security concerns, please see SECURITY.md. Do not open a public GitHub issue for security vulnerabilities.

License

This project is licensed under the MIT License.

About

Microsoft 365 Copilot Connectors for MSR Event Agent API

Resources

License

Code of conduct

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors