Installation Validation: Outlook Calendar#
System Criticality: Business-Critical
Validation Approach: Risk-based
Related Design: DESIGN:OutlookCalendar
Related URS: @URS:OutlookCalendar
Related Risks: RISK:OutlookCalendar
Overview#
This Installation Validation (IV) plan validates the Outlook Calendar MCP server before production deployment. The system provides read-only access to Microsoft 365 calendar events via Microsoft Graph API, using OAuth2 On-Behalf-Of (OBO) authentication.
Key Validation Objectives: 1. Verify user-scoped calendar access (no cross-user data leakage) 2. Confirm read-only enforcement (no create, update, or delete operations) 3. Validate calendar privacy preservation (no event data in logs or DynamoDB) 4. Ensure OBO identity preservation (correct user attribution) 5. Verify authentication enforcement (unauthenticated requests rejected)
Test Strategy#
| Level | Purpose | Owner | When | Coverage |
|---|---|---|---|---|
| Unit | Individual function logic (GraphCalendarClient methods) | Developers | During dev | Python pytest with mocks |
| Integration | OBO flow + Graph API interaction | QA | After unit tests | Dev environment, synthetic test users |
| System | End-to-end MCP tool calls | QA | Before UAT | Staging environment, realistic scenarios |
| Security | Authorization, token handling, privacy | Security | Before go-live | Staging + penetration testing |
| UAT | Business validation by end users | Product Owner | Before prod deploy | Production-like data |
Traceability Matrix#
| URS Scenario | Summary | Risk | Test Case | Type | Status |
|---|---|---|---|---|---|
| @Calendar @ListEvents | List events in date range | R1, R2, R3 | TEST-001 | System | Not Started |
| @Calendar @GetEvent | Get event details by ID | R1, R2, R3 | TEST-002 | System | Not Started |
| @Calendar @MultiDay | List events spanning multiple days | R6 | TEST-003 | System | Not Started |
| @Calendar @ListCalendars | List user calendars | R1, R3 | TEST-004 | System | Not Started |
| @Calendar @Timezone | Handle timezone-aware events | R6 | TEST-005 | System | Not Started |
| @Calendar @OnlineMeetings | Access online meeting details | R1, R2 | TEST-006 | System | Not Started |
| @Security @UserScoped | Enforce user-scoped access | R3 | TEST-SEC-001 | Security | Not Started |
| @Compliance @ReadOnly | Read-only access enforcement | R1, R2 | TEST-SEC-002 | Security | Not Started |
| @Compliance @PrivacyPreservation | No event data in logs/storage | R2, R4 | TEST-SEC-003 | Security | Not Started |
| N/A (authentication) | Token validation | R1 | TEST-SEC-004 | Security | Not Started |
Coverage: 10/9 URS scenarios = 111% (includes 1 additional security test for token expiration) Target: 100% of URS scenarios must have corresponding test cases
Test Cases#
TEST-001: List Calendar Events in Date Range#
Related: URS @Calendar @ListEvents | RISK-R1, RISK-R2, RISK-R3 Risk Level: HIGH
Preconditions:
- Dev/staging environment running at outlook.dev.connectors.novo-genai.com
- Test user authenticated with Azure AD (valid OBO token)
- Test user has at least 3 calendar events between 2026-04-22 and 2026-04-30
Steps:
1. Call MCP tool list_events with parameters:
- start_date: "2026-04-22T00:00:00Z"
- end_date: "2026-04-30T23:59:59Z"
- count: 50
2. Inspect response structure
3. Verify each event includes required fields
4. Check CloudWatch logs for event content leakage
Expected:
- HTTP 200 response from MCP server
- Response contains list of events (≥3 events if test data present)
- Each event includes: id, subject, start, end, location, organizer, attendees, isAllDay
- No events from other users present in response
- CloudWatch logs contain event count and date range, but NOT event subjects, locations, or attendee names
Pass Criteria: - All expected fields present in response - Event data matches Microsoft 365 calendar for authenticated test user - No cross-user data leakage (verified by comparing against second test user's calendar) - No sensitive data in CloudWatch logs (grep for test event subject lines returns zero matches)
TEST-002: Get Specific Calendar Event Details#
Related: URS @Calendar @GetEvent | RISK-R1, RISK-R2, RISK-R3 Risk Level: HIGH
Preconditions: - Dev/staging environment running - Test user authenticated - Event ID obtained from TEST-001 (or known test event ID)
Steps:
1. Call MCP tool get_event with parameters:
- event_id: "<valid-event-id>"
- timezone: "Europe/Copenhagen" (optional)
2. Inspect response structure and content
3. Verify detailed fields present (body, recurrence, onlineMeeting, etc.)
4. Check CloudWatch logs for event body or attendee data
Expected:
- HTTP 200 response
- Response includes: id, subject, start, end, location, body, attendees, organizer, isAllDay, recurrence, onlineMeeting
- Times displayed in requested timezone (if provided)
- No event content logged to CloudWatch
Pass Criteria: - All extended fields present (body, recurrence, onlineMeeting) - Timezone correctly applied if specified - Event data matches Microsoft 365 UI for same event - CloudWatch logs contain event ID and operation type, but NOT body content or attendee details
TEST-003: List Events Spanning Multiple Days#
Related: URS @Calendar @MultiDay | RISK-R6 Risk Level: MEDIUM
Preconditions: - Test user has a multi-day event (e.g., conference from 2026-05-01 to 2026-05-03) - Test user has a recurring daily event (e.g., standup every weekday at 09:00)
Steps:
1. Call list_events with date range covering the multi-day and recurring events:
- start_date: "2026-05-01T00:00:00Z"
- end_date: "2026-05-10T23:59:59Z"
2. Inspect returned events for multi-day event
3. Verify recurring event appears once per day within range
Expected: - Multi-day event appears once (not duplicated per day) - Recurring event appears as 8 separate instances (Mon-Fri in week 1, Mon-Wed in week 2) - All times correctly preserved (no timezone conversion errors)
Pass Criteria:
- Multi-day events correctly represented (single entry with start/end spanning days)
- Recurring events expanded correctly by Microsoft Graph API (via calendarView endpoint)
- No duplicate or missing occurrences
TEST-004: List User Calendars#
Related: URS @Calendar @ListCalendars | RISK-R1, RISK-R3 Risk Level: MEDIUM
Preconditions: - Test user has primary calendar + at least one shared calendar (e.g., team calendar)
Steps:
1. Call MCP tool list_calendars (no parameters)
2. Inspect response for primary and shared calendars
3. Verify each calendar includes: id, name, color, isDefaultCalendar, owner
Expected: - HTTP 200 response - Response includes primary calendar (isDefaultCalendar = true) - Response includes shared calendars (if user has access) - No calendars from other users visible unless explicitly shared
Pass Criteria: - All accessible calendars returned - Primary calendar correctly identified - Shared calendar ownership correctly attributed (owner field shows original owner, not authenticated user) - No unauthorized calendar access (second test user cannot see first user's private calendar)
TEST-005: Handle Timezone-Aware Events#
Related: URS @Calendar @Timezone | RISK-R6 Risk Level: MEDIUM
Preconditions: - Test event created with explicit timezone: "Project Kickoff" on 2026-05-15 14:00-15:00 CET (Europe/Copenhagen)
Steps:
1. Call get_event with event ID, no timezone parameter (should fall back to user's mailbox timezone)
2. Call get_event with event ID, timezone: "America/New_York"
3. Call get_event with event ID, timezone: "UTC"
4. Compare times across responses
Expected: - Step 1: Event time returned in user's mailbox timezone (likely CET) - Step 2: Event time returned as 08:00-09:00 EDT (CET 14:00 = EDT 08:00) - Step 3: Event time returned as 12:00-13:00 UTC (CET 14:00 = UTC 12:00) - Timezone field explicitly included in start/end objects
Pass Criteria:
- Times correctly converted by Microsoft Graph API (no client-side conversion)
- Prefer: outlook.timezone header correctly sent to Graph API (verified in debug logs or network trace)
- No timezone parsing errors in Python code
- DST transitions handled correctly (test event on DST boundary date)
TEST-006: Access Online Meeting Details#
Related: URS @Calendar @OnlineMeetings | RISK-R1, RISK-R2 Risk Level: MEDIUM
Preconditions: - Test event with Teams online meeting link (created via Outlook/Teams)
Steps:
1. Call get_event for online meeting event
2. Inspect location and onlineMeeting fields
3. Verify meeting URL accessible
Expected:
- location field includes online meeting URL or "Microsoft Teams Meeting"
- onlineMeeting object present with joinUrl field
- Join URL is valid Teams meeting link (starts with https://teams.microsoft.com/)
Pass Criteria: - Online meeting details correctly returned - Join URL not logged to CloudWatch - No modification or truncation of meeting URL
Security Tests#
TEST-SEC-001: Enforce User-Scoped Calendar Access#
Related: URS @Security @UserScoped | RISK-R3 Risk Level: CRITICAL
Preconditions: - Two test users: User A and User B - Each user has private calendar events not shared with the other
Steps:
1. Authenticate as User A
2. Call list_events for date range containing User A's private events
3. Verify response contains only User A's events
4. Attempt to call get_event with an event ID from User B's calendar (obtained separately)
5. Verify access denied
Expected: - User A sees only own events (no User B events in list) - Attempting to access User B's event by ID returns HTTP 404 or 403 (Microsoft Graph API enforces access control) - No cross-user data leakage in any scenario
Pass Criteria: - Zero events from User B visible to User A - Explicit cross-user access attempt rejected by Microsoft Graph API (HTTP 403/404) - Shared calendar access works correctly (if User A has permission to User B's shared calendar, those events are visible)
CRITICAL: This test must pass before production deployment. Cross-user access is a show-stopper issue.
TEST-SEC-002: Read-Only Access Enforcement#
Related: URS @Compliance @ReadOnly | RISK-R1, RISK-R2 Risk Level: HIGH
Preconditions: - Dev/staging environment running - Test user authenticated
Steps:
1. Attempt to create a new event (if MCP tool exists for write operations — should NOT exist)
2. Verify Microsoft Graph API token scopes: call /me/ endpoint and inspect scopes returned
3. Attempt to modify existing event via direct Graph API call using OBO token (simulate malicious client)
4. Verify DynamoDB table contains only OAuth tokens (no event data)
Expected:
- No MCP tools for event creation, modification, or deletion exist in server.py
- Token scopes limited to Calendars.Read only (verified via /.default scope request or token introspection)
- Direct Graph API write attempts (POST/PATCH/DELETE) return HTTP 403 Forbidden (insufficient permissions)
- DynamoDB table mcp-oauth-storage-outlook-mcp-dev contains only token records (partition key = user_id, attributes = access_token, refresh_token, expires_at)
Pass Criteria: - No write operations possible via MCP tools - Microsoft Graph API enforces read-only access at token scope level - DynamoDB contains zero event content (verified by scanning table and inspecting attribute names)
TEST-SEC-003: Calendar Privacy Preservation (No Event Data in Logs)#
Related: URS @Compliance @PrivacyPreservation | RISK-R2, RISK-R4 Risk Level: HIGH
Preconditions:
- Test event created with identifiable subject: "Confidential Salary Review - John Doe"
- Test event includes attendees: sensitive.attendee@novonordisk.com
Steps:
1. Call list_events and get_event for the test event
2. Download CloudWatch logs for the session (log group: /ecs/outlook-mcp-dev)
3. Search logs for:
- Event subject: "Confidential Salary Review"
- Attendee email: "sensitive.attendee@novonordisk.com"
- Event location: (any location string from test events)
- Event body content: (any content from test event body)
4. Verify structured logging excludes these fields
Expected:
- CloudWatch logs contain operational metadata:
- calendar_events_listed with count, date range, user identity
- calendar_event_fetched with event ID, user identity
- graph_request with HTTP method, endpoint, status code
- CloudWatch logs do NOT contain:
- Event subject lines
- Attendee names or email addresses
- Event body content
- Location details
- Online meeting URLs
Pass Criteria:
- Zero matches for sensitive test strings in CloudWatch logs
- Log retention set to 90 days (verified in CloudWatch log group settings)
- Log encryption enabled (KMS key used)
- LOG_LEVEL=INFO in ECS task definition (DEBUG disabled)
CRITICAL: GDPR compliance depends on this test. Logging PII creates Article 5 violation.
TEST-SEC-004: Expired Token Rejection#
Related: N/A (authentication foundational requirement) | RISK-R1 Risk Level: HIGH
Preconditions: - Test user has an expired OAuth2 access token (> 60 minutes old, no refresh attempted)
Steps:
1. Obtain OAuth2 access token for test user
2. Wait for token expiration (or manually set token expiry time in past)
3. Call list_events with expired token
4. Verify MCP server returns authentication error
Expected: - HTTP 401 Unauthorized response from MCP server - Microsoft Graph API rejects expired token (upstream validation) - Error message: "Authentication failed" or "Token expired" (user-friendly, no stack trace) - OBO token exchange fails before Graph API call attempted
Pass Criteria: - Expired token rejected by Microsoft Graph API or OBO flow - No calendar data returned with invalid authentication - Error logged to CloudWatch (without sensitive data)
Quality Gates#
Before production deployment, the following must be satisfied:
- All 10 test cases executed (TEST-001 through TEST-006, TEST-SEC-001 through TEST-SEC-004)
- All CRITICAL and HIGH risk tests passed (TEST-SEC-001, TEST-SEC-002, TEST-SEC-003, TEST-SEC-004, TEST-001, TEST-002)
- Zero critical defects open
- < 3 medium defects open (and risk-accepted by Product Owner)
- CloudWatch log inspection passed (no PII found in sample logs)
- DynamoDB table inspection passed (no event data stored)
- Security team sign-off obtained (RISK:OutlookCalendar reviewed and accepted)
- UAT completed by at least 2 end users (one from IT, one from business)
- Azure AD app registration scopes verified:
Calendars.Read,MailboxSettings.Read,offline_accessonly (noCalendars.ReadWriteorCalendars.ReadWrite.Shared) - ECS task definition secrets correctly configured (SSM parameters for
client_id,client_secret,dynamodb_table_name)
Go/No-Go Decision: Product Owner, Engineering Lead, Security representative must approve before production deployment.
Test Environment#
Development#
- URL:
https://outlook.dev.connectors.novo-genai.com/mcp - ECS Service:
outlook-mcp-dev(0.25 vCPU / 512 MB) - CloudWatch Logs:
/ecs/outlook-mcp-dev - DynamoDB Table:
mcp-oauth-storage-outlook-mcp-dev(on-demand billing) - Token Storage: In-memory (for local dev) or DynamoDB (for deployed dev)
- Test Users:
testuser1@novonordisk.com(primary test user)testuser2@novonordisk.com(cross-user access test)
Staging (Production-like)#
- URL:
https://outlook.connectors.novo-genai.com/mcp(same as prod) - ECS Service:
outlook-mcp-prod(0.5 vCPU / 1 GB) — blue/green deployment for testing - CloudWatch Logs:
/ecs/outlook-mcp-prod - DynamoDB Table:
mcp-oauth-storage-outlook-mcp-prod - Test Users: Real Novo Nordisk users (volunteer testers from UAT group)
Test Data Requirements#
- At least 10 calendar events per test user covering:
- Single-day events (appointments, meetings)
- Multi-day events (conferences, vacations)
- Recurring events (daily standups, weekly team meetings)
- Online meetings (Teams links)
- All-day events (holidays, out-of-office)
- Events in different timezones (CET, EST, UTC)
- At least one shared calendar accessible to test users
Acceptance Criteria#
System is ready for production when: 1. All quality gates passed 2. No CRITICAL or HIGH severity defects open 3. Security team approval obtained 4. UAT sign-off from business users 5. Risk assessment reviewed and residual risks accepted by Product Owner 6. Monitoring dashboards configured (CloudWatch alarms for error rate > 5%, Graph API 5xx > 10/min) 7. Runbook created for on-call team (OAuth token troubleshooting, Graph API outage response)
Rollback Plan: - If critical defect found post-deployment: revert ECS service to previous task definition version - If Microsoft Graph API integration fails: disable MCP server at ALB (route to maintenance page) - If GDPR violation detected (PII in logs): immediate shutdown, investigate root cause, delete affected logs
Version: 1.0 Date: 2026-04-22 Author: AILab Engineering Team Approved by: [Pending Product Owner, Security, Engineering Lead sign-off]