This comprehensive lab environment demonstrates Microsoft Purview Information Protection through automated simulation of enterprise-scale data governance workflows. Through eight progressive hands-on labs (00–07) — including nine discovery method tracks (05a–05i) and an optional temporal analysis — you'll implement SharePoint provisioning, realistic document generation, classification validation, multi-method data discovery, reporting and remediation, and comprehensive monitoring. All labs use browser-based authentication and configuration-driven architecture for complete environment portability.
Target Audience: Security engineers, compliance administrators, consultants, and IT professionals implementing Microsoft Purview Information Protection in enterprise environments.
Approach: Fully automated simulation environment with configuration-driven design. Generate thousands of realistic HR/PII documents, validate built-in Sensitive Information Type (SIT) detection, implement DLP policies, and monitor effectiveness - all without writing custom SITs or managing credentials. Complete the simulation cycle in 1-2 days OR scale to enterprise-level demonstrations with 20,000+ documents for production validation.
What Makes This Different:
- Configuration-Driven: Single
global-config.jsonfile controls all environment values - no hardcoded tenants, URLs, or emails. - Complete Portability: Move between Dev/Test/Prod by updating one config file. Multi-tenant ready for consultants.
- Browser-Based Auth: Zero secrets, credentials, or app registrations. All authentication via interactive browser flow.
- Scalable Simulation: Three scale levels (Small/Medium/Large) adjust complexity automatically from demos to enterprise simulations.
- Built-In SITs Focus: Leverages Microsoft's built-in Sensitive Information Types - no custom SIT creation required.
- Repeatable Cleanup: Complete environment reset for multiple demonstrations and testing iterations.
Before starting this project, understand these key factors:
| Consideration | Impact | Planning |
|---|---|---|
| ⏱️ Document Generation | 5-10 min (Small) to 1-2 hours (Large) | Start Small for testing, scale up for demos |
| ⏱️ Document Upload | 20-40 min (Small) to 3-5 hours (Large) | Plan upload during low-network activity |
| ⏱️ Classification Time | Up to 7 days (all scales) | On-Demand Classification async process; Content Explorer updates within 7 days |
| 💾 Storage Impact | 100 MB - 1 GB (Small) to 10-15 GB (Large) | Monitor SharePoint storage quotas |
| 🔄 Throttling | PnP PowerShell: 600 req/min, 2500 req/hour | Scripts include automatic retry logic |
⏱️ Recommended Timeline: 7-14 days for complete workflow with classification
- Day 1: Complete Labs 00-03 (Setup, Sites, Generation, Upload: ~2-4 hours hands-on)
- Days 2-7: On-Demand Classification processing (up to 7 days async, Content Explorer updates within 7 days)
- Day 2+ (Parallel): Lab 05a immediate regex discovery (88-95% accuracy, no classification wait)
- Day 3+ (24hr wait): Lab 05b eDiscovery search + Lab 05c Graph API discovery (100% Purview SITs after SharePoint Search indexing)
- Day 3+ (ongoing): Lab 05d Activity Explorer monitoring (24-48h population); Lab 05e Auto-Apply policy simulation (1-7 days)
- Anytime (portal-based): Lab 05f Oversharing posture; Lab 05g Custom Detection; Lab 05h On-Prem Scanner (optional); Lab 05i Defender for Cloud Apps (optional)
- Days 7-14: Lab 04 Content Explorer validation (after Microsoft Search indexing)
- After discovery: Lab 06 Reporting and Remediation (1-2 hours)
- Cleanup: Lab 07 anytime (15-30 minutes)
💡 Accelerated Option: Complete Labs 00-03 + 05a in 4-6 hours for immediate regex-based discovery (88-95% accuracy) without waiting for classification or indexing.
Resource Management:
- Start Small: Begin with Small scale (1,000 documents) for testing
- Monitor Storage: Check SharePoint storage consumption before Large scale
- Throttling Awareness: Scripts handle throttling automatically with retry logic
- Cleanup Critical: Always run Lab 07 cleanup to remove simulation data
Duration: 10-15 minutes | Background setup
Objective: Validate environment readiness and initialize simulation infrastructure
What You'll Learn:
- Validate Microsoft 365 E5 licensing and required Purview permissions.
- Configure global-config.json with tenant-specific information.
- Test PowerShell module versions and service connectivity.
- Initialize directory structure and logging infrastructure.
- Understand configuration-driven architecture and portability design.
Key Deliverables:
global-config.jsonupdated with tenant information (TenantUrl, AdminEmail, etc.).- PowerShell modules validated: PnP.PowerShell v2.0+, ExchangeOnlineManagement v3.0+.
- Directory structure created (logs, output, generated-documents, reports, temp).
- Logging infrastructure initialized for simulation tracking.
Critical Success Factor: ✅ Update global-config.json FIRST with your tenant details. All subsequent labs read from this single configuration file.
Duration: 10-15 minutes (5 sites) to 30-45 minutes (25 sites)
Objective: Provision SharePoint sites representing organizational departments
What You'll Learn:
- Create SharePoint sites from
global-config.jsonspecifications. - Configure site templates (Communication vs Team sites).
- Set site permissions and ownership automatically.
- Use
-SkipExistingparameter for idempotent re-runs. - Validate site creation success and readiness.
Key Deliverables:
- SharePoint sites created (default: HR, Finance, Legal, Marketing, IT).
- Site permissions configured with specified owners.
- Sites ready for document upload in Lab 03.
Prerequisites: Lab 00 completed (configuration validated)
Duration: 5-10 minutes (Small) to 1-2 hours (Large)
Objective: Generate realistic documents containing built-in Sensitive Information Types
What You'll Learn:
- Generate HR documents with U.S. Social Security Numbers (SSN).
- Create financial records with Credit Card numbers, Bank Accounts, ABA Routing Numbers.
- Produce identity documents with Passport numbers, Driver's Licenses, ITIN.
- Generate mixed-format documents (.docx, .xlsx, .pdf, .txt) with varying PII density.
- Understand built-in SIT patterns and confidence levels.
Key Deliverables:
- Generated documents stored in
./generated-documentsdirectory. - Document distribution: 45% docx, 30% xlsx, 15% pdf, 10% txt (configurable).
- Built-in SITs embedded: SSN, Credit Cards, Passports, Driver's Licenses, ITIN, Bank Accounts, Routing Numbers.
- Scale-appropriate document counts (500-1,000 Small | 5,000 Medium | 20,000 Large).
Prerequisites: Lab 00 completed (environment validated, directory structure initialized), Lab 01 optional (sites not required for local document generation - only needed later for Lab 03 upload)
Duration: 20-40 minutes (Small) to 3-5 hours (Large)
Objective: Upload generated documents to SharePoint sites with intelligent distribution
What You'll Learn:
- Execute bulk document upload operations with throttling management.
- Distribute documents realistically across department sites (HR docs → HR site).
- Apply SharePoint metadata to uploaded documents (Department, ContentType, PIIDensity).
- Validate upload completion and distribution accuracy.
- Understand PnP PowerShell throttling limits and retry logic.
Key Deliverables:
- Documents uploaded to SharePoint sites from Lab 01.
- Intelligent distribution: HR documents → HR site, Financial → Finance site, etc.
- Custom SharePoint metadata applied: Department, ContentType, PIIDensity, GeneratedDate.
- Upload validation report confirming distribution success.
Prerequisites: Lab 01 completed (sites exist), Lab 02 completed (documents generated)
Duration: 30-60 minutes hands-on | Up to 7 days background classification
Objective: Execute On-Demand Classification and validate official Purview SIT detection
What You'll Learn:
- Create and configure On-Demand Classification scans in Microsoft Purview portal (purview.microsoft.com).
- Review estimation results before starting classification (items for review, cost analysis).
- Start classification manually using "Start classification" workflow.
- Monitor classification progress through portal status updates (Estimating → Classifying → Completed).
- Query Content Explorer for classification results after completion.
- Validate official Purview SIT detection accuracy (100% accuracy with built-in SITs).
- Generate classification coverage reports and compliance metrics.
Key Deliverables:
- On-Demand Classification scan created and estimation completed (typically 300-500 items for review).
- Cost analysis validated before starting classification (typically $5-15 for simulation scale).
- Classification executed successfully across all simulation sites (7-day process).
- Content Explorer validation showing classified documents and detected SITs (updates within 7 days).
- Classification coverage report: detection counts by SIT type, confidence levels, and distribution.
- SIT effectiveness analysis for built-in types (High/Medium/Low confidence distribution).
Prerequisites: Lab 03 completed (documents uploaded to SharePoint), BuiltInSITs configured in global-config.json
Timing Note: ⏱️ Classification runs asynchronously over up to 7 days. Process includes estimation phase (minutes), then classification phase (up to 7 days), with Content Explorer updates appearing within 7 days of scan completion. Portal at purview.microsoft.com provides progress tracking.
💡 Strategic "Fire and Forget" Workflow:
- Start Lab 04 Immediately: Initiate the classification scan (Step 1).
- Do Not Wait: Proceed immediately to Lab 05a or 05b (which do not require classification).
- Return Later: Come back in 7 days to validate results in Content Explorer.
Why? Lab 04 is the only way to populate Content Explorer (Visualization), but Lab 05 provides CSV Reports (Discovery). Lab 04 serves as the "Gold Standard" baseline to validate the accuracy of your Lab 05 discovery reports.
💡 Parallel Execution: While Lab 04 classification runs asynchronously over 7 days, you can proceed with discovery labs:
- Lab 05a: Immediate regex-based discovery (88-95% accuracy, no classification required)
- Lab 05b: 24-hour eDiscovery search (100% Purview SITs, no classification required)
- Lab 05c: 24-hour Graph API discovery (100% Purview SITs, no classification required)
- Lab 05d: 24-48 hour Activity Explorer monitoring (audit-trail events, no classification required)
- Lab 05e: Auto-Apply policy simulation (1-7 days, ongoing label enforcement)
- Labs 05f/05g/05h/05i: Oversharing posture, custom detection, on-prem scanner, and Defender for Cloud Apps (portal-based or advanced-setup tracks)
Duration: Varies by path (5 minutes to 14 days depending on method and indexing requirements)
Objective: Discover and report sensitive data using nine distinct tracks with different timelines, accuracy levels, and use cases
What You'll Learn:
- Choose between nine discovery methods based on timing, accuracy, and scope requirements.
- Execute immediate regex-based discovery (05a), 24-hour eDiscovery and Graph API searches (05b/05c), activity monitoring (05d), and auto-apply policy simulation (05e).
- Assess access risk and oversharing posture with DSPM and DAG (05f).
- Extend discovery to custom patterns, EDM, and trainable classifiers (05g), on-premises file shares (05h), and third-party SaaS apps (05i).
- Generate comprehensive CSV reports for compliance audits and security operations.
Nine Discovery Tracks:
| Track | Method | Timeline | Best For |
|---|---|---|---|
| 05a: PnP Direct File Access | Direct file enumeration + regex | Immediate | Learning, quick interim results |
| 05b: eDiscovery Manual | Portal UI with Purview SITs | 24 hours | One-time compliance searches |
| 05c: eDiscovery API | Microsoft Graph eDiscovery API | 24 hours | Recurring scans, API integration |
| 05d: Activity Explorer Monitoring | Audit-trail activity events | 24-48 hours | Monitoring data movement and use |
| 05e: Auto-Apply Policies | Auto-apply retention policy simulation | 1-7 days | Ongoing label enforcement |
| 05f: Access and Oversharing | DSPM, DSPM for AI, DAG, RCD | Portal review | Oversharing and access risk posture |
| 05g: Custom Detection | Custom SITs, EDM, trainable classifiers, OCR | Portal setup | Data the built-ins miss |
| 05h: On-Prem Scanner | IP scanner on Windows host | Scanner host | On-premises file shares (advanced) |
| 05i: Defender for Cloud Apps | MDCA file policy + content inspection | Portal review | SITs in third-party SaaS (optional) |
Key Deliverables:
- CSV discovery reports with SIT types, file paths, confidence levels, and metadata.
- Automation scripts for recurring discovery (Lab 05c with Microsoft Graph API).
- Discovery comparison analysis across multiple methods and accuracy levels.
- Oversharing and access risk assessment (Lab 05f).
- Compliance audit reports suitable for stakeholder presentations.
Prerequisites: Lab 04 (step 1) completed (classification active for portal-based discovery)
Duration: 1-2 hours
Objective: Consolidate discovery findings into a stakeholder report, classify files by severity, and safely remediate low-risk SharePoint content
What You'll Learn:
- Consolidate discovery results from 05a/05b/05c into a unified remediation plan.
- Classify detected files into High / Medium / Low severity tiers using a documented decision matrix.
- Preview deletions safely with a dry-run, then apply an explicit
-Applygate. - Produce a stakeholder summary and an audit trail for restoration.
Key Deliverables:
RemediationPlan-<timestamp>.csvwith severity and recommended action per file.- Stakeholder summary report and deletion audit trail in
reports/. - SharePoint Recycle Bin used for reversible cleanup (items remain recoverable until purged).
Prerequisites: At least one discovery track complete (05a/05b/05c) with a CSV export; PnP.PowerShell v2.0+ for the remediation step
Duration: 15-30 minutes
Objective: Remove simulation resources and restore environment to clean state
What You'll Learn:
- Execute targeted cleanup of specific simulation resources.
- Perform complete environment reset (sites, documents, logs).
- Restore
global-config.jsonto default template values. - Validate cleanup completion and resource removal.
- Export cleanup documentation report.
Key Deliverables:
- Simulation sites removed from SharePoint tenant.
- Generated documents deleted from local directories.
- Configuration restored to default template (
global-config.json.template). - Cleanup validation report confirming resource removal.
- Environment ready for next simulation iteration.
Prerequisites: Any labs completed (can run cleanup at any stage)
Safety Features: WhatIf support, confirmation prompts, backup creation before deletion
This project provides hands-on practical experience with core Microsoft Purview discovery methods simulation capabilities, focusing on:
- Built-In Sensitive Information Types (leveraging Microsoft's pre-configured SITs with 100% accuracy)
- On-Demand Classification (7-day portal-based classification with estimation and cost analysis)
- Content Explorer Validation (classification coverage analysis with 7-day update timing)
- Nine Discovery Tracks (immediate regex, 24hr eDiscovery, 24hr Graph API, Activity Explorer, Auto-Apply, Oversharing, Custom Detection, On-Prem Scanner, Defender for Cloud Apps)
- Modern eDiscovery Portal (purview.microsoft.com with Cases preview and Condition builder)
- Microsoft Graph API (automated tenant-wide discovery)
- SharePoint Online Governance (automated site provisioning, document distribution, metadata management)
- PowerShell & API Automation (Graph SDK, PnP PowerShell, OAuth 2.0 authentication)
- Configuration-Driven Architecture (environment portability, multi-tenant support, scalable simulations)
Coverage Depth: ~45% of total Microsoft Purview capability landscape with deep hands-on simulation experience in covered areas (production-ready automation patterns with multiple discovery methods, not theoretical overview).
Project Focus: Automated simulation environment suitable for consultants, pre-sales engineers, compliance administrators, and IT professionals building practical Purview demonstration and POC capabilities with emphasis on rapid deployment, realistic data governance scenarios, and comprehensive discovery method comparison.
| Capability | Coverage Level | Project Section(s) |
|---|---|---|
| Built-In SITs (Pre-configured) | ✅ COMPREHENSIVE | Lab 02 (SSN, Credit Cards, Passport, ITIN, Bank Accounts) |
| On-Demand Classification | ✅ EXTENSIVE | Lab 04 (estimation, cost analysis, 7-day classification, Content Explorer) |
| Content Explorer | ✅ EXTENSIVE | Lab 04 (classification metrics, 7-day updates, coverage reporting) |
| Synthetic PII Generation | ✅ COMPREHENSIVE | Lab 02 (realistic HR/Financial document creation) |
| Classification Validation | ✅ DETAILED | Lab 04 (100% Purview SIT accuracy, confidence analysis) |
| Multi-SIT Document Generation | ✅ COMPREHENSIVE | Lab 02 (mixed content with multiple PII types) |
| Modern Purview Portal | ✅ COMPREHENSIVE | Labs 04, 05b (purview.microsoft.com, eDiscovery Cases) |
| Capability | Coverage Level | Project Section(s) |
|---|---|---|
| eDiscovery Compliance Search | ✅ COMPREHENSIVE | Lab 05b (modern Cases preview, Condition builder, 24hr results) |
| Direct File Access Discovery | ✅ COMPREHENSIVE | Lab 05a (PnP PowerShell, immediate regex-based detection) |
| Microsoft Graph API Discovery | ✅ EXTENSIVE | Lab 05c (automated tenant-wide scans, SIEM integration) |
| Activity Explorer Monitoring | ✅ EXTENSIVE | Lab 05d (audit-trail events, cross-workload monitoring) |
| Auto-Apply Policy Simulation | ✅ COMPREHENSIVE | Lab 05e (SIT-triggered label enforcement, 1-7 day timeline) |
| Access and Oversharing Posture | ✅ COMPREHENSIVE | Lab 05f (DSPM, DSPM for AI, DAG, RCD portal review) |
| Custom Detection | ✅ DETAILED | Lab 05g (custom SITs, EDM, trainable classifiers, OCR) |
| On-Premises Scanner | ✅ EXTENSIVE | Lab 05h (IP scanner, on-prem file shares, advanced track) |
| Defender for Cloud Apps | ✅ DETAILED | Lab 05i (MDCA file policy, third-party SaaS content inspection) |
| Multi-Method Discovery Comparison | ✅ COMPREHENSIVE | Lab 05 overview (timing/accuracy matrix, decision guide) |
| CSV Report Generation | ✅ COMPREHENSIVE | Labs 05a/05c (Excel-ready discovery reports) |
| Discovery Automation | ✅ EXTENSIVE | Lab 05c (scheduled scans, recurring discovery) |
| Capability | Coverage Level | Project Section(s) |
|---|---|---|
| Automated Site Provisioning | ✅ COMPREHENSIVE | Lab 01 (5-25 sites, department-based organization) |
| Bulk Document Upload | ✅ COMPREHENSIVE | Lab 03 (throttling management, distribution logic) |
| Document Distribution | ✅ EXTENSIVE | Lab 03 (realistic cross-site content placement) |
| Metadata Management | ✅ DETAILED | Lab 03 (document properties, organizational context) |
| Site Permission Configuration | ✅ EXTENSIVE | Lab 01 (ownership, access management) |
| Capability | Coverage Level | Project Section(s) |
|---|---|---|
| Browser-Based Authentication | ✅ COMPREHENSIVE | All labs (interactive auth, OAuth 2.0) |
| PnP PowerShell Operations | ✅ COMPREHENSIVE | Labs 01, 03, 05a (sites, upload, discovery) |
| Microsoft Graph SDK | ✅ EXTENSIVE | Lab 05c (Graph Search API, delegated permissions) |
| Bulk Document Generation | ✅ COMPREHENSIVE | Lab 02 (500-20,000 documents with realistic PII) |
| Throttling Management | ✅ COMPREHENSIVE | Lab 03 (automatic retry logic, rate limiting) |
| Indexing Status Validation | ✅ COMPREHENSIVE | Lab 05b (real SharePoint scanning, readiness checks) |
| Discovery Automation | ✅ EXTENSIVE | Lab 05c (scheduled scans, recurring discovery) |
| Environment Cleanup | ✅ COMPREHENSIVE | Lab 07 (complete resource removal, reset operations) |
| Capability | Coverage Level | Project Section(s) |
|---|---|---|
| Configuration-Driven Design | ✅ COMPREHENSIVE | All labs (global-config.json single source of truth) |
| Environment Portability | ✅ COMPREHENSIVE | Documentation (Dev/Test/Prod migration) |
| Multi-Tenant Support | ✅ EXTENSIVE | Configuration (consultant-friendly patterns) |
| Scalable Simulations | ✅ COMPREHENSIVE | Configuration (Small/Medium/Large scale levels) |
| Template-Based Reset | ✅ COMPREHENSIVE | Lab 07 (pristine configuration restoration) |
The following capabilities require custom development, advanced configuration, or specialized scenarios beyond this project's simulation scope:
| Capability | Complexity | Why Not Covered |
|---|---|---|
| Custom SITs (Regex-based) | INTERMEDIATE | Project uses built-in SITs only; custom SIT creation requires advanced configuration |
| Exact Data Match (EDM) SITs | ADVANCED | Complex EDM schema design beyond simulation scope |
| SIT Confidence Tuning | INTERMEDIATE | Uses default confidence levels for built-in SITs |
| Keyword Dictionaries | INTERMEDIATE | Custom dictionaries not required for built-in SIT simulation |
| Capability | Complexity | Why Not Covered |
|---|---|---|
| Sensitivity Labels | INTERMEDIATE | Project focuses on classification and DLP, not document encryption |
| Label Encryption | ADVANCED | Encryption workflows not part of basic governance simulation |
| Label Inheritance | INTERMEDIATE | Advanced labeling scenarios beyond scope |
| Double Key Encryption | EXPERT | Requires external key server, enterprise-only feature |
| Capability | Complexity | Why Not Covered |
|---|---|---|
| Retention Labels | INTERMEDIATE | Classification and DLP focus; retention lifecycle management not simulation objective |
| Auto-Apply Retention Policies | INTERMEDIATE | Retention lifecycle management not simulation objective |
| Disposition Reviews | ADVANCED | Manual approval workflows beyond simulation scope |
| File Plan Descriptors | INTERMEDIATE | Advanced records metadata not covered |
| Capability | Complexity | Why Not Covered |
|---|---|---|
| Endpoint DLP (Windows/macOS) | ADVANCED | Requires device onboarding, client deployment beyond simulation scope |
| Cloud App DLP | ADVANCED | Third-party SaaS app integration beyond SharePoint Online focus |
| Email DLP (Exchange) | INTERMEDIATE | Email protection not part of SharePoint-focused simulation |
| Teams DLP | INTERMEDIATE | Teams content governance beyond simulation scope |
| Capability | Complexity | Why Not Covered |
|---|---|---|
| DLP Policies (SharePoint) | INTERMEDIATE | Project focuses on discovery and classification; DLP policy creation beyond scope |
| Policy Templates | INTERMEDIATE | DLP policy configuration not part of governance simulation |
| DLP Incident Management | INTERMEDIATE | Focus is on data discovery, not prevention policies |
| Test Mode Policies | BASIC | DLP testing workflows not included in discovery-focused project |
| Capability | Complexity | Why Not Covered |
|---|---|---|
| eDiscovery (Premium) | ADVANCED | Basic eDiscovery covered in Lab 05b; Premium features (custodians, review sets, advanced holds) beyond scope |
| Communication Compliance | EXPERT | Regulatory monitoring (SEC, FINRA) beyond scope |
| Insider Risk Management | EXPERT | User behavior analytics, risk scoring not covered |
| Information Barriers | EXPERT | Organizational segmentation beyond simulation needs |
| Privileged Access Management | EXPERT | Just-in-time admin access beyond scope |
Primary Mission: Provide repeatable, scalable Microsoft Purview Information Protection simulation environment for rapid POC deployment, demonstration preparation, and data discovery workflow validation using built-in capabilities and multiple discovery methods.
Ideal For: Consultants preparing client demonstrations, pre-sales engineers validating Purview discovery capabilities, IT professionals learning classification and search workflows, compliance teams comparing discovery methods before production deployment.
Not Suitable For: Custom SIT development, DLP policy implementation, advanced retention lifecycle management, or production compliance configurations requiring custom pattern development.
| Skill Area | Labs | Proficiency Level |
|---|---|---|
| Microsoft Purview Configuration | 00, 01, 04, 05 | Intermediate to Advanced |
| PowerShell Automation | All Labs | Intermediate to Advanced |
| Microsoft Graph API Integration | 05c | Intermediate |
| SharePoint Administration | 01, 03, 07 | Intermediate |
| PII Data Generation | 02 | Foundational |
| Classification Workflows | 04 | Intermediate to Advanced |
| Data Discovery Methods | 05a, 05b, 05c, 05d, 05e, 05f, 05g, 05h, 05i | Advanced |
| eDiscovery Compliance Search | 05b | Advanced |
| Compliance Reporting | 06 | Intermediate |
| Environment Portability | 00, 07 | Advanced |
This project complements official Microsoft Learn paths:
- Microsoft Purview Information Protection - Classification and sensitivity labels
- Data Loss Prevention in Microsoft 365 - DLP policy creation and monitoring
- Manage Microsoft Purview - Governance and compliance management
Beginner → Intermediate: Start with this project's Small scale simulation to understand core Purview workflows, then explore:
- Lab 05h: On-Premises Scanner Discovery for hybrid on-premises scanner deployment
Intermediate → Advanced: Complete this project's Large scale simulation, then advance to:
- Custom SIT development (regex patterns, EDM)
- Hybrid scanning scenarios (on-premises scanner + cloud) — see Lab 05h
- Production automation with PowerShell at scale
Complete simulation in 5 steps:
- Configure: Update
global-config.jsonwith your tenant information (see Lab 00). - Validate Environment: Run
.\00-Prerequisites-Setup\Test-Prerequisites.ps1. - Create Sites: Run
.\01-SharePoint-Site-Creation\New-SimulatedSharePointSites.ps1 -SkipExisting. - Generate & Upload Documents: Execute Labs 02-03 scripts in sequence.
- Execute Classification & Discovery: Complete Labs 04-05 (7-day classification, then choose one or more discovery tracks based on timing and scope), then Lab 06 to consolidate findings.
Full Sequential Execution (copy-paste for complete simulation):
# Lab 00: Validate prerequisites
.\00-Prerequisites-Setup\Test-Prerequisites.ps1
# Lab 01: Create SharePoint sites
.\01-SharePoint-Site-Creation\New-SimulatedSharePointSites.ps1 -SkipExisting
# Lab 02: Generate test documents
.\02-Test-Data-Generation\New-TestDocuments.ps1
# Lab 03: Upload to SharePoint
.\03-Document-Upload-Distribution\Upload-DocumentsToSharePoint.ps1
# Lab 04: On-Demand Classification (up to 7 days for completion)
# Portal: purview.microsoft.com > Data loss prevention > Classifiers > On-demand classification
# Create scan > Review estimation > Start classification
# Lab 05a: Immediate discovery (regex-based, 70-90% accuracy)
.\05-Discovery-Methods\05a-PnP-Direct-File-Access\scripts\Search-SensitiveData.ps1
# Lab 05b: eDiscovery (wait 24 hours after Lab 03 for SharePoint Search index)
.\05-Discovery-Methods\05b-eDiscovery-Compliance-Search\scripts\Test-ContentIndexingStatus.ps1
# Portal: purview.microsoft.com > eDiscovery > Cases (preview) > Create search
# Lab 05c: Graph API Discovery (wait 24 hours after Lab 03 for SharePoint Search index)
# See Lab 05 overview for decision guide on which path to use
# Lab 05: Discover sensitive data across sites (multiple tracks available)
# Choose one or more discovery methods based on timing needs
# Lab 06: Reporting and Remediation (after at least one discovery track complete)
.\06-Reporting-and-Remediation\scripts\Analyze-SeverityBasedRemediation.ps1
# Lab 07: Cleanup (when ready to remove all simulation resources)
.\07-Cleanup-Reset\scripts\Reset-Environment.ps1 -FullReset -Confirm:$false💡 Best Practice: Start with Small scale (1,000 documents) for testing, then scale up to Medium or Large for demonstrations.
Choose your simulation scale in global-config.json based on testing vs demonstration needs:
| Scale | Documents | Sites | Generation Time | Upload Time | Classification Time | Storage | Use Case |
|---|---|---|---|---|---|---|---|
| Small | 500-1,000 | 5 | 5-10 min | 20-40 min | 15-30 min | 100 MB - 1 GB | Testing, POCs, Quick Demos |
| Medium | 5,000 | 10-15 | 45-60 min | 1.5-2 hours | 2-3 hours | 3-5 GB | Standard Demos, Training |
| Large | 20,000 | 20-25 | 1-2 hours | 3-5 hours | 4-6 hours | 10-15 GB | Enterprise Scale Demos |
Recommendation: Start with Small for initial testing, validate workflows work correctly, then scale up for demonstrations.
Purview-Data-Discovery-Methods/
├── 00-Prerequisites-Setup/ # Environment validation and configuration
├── 01-SharePoint-Site-Creation/ # SharePoint site provisioning
├── 02-Test-Data-Generation/ # PII document generation
├── 03-Document-Upload-Distribution/ # Document upload and metadata
├── 04-Classification-Baseline/ # On-Demand Classification and validation
├── 05-Discovery-Methods/ # Data discovery tracks (05a-05i + temporal)
├── 06-Reporting-and-Remediation/ # Cross-method reporting and remediation
├── 07-Cleanup-Reset/ # Environment cleanup and reset
├── global-config.json # Single source of truth for configuration
├── global-config.json.template # Pristine template for reset operations
├── README.md # This file
└── Documentation/ # Technical documentation
├── AUTHENTICATION-STANDARDIZATION.md # Browser-based auth patterns
├── IMPLEMENTATION-STATUS.md # Current project status
├── PORTABILITY-GUIDE.md # Multi-tenant migration guide
└── PROJECT-FLOW-ANALYSIS.md # Dependency and timeline analysis
Prerequisites: M365 E5 licensing, PowerShell modules, admin permissions. See Lab 00: Prerequisites Setup for complete requirements.
Completing this project demonstrates proficiency in the following industry-recognized Microsoft Purview skills, formatted for LinkedIn profiles, resumes, and corporate workforce development systems:
Microsoft Purview Configuration & Administration:
- Built-in Sensitive Information Types (SIT) deployment and validation (100% accuracy).
- On-Demand Classification workflow execution with estimation and cost analysis (7-day processing).
- Modern eDiscovery portal operations (purview.microsoft.com, Cases preview, Condition builder).
- Content Explorer proficiency for classification validation (7-day update timing).
- Nine discovery track implementations (immediate regex, 24hr eDiscovery, 24hr APIs, Activity Explorer, Auto-Apply, Oversharing, Custom Detection, On-Prem Scanner, Defender for Cloud Apps).
- SharePoint Online data governance and compliance management.
- Purview Information Protection configuration and lifecycle management.
Automation & Scripting:
- PowerShell automation for Purview simulation workflows.
- Microsoft Graph PowerShell SDK for tenant-wide discovery automation (OAuth 2.0, delegated permissions).
- PnP PowerShell for SharePoint bulk operations and document management.
- Automated document generation with realistic PII patterns.
- Configuration-driven deployment patterns for environment portability.
- Modern authentication implementation (interactive browser-based, OAuth 2.0, no service principals required).
Testing & Validation:
- Synthetic PII data generation for compliance testing.
- Classification accuracy validation and coverage analysis (100% Purview SIT accuracy).
- Discovery method comparison and effectiveness analysis (regex vs official Purview SITs).
- SharePoint Search indexing status validation and readiness checks.
- Multi-method discovery results comparison (immediate, 24hr timelines).
- Compliance monitoring dashboard development.
- Executive-level reporting for data governance metrics.
Data Governance & Compliance:
- Data classification strategy implementation (7-day On-Demand Classification).
- Discovery method selection and timing strategy (immediate vs 24hr vs 7-14 day).
- eDiscovery compliance search execution and analysis.
- Automated discovery workflows with Microsoft Graph API.
- Purview simulation environments for POC and demonstrations.
- Data discovery validation workflows across multiple methods.
Project & Process Management:
- Configuration management best practices (single source of truth patterns).
- Environment portability strategies for multi-tenant deployments.
- Simulation-to-production workflow design.
- Technical documentation and operational procedures.
- Repeatable demonstration environments for stakeholder presentations.
Environment Architecture:
- Configuration-driven infrastructure design (zero hardcoded values).
- Multi-tenant simulation portability patterns.
- Consultant-friendly deployment frameworks.
- Template-based environment reset and cleanup procedures.
- Scalable simulation architectures (Small/Medium/Large scale levels).
Demonstration & Enablement:
- POC environment rapid deployment (4-6 hours Small scale).
- Enterprise-scale demonstrations (20,000 documents, multiple sites).
- Stakeholder presentation preparation and validation.
- Training environment setup for compliance teams.
This project provides hands-on experience aligned with:
- Microsoft Certified: Information Protection and Compliance Administrator Associate (SC-400).
- Microsoft Certified: Security, Compliance, and Identity Fundamentals (SC-900).
- Roles: Compliance Administrator, Data Governance Analyst, Purview Consultant, Microsoft 365 Administrator, Pre-Sales Engineer (Microsoft Purview demonstrations).
For maximum visibility on LinkedIn and applicant tracking systems (ATS), this project covers:
Microsoft Purview • Information Protection • Data Classification • Built-In SITs • On-Demand Classification • Content Explorer • eDiscovery • Compliance Search • Data Discovery • Microsoft Graph API • Microsoft Graph PowerShell SDK • Compliance Management • Microsoft 365 Administration • SharePoint Online • PowerShell Scripting • PnP PowerShell • API Integration • OAuth 2.0 Authentication • Data Governance • Discovery Automation • Compliance Reporting • Compliance Testing • Simulation Environments • POC Development • Configuration Management • Modern Authentication • Technical Documentation • Demonstration Environments
The global-config.json file is the single source of truth for all environment-specific values. Complete configuration documentation is in Lab 00: Prerequisites Setup.
Key Sections: Environment (tenant URLs), Simulation (scale level), SharePointSites (site definitions), BuiltInSITs (SIT selections), DocumentGeneration (counts, file types, PII density), Paths (directories), Logging (levels, retention).
✅ Critical: Update
global-config.jsonwith your tenant details BEFORE running any lab scripts.
Zero hardcoded values • One-file migration • Multi-tenant ready • Dev/Test/Prod support
Change global-config.json only → Scripts work in new environment automatically. Ideal for consultants maintaining multiple client configurations.
Browser-based authentication (no secrets, certificates, or app registrations required):
- SharePoint:
Connect-PnPOnline -Interactive - Security & Compliance:
Connect-IPPSSession
Eliminates complex service principal configuration.
Personal: SSN, Passport, Driver's License, ITIN
Financial: Credit Cards, Bank Accounts, ABA Routing Numbers
Built-in SITs are immediately available in any M365 E5 tenant.
Microsoft Documentation: Purview Overview • SIT Reference • DLP Policies
PowerShell Modules: PnP PowerShell • Exchange Online Management
Advanced Scenarios: On-prem scanner (05h) and Defender for Cloud Apps (05i) tracks extend discovery beyond SharePoint.
Resource Management: Storage 100 MB - 15 GB • Cleanup critical • Template restoration via global-config.json.template
Performance: Start Small scale • Throttling automatic (600 req/min) • Classification 15 min - 6 hours • Background processing
Security: Test environments only • Browser auth • Global Admin + Compliance Admin required • Synthetic PII patterns
Contributions welcome! Follow repository style guides (PowerShell, Markdown, Parameters) • Configuration-driven architecture • Comprehensive error handling and logging.
This project is part of the Projects repository demonstrating Microsoft Purview capabilities.
This comprehensive Purview Discovery Methods Simulation project was created with the assistance of GitHub Copilot powered by advanced AI language models. The content, PowerShell automation scripts, configuration architecture, and documentation were generated, structured, and refined through iterative collaboration between human expertise and AI assistance within Visual Studio Code, incorporating Microsoft Purview Information Protection best practices, browser-based authentication patterns, and enterprise-grade discovery methods simulation workflows.
AI tools were used to enhance productivity and ensure comprehensive coverage of Purview simulation requirements while maintaining technical accuracy and reflecting established repository standards for infrastructure automation and documentation quality.
Ready to simulate enterprise data governance? Start with Lab 00: Prerequisites Setup!