Sync from dev @ 252c1cf
Source: main (252c1cf) Excluded: live tenant exports, generated artifacts, and dev-only tooling.
This commit is contained in:
198
AGENTS.md
Normal file
198
AGENTS.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Agent Guidance: Intune / Entra Drift Backup
|
||||
|
||||
This repository tracks Git-based snapshots of Microsoft Intune and Entra ID configuration, generates review reports, and drives a rolling pull-request workflow for post-change review and remediation.
|
||||
|
||||
## Project Overview
|
||||
|
||||
The implementation is centered on three Azure DevOps pipelines:
|
||||
|
||||
- `azure-pipelines.yml`: hourly backup/export pipeline with rolling PR management.
|
||||
- `azure-pipelines-review-sync.yml`: 20-minute reviewer-decision sync and post-merge remediation queue.
|
||||
- `azure-pipelines-restore.yml`: manual or auto-queued restore pipeline for approved baseline rollback.
|
||||
|
||||
Workflow at a high level:
|
||||
1. Export Intune and Entra configuration into `tenant-state/`.
|
||||
2. Generate Markdown/CSV reports in `tenant-state/reports/`.
|
||||
3. Filter known non-actionable drift noise before commit.
|
||||
4. Commit workload drift to `drift/intune` and `drift/entra`.
|
||||
5. Create or update one rolling PR per workload into `main`.
|
||||
6. Refresh the PR description with deterministic change/risk summary and optional Azure OpenAI narrative.
|
||||
7. Apply reviewer `/reject` or `/accept` decisions and queue restore when needed.
|
||||
|
||||
## Technology Stack
|
||||
|
||||
- **Python 3**: primary language for all automation scripts.
|
||||
- **Azure DevOps Pipelines**: YAML-based CI/CD (`azure-pipelines.yml`, `azure-pipelines-review-sync.yml`, `azure-pipelines-restore.yml`).
|
||||
- **PowerShell & Bash**: inline pipeline steps for Git operations, token retrieval, and conditional logic.
|
||||
- **IntuneCD** (Python package, pinned to `2.5.0`): exports Intune configuration and restores baseline state.
|
||||
- **Microsoft Graph API**: reads/writes tenant configuration and resolves references.
|
||||
- **Node.js / md-to-pdf** (v5.2.5): generates HTML and PDF documentation artifacts from Markdown on full runs.
|
||||
- **Azure OpenAI**: optional PR narrative generation.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
```
|
||||
.
|
||||
├── azure-pipelines.yml # Main hourly backup pipeline
|
||||
├── azure-pipelines-review-sync.yml # 20-minute review sync
|
||||
├── azure-pipelines-restore.yml # Baseline restore pipeline
|
||||
├── scripts/ # Python automation helpers
|
||||
├── tests/ # unittest coverage for scripts
|
||||
├── tenant-state/ # Committed JSON exports and reports
|
||||
│ ├── intune/
|
||||
│ ├── entra/
|
||||
│ └── reports/
|
||||
├── docs/ # Security review docs and roadmap
|
||||
├── md2pdf/ # HTML/PDF styling and configs
|
||||
├── prod-as-built.md # Generated as-built source
|
||||
└── README.md # Operational overview for humans
|
||||
```
|
||||
|
||||
### Key Scripts
|
||||
|
||||
- `export_entra_baseline.py`: Graph API export for Entra objects (Named Locations, Authentication Strengths, Conditional Access, App Registrations, Enterprise Applications).
|
||||
- `commit_entra_drift.py`: commits Entra drift with author attribution from audit logs.
|
||||
- `resolve_ca_references.py`: resolves Conditional Access GUID references to human-readable names.
|
||||
- `filter_entra_enrichment_noise.py`: reverts JSON churn caused by best-effort Graph enrichment (owners, app roles).
|
||||
- `filter_intune_partial_settings_noise.py`: reverts partial Settings Catalog exports.
|
||||
- `generate_assignment_report.py`: produces Markdown and CSV assignment inventories.
|
||||
- `generate_app_inventory_report.py`: produces Entra apps inventory CSV.
|
||||
- `generate_object_inventory_reports.py`: produces per-category object inventory CSVs.
|
||||
- `validate_backup_outputs.py`: asserts required files exist after export.
|
||||
- `ensure_rolling_pr.py`: creates or updates one rolling drift PR per workload.
|
||||
- `update_pr_review_summary.py`: refreshes PR descriptions with change counts, risk assessment, and optional AI narrative.
|
||||
- `apply_reviewer_rejections.py`: processes `/reject` and `/accept` reviewer thread commands.
|
||||
- `queue_post_merge_restore.py`: queues restore pipeline after merged PRs that contained `/reject` decisions.
|
||||
|
||||
## Code Style and Conventions
|
||||
|
||||
- Every Python file starts with `#!/usr/bin/env python3` and `from __future__ import annotations`.
|
||||
- Type hints are used throughout (`typing.Any`, `argparse.Namespace`, etc.).
|
||||
- Internal helper functions are prefixed with `_`.
|
||||
- Common environment parsing helpers appear in multiple scripts:
|
||||
- `_env_text(name, default="")` – reads and sanitizes env vars, treating unresolved Azure DevOps macros `$(...)` as empty.
|
||||
- `_env_bool(name, default=False)` – interprets `1`, `true`, `yes`, `on` as boolean true.
|
||||
- Arguments use `argparse` with typed flags; pipeline variables are passed as env vars or CLI args.
|
||||
- JSON is written with `indent=4` or `indent=5` and `ensure_ascii=False`.
|
||||
- HTTP calls to Graph or Azure DevOps REST APIs use `urllib.request` (no external HTTP library).
|
||||
|
||||
## Testing
|
||||
|
||||
Tests are written with the Python standard library `unittest` framework. There is **no pytest configuration** (`pyproject.toml`, `setup.py`, or `pytest.ini` are absent). Modules are loaded dynamically in tests using `importlib.util.spec_from_file_location` so that scripts in `scripts/` do not need to be on `PYTHONPATH`.
|
||||
|
||||
### Run Tests
|
||||
|
||||
```bash
|
||||
python3 -m unittest discover -s tests -v
|
||||
```
|
||||
|
||||
### Test Coverage Areas
|
||||
|
||||
- `test_ensure_rolling_pr.py`: rolling PR creation, draft publishing, merge strategy logic.
|
||||
- `test_export_entra_baseline.py`: Entra export parsing, concurrent export behavior, error handling.
|
||||
- `test_filter_entra_enrichment_noise.py`: enrichment-only churn detection and reversion.
|
||||
- `test_filter_intune_partial_settings_noise.py`: partial Settings Catalog export filtering.
|
||||
- `test_queue_post_merge_restore.py`: post-merge restore queueing logic.
|
||||
- `test_update_pr_review_summary.py`: semantic diffing, AI thread management, PR description upserts.
|
||||
- `test_validate_backup_outputs.py`: validation rules for Intune and Entra outputs.
|
||||
|
||||
## Build and Runtime Architecture
|
||||
|
||||
There is no traditional build step for the Python code. The pipelines install runtime dependencies on each run:
|
||||
|
||||
```bash
|
||||
pip3 install "IntuneCD==2.5.0"
|
||||
```
|
||||
|
||||
For local development, only a Python 3 interpreter is required; scripts use the standard library except for the optional IntuneCD package.
|
||||
|
||||
### Pipeline Jobs
|
||||
|
||||
- **Intune backup job** (`backup_intune`):
|
||||
1. Prepare `drift/intune` branch from `main`.
|
||||
2. Decide light vs full mode (configured full-run hour or `forceFullRun=true`).
|
||||
3. Run `IntuneCD-startbackup`.
|
||||
4. Filter partial Settings Catalog exports.
|
||||
5. Resolve assignment group names from Graph.
|
||||
6. Generate assignment and object inventory reports.
|
||||
7. Validate outputs.
|
||||
8. Commit drift and update rolling PR.
|
||||
|
||||
- **Entra backup job** (`backup_entra`):
|
||||
1. Prepare `drift/entra` branch from `main`.
|
||||
2. Export selected categories with `export_entra_baseline.py`.
|
||||
3. Resolve Conditional Access references.
|
||||
4. Generate reports.
|
||||
5. Validate outputs.
|
||||
6. Filter enrichment noise and commit drift.
|
||||
|
||||
- **Review sync jobs** (`sync_intune_review_decisions`, `sync_entra_review_decisions`):
|
||||
1. Apply `/reject` decisions.
|
||||
2. Update automated PR summary.
|
||||
3. Queue post-merge restore when needed.
|
||||
|
||||
- **Restore job** (`restore_from_baseline`):
|
||||
1. Checkout approved baseline snapshot (branch, tag, or commit).
|
||||
2. Prepare restore scope (`full` or `selective`).
|
||||
3. Normalize payload JSON and strip display-only assignment labels.
|
||||
4. Run `IntuneCD-startupdate` with optional `--entraupdate`.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Token handling**: Graph tokens are obtained via `Get-AzAccessToken` in PowerShell and passed as secret pipeline variables. Token payload is decoded to validate required application permissions before use.
|
||||
- **Service connection**: Azure DevOps service connection (e.g. `sc-astral-backup`) uses workload federated credentials.
|
||||
- **Permissions**: read-only permissions for backup; read-write permissions (`...ReadWrite.All`) for restore. Missing roles are surfaced as pipeline errors before any Graph mutations occur.
|
||||
- **Path traversal**: selective restore paths are normalized and validated against `..` segments before file copy.
|
||||
- **Dry run**: restore pipeline defaults to `dryRun=true` and must be explicitly overridden to push changes.
|
||||
- **Access token scope**: `System.AccessToken` is required for PR and thread management via Azure DevOps REST APIs.
|
||||
|
||||
## Common Development Tasks
|
||||
|
||||
### Generate Entra export locally
|
||||
|
||||
```bash
|
||||
python3 ./scripts/export_entra_baseline.py \
|
||||
--root ./tenant-state/entra \
|
||||
--token "$GRAPH_TOKEN" \
|
||||
--enterprise-app-workers 8
|
||||
```
|
||||
|
||||
### Resolve Conditional Access references locally
|
||||
|
||||
```bash
|
||||
python3 ./scripts/resolve_ca_references.py \
|
||||
--root ./tenant-state/entra \
|
||||
--token "$GRAPH_TOKEN"
|
||||
```
|
||||
|
||||
### Generate assignment report locally
|
||||
|
||||
```bash
|
||||
python3 ./scripts/generate_assignment_report.py \
|
||||
--root ./tenant-state/intune \
|
||||
--output-dir ./tenant-state/reports/intune
|
||||
```
|
||||
|
||||
### Validate backup outputs locally
|
||||
|
||||
```bash
|
||||
python3 ./scripts/validate_backup_outputs.py \
|
||||
--workload intune \
|
||||
--mode light \
|
||||
--root ./tenant-state/intune \
|
||||
--reports-root ./tenant-state/reports/intune
|
||||
```
|
||||
|
||||
## Key Environment / Pipeline Variables
|
||||
|
||||
- `BASELINE_BRANCH` (default: `main`)
|
||||
- `DRIFT_BRANCH_INTUNE` (default: `drift/intune`)
|
||||
- `DRIFT_BRANCH_ENTRA` (default: `drift/entra`)
|
||||
- `BACKUP_FOLDER` (default: `tenant-state`)
|
||||
- `ENABLE_WORKLOAD_INTUNE` / `ENABLE_WORKLOAD_ENTRA`
|
||||
- `ENABLE_PR_REVIEW_SUMMARY` / `ENABLE_PR_REVIEWER_DECISIONS`
|
||||
- `AUTO_REMEDIATE_AFTER_MERGE` / `AUTO_REMEDIATE_DRY_RUN`
|
||||
- `ENABLE_PR_AI_SUMMARY` + `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_DEPLOYMENT`, `AZURE_OPENAI_API_KEY`
|
||||
- `ROLLING_PR_DELAY_REVIEWER_NOTIFICATIONS` / `ROLLING_PR_MERGE_STRATEGY`
|
||||
|
||||
See the top of each pipeline YAML for the full variable list and defaults.
|
||||
Reference in New Issue
Block a user