Capstone: The Complete Partner Feed Guardian Testing Strategy
π¦ Run it yourself β the complete companion package is organized under
companion/, with source, test layers, Pester configurations, and gate wrappers.
A pile of passing tests is not automatically a production testing strategy.
A strategy connects each test to:
- A risk
- A lifecycle moment
- An execution identity
- A permitted level of access
- A decision
- An owner
- A recovery action
This capstone assembles the Partner Feed Guardian into one complete model. The goal is not to convince you to build this exact system. The goal is to give you a structure you can apply to your own PowerShell automation.
What you will learn
- How the complete test layers fit together
- Which suites run at each lifecycle stage
- How Pester results control external orchestration
- How to maintain a test-decision record
- How to avoid duplicating the same test at every layer
- How to determine when the strategy is ready for production use
The production workflow
The Partner Feed Guardian receives a partner batch and moves it through:
Intake
-> Inspection
-> Eligibility decision
-> Preflight
-> Publication
-> Acknowledgement
-> Reconciliation
-> Evidence
Failure can route the batch to:
Quarantine
Retry acknowledgement
Reconcile unknown publication
Suspend partner intake
Rollback module
Incident investigation
The system has enough real complexity to make testing meaningful without relying on the traditional ping, service, disk-space, backup, or website examples.
Repository structure
PartnerFeedGuardian/
βββ src/
β βββ PartnerFeedGuardian.psd1
β βββ PartnerFeedGuardian.psm1
βββ tests/
β βββ Unit/
β βββ Integration/
β βββ Preflight/
β βββ Deployment/
β βββ PostDeploy/
β βββ Production/
β βββ Drift/
β βββ Recovery/
βββ config/
β βββ pester.unit.ps1
β βββ pester.integration.ps1
β βββ pester.preflight.ps1
β βββ pester.postdeploy.ps1
β βββ pester.production.ps1
β βββ pester.recovery.ps1
βββ artifacts/
βββ Invoke-Gate.ps1
Each test file must be independently executable. It imports what it needs and does not depend on another test fileβs discovery-time state.
The test layers
1. Unit and fast-gate tests
Purpose: Protect decisions, contracts, orchestration rules, and known regressions.
Runs: Developer workstation and clean CI runner.
Touches: In-memory objects and mocks only.
Typical duration: Seconds.
Failure action: Block pull request.
Examples:
- Unsupported schema is rejected
- Duplicate batch is rejected
- Missing payload is reported
- Rejected batch produces no change plan
-WhatIfprevents state-changing adapters- Publication occurs before acknowledgement
- A confirmed publication is not repeated
- Incident
INC-2417remains fixed
This is where the broad decision matrix belongs. Do not repeat every rule through the real provider.
2. Integration tests
Purpose: Prove the real adapter, identity, authentication, serialization, and controlled provider resource work together.
Runs: Dedicated integration environment under a production-like service identity.
Touches: Isolated test partner and disposable test data.
Failure action: Block release; route environment failures to the integration-platform owner.
Examples:
- Service identity can read the test key
- Host-key validation works
- Real provider returns the documented object shape
- Interrupted transfer can be identified and resumed
- Provider error behavior is captured correctly
- Stale test batches can be located and removed
A mock is removed only for the boundary this suite intends to prove.
3. Preflight tests
Purpose: Decide whether one specific production operation may begin.
Runs: Immediately before processing or deployment, under the actual execution identity.
Touches: Production read-only by default.
Failure action: Stop before the first production change.
Examples:
- Correct environment and tenant
- Correct identity
- Approved partner
- Supported schema
- Unique batch ID
- Valid manifest signature
- Exact payload inventory
- Effective date reached
- No unresolved processing lock
- Quarantine and recovery path available
A required preflight rejects failed, skipped, inconclusive, not-run, failed-block, and failed-container counts.
4. Deployment tests
Purpose: Prove the intended artifact landed and resolves correctly.
Runs: First on one canary runner, then each rollout wave.
Touches: Deployed files and module-loading environment; read-only after installation.
Failure action: Stop rollout and investigate or roll back the current wave.
Examples:
- Exact module version exists
- Package hash or signature matches
- Fresh session resolves the expected version first
- Module loads from the approved path
- Approved commands are exported
- Required dependency versions are present
These tests import the deployed artifact, not the repository source.
5. Post-deployment tests
Purpose: Prove the deployed module completes controlled behavior under the runner identity.
Runs: After each rollout wave.
Touches: Canary or controlled workload.
Failure action: Stop rollout; preserve evidence; potentially roll back.
Examples:
- Deployed module rejects a controlled unsupported batch
- Deployed module produces the approved output contract
- Dry-run or controlled adapter path works
- One canary workload reaches the expected final state
Deployment and post-deployment failures have different diagnoses, so keep the suites separate.
6. Production synthetic tests
Purpose: Prove the real production path works end to end.
Runs: On demand after deployment and on a carefully chosen schedule.
Touches: Dedicated synthetic production identity, partner, destination, and data.
Failure action: Depends on stage; may open an urgent ticket, suspend only the synthetic partner, stop rollout, or preserve state for reconciliation.
Safeguards:
- Explicit production opt-in
- Stable production environment ID
- Dedicated least-privileged identity
- Required synthetic partner and batch prefix
- Maximum payload and operation count
- Reporting and billing exclusions
- Correlation ID
- Independent cleanup process
- Sequential execution by default
This suite is intentionally small. Production is not where the full decision matrix belongs.
7. Drift tests
Purpose: Compare approved production configuration with actual production configuration.
Runs: After deployment and on a risk-based schedule.
Touches: Production read-only.
Failure action: Varies by difference type.
Examples:
- Destination changed
- Transform rule set changed
- Partner missing or unexpected
- Schema version differs
- Batch-size policy differs
Pester returns structured differences. A separate operational system retains history and routes alerts.
8. Recovery tests
Purpose: Prove rollback prerequisites, restored artifact, state reconciliation, and incident regression.
Runs: Before release in a controlled exercise and during an actual recovery.
Touches: Disposable or canary state; production only under the incident plan.
Failure action: Block release when recovery evidence is required; escalate during incident recovery.
Examples:
- Prior package and approved hash exist
- Prior version can read current state
- Rollback command works
- Restored version loads from the approved path
- Controlled workload succeeds after rollback
- Incident regression fails on defective code and passes on corrected code
The production test-decision record
Every important suite or test family should answer:
| Field | Required answer |
|---|---|
| Risk | What production failure is being addressed? |
| Claim | What exact statement does the test evaluate? |
| Layer | Unit, integration, preflight, deployment, production, drift, or recovery |
| Location | Where does it run? |
| Identity | Who runs it? |
| Timing | When does it run? |
| Access | What may it read or change? |
| Evidence | What result and metadata are retained? |
| Failure action | What does failure block or trigger? |
| Owner | Who investigates? |
| Limitation | What does the test not prove? |
| Recovery | What if the test itself leaves state behind? |
A sample:
name: Synthetic partner publication
risk: Production intake path is deployed but cannot complete a real transaction
claim: A controlled synthetic batch publishes, acknowledges, and reconciles through production
layer: production-synthetic
location: production
authority: svc-pfg-synthetic
timing: after canary deployment and daily
access: state-changing, synthetic partner only
failureAction: stop rollout when post-deployment; otherwise open urgent partner-platform ticket
owner: Partner Platform
limitations:
- Does not prove every real partner works
- Does not test production load
- Does not validate every schema variation
recovery: reconcile by correlation id; stale synthetic cleanup after 24 hoursGate orchestration
The companion wrapper separates evaluation from orchestration:
[CmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateSet(
'unit',
'integration',
'preflight',
'postdeploy',
'production',
'recovery'
)]
[string]$Name
)
Import-Module "$PSScriptRoot/src/PartnerFeedGuardian.psd1" -Force
$config = & "$PSScriptRoot/config/pester.$Name.ps1"
$result = Invoke-Pester -Configuration $config
Assert-PfgGateResult -Result $result -GateName $NameThe deployment pipeline can call:
./Invoke-Gate.ps1 -Name unit
./Invoke-Gate.ps1 -Name integration
./Invoke-Gate.ps1 -Name preflightAfter a canary deployment:
./Invoke-Gate.ps1 -Name postdeployPester does not deploy the next wave from inside the test. The pipeline continues only after the gate returns successfully.
Strict gate policy
The helper rejects incomplete evidence:
$problems = [ordered]@{
FailedTests = $Result.FailedCount
SkippedTests = $Result.SkippedCount
InconclusiveTests = $Result.InconclusiveCount
NotRunTests = $Result.NotRunCount
FailedBlocks = $Result.FailedBlocksCount
FailedContainers = $Result.FailedContainersCount
}
if ($problems.Values | Where-Object { $_ -gt 0 }) {
throw "$GateName did not complete successfully."
}If a suite contains optional tests, put them in an advisory profile. Do not make a required gate ambiguous.
Tagging standard
Use tags for distinct dimensions:
Layer.Unit
Layer.Integration
Gate.PullRequest
Gate.Preflight
Gate.Deployment
Gate.PostDeploy
Gate.Production
Gate.Drift
Gate.Recovery
Environment.Lab
Environment.Canary
Environment.Production
Risk.ReadOnly
Risk.StateChanging
Regression
INC-2417
A test can carry more than one:
Describe 'Synthetic partner batch' -Tag @(
'Gate.Production'
'Environment.Production'
'Risk.StateChanging'
) {
# ...
}Avoid tags that mix meaning, such as ProductionIntegrationCriticalFast.
Version policy
The capstone pins:
- Pester 6.0.0
- PowerShell 7.4 or later for the primary path
- Windows PowerShell 5.1 only where a real production host requires it
Configurations set:
$config.Should.DisableV5 = $trueThis keeps new series code consistently on the Pester 6 assertion family.
Do not enable experimental parallel execution for stateful or production-facing suites by default.
Mock policy
Use mocks to:
- Isolate decision logic
- Inject failures
- Verify dangerous calls do not occur
- Verify important order, count, target, and correlation behavior
Do not use mocks to claim:
- Real authentication works
- Provider schemas are unchanged
- The production identity has access
- The deployment contains the tested code
- Cleanup works against the real provider
For dangerous dependencies, use an explicit default mock that throws when an unexpected call occurs.
ShouldProcess policy
Every state-changing public command should:
- Declare
SupportsShouldProcess - Call
$PSCmdlet.ShouldProcess()close to each change - Use meaningful action and target text
- Return a structured result in
-WhatIfmode - Be tested to prove the state-changing dependency was not called
-WhatIf is a safety preview. It is not production-path evidence.
State and retry policy
The operation state must distinguish at least:
NotStarted
Publishing
PublishedPendingAcknowledgement
Completed
Failed
A timeout during Publishing requires reconciliation before retry.
A retry from PublishedPendingAcknowledgement must not publish again.
Every state-changing request uses a correlation ID and durable evidence.
Cleanup policy
Test-created state must have:
- Synthetic or integration prefix
- Correlation ID
- Creation time
- Expiration time
- Owner
- Independent cleanup command
AfterAll is useful but not sufficient.
Monitoring boundary
Pester evaluates:
- Did this bounded expectation pass?
- What drift exists?
- Did the synthetic transaction reach the expected state?
Other systems manage:
- Historical trends
- Dashboards
- Deduplication
- Alert routing
- Maintenance suppression
- Escalation
- Long-term retention
Do not let the test runner quietly become an unmaintainable monitoring platform.
Avoid duplicated evidence
The same behavior should not be tested at every layer merely to increase the number of tests.
Example: unsupported schema rejection.
- Full decision matrix: unit suite
- Real manifest parsing of schema field: integration suite
- One controlled rejection by deployed module: post-deployment suite
- Not necessary in every scheduled production synthetic run
Each layer proves the part that only that layer can prove.
Readiness checklist
Before calling the strategy production-ready, confirm:
Code design
- Inspection, decision, planning, apply, and verification are separable
- State-changing commands implement
ShouldProcess - Result objects are stable and useful
- Correlation identifiers are preserved
- Partial states are durable
Fast gate
- High-risk decisions are covered
- Boundary contracts are protected
- Known incidents have regressions
- Dangerous dependencies cannot escape mocks
- Required incomplete runs fail the gate
Integration
- Real adapter is tested
- Production-like identity is used
- Isolated resources exist
- Provider error shapes are understood
- Stale test state can be cleaned independently
Preflight
- Target and identity are verified
- Required input and starting state are proven
- Recovery prerequisites are present
- No state change begins before success
Deployment
- Artifact integrity and version are verified
- Module-path precedence is tested
- Canary workload is defined
- Stop and rollback conditions are documented
Production tests
- Dedicated synthetic identity and target exist
- Volume and frequency are limited
- Business reporting exclusions exist
- Cleanup and unknown-outcome reconciliation are tested
Operations
- Every scheduled test has an owner
- Failure actions are actionable
- Monitoring receives structured results
- Flaky tests are treated as defects
- Obsolete tests are retired
Recovery
- Prior artifact is available and compatible
- Rollback is practiced
- Restored code and state are verified
- Incident regressions are added at the cheapest effective layer
What the complete strategy proves
It provides layered evidence that the code makes the intended decisions, the real dependencies work under controlled conditions, the target is ready, the deployed artifact is correct, selected production paths operate, configuration remains approved, and recovery procedures have been exercised.
What it does not prove
It cannot prove production will never fail.
It cannot model every provider behavior, data shape, timing condition, human decision, or future change.
A good test strategy reduces unknowns and makes failures safer to detect, stop, diagnose, and recover from. It does not grant immortality to the system.
Try it yourself
Take one production PowerShell project and create the directory structure from this capstone.
Do not fill every folder immediately.
Start with one test family in each necessary layer:
- One high-risk decision test
- One real-dependency integration test
- One preflight gate
- One deployment-verification test
- One controlled post-deployment workload
- One read-only drift check
- One practiced recovery test
For every test, complete the decision record before scheduling or gating it.
Common mistakes
Measuring maturity by test count. Measure the important risks and decisions covered.
Repeating the full decision matrix in every layer. Each layer should prove what only it can prove.
Using a successful Pester summary as the entire gate policy. Inspect incomplete and container-level failures.
Leaving owners and failure actions outside the test strategy. Unowned evidence becomes noise.
Treating production tests as the final layer only. Most production protection happens before production.
Final recap
The central lesson of this series is not a new assertion command or a clever mock.
It is this:
A production-facing test should prove one necessary claim, at the safest useful layer, and control a clearly defined decision.
Ask every time:
- What risk are we addressing?
- What exactly are we proving?
- Where and when should this run?
- Which identity should run it?
- What may it touch?
- What happens when it fails?
- Who owns the result?
- What does it not prove?
That is how Pester moves from βtests around a scriptβ to a system that genuinely helps protect production.