We run the same test suite in pwsh and in DetentShell

Anyone who’s used an embedded PowerShell console has a story about the day it behaved differently from the real one. The engine is identical; the hosting around it is where the drift creeps in. A format call comes back odd, a path resolves against the wrong base, $PSScriptRoot is mysteriously empty.

DetentShell’s answer is mechanical. We keep a battery of engine-behavior checks, about ninety of them across six areas: output formatting, pipelines and splatting, error streams, threading, .NET interop, host UI behavior, and scoping. Every release, the battery runs twice. Once in stock PowerShell 7 from the command line, once pasted into DetentShell’s own console. The two tallies have to match exactly. A mismatch is by definition a hosting bug, and it blocks the release.

This has caught real problems. The best example: for a stretch of time, capturing formatted output inside the app returned strings with ANSI color codes embedded in them. $text = $thing | Format-List | Out-String came back with escape sequences that stock pwsh would have stripped. The cause was our console color plumbing, the effect was that scripts parsing their own output worked in Windows Terminal and failed in DetentShell, and the parity battery flagged it with a specific failing check. Fixing it properly meant reworking the app’s whole output path. There’s a separate post on that. The relevant part here is that a standing check caught it, rather than a user three weeks after release.

Some of the parity details are things you’d only notice when they break. F5 on a saved, unmodified file runs the file itself, dot-sourced, so $PSScriptRoot resolves and relative-path includes work. Variables from a script run stay in the session, so $results is still there at the prompt afterward. Redirecting formatted output with > writes a clean file. Write-Progress collapses to a progress line instead of spraying record dumps. Each of these has a test with its name on it.

Where the host can’t be transparent, the app says so instead of approximating. The in-process engine can’t spawn Start-Job’s worker process, so calling Start-Job fails with one line telling you to use the bundled Start-ThreadJob, which works. In a live Windows PowerShell 5.1 session, Start-Job runs natively, because that session is an out-of-process Windows PowerShell.

“It’s really PowerShell” appears on a lot of product pages. Ours comes with a battery you can run yourself: the bunch scripts ship in the repo, and they behave the same in both places or the build doesn’t go out.

← All posts