The publisher runs on the provider's macOS machine and is a separate
workflow from the client. It carried its own Azure SAS and SigV4 S3
upload code, parallel uploads via ForEach-Object -Parallel with
helper functions injected through Invoke-Expression, an -UploadOnly
mode, and storage defaults read from the client's ElysiumSettings.txt
- all through the client's Elysium.Common.ps1 loader.
It now only builds the package (shards, manifest.json, khdb-clean.txt,
invalid-hashes.txt) and depends on nothing outside itself: no loader,
no relaunch, no settings file, no credentials. Uploading is left to
the storage provider's CLI, documented in the README for aws s3 /
rclone (S3-compatible) and azcopy (Azure): shards first, manifest
last, so a client never reads a manifest that points at shards that
are not there yet. -ShardRemotePrefix stays, since it is written into
the manifest and tells clients where to fetch shards from.
Removed: Upload-AzureBlob, Invoke-S3HttpUpload, the settings and
parallel-helper setup, the -UploadOnly branch and the upload section;
parameters StorageProvider, StorageAccountName, ContainerName,
SasToken, S3*, ManifestRemotePath, SkipUpload, UploadOnly,
MaxParallelTransfers, SettingsPath. 1,191 -> 770 lines. The shard
splitter and manifest writer are untouched.
BREAKING for the provider workflow only: existing invocations with
-SkipUpload or storage parameters now fail with "parameter cannot be
found"; clients are unaffected.
Verified: an AST check finds no calls to anything not defined in the
script or built into PowerShell; the publish -> update round-trip
test passes without -SkipUpload; 23/23 Pester tests pass.
The summary says "Invalid lines saved to .../invalid-hashes.txt" and
prints samples, but the file was always empty. $ensureInvalidWriter
is a scriptblock invoked with `&`, so it runs in a child scope:
`$invalidWriter = New-Object StreamWriter(...)` created a variable
local to that call, the outer $invalidWriter stayed $null, and every
`if ($invalidWriter) { ...WriteLine() }` was skipped. Same class of
bug as the $meta counter fixed in 06e7607.
Side effect: each invalid line opened another StreamWriter on the
same file (FileShare.Read) and never disposed it. macOS does not
enforce .NET share modes, so this only showed up as the empty file
here; on Windows the second open is expected to throw a sharing
violation and abort the publish once the input has two or more
invalid lines (not verified on Windows).
The writer now lives in a hashtable (`$invalidOutput.Writer`), which
the scriptblock mutates by reference; all 14 uses switched over.
Found by the new publish -> update round-trip test, included here:
tools/Prepare-KHDBStorage.ps1 -SkipUpload on 400 pre-sorted hashes
with mixed case, HASH:count suffixes, adjacent duplicates and a junk
line, then Update-ElysiumKhdb consumes the package. Asserts the junk
line lands in invalid-hashes.txt and the client rebuilds exactly the
sorted, unique, upper-case set. Before the fix: invalid-hashes.txt
empty (0 bytes with two junk lines). After: both lines written;
23/23 Pester tests pass.
Prepare-KHDBStorage.ps1 (publishes the KHDB) and decrypt.py
(decrypts client hash exports on the provider's cracking machine)
are only ever run by the tool provider, but sat next to the client
scripts that every customer clones. Moved both to tools/ so the
client surface (root wrappers + src/Elysium) is separate from
provider tooling; packaging the module later ships only src/Elysium.
Prepare-KHDBStorage.ps1 now resolves Elysium.Common.ps1 and the
default ElysiumSettings.txt from the repo root (one level up) instead
of its own folder. tests/test_decrypt.py imports decrypt from tools/.
README and AGENTS paths updated; AGENTS' structure listing also gets
the module lines back after the settings sample, where they belong.
Verified: decrypt round trip passes from the new location, and
tools/Prepare-KHDBStorage.ps1 -SkipUpload runs from the repo root and
produces a package (manifest, 256 shards, khdb-clean.txt).