Fix `test_upgrade_app` fails on Windows #9129

Job	Run time
build-macos	4m 2s
prepare-linux	3s
build-windows (windows-latest, x64)	8m 40s
build-windows (self-hosted, ARM64, Windows, arm64)	7m 20s
build-linux (stable)	4m 51s
build-linux (beta)	5m 12s
build-linux (nightly)	4m 13s
	34m 21s

Provide feedback