Integrate tell-tale lights to warn the player about issue mid-game

[Kreyren]

Lot of competetive games have ""tell-tale lights"" which are flashing on the screen in case there is an issue that influences the player's gameplay (e.g. latency beyond an expected threshold) for them to make decision to reduce the consequences of such issue e.g. retreating under turret in MOBA games instead of proceeding with an attack.

Example: https://www.youtube.com/watch?v=UzsNlKiSNVE?t=1 (notice the red blinking speedometer icon top right indicating high latency)

Proposing such integration in StadiaEnhanced with time logging to be used as a feedback to Google/ISP.

Motivation

Too many ruined World Records in TheCrew 2 on Stadia.

Theoretical integration

on-demand infinite loop that checks:

• Latency
• Compression time
• Packet Loss
• Lost connection

every X amount of miliseconds (every 250ms by default seems to be optimal in terms of processing resources) and shows the tell-tale light if it's beyond a set value.

---

CC @ChristopherKlay

[ChristopherKlay]

Something similar is actually already part of the extension.

If you open up the stream monitor during a game and doubleclick it (with the Stadia menu open), it switches to the compact view - which features a connection symbol that changes colors based on latency, frame drops, decoding time and other factors.

Proposing such integration in StadiaEnhanced with time logging to be used as a feedback to Google/ISP.

Maybe a option to save a performance log of the last session, that includes logs with date/time and whatever caused the performance issue?

[Kreyren]

If you open up the stream monitor during a game and doubleclick it (with the Stadia menu open), it switches to the compact view - which features a connection symbol that changes colors based on latency, frame drops, decoding time and other factors. -- @ChristopherKlay (#172 (comment))

I wasn't aware of this and tried to run:

1. sleep ${RANDOM:0:2} && while true; do ping localhost & done - To reproduce an unexpected input delay issue (got ~5 sec delay)
2. https://librespeed.org/ - To reproduce high ping (got 19ms -> 67ms)

Both of which only show gray circle in this view which is not sufficient as it's barely noticeable where the idea of tell-tale lights is to flash a warning for a split second decisions.

To clarify I am unable to react fast enough to the issue with the current implementation in competitive setting.

---

Maybe a option to save a performance log of the last session, that includes logs with date/time and whatever caused the performance issue? -- @ChristopherKlay (#172 (comment))

This is meant for a network monitoring to be able to sent a report so rather using a network latency and packet loss with time with date.

The time should be set on mili-seconds as my ISP (and many other ISPs) have an ongoing issues with "micro-spikes" (their terminology) that cause a high latency or packet loss which is hard to detect usually as it happens in less than 1 ms due to various reasons like overwhelmed scheduler on the lag server and alike.

FWIW these micro-spikes happens due to the use of "lag servers" (their terminology) that in practice works as a multiple roads that all lead to the same place where each of them can only take X amount of traffic so if the data is too big for one road it is split and sent across multiple roads and then combined on the lag server.

[ChristopherKlay]

To clarify I am unable to react fast enough to the issue with the current implementation in competitive setting.

While a bigger (optional) warning might make sense for people who don't have the streaming monitor running (and will likely get implemented down the line), i don't think it would help in your situation.

"Tell-tale lights" as you call them are not meant to avoid issues, but inform the user that those issues exist and that they might have an impact on your performance. Which is why rocket league for example displays packet loss, latency variation or a simple "high latency" warning - these are not "accurate" (as in; showing up on the same frame the issue appears) either, but based on average connection data across a given timespan - often times multiple seconds. In almost all cases a "high latency" warning shows up after the spike that caused it is already over to begin with.

The time should be set on mili-seconds as my ISP (and many other ISPs) have an ongoing issues with "micro-spikes" (their terminology) that cause a high latency or packet loss which is hard to detect usually as it happens in less than 1 ms due to various reasons like overwhelmed scheduler on the lag server and alike.

If you require log files with anywhere remotely close to that level of precision, you are far better off looking into the ability to log WebRTC connections via your browser and the data via chrome://webrtc-internals, because Stadia Enhanced isn't going to replace a dedicated monitoring tool when it comes to issues like this by any means.

The main goal of the extension is to provide a easy access point to the stream data to help with solving user issues related to networking and decoding, while providing a warning (and explanation) when the stream is unstable. All that, while having a minimal impact on the actual performance of Stadia on the given hardware.

[Kreyren]

"Tell-tale lights" as you call them are not meant to avoid issues, but inform the user that those issues exist and that they might have an impact on your performance ... often times multiple seconds. In almost all cases a "high latency" warning shows up after the spike that caused it is already over to begin with. -- @ChristopherKlay (#172 (reply in thread))

I agree that theoretically it's not possible for the tell-tale lights ("lights") to warn the player to reduce the impact of the issue since they always flash after the issue already happens.

But practically the delay of 200ms is sufficient headsup for the player to react and reduce the impact of the issue as:

Issue happens -> 200ms later tell-tale light flashes -> 750 (+-200) ms later player has the chance to react.

Where majority of competitive games have a window of average 3 seconds for a decision which can be adjusted through a playstyle so even if the warning comes 200ms late it's enough to reduce the impact of it.

In real-life examples:

• TheCrew2: Playing TheCrew2, seeing light flashing -> pressing pause button until the light stops flashing and then resume playing which can be done for a WR runs in-game.
• League of Legends: Playing on middle lane, about to poke (Cast a spell or use the regular attack to deal damage to the enemy player) an enemy player. If the light flashes then the player has around 1.8 sec to retreat and avoid getting damaged.

FWIW i do agree that the lights in rocket league are too slow to warn about the issue, but e.g. League of Legends it is fast enough.

If you require log files with anywhere remotely close to that level of precision, you are far better off looking into the ability to log WebRTC connections via your browser and the data via chrome://webrtc-internals, because Stadia Enhanced isn't going to replace a dedicated monitoring tool when it comes to issues like this by any means. -- @ChristopherKlay (#172 (reply in thread))

I agree that the data from chrome://webrtc-internals so far seem as sufficient. Thanks for informing me about it.