diff --git a/index.bs b/index.bs index 061124980..501cf0d55 100644 --- a/index.bs +++ b/index.bs @@ -663,6 +663,9 @@ The interfaces defined are: {{AudioNode}} which applies a non-linear waveshaping effect for distortion and other more subtle warming effects. +* An {{AudioPlaybackStats}} interface, which provides statistics about the audio + played from the {{AudioContext}}. + There are also several features that have been deprecated from the Web Audio API but not yet removed, pending implementation experience of their replacements: @@ -1488,6 +1491,7 @@ interface AudioContext : BaseAudioContext { [SecureContext] readonly attribute (DOMString or AudioSinkInfo) sinkId; attribute EventHandler onsinkchange; attribute EventHandler onerror; + [SameObject] readonly attribute AudioPlaybackStats playbackStats; AudioTimestamp getOutputTimestamp (); Promise resume (); Promise suspend (); @@ -1533,6 +1537,10 @@ and to allow it only when the {{AudioContext}}'s [=relevant global object=] has :: An ordered list to store pending {{Promise}}s created by {{AudioContext/resume()}}. It is initially empty. + + : [[playback stats]] + :: + A slot where the instance of {{AudioPlaybackStats}} is stored.

@@ -1633,6 +1641,9 @@ Constructors

1. If |context| is allowed to start, send a control message to start processing. + 1. Set {{[[playback stats]]}} to a new instance of + {{AudioPlaybackStats}}. + 1. Return |context|. @@ -1769,6 +1780,12 @@ Attributes the context is {{AudioContextState/running}}. * When the operating system reports an audio device malfunction. + : playbackStats + :: + An instance of {{AudioPlaybackStats}} for this {{AudioContext}}. + Returns the value of the {{[[playback stats]]}} internal slot. + +

@@ -11536,6 +11553,246 @@ context.audioWorklet.addModule('vumeter-processor.js').then(() => { }); +

+The {{AudioPlaybackStats}} Interface

+ +Provides audio underrun and latency statistics for audio played through the +{{AudioContext}}. + +When audio is not delivered to the playback device on time, this causes an +audio underrun. This causes a discontinuity in the played signal, which produces an +audible "click", commonly called a "glitch". These glitches are bad +for the user experience, so if any of these occur it +can be useful for the application to be able to detect them and possibly +take some action to improve the playback. + +{{AudioPlaybackStats}} is a dedicated object for audio statistics reporting; +it reports audio underrun and playback latency statistics for the +{{AudioContext}}'s playback path via the +{{AudioDestinationNode}} and its associated audio output device. This allows +applications to measure underruns, which can occur due to the +following reasons: +- The audio graph is too complex for the system to generate audio on time, + causing underruns. +- There is some external problem causing underruns. Examples of such problems are: + - Another program playing audio to the same playback device is malfunctioning. + - There is a global system CPU overload. + - The system is overloaded due to thermal throttling. + +Underruns are defined in terms of [=underrun frames=] and [=underrun events=]: +- An underrun frame is an audio frame played by the output device + that was not provided by the {{AudioContext}}. + This happens when the playback path fails to provide audio frames + to the output device on time, in which case it will still have to play something. + + NOTE: Underrun frames are typically silence. +- An underrun event is the playback of a continuous sequence of + [=underrun frames=]. + The duration of the [=underrun event=] is the total duration of + the sequence of [=underrun frames=]s. + +
+[Exposed=Window, SecureContext]
+interface AudioPlaybackStats {
+    readonly attribute double underrunDuration;
+    readonly attribute unsigned long underrunEvents;
+    readonly attribute double totalDuration;
+    readonly attribute double averageLatency;
+    readonly attribute double minimumLatency;
+    readonly attribute double maximumLatency;
+    undefined resetLatency();
+    [Default] object toJSON();
+};
+
+ +Each {{AudioContext}} possesses exactly one {{AudioPlaybackStats}}. + +{{AudioPlaybackStats}} has the following internal slots: + +
+ : [[audio context]] + :: + The {{AudioContext}} that this instance of {{AudioPlaybackStats}} is + associated with. Initialized to the owning {{AudioContext}} upon + creation. + + : [[underrun duration]] + :: + The total duration of all [=underrun events=] that have occurred in + {{[[audio context]]}} playback as of the last stat update, a double. + Initialized to 0. + + : [[underrun events]] + :: + The total number of [=underrun events=] that have occurred in + {{[[audio context]]}} playback as of the last stat update, an int. + Initialized to 0. + + : [[total duration]] + :: + The total duration in seconds of all frames in {{[[audio context]]}} + playback as of the last stat update, a double, defined as + {{[[underrun duration]]}} + {{BaseAudioContext/currentTime}}. + Initialized to 0. + + : [[average latency]] + :: + The average audio output latency of {{[[audio context]]}} + over the {{currently tracked interval}}, a double. + + : [[minimum latency]] + :: + The minimum audio output latency of + {{[[audio context]]}} over the {{currently tracked interval}}, a double. + Initialized to 0. + + : [[maximum latency]] + :: + The maximum audio output latency of + {{[[audio context]]}} over the {{currently tracked interval}}, a double. + Initialized to 0. + + : [[latency reset time]] + :: + The time when the latency statistics were last reset, a + double. This is in the clock domain of {{BaseAudioContext/currentTime}}. + Initialized to 0. + + The currently tracked interval is the interval from + {{[[latency reset time]]}} to the current {{BaseAudioContext/currentTime}}. +
+ +

+Attributes

+ +Note: These attributes update only once per second and under specific +conditions. See [[#update-audio-stats]] and [[#AudioPlaybackStats-mitigations]] +for details. + +
+ : underrunDuration + :: + Returns the value of the {{[[underrun duration]]}} internal slot. + + NOTE: This metric can be used together with {{totalDuration}} to + calculate the percentage of played out media that was not provided by + the {{AudioContext}}. + + +
+ : underrunEvents + :: + Returns the value of the {{[[underrun events]]}} internal slot. + +
+ : totalDuration + :: + Returns the value of the {{[[total duration]]}} internal slot. + +
+ : averageLatency + :: + Returns the value of the {{[[average latency]]}} internal slot. + +
+ : minimumLatency + :: + Returns the value of the {{[[minimum latency]]}} internal slot. + +
+ : maximumLatency + :: + Returns the value of the {{[[maximum latency]]}} internal slot. + +

+Methods

+ +
+ : resetLatency() + :: + Sets the start of the interval that latency stats are tracked over to + the current time. + When {{resetLatency}} is called, run the following steps: + + 1. Set {{[[latency reset time]]}} to {{BaseAudioContext/currentTime}}. + 1. Let currentLatency be the playback latency of the last + frame played by {{[[audio context]]}}, or 0 if no frames have been + played out yet. + 1. Set {{[[average latency]]}} to currentLatency. + 1. Set {{[[minimum latency]]}} to currentLatency. + 1. Set {{[[maximum latency]]}} to currentLatency. + +

Updating the stats

+
+Once per second, execute the update audio stats algorithm: +1. If {{[[audio context]]}} is not running, abort these steps. +1. Let canUpdate be false. +1. Let document be the current [=this=]'s + [=relevant global object=]'s [=associated Document=]. +If document is [=Document/fully active=] and document's + [=Document/visibility state=] is `"visible"`, set canUpdate to + true. +1. Let permission be the [=permission state=] for the permission + associated with [="microphone"=] access. +If permission is "granted", set canUpdate to true. +1. If canUpdate is false, abort these steps. +1. Set {{[[underrun duration]]}} to the total duration of all [=underrun events=] + (in seconds) + that have occurred in {{[[audio context]]}} playback since its construction. +1. Set {{[[underrun events]]}} to the total number of [=underrun events=] + that have occurred in {{[[audio context]]}} playback since its + construction. +1. Set {{[[total duration]]}} to {{[[underrun duration]]}} + + {{[[audio context]]}}.{{BaseAudioContext/currentTime}}. +1. Set {{[[average latency]]}} to the average playback latency (in seconds) of + {{[[audio context]]}} playback over the {{currently tracked interval}}. +1. Set {{[[minimum latency]]}} to the minimum playback latency (in seconds) of + {{[[audio context]]}} playback over the {{currently tracked interval}}. +1. Set {{[[maximum latency]]}} to the maximum playback latency (in seconds) of + {{[[audio context]]}} playback over the {{currently tracked interval}}. +
+ +

Privacy considerations for {{AudioPlaybackStats}}

+ +
Risk
+Audio underrun information could be used to form a cross-site +covert channel between two cooperating sites. +One site could transmit information by intentionally causing audio glitches +(by causing very high CPU usage, for example) while the other site +could detect these glitches. + +Note: This covert channel depends on specific system characteristics. +It typically requires a shared linearization point (such as an OS or browser +audio mixer) and callbacks that are effectively synchronous from that point, +without intermediate buffering that would flatten load spikes. +
Mitigations
+To inhibit the usage of such a covert channel, the API implements these +mitigations. +- The values returned by the API MUST not be updated more than once per + second. +- The API MUST be restricted to sites that fulfill at least one of the following + criteria: + 1. The site has obtained + getUserMedia + permission. + + Note: The reasoning is that if a site has obtained + getUserMedia + permission, it can receive glitch information or communicate + efficiently through use of the microphone, making access to the + information provided by {{AudioPlaybackStats}} redundant. These options + include detecting glitches through gaps in the microphone signal, or + communicating using human-inaudible sine waves. If microphone access is + ever made safer in this regard, this condition should be reconsidered. + 1. The document is [=Document/fully active=] and its + [=Document/visibility state=] is `"visible"`. + + Note: Assuming that neither cooperating site has microphone permission, + this criterion ensures that the site that receives the covert signal + must be visible, restricting the conditions under which the covert + channel can be used. It makes it impossible for sites to communicate + with each other using the covert channel while not visible. +

Processing model