Skip to content

Add orientation metadata to VideoDecoderConfig, update algorithms #874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dalecurtis merged 2 commits into from
Jan 29, 2025
Merged

Add orientation metadata to VideoDecoderConfig, update algorithms #874

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
80f011b
Add orientation metadata to VideoDecoderConfig, update algorithms
dalecurtis Jan 27, 2025
0184fb5
Handle case with no rotation/flip for ImageDecoder.
dalecurtis Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 33 additions & 12 deletions index.src.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -922,10 +922,13 @@
5. If {{VideoDecoderConfig/colorSpace}} [=map/exists=] in the
{{VideoDecoder/[[active decoder config]]}}, assign its value to
|colorSpace|.
6. Let |frame| be the result of running the [=Create a VideoFrame=]
6. Assign the values of {{VideoDecoderConfig/rotation}} and
{{VideoDecoderConfig/flip}} to |rotation| and |flip| respectively.
7. Let |frame| be the result of running the [=Create a VideoFrame=]
algorithm with |output|, |timestamp|, |duration|,
|displayAspectWidth|, |displayAspectHeight|, and |colorSpace|.
7. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
|displayAspectWidth|, |displayAspectHeight|, |colorSpace|,
|rotation|, and |flip|.
8. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
</dd>
<dt><dfn>Reset VideoDecoder</dfn> (with |exception|)</dt>
<dd>
Expand Down Expand Up @@ -1970,6 +1973,8 @@
VideoColorSpaceInit colorSpace;
HardwareAcceleration hardwareAcceleration = "no-preference";
boolean optimizeForLatency;
double rotation = 0;
boolean flip = false;
};
</xmp>

Expand Down Expand Up @@ -2059,6 +2064,14 @@
bitstreams require a minimum number of inputs before any output can be
produced.
</dd>
<dt><dfn dict-member for=VideoDecoderConfig>rotation</dfn></dt>
<dd>
Sets the {{VideoFrame/rotation}} attribute on decoded frames.
</dd>
<dt><dfn dict-member for=VideoDecoderConfig>flip</dfn></dt>
<dd>
Sets the {{VideoFrame/flip}} attribute on decoded frames.
</dd>
</dl>


Expand Down Expand Up @@ -3910,7 +3923,7 @@
with {{VideoFrame/[[metadata]]}}.

### Algorithms ###{#videoframe-algorithms}
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|)
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, |colorSpace|, |rotation|, and |flip|)
:: 1. Let |frame| be a new {{VideoFrame}}, constructed as follows:
1. Assign `false` to {{platform object/[[Detached]]}}.
2. Let |resource| be the [=media resource=] described by |output|.
Expand Down Expand Up @@ -3940,6 +3953,8 @@
11. Assign {{VideoFrame/[[color space]]}} with the result of running the
[=VideoFrame/Pick Color Space=] algorithm, with |colorSpace| and
{{VideoFrame/[[format]]}}.
12. Assign {{VideoFrame/rotation}} and {{VideoFrame/flip}} to |rotation|
and |flip| respectively.
2. Return |frame|.


Expand Down Expand Up @@ -5742,9 +5757,12 @@
3. If a |timestamp| can otherwise be trivially generated from
metadata without further decoding, assign that to |timestamp|.
4. Otherwise, assign `0` to |timestamp|.
4. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
|duration|.
4. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
describe it as |rotation| and |flip|, otherwise set |rotation| to 0
and |flip| to false.
5. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|,
|duration|, |rotation|, and |flip|.
12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
|decodeResult|.

Expand Down Expand Up @@ -5808,11 +5826,14 @@
3. If a |timestamp| can otherwise be trivially generated from
metadata without further decoding, assign that to |timestamp|.
4. Otherwise, assign `0` to |timestamp|.
17. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
|duration|.
18. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
19. Resolve |promise| with |decodeResult|.
17. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
describe it as |rotation| and |flip|, otherwise set |rotation| to 0
and |flip| to false.
18. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|,
|duration|, |rotation|, and |flip|.
19. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
20. Resolve |promise| with |decodeResult|.

: <dfn for=ImageDecoder>Resolve Decode</dfn> (with |promise| and |result|)
:: 1. [=Queue a task=] to perform these steps:
Expand Down