Subject: Re: [payload] Submission of new versions of H.265/HEVC payload format

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 4 Jul 2013 15:54:48 +0000

Of course, one advantage of the new-FU based approach is that it can be applied to pre-encoded or stored content without the need of transcoding. However, the purpose here is for improved error resilience, which IMO is most relevant for converstational applications like video telephony, video conferencing, and telepresence where content is real-time encoded. 

BR, YK

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Wang, Ye-Kui
> Sent: Thursday, July 04, 2013 8:34 AM
> To: Thomas Davies (thdavies); Mo Zanaty (mzanaty); Rickard Sjöberg; Stephan
> Wenger; Paul Bright-Thomas (paubrigh)
> Cc: [email-masked]
> Subject: Re: [payload] Submission of new versions of H.265/HEVC payload
> format
> 
> Hi Thomas,
> 
> My point was really about using dependent slices, not about 1 or N tiles per
> packets (sorry that I did not make that clearer). Whatever strategy you would
> like to do in terms of the number of tiles in a packet, using dependent slices can
> do the same, as each dependent slice can include 1 or N tiles.
> 
> I see the following advantages with the approach using dependent slices
> instead of defining new FUs:
> 
> - First of all, no need to define new payload structures.
> 
> - The RTP packetizer does not need to parse slice headers to identify boundaries
> of tiles within a coded slice, as tiles of one SLICE are already in separate NAL
> units each containing a DEPENDENT SLICE.
> 
> - No additional overhead in packetization (the 1-byte FU header and the 2- or
> 4-byte FU offset). The overhead of the short slice header for dependent slice
> NAL units is compensated by less number of tile entry offsets signalled, as the
> entry offset of the first tile in each NAL unit is not signalled in the slice header.
> Using dependent slices would need more two-byte NAL unit headers, while
> using new FUs would need more two-byte packet payload headers, and the
> overhead bits here are the same for both methods.
> 
> Am I missing anything here?
> 
> BR, YK
> 
> > -----Original Message-----
> > From: Thomas Davies (thdavies) [mailto:[email-masked]]
> > Sent: Thursday, July 04, 2013 1:49 AM
> > To: Wang, Ye-Kui; Mo Zanaty (mzanaty); Rickard Sjöberg; Stephan
> > Wenger; Paul Bright-Thomas (paubrigh)
> > Cc: [email-masked]
> > Subject: RE: [payload] Submission of new versions of H.265/HEVC
> > payload format
> >
> > Yes, we did think about one tile per packet or N tiles per packet,
> > also: I guess that is where we started from. Then you could indeed
> > align missing tiles with missing sequence numbers.
> >
> > The issue is that you need to guarantee that you are doing this for it
> > to be useful, and the decoder needs to know that this is guaranteed
> > through some form of other signalling. However, the bitstream chunk
> > associated to a tile is extremely variable, and many tile chunks would
> > be very small. So there is significant packetization overhead for them if there
> is one tile per packet.
> >
> > Having N>1 reduces this overhead, but it is still there and as N
> > increases you run the risk of breaking the MTU size which then
> > requires that you have to re- encode to fit things in and meet the
> > guarantee. There is definitely no time to re-encode whole tiles,
> > especially if you are already using them for parallelization. Thus you
> > might have to break the guarantee and fragment the packet anyhow, losing
> the resilience.
> >
> > The advantage that we see with the proposal is that you can get fairly
> > similar resilience to regular slice MTU-size matching, but with much
> > lower overheads as you can have a single regular slice header per frame, yet
> have a "dumb"
> > packetization process and no need to re-encode to hit MTU targets. In
> > H264 MTU matching usually means re-encoding just one or two 16x16 MBs.
> > In H265 it involves re-encoding at least one 64x64 CTB, and if you are
> > doing tiles then you can get tiny "dangling slices" at the end of a
> > tile, which means more small packets.
> >
> > Best regards
> >
> > Thomas
> >
> > -----Original Message-----
> > From: Wang, Ye-Kui [mailto:[email-masked]]
> > Sent: 04 July 2013 07:02
> > To: Mo Zanaty (mzanaty); Rickard Sjöberg; Thomas Davies (thdavies);
> > Stephan Wenger; Paul Bright-Thomas (paubrigh)
> > Cc: [email-masked]
> > Subject: RE: [payload] Submission of new versions of H.265/HEVC
> > payload format
> >
> >
> > An interesting idea. Using dependent slices (again :-) together with
> > tiles would do the trick too (each tile in one dependent slice, and
> > then each dependent slice NAL unit in one single NAL unit packet).
> >
> > Have you guys done an analysis comparing the pros and cons of the two
> > alternative approaches?
> >
> > BR, YK
> >
> > > -----Original Message-----
> > > From: Mo Zanaty (mzanaty) [mailto:[email-masked]]
> > > Sent: Wednesday, July 03, 2013 9:51 PM
> > > To: Rickard Sjöberg; Wang, Ye-Kui; Thomas Davies (thdavies); Stephan
> > > Wenger; Paul Bright-Thomas (paubrigh)
> > > Cc: [email-masked]
> > > Subject: RE: [payload] Submission of new versions of H.265/HEVC
> > > payload format
> > >
> > > Thomas, Paul and I have been working on improving packet loss
> > > resilience using fragments and tiles. I mentioned this to Stephan at
> > > IETF 86 in Orlando, and promised to have draft text ready before
> > > Berlin. Here is the proposed text along with an overview of the rationale.
> > >
> > > Rationale:
> > >
> > > 1. Resilience to packet loss requires independent slices or tiles.
> > > Since independence requires restrictions on some coding tools, both
> > > potentially incur slight penalties in coding efficiency, with a
> > > smaller penalty
> > for tiles.
> > >
> > > 2. Fragmentation in a higher layer like RTP can have several
> > > advantages over similar features in a codec layer like slices
> > > (independent or
> > dependent).
> > > a. Efficiency: RTP fragmentation has finer (byte) granularity while
> > > slices have much larger (CTB or tile) granularity. It also has less
> > > slice segment header overhead, especially for independent slices.
> > > This can impact the efficiency of the packetized stream in terms of
> > > both bits and
> > packets per second.
> > > b. Performance: Encoders must perform extra processing to limit
> > > slice sizes, which can impact encoder performance. Some
> > > implementations speculatively end a slice based on computed
> > > statistics (which aggravates a.), while others recode a slice if the
> > > size is exceeded (which
> > aggravates b.).
> > > c. Decoupling: Not all encoders will support limiting slice size to MTU size.
> > > Applications (for live or stored content) using such encoders can
> > > still support MTU limits using RTP fragmentation since it is
> > > decoupled from
> > the encoder.
> > > d. Multi-party: The encoded content may go to multiple receivers
> > > with different MTUs. RTP fragmentation can handle this with
> > > lightweight repacketization at middle boxes whereas slices incur
> heavyweight transcoding.
> > >
> > > 3. Combining fragmentation and tiles can improve resilience while
> > > retaining the above advantages, if fragments are enhanced to contain
> > > offsets (similar to IP fragmentation). This is described in the
> > > proposed draft text below, which would immediately follow section
> > > 4.7
> > Fragmentation Units.
> > >
> > > Draft Text:
> > >
> > > 4.7.1 Fragmentation Units with Fragment Offsets
> > >
> > > If a fragmented NAL unit contains tiles, its slice segment header
> > > contains offsets to the data of each tile. These offsets can be used
> > > for random access to any tile for parallel processing, region of
> > > interest extraction, and resilience to errors in other tiles. These
> > > offsets can also be used to provide resilience to packet loss, in
> > > conjunction with fragment offsets contained in two types of
> > > fragmentation
> > units: FU-A2 and FU-A4.
> > >
> > > Fragmentation units FU-A2 and FU-A4 contain a fragment offset before
> > > the FU payload. FU-A2 contains a 2-byte offset as shown in Figure X.
> > > FU-A4 contains a 4-byte offset as shown in Figure Y. The offset
> > > indicates the total number of bytes in all prior FU payloads of the
> > > fragmented
> > NAL unit.
> > >
> > >
> > >     0                   1                   2                   3
> > >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    |      FU-A2 NAL HDR (type=52)  |   FU header   | FU-A2 offset  |
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    | FU-A2 offset  |     DONL (optional)           |               |
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
> > >    |                                                               |
> > >    |                         FU payload                            |
> > >    |                                                               |
> > >    |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    |                               :...OPTIONAL RTP padding        |
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >
> > >                   Figure X   RTP payload format for FU-A2
> > >
> > >
> > >     0                   1                   2                   3
> > >     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    |      FU-A4 NAL HDR (type=53)  |   FU header   | FU-A4 offset  |
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    |      FU-A4 offset                             | DONL(optional)|
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    | DONL(optional)|                                               |
> > >    +-+-+-+-+-+-+-+-+         FU payload                            |
> > >    |                                                               |
> > >    |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >    |                               :...OPTIONAL RTP padding        |
> > >    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > >
> > >                   Figure Y   RTP payload format for FU-A4
> > >
> > >
> > > Since the offset of the first FU payload of a fragmented NAL unit is
> > > zero, and the Start (S) bit in the FU header is sufficient to
> > > indicate this, the first fragmentation unit of a fragmented NAL unit
> > > SHOULD use FU-A, but MAY use
> > > FU-A2 or FU-A4 with an offset of zero. Subsequent fragmentation
> > > units of a fragmented NAL unit MAY use FU-A2 or FU-A4 with a
> > > non-zero offset. The Start
> > > (S) bit in the FU header MUST NOT be set if the FU-A2 or FU-A4
> > > contains a non- zero offset, and MUST be set if it contains a zero offset.
> > >
> > > If a fragmentation unit is lost, tiles in the lost fragmentation unit are also
> lost.
> > > However, tiles which are successfully received in their entirety can
> > > be decoded if their slice segment header containing tile offsets is
> > > successfully received, despite lost fragmentation units. Fragment
> > > offsets in FU-A2 or FU-A4 are required in order to recover from loss
> > > of prior fragmentation units and continue decoding subsequent tiles.
> > >
> > > The following heuristic SHOULD be applied to determine if the lost
> > > fragmentation units are all part of the same fragmented NAL unit as
> > > subsequently received fragmentation units with identical RTP
> > > timestamps and identical values of NAL unit header layer ID and temporal
> ID.
> > >
> > > 1. Let N be the number of missing sequence numbers between two
> > > received fragmentation units with known offsets.
> > > 2. Let P be the FU payload size of the first of the two received FUs.
> > > 3. Let D be the difference in the offsets, minus P.
> > >    (This is the number of missing FU payload bytes.) 4. Let E be N times P.
> > >    (This is the estimated number of missing FU payload bytes.) 5.
> > > Let ERROR be the absolute difference between D and E, divided by D.
> > > 6. If ERROR < 50%, it is likely all FUs are part of the same
> > > fragmented NAL unit, otherwise it is unlikely.
> > >
> > > Cheers,
> > > Mo
> > >
> > > -----Original Message-----
> > > From: [email-masked] [mailto:[email-masked]] On
> > > Behalf Of Rickard Sjöberg
> > > Sent: Wednesday, July 03, 2013 3:38 PM
> > > To: Wang, Ye-Kui; Thomas Davies (thdavies); Stephan Wenger
> > > Cc: [email-masked]
> > > Subject: Re: [payload] Submission of new versions of H.265/HEVC
> > > payload format
> > >
> > > Dear all,
> > >
> > > I wrote slices and not dependent slices since the slice granularity
> > > and slice/tile rule applies to both slices and dependent slices.
> > > Sorry for the confusion. I will now use 'independent slices' and
> > > 'dependent slices' to distinguish between them.
> > >
> > > I do not question the use of independent slices, I find them useful
> > > in some applications.
> > >
> > > But the current RTP payload text says:
> > >
> > > "FUs SHOULD NOT be applied in live-encoding scenarios such as
> > >       video telephony, video conferencing, live streaming and live
> > >       broadcast, in which cases dependent slice segments SHOULD be used
> > >       when a slice should be transported in multiple RTP packets."
> > >
> > > I believe that FUs are a viable option to dependent slices for the
> > > reasons I listed in my previous e-mail and think that we should not
> > > recommend dependent slices over FUs but leave it to the implementer.
> > >
> > > BR
> > > Rickard
> > >
> > >
> > > -----Original Message-----
> > > From: [email-masked] [mailto:[email-masked]] On
> > > Behalf Of Wang, Ye-Kui
> > > Sent: den 3 juli 2013 20:43
> > > To: Thomas Davies; Stephan Wenger
> > > Cc: [email-masked]
> > > Subject: Re: [payload] Submission of new versions of H.265/HEVC
> > > payload format
> > >
> > > Hi All,
> > >
> > > I drafted an email replying directly to Rickard but did not finish
> > > and there was a meeting (and then Stephan's and Thomas' emails came).
> > >
> > > I wanted to comment that Rickard was comparing the use of FUs vs
> > > slices, not FUs vs dependent slices, while the recommendation in the
> > > draft is recommending use of dependent slices over FUs in
> > > live-encoding scenarios (so this answers Thomas question: the
> > > intention was
> > dependent slices ).
> > >
> > > In any case, it does not make much sense to try MTU size matching
> > > and at the same time use either dependent slices or FUs, because
> > > dependent slices and FUs should not be used when there are
> > > considerable packet losses, while MTU size matching should be
> > > applied when there are
> > considerable packet losses.
> > >
> > > For live encoding, whenever splitting of a regular slice is needed,
> > > the splitting can be done by the encoder using dependent slices -
> > > and the encoder has more knowledge and is more powerful than the RTP
> > > packetizer and thus can do a better job for the splitting.
> > >
> > > BR, YK
> > >
> > >
> > > > -----Original Message-----
> > > > From: [email-masked] [mailto:[email-masked]]
> > > > On Behalf Of Thomas Davies
> > > > Sent: Wednesday, July 03, 2013 9:34 AM
> > > > To: Stephan Wenger
> > > > Cc: [email-masked]
> > > > Subject: Re: [payload] Submission of new versions of H.265/HEVC
> > > > payload format
> > > >
> > > > Hi Stephan
> > > >
> > > > I think I agree with Rickard.
> > > >
> > > > I think the text in question states a preference for _dependent_
> > > > slice segments, which have no error resiliency advantage over FUs.
> > > > Is this just a
> > > typo?
> > > >
> > > > <snip>
> > > >
> > > > FUs SHOULD NOT be applied in live-encoding scenarios such as
> > > >        video telephony, video conferencing, live streaming and live
> > > >        broadcast, in which cases dependent slice segments SHOULD be used
> > > >        when a slice should be transported in multiple RTP packets.
> > > >
> > > >
> > > >
> > > > </snip>
> > > >
> > > > In general FUs may be used where error conditions are known to be
> > > > benign, for greater efficiency, and also where for whatever reason
> > > > the encoder cannot support MTU matching.
> > > >
> > > > We also are formulating some more general comments on this area
> > > > which we hope to provide more fully soon.
> > > >
> > > > best regards
> > > >
> > > > Thomas
> > > >
> > > >
> > > > On 03/07/13 17:22, Stephan Wenger wrote:
> > > > > Hi Rickard,
> > > > > Commenting only on slices vs. FUs.
> > > > > The preference for slices over FUs, historically, was pushed by
> > > > > myself into RFC 3984 for reasons of error resilience, and was
> > > > > moved over to this draft for the same reason.  Loose a slice,
> > > > > and you can recover at the next slice boundary; loose an FU, and
> > > > > you have to wait for the next slice header and throw away all
> > > > > trailing FUs.  RTP is in virtually all cases run over UDP, and
> > > > > in the typical Internet scenario UDP is lossy (in contrast to
> > > > > private, managed IP-network, which may have other constraints,
> > > > > but are not the IETF's main
> > > > > concern.) To set your remarks into context, let's first
> > > > > understand what we are talking
> > > > > about: the cost of slices is the overhead stemming from the
> > > > > insertion of slice headers (in-picture syntax prediction) and
> > > > > from the packet headers themselves.  And, of course,
> > > > > implementation complexity, but we should not be in the business
> > > > > to encourage implementor
> > > laziness when
> > > > > there would be a more complex, but better suited tool available.    The
> > > > > additional packetization overhead is only incurred when, as a
> > > > > result of incompetently filled packets (which, I agree, is more
> > > > > likely with
> > > > > 64x64 CUs than with 16/16 macroblocks), an additional packet
> > > > > needs to be inserted.  You may remember a quick analysis of mine
> > > > > we discussed way back early in JCT, where we (I think) agreed
> > > > > that the practical impact is negligible in almost all non-tile
> > > > > scenarios (mostly because an additional overhead of 50 bytes or
> > > > > so--3% or less of the coded picture size) is incurred
> > > > > statistically only rarely (only in those cases where an
> > > > > additional packet would need to be generated because of the
> > > > > video coding related overhead for the use of slices).  The
> > > > > prediction overhead of the use of regular slices has been shown
> > > > > to be substantial--the price one needs to pay for independent
> > > > > decodability--but for entropy slices that overhead virtually
> > > does not exist.
> > > > > With respect to tiles, I think you have a point though,
> > > > > especially when considering the type of massive parallel
> > > > > implementations for relatively small picture sizes some folks have been
> considering.
> > > > > OTOH, because FUs are trivial to implement--some say in contrast
> > > > > to slices--and because (I hope) we all agree that using FUs in
> > > > > error prone scenarios is a bad idea if you could use slices (but
> > > > > for reasons like the tile connection you mentioned), the draft
> > > > > should IMO continue to set a preference for the use of regular
> > > > > slices over FUs.  To avoid underperforming systems due to
> > > > > implementer
> > laziness.
> > > > > However, this is the IETF.  We don't have to worry about the
> > > > > word-count of explanatory language.  In fact, explaining such
> > > > > complex tradeoffs and relationships is generally encouraged here.
> > > > > So let's
> > > just do that:
> > > > > express a preference of the use of regular slices (SHOULD) when
> > > > > you expect losses and can use them (real-time encoding, no
> > > > > tiling structure that would lead to unacceptable packetization
> > > > > overhead); and suggest (MAY) the use of FUs in other scenarios.
> > > > > Accompanied by a more coherently expressed analysis.
> > > > > Stephan
> > > > >
> > > > >
> > > > > On 7.3.2013 06:46 , "Rickard
> > > > > Sjöberg"<[email-masked]>
> > > wrote:
> > > > >
> > > > >> Hi Ye-Kui,
> > > > >>
> > > > >> Thanks for your feedback on my comments, your suggestions look
> > > > >> good to
> > > > me.
> > > > >>
> > > > >> Regarding discouraging fragmentation units (FUs) for
> > > > >> live-encoding scenarios in section 4.7, I think using FUs does
> > > > >> make sense when you do not have many non-VLC NAL units for
> > > > >> aggregation. The spatial granularity of slices was 16x16 pixels
> > > > >> in H.264 but is typically
> > > > >> 64x64 for HEVC which means that MTU size matching is done with
> > > > >> units that are 16x larger. This may lead to a larger number of
> > > > >> smaller packets when slices are used compared to FUs. In
> > > > >> addition, there is the HEVC rule of slice and tile boundaries
> > > > >> that makes the slice granularity equal to the tile granularity
> > > > >> for cases when slices span multiple tiles. Finally, FUs are
> > > > >> easier to implement since you do not need to care about when to
> > > > >> break your slice to not exceed the MTU size limit. I think both
> > > > >> slices and FUs has their merits and the choice between them for
> > > > >> live-encoding should be left for
> > > the implementer.
> > > > >>
> > > > >> Regarding the use of PicOrderCntVal for feedback in 8.2 and 8.2
> > > > >> and the POC MSB sync issue, I agree that this is a corner case
> > > > >> that we probably will not see much of in practice. I have no
> > > > >> text change suggestion at the moment.
> > > > >>
> > > > >> BR
> > > > >> Rickard
> > > > >>
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Wang, Ye-Kui [mailto:[email-masked]]
> > > > >> Sent: den 24 juni 2013 01:00
> > > > >> To: Rickard Sjöberg; [email-masked]
> > > > >> Subject: RE: Submission of new versions of H.265/HEVC payload
> > > > >> format
> > > > >>
> > > > >> Hi Rickard,
> > > > >>
> > > > >> Thanks for the careful review and the comments. Please see inline
> below.
> > > > >>
> > > > >> BR, YK
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Rickard Sjöberg [mailto:[email-masked]]
> > > > >>> Sent: Thursday, June 20, 2013 9:17 AM
> > > > >>> To: Wang, Ye-Kui; [email-masked]
> > > > >>> Subject: RE: Submission of new versions of H.265/HEVC payload
> > > > >>> format
> > > > >>>
> > > > >>> Dear all,
> > > > >>>
> > > > >>> I have taken a first look at draft-schierl-payload-rtp-h265-03
> > > > >>> and have some questions/comments.
> > > > >>>
> > > > >>> My first question is what extensions of HEVC that
> > > > >>> draft-schierl-payload-rtp-
> > > > >>> h265 is intended to cover? The draft mentions "future
> > > > >>> scalable or 3D  video coding  extensions  of  this
> > > > >>> specification", what extensions do you refer to?
> > > > >>> Is the intent to cover the all HEVC extensions in a single
> > > > >>> payload specification?
> > > > >>>
> > > > >> [YK] That is in the semantics of LayerId (nuh_layer_id), where
> > > > >> "this specification" should be replaced with "[HEVC]". It is
> > > > >> sort of copy-and-paste error. Thanks for the good catching. No,
> > > > >> there is no intention to cover HEVC extensions in this payload
> > > > >> specification, though we intentionally to have the
> > > > >> depacketization process and the use with feedback messages to
> > > > >> work for HEVC scalability
> > > and 3DV extensions.
> > > > >>
> > > > >>> Another question for my understanding is why "Receivers MUST
> > > > >>> pass picture timing SEI messages and decoding unit information
> > > > >>> SEI messages to the decoder"?
> > > > >> [YK] Generally, RTP receivers should/must pass all NAL units
> > > > >> specified by the video coding spec to the video decoders. This
> > > > >> is emphasized here for the two timing related SEI messages
> > > > >> after it is said that picture output timing in RTP timestamps
> > > > >> should be used instead (to avoid giving a wrong impression to
> > > > >> readers that the SEI messages should be ignored). If an RTP
> > > > >> receiver discards the SEI messages, then HRD conformance
> > > > >> checking for the bitstream that was possible would be disabled,
> > > > >> and also the information related to frame
> > > > doubling or tripping would be lost.
> > > > >>
> > > > >>> In "5. Packetization Rules" in Section 4.7, there are 5 rules.
> > > > >>> The fourth one does not seem to be a rule since there is no
> > > > >>> "MUST" or "SHOULD" in the corresponding text.
> > > > >> [YK] I see your point, it is only a "MAY" rule. An alternative
> > > > >> to put this as a packetization rule is to put it as part of the
> > > > >> semantics of NAL unit header (1.1.4), but that sub-section
> > > > >> belongs to an overview wherein normative language (even "MAY")
> > > > >> should not appear. One solution to this is to insert "Payload
> > > > >> Header
> > Usage"
> > > > >> after 4.1 "RTP Header Usage" and include at least this item there.
> > > > >> I will do this in the next version unless there is a better suggestion.
> > > > >>
> > > > >>> Further, the fifth one discourages using FUs, what are the
> > > > >>> reasons behind that?
> > > > >> [YK] The bullet item only discourages using FUs for
> > > > >> live-encoding scenarios, wherein dependent slice segments should be
> used instead.
> > > > >> This was added after a discussion (among the authors) on the
> > > > >> need of whether to specify a payload structure for mixed FU and
> > > > >> aggregation packets and
> > > > > >from that discussion we concluded that encoding dependent slice
> > > > > >segments
> > > > >> at source coding level already allows for the use cases and
> > > > >> using of FUs for live-encoding does not make sense. Please let
> > > > >> us know if you think differently.
> > > > >>
> > > > >>> In 8.2 and 8.3, the feedback messages are using PicOrderCntVal.
> > > > >>> Isn't there a possiblity that the MSB of PicOrderCntVal may
> > > > >>> differ between the encoder and decoder?
> > > > >> [YK] Good question. In theory this is indeed possible when
> > > > >> random accessing to a bitstream is performed. However, it would
> > > > >> not happen in conversational applications where the feedback
> > > > >> messages may
> > be used.
> > > > >> Therefore, to me this would just work just fine. But please
> > > > >> feel free to make other suggestions.
> > > > >>
> > > > >>> The use of spatial-segmentation-idc may need some updates to
> > > > >>> be useful, let me come back to that later on.
> > > > >>>
> > > > >> [YK] OK. Look forward to seeing your input herein.
> > > > >>
> > > > >>> I have also a number of spelling suggestions for your considerations:
> > > > >>>
> > > > >>> Section 1.1:
> > > > >>> Replace "community" by "communities"
> > > > >> [YK] I guess both are OK. But anyway I just changed it per your
> > > > >> suggestion (for the next version).
> > > > >>
> > > > >>> Section 1.1.1:
> > > > >>> Replace "In addition, interpolation filter" by "In addition,
> > > > >>> the interpolation filter"
> > > > >> [YK] Thanks - done.
> > > > >>
> > > > >>> Replace "skipping the transform coding" by "skipping the transform"
> > > > >>> (transform_skip_flag of HEVC skips the transform, not the
> > > > >>> coding)
> > > > >>>
> > > > >> [YK] Done.
> > > > >>
> > > > >>> Section 1.1.2:
> > > > >>> Replace:
> > > > >>> "2) An indication of whether there is any parameter set within
> > > > >>> the current coded video sequence that updates another
> > > > >>> parameter set of the same type preceding in decoding order."
> > > > >>> by:
> > > > >>> "2) An indication of whether there is no parameter set update
> > > > >>> in the coded video sequence."
> > > > >>> since that is what no_parameter_set_update_flag in HEVC indicates.
> > > > >>>
> > > > >> [YK] Changed the wording to "An indication of whether there is
> > > > >> no parameter set within the current coded video sequence that
> > > > >> updates another parameter set of the same type preceding in
> > > > >> decoding
> > order."
> > > > >>
> > > > >>> Section 4.7:
> > > > >>> Replace "Type" in Figure 10, and "Type:  6 bits" by "FU Type"
> > > > >>> and "FU
> > > > >>> Type:  6 bits" to avoid confusion with the Type in the payload header.
> > > > >>>
> > > > >> [YK] Good point again. I will change the field name to be "FuType".
> > > > >>
> > > > >>> Section 6:
> > > > >>> Replace "recovery" by "recover"
> > > > >> [YK] Done.
> > > > >>
> > > > >>> Section 7.1:
> > > > >>> Replace "level-id" by "level-idc"
> > > > >> [YK] We are consistently using "id" for profile-id, level-id,
> > > > >> max-recv-level-id etc, similarly as in RFC 6184, instead of "idc".
> > > > >>
> > > > >>> Replace "sub_layer_leve_idc[j]" by "sub_layer_level_idc[j]"
> > > > >> [YK] Done.
> > > > >>
> > > > >>> Section 8:
> > > > >>> Replace "two  payload-specific  feedback  messages" by "a
> > > > >>> payload-specific feedback  message", since only SPLI is
> > > > >>> "Assigned in this memo".
> > > > >>> Also, consider removing references to H.264 in the last
> > > > >>> paragraph of Section
> > > > >>> 8 since the memo is about HEVC.
> > > > >> [YK] Good catch again. Changed.
> > > > >>
> > > > >>> Section 8.1:
> > > > >>> Seems to me that "There MUST be exactly one RPSI contained in
> > > > >>> the FCI field" should be "There MUST be exactly one SPLI
> > > > >>> contained in the FCI field"
> > > > >> [YK] Another good catch. Changed.
> > > > >>
> > > > >>>
> > > > >>> Best Regards,
> > > > >>> Rickard Sjöberg
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> -----Original Message-----
> > > > >>> From: [email-masked]
> > > > >>> [mailto:[email-masked]] On Behalf Of Wang, Ye-Kui
> > > > >>> Sent: den 11 juni 2013 19:00
> > > > >>> To: [email-masked]
> > > > >>> Subject: [payload] Submission of new versions of H.265/HEVC
> > > > >>> payload format
> > > > >>>
> > > > >>> Dear All,
> > > > >>>
> > > > >>> We have just submitted versions 02 and 03 of
> > > > >>> draft-schierl-payload-rtp-h265, for which the links are as follows:
> > > > >>>
> > > > >>> Version 02:
> > > > >>> URL:
> > > > >>> http://www.ietf.org/internet-drafts/draft-schierl-payload-rtp-
> > > > >>> h265-02.txt
> > > > >>> Htmlized:
> > > > >>> http://tools.ietf.org/html/draft-schierl-payload-rtp-h265-02
> > > > >>> Diff:
> > > > >>> http://www.ietf.org/rfcdiff?url2=draft-schierl-payload-rtp-h26
> > > > >>> 5-
> > > > >>> 02
> > > > >>>
> > > > >>> Version 03:
> > > > >>> URL:
> > > > >>> http://www.ietf.org/internet-drafts/draft-schierl-payload-rtp-
> > > > >>> h265-03.txt
> > > > >>> Htmlized:
> > > > >>> http://tools.ietf.org/html/draft-schierl-payload-rtp-h265-03
> > > > >>> Diff:
> > > > >>> http://www.ietf.org/rfcdiff?url2=draft-schierl-payload-rtp-h26
> > > > >>> 5-
> > > > >>> 03
> > > > >>>
> > > > >>> Version 02 includes huge changes compared to the earlier
> > > > >>> submitted version 01. In a short summary, the authors have
> > > > >>> worked hard to try to make the design complete, from both the
> > > > >>> payload structure and the signaling points of view. Compared
> > > > >>> to version 02, version
> > > > >>> 03 only contains a correction of the email address of an author.
> > > > >>>
> > > > >>> Due to that the industry is eager to deploy H.265/HEVC and
> > > > >>> other standards bodies such as 3GPP rely on the payload format
> > > > >>> for support of H.265/HEVC in RTP based video services, we need
> > > > >>> to progress this draft relatively quickly.
> > > > >>> Therefore, we would like to request quick reviews from
> > > > >>> interested parties and also request for the WG status of this draft.
> > > > >>>
> > > > >>> BR, YK (on behalf of the authors)
> > > > >>> _______________________________________________
> > > > >>> payload mailing list
> > > > >>> [email-masked]
> > > > >>> https://www.ietf.org/mailman/listinfo/payload
> > > > >>>
> > > > >> _______________________________________________
> > > > >> payload mailing list
> > > > >> [email-masked]
> > > > >> https://www.ietf.org/mailman/listinfo/payload
> > > > > _______________________________________________
> > > > > payload mailing list
> > > > > [email-masked]
> > > > > https://www.ietf.org/mailman/listinfo/payload
> > > >
> > > > _______________________________________________
> > > > payload mailing list
> > > > [email-masked]
> > > > https://www.ietf.org/mailman/listinfo/payload
> > > _______________________________________________
> > > payload mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/payload
> > > _______________________________________________
> > > payload mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/payload
> _______________________________________________
> payload mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/payload

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Thu, 4 Jul 2013 16:50:11 +0000

Hi YK,

You are correct, tiles can indeed be used with dependent slices to improve resilience, which is already supported by the current draft. But you lose the advantages of fragments described in 2 below. You are not missing anything if you doubt or don't care about these advantages.

The proposal is for applications which care about the advantages of fragments. Many (perhaps most) applications, including live encoding, use encoders outside the application. For example, hardware accelerators or external software components. These encoders don't always support limiting slice size to MTU size (2c), or it may lower performance (2b), or entail other restrictions. For example, I'm aware of very widely deployed H.264 silicon which can't do CABAC when MTU matching, so apps must choose between mode 0 and HP. And it took a long time for popular silicon and software libraries to even support MTU matching at all. So even if you doubt any efficiency advantages (2a), there are more reasons apps (including live encoding) may prefer fragments over dependent slices. Hence the proposal to provide this option.

Cheers,
Mo

On Jul 4, 2013, at 11:55 AM, "Wang, Ye-Kui" <[email-masked]> wrote:

BR, YK