Subject: [decade] Object naming in -req and -arch

From: "Zhang Peng" <[email-masked]>
Date: Tue, 3 Jul 2012 00:34:02 -0500

Dear DECADE participants,

As pointed out by Richard in previous discussions, both the -req and -arch documents has some paragraphs describing content naming, and they need better organization: some generic requirements should be in -req, while some more specific ones should be in -arch. Here are some suggestions about how to split the content on naming in these two documents. 

The rationale for the following split is that I recognize there are two generic requirements, that is, uniqueness, and binding validation. They should be placed in the -req. In -arch, we should present the specific requirements to enable uniqueness, and binding validation, respectively. There are also some more requirements on how to improve the responsiveness of the system, i.e., the support of early name generation. See below for details (references to the original documents are given after each specific requirements):

Generic requirements (to be put in -req)

1. It SHOULD ensure that uniqueness of object names. 
2. It SHOULD support the validation of name-object binding.
3. It SHOULD support the operation of DECADE-compatible servers under diverse administrative domains with no coordination. (not sure about what operations are meant here)

Specific requirements (to be put in -arch)

Requirements to enable uniqueness: 
1. It MAY enable applications to construct unique names by themselves with a high probability (ref: last sentence, pp. 11, -arch). If this is the case, it SHOULD indicate conflicts when duplicate names are constructed (ref: first para, 4.5.1, -req).
2. It MAY support content-dependent hash-based schemes (for general immutable objects), and content-independent enumeration-based schemes (for live streaming) 
(ref: 2nd para, pp. 12, -arch)

Requirements to enable name-object binding validation:
3. Different name-object binding validation mechanisms MAY be supported, and an application can decide which mechanism to use. (ref: 2nd para, 4.4, -arch)
4. Names MAY be self-describing so that a receiving entity (Content Consumer) knows which mechanism is used (ref: 1st para, pp. 12, -arch)
5. It SHOULD provide the following name elements: (1) A "type" field, (2) Cryptographic data, (3) Application or publisher information. (ref: pp. 12, -arch)

More requirements about performance:
6. It SHOULD support the scenarios where a client needs to know the name of a data object before it is completely stored at the server. (This is to let clients advertise the object before having fully uploaded it) (ref: second last para, pp. 12, -arch)
7. It SHOULD support the scenarios where a client need to know the name of a data object before it is locally created. (This is to support live streaming) (ref: last para, pp. 12, -arch)

Any comments are welcome, thanks.

Peng.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 03 Jul 2012 09:26:43 +0100

Hiya,

I guess I hope that our draft (just exited IETF LC) meets
these requirements. If not, I'd be interested in what's
missing.

Cheers,
S.

[1] http://tools.ietf.org/html/draft-farrell-decade-ni

---

From: Wangdanhua <[email-masked]>
Date: Wed, 4 Jul 2012 09:14:13 +0000

Hi Peng,

As to Q2, I agree with you that the content naming requirement part in –arch had better be moved to –req, and the “immutable data object” should be remained in –arch.

As to the generic requirements vs. specific requirements you enumerated, I’m a little confused. For example, as to the “Requirements to enable uniqueness”, I think “it SHOULD indicate conflicts when duplicate names are constructed (ref: first para, 4.5.1, -req)” is also the detailed description/demand of the generic requirement “ensure the uniqueness of object names”. In my opinion, such requirements should also be presented in –req. I wonder if my understanding is right.

Best wishes,
Danhua

发件人: [email-masked] [mailto:[email-masked]] 代表 Zhang Peng
发送时间: 2012年7月3日 13:34
收件人: [email-masked]
主题: [decade] Object naming in -req and -arch

Peng.

---

From: "Zhang Peng" <[email-masked]>
Date: Wed, 4 Jul 2012 14:24:04 -0400

Hi Danhua,

Thanks for your comments. I think your comments are very reasonable. But I still think the requirement "It MAY enable applications to construct unique names w.h.p." + "the naming scheme should indicated conflicts" is just one approach to ensure uniqueness. It is because we can also let the DECADE service providers assign the name to ensure uniqueness. Thus, I think we better just state the generic requirement that "names should be unique" without going into details of the design in -req, and give the detailed requirements for our design (i.e., enable apps to given names w.h.p., and indicate conflicts when necessary) in -arch. What would you think? 

Thanks,

Peng.  

发件人： Wangdanhua
发送时间： 2012-07-04 05:14
收件人： pzhang.thu; [email-masked]
主题： 答复: [decade] Object naming in -req and -arch
Hi Peng,

 
Best wishes,
Danhua

 

 

 

 
Peng.

---

From: "Zhang Peng" <[email-masked]>
Date: Wed, 4 Jul 2012 16:05:41 -0400

Hi Stephen,

I have read your documents on hash-based naming. It designs a good URI format and mapping scheme to HTTP URL. For DECADE, since our requirements include uniqueness, name-object binding validation, I wonder how we can apply your scheme to meet them. 
BTW, can you give more details on what you mean by "meets these requirements"? Thanks.

B,

Peng.

From: Stephen Farrell
Date: 2012-07-03 03:26
To: pzhang.thu
CC: [email-masked]
Subject: Re: [decade] Object naming in -req and -arch

Hiya,

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 04 Jul 2012 22:08:37 +0100

> I have read your documents on hash-based naming. It designs a good URI format and mapping scheme to HTTP URL. For DECADE, since our requirements include uniqueness, name-object binding validation,…

ni URIs are as unique as the collision resistance of your hash
function supports basically. If you pick sha-256 that won't be
an issue.

ni URis support name-data integrity (incl. in the code we've
released) which I think is the same as what you're calling
name-object binding validation.

So my answer for both is: just use ni URIs to meet these
requirements:-)

> BTW, can you give more details on what you mean by "meets these requirements"? Thanks.

I meant: if there's some requirement that can't be met using ni URIs,
then I'd like to know about that before its too late. (I realise
that the decade WG might still change reqs of course but wanted to
check now before we're done with our doc.)

S

---

From: "Zhang Peng" <[email-masked]>
Date: Wed, 4 Jul 2012 17:40:22 -0400

Hi Stephen,

Yes, for most cases, hash-based naming scheme is ok for in-network storage system which holds immutable objects. 
Bur for some application such as live streaming, we expect that the uploader/streamer can know the object name before it uploads the object to decade server. This will allow the streamer to advertise the file before fully uploading it, and then the viewer can fetch the object using the advertised name. If each object contains several seconds of video data, this feature can considerably reduce the latency. Thus, we may need a enumeration-based scheme besides the hash-based one. Can we add this feature in your proposed design? Thanks.

B,

Peng.

From: Stephen Farrell
Date: 2012-07-04 17:08
To: pzhang.thu
CC: [email-masked]

S

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 04 Jul 2012 22:42:26 +0100

> Bur for some application such as live streaming, we expect that the uploader/streamer can know the object name before it uploads the object to decade server. This will allow the streamer to adverti…

Ah gotcha. Yes there are ways, basically by defining a new
hash alg and using a signature. There's one documented in
draft-hallambaker-decade-ni-params [1] section 2.1.

Cheers,
S.

[1] http://tools.ietf.org/html/draft-hallambaker-decade-ni-params-03

---

From: "Zhang Peng" <[email-masked]>
Date: Wed, 4 Jul 2012 17:57:39 -0400

Ok, it seems to work. But it just includes the hash of public key. How to name a sequence of dynamic names? For example, the stream is divided into object by seconds: 1s, 2s, 3s,...? 

B,

Peng.
From: Stephen Farrell
Date: 2012-07-04 17:42
To: pzhang.thu
CC: [email-masked]

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 04 Jul 2012 23:01:29 +0100

> Ok, it seems to work. But it just includes the hash of public key. How to name a sequence of dynamic names? For example, the stream is divided into object by seconds: 1s, 2s, 3s,...? 

Some work TBD:-)

I suspect that the details will need work in the decade WG.

I'm confident that once we know we can handle dynamic
content as in draft-hallambaker, then we can use the same
approach for whatever decade-specifics are needed later.

In the case you mention, maybe a chain of ni URIs might
be needed included as some decade metadata with each
1s chunk and perhaps giving the name of the next chunk.
But I suspect you've gone beyond naming requirements
when you start to consider sets of related (parts of)
objects.

S.

---

From: "Y. Richard Yang" <[email-masked]>
Date: Thu, 5 Jul 2012 10:37:36 +0800

Hi Stephen, Peng,

On Thursday, July 5, 2012, Stephen Farrell wrote:

> > Ok, it seems to work. But it just includes the hash of public key. How
> to name a sequence of dynamic names? For example, the stream is divided
> into object by seconds: 1s, 2s, 3s,...?
>
> Some work TBD:-)
>

PPSP uses Merkle Hash tree for (integrity check on) a sequence of data:

http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-02
Section 5.

How relevant is this for decade naming?

> Personally, I agree. But I am not sure any feedback from iesg or the
> general community.
>
> -yry
> > B,
> >
> > Peng.
> > From: Stephen Farrell
> > Date: 2012-07-04 17:42
> > To: pzhang.thu
> > CC: [email-masked]
> >> Yes, for most cases, hash-based naming scheme is ok for in-network
> storage system which holds immutable objects.
> >> Bur for some application such as live streaming, we expect that the
> uploader/streamer can know the object name before it uploads the object to
> decade server. This will allow the streamer to advertise the file before
> fully uploading it, and then the viewer can fetch the object using the
> advertised name. If each object contains several seconds of video data,
> this feature can considerably reduce the latency. Thus, we may need a
> enumeration-based scheme besides the hash-based one. Can we add this
> feature in your proposed design? Thanks.
> >
> >>> I have read your documents on hash-based naming. It designs a good URI
> format and mapping scheme to HTTP URL. For DECADE, since our requirements
> include uniqueness, name-object binding validation, I wonder how we can
> apply your scheme to meet them.
> >>
> >>>> As pointed out by Richard in previous discussions, both the -req and
> -arch documents has some paragraphs describing content naming, and they
> need better organization: some generic requirements should be in -req,
> while some more specific ones should be in -arch. Here are some suggestions
> about how to split the content on naming in these two documents.
> >>>>
> >>>> The rationale for the following split is that I recognize there are
> two generic requirements, that is, uniqueness, and binding validation. They
> should be placed in the -req. In -arch, we should present the specific
> requirements to enable uniqueness, and binding validation, respectively.
> There are also so

---

From: Wangdanhua <[email-masked]>
Date: Thu, 5 Jul 2012 08:47:51 +0000

Hi Peng,
Your explanation makes sense to me. Thanks a lot.

Best wishes,
Danhua

发件人: Zhang Peng [mailto:[email-masked]]<mailto:[mailto:[email-masked]]>
发送时间: 2012年7月5日 2:24
收件人: Wangdanhua; [email-masked]<mailto:[email-masked]>
主题: 回复: 答复: [decade] Object naming in -req and -arch

Hi Danhua,

Thanks,

Peng.

发件人： Wangdanhua<mailto:[email-masked]>
发送时间： 2012-07-04 05:14
收件人： pzhang.thu<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
主题： 答复: [decade] Object naming in -req and -arch
Hi Peng,

Best wishes,
Danhua

发件人: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> 代表 Zhang Peng
发送时间: 2012年7月3日 13:34
收件人: [email-masked]<mailto:[email-masked]>
主题: [decade] Object naming in -req and -arch

Peng.

---

From: "Zhang Peng" <[email-masked]>
Date: Fri, 6 Jul 2012 22:35:08 -0400

Without considering the optimization of naming (i.e., letting applications know the object name before commitment of uploading), I think the hash-based you proposed is good enough for the requirements (i.e., uniqueness, and object-id binding). One problem, which is also criticized in Content-Centric Networking, is that such flat names are not human-readable, and are not suitable for aggregations.  There should be a mapping from meaningful names to them. We can leave this part to applications.

On the other hand, if we are encouraging the optimization, we should more sophisticated naming schemes. Since we would like to give the name for the object that is not fully uploaded, then we should use the following scheme inspired by the naming schemes of Content-Centric Networking. 

We assume that the uploading client or a publisher holds a private/public key pair.
(1) The object name is a the hash of the public key of the uploading client, together with the locally unique identifier assigned to the object.
(2) The object content and the name are signed together using the private key of the uploading client. 
(3) Some information about the location of the key may be included in the object, so that a client that requests the object can retrieve the public key to verify the name-object binding.

ref:  [1] Naming in Content-Oriented Architectures, ICN11
[2] Networking Named Content, Conext09

B,

Peng.

From: Stephen Farrell
Date: 2012-07-04 18:01
To: pzhang.thu
CC: [email-masked]

Some work TBD:-)

S.

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 07 Jul 2012 13:31:10 +0100

Hiya,

> (3) Some information about the location of the key may be included in the object, so that a client that requests the object can retrieve the public key to verify the name-object binding.

I think that scheme is the same as the one in draft-hallambaker I
mentioned before (if not, please tell me what I've missed), and,
in our case, is based on previous work (mainly by Christian) in
the FP7 4WARD project. So the ideas have been around for a
while in various places for how to handle name-data integrity
for dynamic content using asymmetric keys.

However, I'm not actually that fond of those schemes, mainly because
they create an often-ignored requirement for public key revocation,
which is an essentially unsolved problem in this space, and is still
problematic even in X.509 based PKIs where we've had real difficulty
getting clients to hard-fail. Those schemes also of course add a
requirement for private key maintenance, which is considerably
more burdensome compared to just using hashes. IMO, we're getting
very close to research topics with such schemes and it might be
better to focus on static content cases for now where easier
and more practical solutions clearly exist. Having said that,
I could understand the wg might not want to focus in like that,
but I suspect solving the clearer, better scoped problem might
be a better approach in the first instance.

Cheers,
S.

PS: I hope, if there's time, to start some discussion on the
use of crypto in names in the icnrg meeting in Vancouver.

---

From: Zhang Peng <[email-masked]>
Date: Sun, 8 Jul 2012 00:17:46 -0400

Yes, I think the scheme I listed is basically the same with that you mentioned. It is more detailed in that it says that the client can use the hash of the public key as a global unique identifier and then attach it with a locally unique label to name the object. It is not a big issue, and I think it is also considered in draft-hallambaker-decade-ni-params.

For the self-verifying scheme in which public key is hashed in the name, it may have some concerns about the key management problems, but they may be inherent in the public-key cryptosystems, and I don't quite agree that we should abandon the use of it because it has some vulnerability. But I agree that we should not go too far in the naming scheme design. Since we are focused on immutable objects in the DECADE, we may just claim the basic design of hash-based flat naming scheme in the -arch documents. 

Thanks,

Peng. 

On Jul 7, 2012, at 8:31 AM, Stephen Farrell wrote:

> project

---

From: Dirk Kutscher <[email-masked]>
Date: Mon, 9 Jul 2012 16:35:14 +0000

Hi Peng and all,

Thanks for the discussion.

So, I think it's good to check the requirements and assess draft-farrell-decade-ni accordingly. I personally that it meets the ones you listed in your earlier message.

One of the interesting features is that the NI specification defines a common format that enables the integration of hashes (or other cryptographic functions' output) into names and corresponding validation steps on nodes. The explicit signaling of the function that is actually used allows for application-specific usage (if so needed) and for some agility regarding crypto algorithms. A DECADE protocol specification could specify the usage of the scheme, MANDATORY to implement algorithms etc more specifically.

One of the, IMO, important things that have not been discussed sufficiently is the question about PPSP compatibility that Richard Yang brought up.

My view on this is as follows: the PPSP Merkle Hash Tree approach for static and live content has requirements on the *object* and *meta-data* format.  For example, chunks have to carry sibling and uncle hashes. Specifically, for live streaming with dynamic trees, chunks have to provide chunk signatures (block signature, chunk position in the block, and the digests of the sibling to the way to the root).

For both static object and live streaming with PPSP, I think, a hash-based naming scheme could be employed. Now, it can be worthwhile to employ a naming scheme that signals the existence of the specific object format (to aid receiving nodes). This would all be feasible with the scheme in draft-farrell-decade-ni - but it could be good to extend this discussion to PPSP.

Best regards,
Dirk

From: [email-masked] [mailto:[email-masked]] On Behalf Of Zhang Peng
Sent: Sonntag, 8. Juli 2012 06:18
To: Stephen Farrell
Cc: [email-masked]
Subject: Re: [decade] Object naming in -req and -arch

Thanks,

Peng.

project

---

From: Peng Zhang <[email-masked]>
Date: Tue, 10 Jul 2012 00:33:04 -0400

Hi Dirk and all,

	I agree that the NI specification meets the basic requirement of DECADE (without optimization on "early name generation"). 

	As for the Merkle Hash Tree, or MTH, it is a integrity-assurance method for a sequence of objects. It is critical to the PPSP protocol. But i wonder wether we should incorporate it in our design:

	First, DECADE is targeted at general content distribution applications, and for applications other than P2P streaming, there is no great value of using Merkle Hash Tree. It may cause high overhead to these applications due "meta data" including signatures and full hashes should be exchanged. 

	Still, we can discuss more on how to better incorporate PPSP based on NI without hurting the general application of DECADE. Thanks.

B,

Peng.

On Jul 9, 2012, at 12:35 PM, Dirk Kutscher wrote:

> So, I think it’s good to check the requirements and assess draft-farrell-decade-ni accordingly. I personally that it meets the ones you listed in your earlier message.
>  
> One of the interesting features is that the NI specification defines a common format that enables the integration of hashes (or other cryptographic functions’ output) into names and corresponding validation steps on nodes. The explicit signaling of the function that is actually used allows for application-specific usage (if so needed) and for some agility regarding crypto algorithms. A DECADE protocol specification could specify the usage of the scheme, MANDATORY to implement algorithms etc more specifically.
>  
> For both static object and live streaming with PPSP, I think, a hash-based naming scheme could be employed. Now, it can be worthwhile to employ a naming scheme that signals the existence of the specific object format (to aid receiving nodes). This would all be feasible with the scheme in draft-farrell-decade-ni – but it could be good to extend this discussion to PPSP.
>  
> Best regards,
> Dirk

---

From: "Y. Richard Yang" <[email-masked]>
Date: Tue, 10 Jul 2012 13:23:05 +0800

Hi Peng, Dirk,

I am cc'ing the ppsp list as well. You raised a good point on the
distinction between one object and a sequence of objects. To generalize, we
can discuss even a set of objects (no ordering), and a set of equivalence
objects (dynamic streaming that interleaves different resolutions). Your
arguement against MTH is higher overhead in the general case (end to end
arguements). How much exactly is the overhead? Decade may benefit from the
analysis from ppsp. Since streaming is considered as the main app, how much
overhead if decade has to build on top?

Thanks!

Richard

On Tuesday, July 10, 2012, Peng Zhang wrote:

> Hi Dirk and all,
>
> I agree that the NI specification meets the basic requirement of DECADE
> (without optimization on "early name generation").
>
> As for the Merkle Hash Tree, or MTH, it is a integrity-assurance method
> for a sequence of objects. It is critical to the PPSP protocol. But i
> wonder wether we should incorporate it in our design:
>
> First, DECADE is targeted at general content distribution applications,
> and for applications other than P2P streaming, there is no great value of
> using Merkle Hash Tree. It may cause high overhead to these applications
> due "meta data" including signatures and full hashes should be exchanged.
>
> Still, we can discuss more on how to better incorporate PPSP based on NI
> without hurting the general application of DECADE. Thanks.
>
> B,
>
> Peng.
>
>
> Hi Peng and all,****
> ** **
> Thanks for the discussion.****
> ** **
> So, I think it’s good to check the requirements and assess
> draft-farrell-decade-ni accordingly. I personally that it meets the ones
> you listed in your earlier message.****
> ** **
> One of the interesting features is that the NI specification defines a
> common format that enables the integration of hashes (or other
> cryptographic functions’ output) into names and corresponding validation
> steps on nodes. The explicit signaling of the function that is actually
> used allows for application-specific usage (if so needed) and for some
> agility regarding crypto algorithms. A DECADE protocol specification could
> specify the usage of the scheme, MANDATORY to implement algorithms etc more
> specifically.****
> ** **
> One of the, IMO, important things that have not been discussed
> sufficiently is the question about PPSP compatibility that Richard Yang
> brought up.****
> ** **
> My view on this is as follows: the PPSP Merkle Hash Tree approach for
> static and live content has requirements on the **object** and **meta-data
> ** format.  For example, chunks have to carry sibling and uncle hashes.
> Specifically, for live streaming with dynamic trees, chunks have to provide
> chunk signatures (block signature, chunk position in the block, and the
> digests of the sibling to the way to the root).****
> ** **
> For both static object and live streaming with
>
>