Subject: [decade] Open discussion: Application agnostic content distribution

From: Songhaibin <[email-masked]>
Date: Thu, 14 Jun 2012 06:42:56 +0000

Hi guys,

I think this is a very general and open question to all of the working group. Which is also somewhat the high level goal of the working group. In order to do a general application agnostic content distribution platform inside the network, and the content distribution applications will be likely to use it. what kind of functions and components it should support? Or more general, what does the platform need? If you can answer with use case, that would be good. The architecture document is a good base. But I do not want the limit.

BR,
-Haibin (as individual contributor)

---

From: "Rahman, Akbar" <[email-masked]>
Date: Thu, 14 Jun 2012 13:19:32 -0400

Hi Haibin,

I agree that this is a very important question that you are asking.  In
terms of an important use case, I think that we should consider if
DECADE can be applied to work as in-network storage as part of a cloud
computing environment.  To me this would be the most "general
application agnostic content distribution platform inside the network"
that we could aim for.  I remember that in some of the previous WG
discussions, we considered the possibility of supporting the cloud
computing model as part of the WG re-charter.  I for one would vote in
favor of this.

Akbar

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Songhaibin
Sent: Thursday, June 14, 2012 2:43 AM
To: [email-masked]
Subject: [decade] Open discussion: Application agnostic content
distribution

Hi guys,

I think this is a very general and open question to all of the working
group. Which is also somewhat the high level goal of the working group.
In order to do a general application agnostic content distribution
platform inside the network, and the content distribution applications
will be likely to use it. what kind of functions and components it
should support? Or more general, what does the platform need? If you can
answer with use case, that would be good. The architecture document is a
good base. But I do not want the limit.

BR,

---

From: "Y. Richard Yang" <[email-masked]>
Date: Thu, 14 Jun 2012 20:54:13 -0400

Hi Haibin, Akbar,

Good question and good discussion. Looking at the term agnostic, and
looking at DECADE in a "layered" architecture setting, I would imagine that
DECADE (or a general network storage layer, but I use DECADE as the
Term) is at a base layer, and multiple Apps on top of it. Here by agnostic,
we may mean DECADE supports the intersection or the union of the
features required by the Apps. Clearly we may need to consider end-to-end
arguements. A general storage system and a multimedia storage system may
 have different requirements. For multimedia, some QoS/DiffServ features
may be essential. The workload (read, write) from different apps can be
different,
and hence one may optimize differently. The requirements on reliability
(persistence)
will also have impacts on the design. I feel that some precedng differences
can be
internal internal differences while others demand different APIs. A good
discussion
on this topic during rechartering will be very helpful indeed.

Richard

On Friday, June 15, 2012, Rahman, Akbar wrote:

> From: [email-masked] <javascript:;> [mailto:
> [email-masked] <javascript:;>] On Behalf
> Of Songhaibin
> To: [email-masked] <javascript:;>

---

From: Songhaibin <[email-masked]>
Date: Mon, 18 Jun 2012 11:04:13 +0000

Hi Akbar,

Thank you for the response. Cloud storage is an interesting topic. If the existing goal of the WG is application agnostic data distribution, it is somewhat part like cloud storage. But if we go for cloud storage, does it mean a cloud storage data protocol and a cloud storage management protocol?  And what does a cloud storage data protocol need to support? Does it need to support database operations? Our resource model in the current architecture is about the resource allocation for data distribution, is it part of the management protocol or data protocol?

I'm still not sure what the resource model would like in detail, how to allocate the bandwidth, storage space, connections and etc. resources to users and applications. I can image some different mechanisms. Maybe the service provider can choose one mechanism or the combination of them.

(1) If there are different subscription levels among users/applications, the service provider can allocate the relative amount of resources according to the levels. This mechanism is simple to design and implement.

(2) The service provider can assign different weight to different users/applications, but at the same time, the amount of resource to guarantee an application normal performance should be the minimum value for that application, this value can be indicated by user/application according to its context

(3) A flexible resource allocation (pay as you consume, by this concept, it also somewhat like a cloud storage or cloud distribution?). Assume a reasonable pricing mechanism is provided, the user can use as much idle resource as it wants, the upper bound is based on the physical limit of the equipments and the network, and the lower bound is based on the application performance, e.g. the provider can allocate as much bandwidth to a VoD user as it can, but the minimum is to guarantee a fluent play according to the encoding rate, and this can be indicated by the app software installed in the end user's machine. The bandwidth or other resources allocation can be dynamically, e.g. in the adaptive streaming scenario.
 
I also imagine that if a local data locker server/in-network storage server cannot provide enough resources to its end users, its neighbor data locker servers can be used. There is something like a central controller for the request routing and resource reservation (for distribution) in certain scenarios.

BR,

> From: Rahman, Akbar [mailto:[email-masked]]
> Sent: Friday, June 15, 2012 1:20 AM
> To: Songhaibin
> Cc: [email-masked]
> Subject: RE: [decade] Open discussion: Application agnostic content distribution
> 
> Hi Haibin,
> 
>

---

From: Songhaibin <[email-masked]>
Date: Mon, 18 Jun 2012 11:14:26 +0000

Hi Richard,

From: [email-masked] [mailto:[email-masked]] On Behalf Of Y. Richard Yang
Sent: Friday, June 15, 2012 8:54 AM
To: Rahman, Akbar
Cc: Songhaibin; [email-masked]
Subject: Re: [decade] Open discussion: Application agnostic content distribution

Hi Haibin, Akbar,

features required by the Apps.

[Haibin] Yes.

Clearly we may need to consider end-to-end
arguements. A general storage system and a multimedia storage system may
 have different requirements. For multimedia, some QoS/DiffServ features
may be essential. The workload (read, write) from different apps can be different,
and hence one may optimize differently.

[Haibin] I agree.

The requirements on reliability (persistence)
will also have impacts on the design. I feel that some precedng differences can be
internal internal differences while others demand different APIs. A good discussion
on this topic during rechartering will be very helpful indeed.

[Haibin] So beyond the general requirements, do you think it is required to classify the applications to different types, and the protocol design should meet the requirements to each type of apps in general? Or is there some other way to do it? This could be a good method to go and review the work IMO.

BR,

Richard

Akbar

-----Original Message-----
From: [email-masked]<javascript:;> [mailto:[email-masked]<javascript:;>] On Behalf
Of Songhaibin
Sent: Thursday, June 14, 2012 2:43 AM
To: [email-masked]<javascript:;>
Subject: [decade] Open discussion: Application agnostic content
distribution

Hi guys,

BR,

---

From: Haiwei Xue <[email-masked]>
Date: Mon, 18 Jun 2012 16:11:50 -0400

Hi Haibin,

I am confused, who dose 'application agnostic content distribution platform'
face with? what kinds of applications do mean by 'content distribution
applications will be likely to use it'. Do you mean content provider?

It my understanding that 'application agnostic content distribution platform'
is doing something CDN do. So what is new for 'application agnostic content
distribution platform', who will benefit from it?

Thanks.

Haiwei

On Thu, Jun 14, 2012 at 2:42 AM, Songhaibin <[email-masked]> wrote:

> group. Which is also somewhat the high level goal of the working group. In
> order to do a general application agnostic content distribution platform
> inside the network, and the content distribution applications will be
> likely to use it.

 what kind of functions and components it should support? Or more general,
> what does the platform need? If you can answer with use case, that would be
> good. The architecture document is a good base. But I do not want the limit.
>
> BR,

---

From: Songhaibin <[email-masked]>
Date: Tue, 19 Jun 2012 08:52:04 +0000

Hi Haiwei,

Thank you for your questions.

From: Haiwei Xue [mailto:[email-masked]]
Sent: Tuesday, June 19, 2012 4:12 AM
To: Songhaibin
Cc: [email-masked]
Subject: Re: [decade] Open discussion: Application agnostic content distribution

Hi Haibin,

I am confused, who dose 'application agnostic content distribution platform' face with? what kinds of applications do mean by 'content distribution applications will be likely to use it'. Do you mean content provider?

[Haibin] Content providers can be the users. Besides that, applications that do content distribution work can also be the users.

It my understanding that 'application agnostic content distribution platform' is doing something CDN do. So what is new for 'application agnostic content distribution platform', who will benefit from it?

[Haibin] The content distribution applications can benefit from an open distribution paradigm that is easy to use, easy to migrate, and easy to integrate their own distribution polices. The platform service providers will also benefit with open connectivity to other service providers and easy maintenance.

Any thoughts?

BR,
-Haibin

Thanks.

Haiwei
On Thu, Jun 14, 2012 at 2:42 AM, Songhaibin <[email-masked]<mailto:[email-masked]>> wrote:
Hi guys,

I think this is a very general and open question to all of the working group. Which is also somewhat the high level goal of the working group. In order to do a general application agnostic content distribution platform inside the network, and the content distribution applications will be likely to use it.
what kind of functions and components it should support? Or more general, what does the platform need? If you can answer with use case, that would be good. The architecture document is a good base. But I do not want the limit.

BR,

---

From: Haiwei Xue <[email-masked]>
Date: Tue, 19 Jun 2012 10:49:38 -0400

Hi Haibin

Thank you for your reply. While, I am not convinced:

On Tue, Jun 19, 2012 at 4:52 AM, Songhaibin <[email-masked]> wrote:

>  Hi Haiwei,****
>
> ** **
>
> Thank you for your questions.****
>
> ** **
>
> *From:* Haiwei Xue [mailto:[email-masked]]
> *Sent:* Tuesday, June 19, 2012 4:12 AM
> *To:* Songhaibin
> *Cc:* [email-masked]
>
> *Subject:* Re: [decade] Open discussion: Application agnostic content
> distribution****
>
>  ** **
>
> Hi Haibin,****
>
> ** **
>
> I am confused, who dose 'application agnostic content distribution
> platform' face with? what kinds of applications do mean by 'content
> distribution applications will be likely to use it'. Do you mean content
> provider?****
>
> ** **
>
> [Haibin] Content providers can be the users. Besides that, applications
> that do content distribution work can also be the users.****
>
> ** **
>
> It my understanding that 'application agnostic content distribution
> platform' is doing something CDN do. So what is new for 'application
> agnostic content distribution platform', who will benefit from it?****
>
> ** **
>
> [Haibin] The content distribution applications can benefit from an open
> distribution paradigm that is easy to use, easy to migrate, and easy to
> integrate their own distribution polices.
>

     [Haiwei] Q1: If content distribution application integrated their own
distribution polices, then they need to aware that is such a supported
platform, so would the term 'application agnostic' make sense?

     [Haiwei] Q2: What do you mean by 'distribution polices', content
distribution algorithms? or what?  If it is algorithms, then simple API is
not enough since these algorithms could be very complex, which means that
the platform need to run applications‘ program. Then we need talk about
more on the cloud computing.

> The platform service providers will also benefit with open connectivity to
> other service providers and easy maintenance.****
>
> ** **
>
> Any thoughts?
>

   [Haiwei] Q3: I still did not see the benefit of this platform, For
content provider using this platform is no difference with CDN; For content
applications, Can you show me a example/usecase of such a application that
it need to deploy their own  distribution polices?

Thanks.

Haiwei

> ****
>
> ** **
>
> BR,****
>
> -Haibin****
>
> ** **
>
> ** **
>
> ** **
>
> Thanks.****
>
> ** **
>
> Haiwei****
>
> On Thu, Jun 14, 2012 at 2:42 AM, Songhaibin <[email-masked]>
> wrote:****
>
> Hi guys,
>
> likely to use it.****
> -Haibin (as individual contributor)****
>
>  ** **
>

---

From: Zongning <[email-masked]>
Date: Wed, 20 Jun 2012 00:33:49 +0000

Hi, Haiwei,

Thank you for the questions. See my thoughts inline.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Haiwei Xue
Sent: Tuesday, June 19, 2012 10:50 PM
To: Songhaibin
Cc: [email-masked]
Subject: Re: [decade] Open discussion: Application agnostic content distribution

Hi Haibin

On Tue, Jun 19, 2012 at 4:52 AM, Songhaibin <[email-masked]<mailto:[email-masked]>> wrote:
Hi Haiwei,

From: Haiwei Xue [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, June 19, 2012 4:12 AM
To: Songhaibin
Cc: [email-masked]<mailto:[email-masked]>

Hi Haibin,

[Haibin] The content distribution applications can benefit from an open distribution paradigm that is easy to use, easy to migrate, and easy to integrate their own distribution polices.

     [Haiwei] Q1: If content distribution application integrated their own distribution polices, then they need to aware that is such a supported platform, so would the term 'application agnostic' make sense?

[Ning] A1: I think Haibin meant that the content distribution platform should provide general APIs to applications to satisfy content distribution policies such as resource allocation (e.g. bandwidth restriction, how many and when to put data copies) for each content distribution instance. Please refer to DECADE drafts on survey and problem statement.

     [Haiwei] Q2: What do you mean by 'distribution polices', content distribution algorithms? or what?  If it is algorithms, then simple API is not enough since these algorithms could be very complex, which means that the platform need to run applications' program. Then we need talk about more on the cloud computing.

[Ning] A2: I think the content distribution platform will have a clear demarcation with application regarding the distribution policies. The platform will focus on how to efficiently delivery the data in terms of the delivery efficiency, network efficiency. The application will care about the high level flow possibly integrated with business concerns and the properties of the content, e.g. how many sources of the data, where to put the data, etc.

The platform service providers will also benefit with open connectivity to other service providers and easy maintenance.

Any thoughts?

   [Haiwei] Q3: I still did not see the benefit of this platform, For content provider using this platform is no difference with CDN; For content applications, Can you show me a example/usecase of such a application that it need to deploy their own  distribution polices?

[Ning]: A3: For design point of view, an open content distribution platform with general APIs to applications will always give applications more flexible and on-demand capabilities on content distribution. I am not a right person to give you a good example for CP/SP at this moment, but I really believe this is a good point we need to discuss and confirm with CP/SPs.

Thanks.

Haiwei

BR,
-Haibin

Thanks.

Haiwei

BR,