Subject: [Idnet] IETF99 for applying AI/ML into network management: Follow-up

From: Jérôme François <[email-masked]>
Date: Thu, 20 Jul 2017 19:05:17 +0200

Dear all,

Thanks Sheng for having organized the meeting. As we are running out of
time, here are some comments I want to share:
- in the figure with the loop, data to be used could be from other
sources of data out from the network to enrich the analysis
- regarding use cases, it was unclear for me what is your or the group
plan. I think many of us express interests for other use cases but I'm
not sure how to contribute / select use cases (and if we have to ? ->
see next item)
- the immediate plan is to organize a regular workshop (probably within
NMRG). I fully support the idea as this helps to federate the community
first to ambition then coordinated actions (regarding the issues with
datasets for example). But recall that that will be workshop within a RG
group and it is probably too early to identify what should be
standardized  (or look at a WG in that case). Looking at the experience
with flow measurement workshop (within NMRG as well), it should be more
open for presentations and discussions and if a potential standard is
identified, this can be targeted afterwards. 

Best regards,
jerome

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Fri, 21 Jul 2017 02:19:08 +0900

> Dear all,

Dear Jerome,

Thank you very much for your comments. Please find below some hints from
my side.

> sources of data out from the network to enrich the analysis

This is delicate since, strictly speaking, such data cannot form part of
the "control loop" because it is not a variable to control. However, in
a practical view it will be connected to the input of the loop (and I am
currently working on solutions that consider such information in such
point), but it must be clear that it will not form part of the closure
of the loop.

> not sure how to contribute / select use cases (and if we have to ? ->

To my experience, the best option is to find some common requirements to
all use cases we are taking into account and then try to find a smaller
use case that reflects only those requirements. We can then "vote" which
aspects must be considered from the beginning and which can be left for
future use cases.

> identified, this can be targeted afterwards.

That is nice, but has some risks. We must first set a strong topic or we
can find a too high diversion from the issues that IDNET must resolve.

I agree with you in that it is good to gather proposals to resolve the
dataset issues and it can be the main (I propose that only) topic of the
first workshop on IDNET.

Those are my 2 cents. Thanks for reading.

> jerome

Regards,
Pedro

-- 
Pedro Martinez-Julia
Network Science and Convergence Device Technology Laboratory
Network System Research Institute
National Institute of Information and Communications Technology (NICT)
4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
Email: [email-masked]
---------------------------------------------------------
*** Entia non sunt multiplicanda praeter necessitatem ***

---

From: yanshen <[email-masked]>
Date: Fri, 21 Jul 2017 07:32:58 +0000

Dear Jerome,

Thanks for your comment. Please find the reply in below. 

> -----Original Message-----
> From: IDNET [mailto:[email-masked]] On Behalf Of Jér?me Fran?ois
> Sent: Friday, July 21, 2017 1:05 AM
> To: [email-masked]
> Subject: [Idnet] IETF99 for applying AI/ML into network management: Follow-up
> 
> Dear all,
> 
> Thanks Sheng for having organized the meeting. As we are running out of time,
> here are some comments I want to share:
> - in the figure with the loop, data to be used could be from other sources of data
> out from the network to enrich the analysis

It is significant. The data indeed is from not only the network itself but also from other sources. Sometimes I ever thought the "non-network data" influence more, for example, a popular movie or online sales may detonate the traffic. In details, different users can utilize their private data to build their requirements. Such as, a content provider may use subscribers' data to train and optimize its delivery strategy. From the view of standardization, more data source could expand the impact and attract more players joining in the work.

> - regarding use cases, it was unclear for me what is your or the group plan. I think
> many of us express interests for other use cases but I'm not sure how to
> contribute / select use cases (and if we have to ? -> see next item)
> - the immediate plan is to organize a regular workshop (probably within NMRG). I
> fully support the idea as this helps to federate the community first to ambition
> then coordinated actions (regarding the issues with datasets for example). But
> recall that that will be workshop within a RG group and it is probably too early to
> identify what should be standardized  (or look at a WG in that case). Looking at
> the experience with flow measurement workshop (within NMRG as well), it
> should be more open for presentations and discussions and if a potential
> standard is identified, this can be targeted afterwards.

I support the regular workshop (whatever in any form). I fully understand your worried about the "too early". However, I think it is better to find a common or say a potential standardized point to concentrate the focus. After that, we can expand our focus to other valuable area. A details point may help us to clear up the thought, during which we may evolve a clear methodology for the future work. Of course I totally agree with your open attitude. I mean only to concentrate the topic firstly. 

> Jerome

Thanks for reading. And I hope I have caught up all your points here.

Yansen

> 
> _______________________________________________
> IDNET mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idnet

---

From: yanshen <[email-masked]>
Date: Fri, 21 Jul 2017 08:52:33 +0000

Dear Pedro,

Thanks for your hints. Please find some hints from my side.

> From: IDNET [mailto:[email-masked]] On Behalf Of Pedro Martinez-Julia
> Sent: Friday, July 21, 2017 1:19 AM
> To: Jérôme François <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [Idnet] IETF99 for applying AI/ML into network management:
> Follow-up
> 
> Thank you very much for your comments. Please find below some hints from my
> side.
> 
> > Thanks Sheng for having organized the meeting. As we are running out
> > of time, here are some comments I want to share:
> This is delicate since, strictly speaking, such data cannot form part of the "control
> loop" because it is not a variable to control. However, in a practical view it will be
> connected to the input of the loop (and I am currently working on solutions that
> consider such information in such point), but it must be clear that it will not form
> part of the closure of the loop.

I am not sure whether you means that there should be an "interface" or an "entity" between the other source data and control loop?

BTW, have you ever obtain some achievement about such point that can be shared?

> To my experience, the best option is to find some common requirements to all
> use cases we are taking into account and then try to find a smaller use case that
> reflects only those requirements. We can then "vote" which aspects must be
> considered from the beginning and which can be left for future use cases.

I think that since NMRG may be a probably choice, it is better to explore via this thread. 

> 
> > - the immediate plan is to organize a regular workshop (probably
> > within NMRG). I fully support the idea as this helps to federate the
> > community first to ambition then coordinated actions (regarding the
> > issues with datasets for example). But recall that that will be
> > workshop within a RG group and it is probably too early to identify
> > what should be standardized  (or look at a WG in that case). Looking
> > at the experience with flow measurement workshop (within NMRG as
> > well), it should be more open for presentations and discussions and if
> > a potential standard is identified, this can be targeted afterwards.
> 
> That is nice, but has some risks. We must first set a strong topic or we can find a
> too high diversion from the issues that IDNET must resolve.
> 
> I agree with you in that it is good to gather proposals to resolve the dataset issues
> and it can be the main (I propose that only) topic of the first workshop on IDNET.
> 

Thanks for reading. 

Regards,

Yansen

> --
> Pedro Martinez-Julia
> Network Science and Convergence Device Technology Laboratory Network
> System Research Institute National Institute of Information and Communications
> Technology (NICT) 4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
> Email: [email-masked]

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Fri, 21 Jul 2017 18:17:30 +0900

Dear Yanshen,

Please find my reply below.

On Fri, Jul 21, 2017 at 08:52:33AM +0000, yanshen wrote:
> I am not sure whether you means that there should be an "interface" or
> an "entity" between the other source data and control loop?

I'm sorry, I mean that, by definition, a "closed control loop" intends
to maintain a particular variable within some defined values by reading
it and enforcing the necessary actions to affect it. Any external data
cannot be considered as such variable so they cannot be part of the
closed control loop. However, it can be used to help controlling other
variables, but it must be clearly defined how the external data is
integrated and derived into the changes to the other variable.

> BTW, have you ever obtain some achievement about such point that can

My results are trivial and do not shed any light to this issue and I do
not try to fit external data into the closed control loop, I just use it
to complement the control of the "load" variable.

> I think that since NMRG may be a probably choice, it is better to

So let's go that way.

Regards,
Pedro

---

From: yanshen <[email-masked]>
Date: Mon, 24 Jul 2017 08:44:01 +0000

Dear Pedro, 

> Sent: Friday, July 21, 2017 5:18 PM
> To: yanshen <[email-masked]>
> Cc: [email-masked]
> I'm sorry, I mean that, by definition, a "closed control loop" intends to maintain a
> particular variable within some defined values by reading it and enforcing the
> necessary actions to affect it. Any external data cannot be considered as such
> variable so they cannot be part of the closed control loop. However, it can be
> used to help controlling other variables, but it must be clearly defined how the
> external data is integrated and derived into the changes to the other variable.
> 

Indeed. So that there should be a little complement in the loops. [see the attachment]

For compatible, there may be an interface or device to "receive" the external data? 

Regards, 

Yansen

> My results are trivial and do not shed any light to this issue and I do not try to fit
> external data into the closed control loop, I just use it to complement the control
> of the "load" variable.
>

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Mon, 24 Jul 2017 17:59:57 +0900

Dear Yanshen,

> Indeed. So that there should be a little complement in the loops. [see

I see and I find it a perfect representation and fit for the role that
external data would have (fed directly to the brain!).

> For compatible, there may be an interface or device to "receive" the

I think it would be nice to have such interface and standardize it so
data producers/owners can be easily plugged to data consumers, promoting
the proliferation of different solutions that could be exchanged and/or
complemented to each other...

Regards,
Pedro

---

From: yanshen <[email-masked]>
Date: Tue, 25 Jul 2017 14:10:26 +0000

Dear Pedro,

Sorry for replying late. 

> From: Pedro Martinez-Julia [mailto:[email-masked]]
> Sent: Monday, July 24, 2017 5:00 PM
> I see and I find it a perfect representation and fit for the role that external data
> would have (fed directly to the brain!).
> 

Recently I meet an interesting question is what data should belong to internal and what data should be external?

A simple case is the ML algorithm, such as reinforcement learning, needs "rewards" to  correct the error and improve its model during iterations. Some reward parameters may be from subscribers. For example, when we use the QoE value to present the reward in each period. These feedback QoE values should be internal or external? 

BTW, I think there should be some methods to support the "reward" feedback. For instance, there is some proposal to use RSVP protocol's option to transmit the QoS value. And in the scenarios of video transmission, some proposal uses TCP option to feedback the QoS of current flow. 

Maybe there is room for the future.

Thanks for your reply and suggestions.

Yansen

> I think it would be nice to have such interface and standardize it so data
> producers/owners can be easily plugged to data consumers, promoting the
> proliferation of different solutions that could be exchanged and/or

---

From: Jérôme François <[email-masked]>
Date: Tue, 25 Jul 2017 16:26:00 +0200

Hi,

> I support the regular workshop (whatever in any form). I fully understand your worried about the "too early". However, I think it is better to find a common or say a potential standardized point to…
So, why not targeting a WG (rather than a RG because in my understanding
primary focus of RG is not standardization) ?

best regards,
jerome

---

From: Jérôme François <[email-masked]>
Date: Tue, 25 Jul 2017 16:28:08 +0200

Hi

> For compatible, there may be an interface or device to "receive" the external data? 
I'm not sure wheter the external data loop should be linked to the brain
or data analysis or both.

jerome

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Wed, 26 Jul 2017 09:12:09 +0900

> or data analysis or both.

It is supposed to be already analyzed (not raw data but events), so it
is better to be inputted to the brain.

Regards,
Pedro

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Wed, 26 Jul 2017 09:24:17 +0900

Dear Yanshen,

Thanks for your answer, do not worry about the timing, and find my
answer in-line :-).

On Tue, Jul 25, 2017 at 02:10:26PM +0000, yanshen wrote:
> Dear Pedro,
> 
> Sorry for replying late. 
> 
> Recently I meet an interesting question is what data should belong to
> internal and what data should be external?

For me that depends on the algorithmic representation. We have two types
of variables:

- Measurable and controllable variables, such as the load of a resource
  managed by the solution, which can be known and can be altered by some
  decision of such solution (increase/decrease the amount of resources).

- Measurable but non-controllable, which are outside the control of the
  solution. They can be known and used to take decisions but cannot be
  altered (directly or indirectly) by the solution. A simple example can
  be found in the occurrence of some event that can affect to a system,
  such as the number of attendees to a baseball match. The management
  solution cannot control such variable but uses it to determine the
  amount of network resources assigned to the network of the stadium.

> A simple case is the ML algorithm, such as reinforcement learning,
> needs "rewards" to  correct the error and improve its model during
> iterations. Some reward parameters may be from subscribers. For
> example, when we use the QoE value to present the reward in each
> period. These feedback QoE values should be internal or external? 

Those rewards are "controllable variables", and thus must be introduced
to the whole loop, so the management solution uses them, and maybe other
information, to determine actions that will change/improve them.

> BTW, I think there should be some methods to support the "reward"
> feedback. For instance, there is some proposal to use RSVP protocol's
> option to transmit the QoS value. And in the scenarios of video
> transmission, some proposal uses TCP option to feedback the QoS of
> current flow.

That is a good option for a new draft, but it would be more related to
other WG than to IDNET "per se". I find it very interesting so, if you
plan to work on it, please count on me. I think we can circulate some
initial version to the NMRG and from there we can see where it fits.

> Thanks for your reply and suggestions.

Let's be in contact...

Regards,
Pedro

---

From: Jérôme François <[email-masked]>
Date: Wed, 26 Jul 2017 06:50:29 +0200

Hi Pedro,

> is better to be inputted to the brain.
Assuming the case where you have flow records (raw event) you want to
label regarding the matching of IP addresses in external blacklists, I
would say it has to be done to the data analysis.

jerome

---

From: Sheng Jiang <[email-masked]>
Date: Wed, 26 Jul 2017 11:12:22 +0000

> > well), it should be more open for presentations and discussions and if a
> potential standard is identified, this can be targeted afterwards.
> > I support the regular workshop (whatever in any form). I fully understand your
> worried about the "too early". However, I think it is better to find a common or
> say a potential standardized point to concentrate the focus. After that, we can
> expand our focus to other valuable area. A details point may help us to clear up
> the thought, during which we may evolve a clear methodology for the future
> work. Of course I totally agree with your open attitude. I mean only to
> concentrate the topic firstly.
> >
> >

Hi, Jerome,

Of course, we would like to target a WG if we could. However, so far, we have not converged at all. It seems there are full of possibilities to apply AI/ML in networks. It seems everybody is talking about their own use cases. Until we converged with one or two very specific use cases that are also generic and significance, or we might find some fundamental functionalities that are generically support various AI use cases, we may not have anything to standardized. So far, we are still in the way to find these work items. So, let's keep discussing use cases and fundamental functionalities. Hopefully, with more discussion, we could reduce our scope to be more and more focus. Then we may have concrete work items to form a WG.

Best regards,

Sheng

---

From: Jérôme François <[email-masked]>
Date: Wed, 26 Jul 2017 21:35:27 +0200

Hi,

> Of course, we would like to target a WG if we could. However, so far, we have not converged at all. It seems there are full of possibilities to apply AI/ML in networks. It seems everybody is talkin…
I agree that before formally targeting a WG, we need something specific
to be address. I said "something" because it could be a use case where
AI is applied, a generic problem across use case, interfaces....

But my little concern is that I would avoid that planned workshop would
be stricly limited to particular use cases, problems or architecture.
In my opinion, we have to take care of the two items (community building
with the workshop and scope refinement) in parallel but we could, of
course, expect that the workshop gives us some input to refine our
target in the proper way.

jerome

---

From: yanshen <[email-masked]>
Date: Thu, 27 Jul 2017 01:54:58 +0000

Dear Pedro,

> Sent: Wednesday, July 26, 2017 8:24 AM
> Thanks for your answer, do not worry about the timing, and find my answer
> in-line :-).
> 
> For me that depends on the algorithmic representation. We have two types of
> variables:
> 

I have a similar thought. I try to divide the variables into objective and subjective (may corresponding to your controllable and non-controllable. Perhaps a little different). 
	- The objective data means that we can capture, input or measure it periodically from any source (whatever it is). This class of data needs to be focused and formatted, including the context and format and so on. One of the cases is QoS value which I just mentioned. 
	- The subjective data means that it is imported into the machine (AI/Brain/Knowledge system) temporarily or irregularly. This class of data may be high-level and diversity. The solution for them I think should be pushed to application layer. It is not our dishes. Another thought is that we should try to obtain more subjective data by objective way, for example, we should try to change the way of obtaining the "reward" feedback from "user randomly report" to "periodically capture". Unless such, it should not be included into our consideration. 

I think it is significant to combine our two dimensions into one. Of course other dimensions may also considerable. 

> Those rewards are "controllable variables", and thus must be introduced to the
> whole loop, so the management solution uses them, and maybe other

Yeah, it is controllable. But it is troubled by using my benchmark that some of the rewards can be object but others are not.

> That is a good option for a new draft, but it would be more related to other WG
> than to IDNET "per se". I find it very interesting so, if you plan to work on it,
> please count on me. I think we can circulate some initial version to the NMRG and
> from there we can see where it fits.
> 

My pleasure! Keep in touch. 

Yansen

---

From: yanshen <[email-masked]>
Date: Thu, 27 Jul 2017 01:59:02 +0000

Hi Jerome,

> From: Jérôme François [mailto:[email-masked]]
> Sent: Tuesday, July 25, 2017 10:26 PM
> To: yanshen <[email-masked]>; [email-masked]

I am troubled in finding a satisfied WG. Any suggestion? For example, in measurement area.

Thanks for your reply.

Regards,

Yansen

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Thu, 27 Jul 2017 11:16:22 +0900

Dear Yanshen,

Thanks for your comments, please find my reply in-line.

On Thu, Jul 27, 2017 at 01:54:58AM +0000, yanshen wrote:
> I have a similar thought. I try to divide the variables into objective
> and subjective (may corresponding to your controllable and
> non-controllable. Perhaps a little different). 
> 
> 	- The objective data means that we can capture, input or measure it
> 	periodically from any source (whatever it is). This class of data
> 	needs to be focused and formatted, including the context and format
> 	and so on. One of the cases is QoS value which I just mentioned. 
> 
> 	- The subjective data means that it is imported into the machine
> 	(AI/Brain/Knowledge system) temporarily or irregularly. This class
> 	of data may be high-level and diversity. The solution for them I
> 	think should be pushed to application layer. It is not our dishes.
> 	Another thought is that we should try to obtain more subjective data
> 	by objective way, for example, we should try to change the way of
> 	obtaining the "reward" feedback from "user randomly report" to
> 	"periodically capture". Unless such, it should not be included into
> 	our consideration. 
> 
> I think it is significant to combine our two dimensions into one. Of
> course other dimensions may also considerable.

They are different aspects and, of course, they must be combined. My
classification is based on the "control theory", which clearly states
what a controlled variable means and the strict definition of the loop
closure (closed-loop controller), which implies to check such variables
and confirm they are changed according to the intended objective after
some change/s is/are applied to the controlled environment.

On the other hand, your view is more practical in terms of the scope of
the information in relation to the AI algorithm and its needs. It would
be good to simplify/unify both dimensions but also to simplify/unify the
differences within them. I mean that it would be probably a good option
to consider any kind of data homogeneously. I am using a simple ontology
to resolve such problem, so we could probably use it and a corresponding
YANG model, as I proposed in my other message, to achieve such unified
view. Please, let me know your thoughts. Thank you very much.

Regards,
Pedro

---

From: Simone Ferlin <[email-masked]>
Date: Thu, 27 Jul 2017 12:04:58 +0900

Hello everyone,

I have been following this list for some time, but I have not been in
the meetings (f2f nor online) yet due to lack of time.
Do you have a summary of these meetings, including the topics
discussed at IETF'99, so that people like me could engage in the
discussions? I see you started a thread here, but not with much I
could understand the starting point.

Thanks and cheers,
Simone

---

From: Sheng Jiang <[email-masked]>
Date: Thu, 27 Jul 2017 11:25:50 +0000

Hi, Jerome,

I do understand your concern. One of the historic stories is that we had two NMRG sessions/workshops back to IETF 88, 89. It served us both purposes - community building and scope refinement well. Because these two sessions, we developed two RFC 7575 & 7576. It gives us good foundation to form ANIMA WG. The problem we have now is actually TOO open and wide. If we could convergent on GENERIC problem across use case, interfaces, foundation functionalities, it would be the best. If we don't know what might be GENERIC for now, we may have to deeply study various use cases, then try to find the common among them.

Regards,

Sheng

> Sent: Thursday, July 27, 2017 3:35 AM
> To: Sheng Jiang; yanshen; [email-masked]
> > Of course, we would like to target a WG if we could. However, so far, we have
> not converged at all. It seems there are full of possibilities to apply AI/ML in
> networks. It seems everybody is talking about their own use cases. Until we
> converged with one or two very specific use cases that are also generic and
> significance, or we might find some fundamental functionalities that are
> generically support various AI use cases, we may not have anything to
> standardized. So far, we are still in the way to find these work items. So, let's
> keep discussing use cases and fundamental functionalities. Hopefully, with
> more discussion, we could reduce our scope to be more and more focus. Then
> we may have concrete work items to form a WG.
> >
> >
> I agree that before formally targeting a WG, we need something specific to be
> address. I said "something" because it could be a use case where AI is applied,
> a generic problem across use case, interfaces....
> 
> But my little concern is that I would avoid that planned workshop would be
> stricly limited to particular use cases, problems or architecture.
> In my opinion, we have to take care of the two items (community building with
> the workshop and scope refinement) in parallel but we could, of course, expect
> that the workshop gives us some input to refine our target in the proper way.
> 
> jerome

---

From: Jérôme François <[email-masked]>
Date: Fri, 28 Jul 2017 12:04:04 +0200

Hi Sheng,

That's fine with me. So maybe, my suggestion would to have NMRG session
where people present their own use cases without giving details about
specific obtained results or ML algorithms but with a focus of what are
the different processing steps, interfaces among them, data to be
used... in an ideal case trying to see how this fits with the proposed loop.

Would that be a good plan ?

jerome

---

From: yanshen <[email-masked]>
Date: Fri, 28 Jul 2017 10:05:05 +0000

Dear Simone,

Sheng had sent a page of PPT which was mostly focused in the meeting. You can find it in: https://www.ietf.org/mail-archive/web/idnet/current/msg00113.html

BTW, according to the discussion in mailing list, Pedro and I made a little modification and you can find it in the attachment: https://www.ietf.org/mail-archive/web/idnet/current/msg00104.html

It is welcome to point out the drawbacks and improve it.

Yansen

> From: Simone Ferlin [mailto:[email-masked]]
> Sent: Thursday, July 27, 2017 11:05 AM
> To: Pedro Martinez-Julia <[email-masked]>
> Cc: yanshen <[email-masked]>; [email-masked]
> I have been following this list for some time, but I have not been in the meetings
> (f2f nor online) yet due to lack of time.
> Do you have a summary of these meetings, including the topics discussed at
> IETF'99, so that people like me could engage in the discussions? I see you started
> a thread here, but not with much I could understand the starting point.
> 
> Thanks and cheers,
> Simone
> 
> >> I have a similar thought. I try to divide the variables into
> >> objective and subjective (may corresponding to your controllable and
> > closure (closed-loop controller), which implies to check such
> > variables and confirm they are changed according to the intended
> > objective after some change/s is/are applied to the controlled environment.
> >
> > On the other hand, your view is more practical in terms of the scope
> > of the information in relation to the AI algorithm and its needs. It
> > would be good to simplify/unify both dimensions but also to
> > simplify/unify the differences within them. I mean that it would be
> > probably a good option to consider any kind of data homogeneously. I
> > am using a simple ontology to resolve such problem, so we could
> > probably use it and a corresponding YANG model, as I proposed in my
> > other message, to achieve such unified view. Please, let me know your
> thoughts. Thank you very much.
> >
> > Regards,
> > Pedro
> >
> > --
> > Pedro Martinez-Julia
> > System Research Institute National Institute of Information and
> > Communications Technology (NICT) 4-2-1, Nukui-Kitamachi, Koganei,
> > Tokyo 184-8795, Japan
> > Email: [email-masked]