Subject: [Idnet] Summary 20170814 & IDN dedicated session call for case

From: yanshen <[email-masked]>
Date: Mon, 14 Aug 2017 03:36:15 +0000

Dear all,

Here is a summary and some index (2017.08.14). Till now, whatever the case is supported or not, I tried to organize all the content and keep the core part. It is still welcome to contribute and discuss.

If I miss something important, please let me know. Apologized in advance.

Yansen

---------  Roadmap  ---------
***Aug. : Collecting the use cases (related with NM). Rough thoughts and requirements
Sep. : Refining the cases and abstract the common elements
Oct. : Deeply analysis. Especially on Data Format, control flow, or other key points
Nov.: F2F discussions on IETF100
---------  Roadmap End  ---------

1. Gap and Requirement Analysis
    1.1 Network Management requirement
    1.2 TBD
2. Use Cases
    2.1 Traffic Prediction
                   Proposed by: [email-masked]
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
                   Abstract: Collect the history traffic data and external data which may influence the traffic. Predict the traffic in short/long/specific term. Avoid the congestion or risk in previously.

    2.2 QoS Management
                   Proposed by: [email-masked]
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
                   Abstract: Use multiple paths to distribute the traffic flows. Adjust the percentages. Avoid congestion and ensure QoS.

    2.3 Application (and/or DDoS) detection
                   Proposed by: [email-masked]
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00133.html
                   Abstract: Detect the application (or attack) from network packets (HTTPS or plain) Collect the history traffic data and identify a service or attack (ex: Skype, Viber, DDoS attack etc.)

         2.4 QoE Management
                   Proposed by: [email-masked]
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00137.html
                   Abstract: Collect low-level metrics (SNR, latency, jitter, losses, etc) and measure QoE. Then use ML to understand what is the relation between satisfactory QoE and the low-level metrics. As an example learn that when delay>N then QoE is degraded, but when M<delay<N then QoE is satisfactory for the customers (please note that QoE cannot be measured directly over your network). This is useful to understand how the network must be operated to provide satisfactory QoE.

         2.5 (Encrypted) Traffic Classification
                   Proposed by: [email-masked]; [email-masked]
                   Track: [Jerome] https://www.ietf.org/mail-archive/web/idnet/current/msg00141.html ; [Min-Suk Kim] https://www.ietf.org/mail-archive/web/idnet/current/msg00153.html
                   Abstract:
                            [Jerome] collect flow-level traffic metrics such as protocol information but also meta metrics such as distribution of packet sizes, inter-arrival times... Then use such information to label the traffic with the underlying application assuming that the granularity of classification may vary (type of application, exact application name, version...)
                            [Min-Suk Kim] continuously collect packet data, then applying learning process for traffic classification with generating application using deep learning models such as CNN (convolutional neural network) and RNN (recurrent neural network). Data-set to apply into the models are generated by processing with features of information from flow in packet data.

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies
    4.1 Benchmarking Framework
                   Proposed by: [email-masked]
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00146.html
                   Abstract: A proper benchmarking framework comprises a set of reference procedures, methods, and models that can (or better *must*) be followed to assess the quality of an AI mechanism proposed to be applied to the network management/control area. Moreover, and much more specific to the IDNET topics, is the inclusion, dependency, or just the general relation of a standard format enforced to the data that is used (input) and produced (output) by the framework, so a kind of "data market" can arise without requiring to transform the data. The initial scope of input/output data would be the datasets, but also the new knowledge items that are stated as a result of applying the benchmarking procedures defined by the framework, which can be collected together to build a database of benchmark results, or just contrasted with other existing entries in the database to know the position of the solution just evaluated. This increases the usefulness of IDNET.

    4.2 TBD

---

From: Haoyu song <[email-masked]>
Date: Mon, 14 Aug 2017 16:47:34 +0000

Yansen,

I see two key use cases are missing in the current list: root cause analysis and anomaly detection. Those two are likely to use ML-based solutions and the first one has already received a lot of research.

Haoyu

From: IDNET [mailto:[email-masked]] On Behalf Of yanshen
Sent: Sunday, August 13, 2017 8:36 PM
To: [email-masked]
Subject: [Idnet] Summary 20170814 & IDN dedicated session call for case

Dear all,

Yansen

---------  Roadmap End  ---------

                   Proposed by: [email-masked]<mailto:[email-masked]>

    2.2 QoS Management
                   Proposed by: [email-masked]<mailto:[email-masked]>
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
                   Abstract: Use multiple paths to distribute the traffic flows. Adjust the percentages. Avoid congestion and ensure QoS.

    2.3 Application (and/or DDoS) detection
                   Proposed by: [email-masked]<mailto:[email-masked]>
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00133.html
                   Abstract: Detect the application (or attack) from network packets (HTTPS or plain) Collect the history traffic data and identify a service or attack (ex: Skype, Viber, DDoS attack etc.)

         2.4 QoE Management
                   Proposed by: [email-masked]<mailto:[email-masked]>
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00137.html
                   Abstract: Collect low-level metrics (SNR, latency, jitter, losses, etc) and measure QoE. Then use ML to understand what is the relation between satisfactory QoE and the low-level metrics. As an example learn that when delay>N then QoE is degraded, but when M<delay<N then QoE is satisfactory for the customers (please note that QoE cannot be measured directly over your network). This is useful to understand how the network must be operated to provide satisfactory QoE.

                   Proposed by: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies
    4.1 Benchmarking Framework
                   Proposed by: [email-masked]<mailto:[email-masked]>
                   Track: https://www.ietf.org/mail-archive/web/idnet/current/msg00146.html
                   Abstract: A proper benchmarking framework comprises a set of reference procedures, methods, and models that can (or better *must*) be followed to assess the quality of an AI mechanism proposed to be applied to the network management/control area. Moreover, and much more specific to the IDNET topics, is the inclusion, dependency, or just the general relation of a standard format enforced to the data that is used (input) and produced (output) by the framework, so a kind of "data market" can arise without requiring to transform the data. The initial scope of input/output data would be the datasets, but also the new knowledge items that are stated as a result of applying the benchmarking procedures defined by the framework, which can be collected together to build a database of benchmark results, or just contrasted with other existing entries in the database to know the position of the solution just evaluated. This increases the usefulness of IDNET.

    4.2 TBD

---

From: Oscar Mauricio Caicedo Rendon <[email-masked]>
Date: Mon, 14 Aug 2017 12:23:56 -0500

Dear all,

Scaling mechanisms in NFV can leverage ML features. For instance,
Reinforcement Learning could be used to tune policies of VNFs scaling (in
memory, processing, number of instances, and so on), targeted to improve
VNFs performance.

Oscar

On Mon, Aug 14, 2017 at 11:47 AM, Haoyu song <[email-masked]> wrote:

> Yansen,
>
>
>
> I see two key use cases are missing in the current list: root cause
> analysis and anomaly detection. Those two are likely to use ML-based
> solutions and the first one has already received a lot of research.
>
>
>
> Haoyu
>
>
>
> *From:* IDNET [mailto:[email-masked]] *On Behalf Of *yanshen
> *Sent:* Sunday, August 13, 2017 8:36 PM
> *To:* [email-masked]
> *Subject:* [Idnet] Summary 20170814 & IDN dedicated session call for case
>
>
>
> Dear all,
>
>
>
> Here is a summary and some index (2017.08.14). Till now, whatever the case
> is supported or not, I tried to organize all the content and keep the core
> part. It is still welcome to contribute and discuss.
>
>
>
>                    Abstract: Collect the history traffic data and external
> data which may influence the traffic. Predict the traffic in
> short/long/specific term. Avoid the congestion or risk in previously.
>
>
>
>     2.2 QoS Management
>
>                    Abstract: Use multiple paths to distribute the traffic
> flows. Adjust the percentages. Avoid congestion and ensure QoS.
>
>
>
>                    Abstract: Detect the application (or attack) from
> network packets (HTTPS or plain) Collect the history traffic data and
> identify a service or attack (ex: Skype, Viber, DDoS attack etc.)
>
>
>
>          2.4 QoE Management
>
>                    Abstract: Collect low-level metrics (SNR, latency,
> jitter, losses, etc) and measure QoE. Then use ML to understand what is the
> relation between satisfactory QoE and the low-level metrics. As an example
> learn that when delay>N then QoE is degraded, but when M<delay<N then QoE
> is satisfactory for the customers (please note that QoE cannot be measured
> directly over your network). This is useful to understand how the network
> must be operated to provide satisfactory QoE.
>
>
>
>                    Track: [Jerome] https://www.ietf.org/mail-
> archive/web/idnet/current/msg00141.html ; [Min-Suk Kim]
> https://www.ietf.org/mail-archive/web/idnet/current/msg00153.html
>
>                    Abstract:
>
>                             [Jerome] collect flow-level traffic metrics
> such as protocol information but also meta metrics such as distribution of
> packet sizes, inter-arrival times... Then use such information to label the
> traffic with the underlying application assuming that the granularity of
> classification may vary (type of application, exact application name,
> version...)
>
>                             [Min-Suk Kim] continuously collect packet
> data, then applying learning process for traffic classification with
> generating application using deep learning models such as CNN
> (convolutional neural network) and RNN (recurrent neural network). Data-set
> to apply into the models are generated by processing with features of
> information from flow in packet data.
>
>
>
>          2.6 TBD
>
>
>
> 3. Data Focus
>
>     3.1 Data attribute
>
>     3.2 Data format
>
>     3.3 TBD
>
>
>
> 4. Support Technologies
>
>                    Abstract: A proper benchmarking framework comprises a
> set of reference procedures, methods, and models that can (or better
> *must*) be followed to assess the quality of an AI mechanism proposed to be
> applied to the network management/control area. Moreover, and much more
> specific to the IDNET topics, is the inclusion, dependency, or just the
> general relation of a standard format enforced to the data that is used
> (input) and produced (output) by the framework, so a kind of "data market"
> can arise without requiring to transform the data. The initial scope of
> input/output data would be the datasets, but also the new knowledge items
> that are stated as a result of applying the benchmarking procedures defined
> by the framework, which can be collected together to build a database of
> benchmark results, or just contrasted with other existing entries in the
> database to know the position of the solution just evaluated. This
> increases the usefulness of IDNET.
>
>
>
>     4.2 TBD
>
> _______________________________________________
> IDNET mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idnet
>
>

-- 
*Oscar Mauricio Caicedo Rendón*
*PhD Computer Science - Federal University of Rio Grande do Sul*
*Full Profesor - University of Cauca*

-- 

------------------------------

*Hacia una Universidad comprometida con la Paz Territorial*

---

From: Oscar Mauricio Caicedo Rendon <[email-masked]>
Date: Mon, 14 Aug 2017 12:24:46 -0500

Dear all,

Scaling mechanisms in NFV can leverage ML features. In particular, for
instance, Reinforcement Learning could be used to tune policies scaling
target to

--

---

From: Stenio Fernandes <[email-masked]>
Date: Mon, 14 Aug 2017 14:08:58 -0400

Haoyu,

I'm working on the anomaly detection use case and will send it to the
list this week.

Stenio

On Mon, Aug 14, 2017 at 12:47 PM, Haoyu song <[email-masked]> wrote:
> Yansen,
>
>
>
> I see two key use cases are missing in the current list: root cause analysis
> and anomaly detection. Those two are likely to use ML-based solutions and
> the first one has already received a lot of research.
>
>
>
> Haoyu
>
>
>
>                    Track:
> https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
>
>                    Track:
> https://www.ietf.org/mail-archive/web/idnet/current/msg00131.html
>
>                    Track:
> https://www.ietf.org/mail-archive/web/idnet/current/msg00133.html
>
>                    Abstract: Detect the application (or attack) from network
> packets (HTTPS or plain) Collect the history traffic data and identify a
> service or attack (ex: Skype, Viber, DDoS attack etc.)
>
>
>
>          2.4 QoE Management
>
>                    Track:
> https://www.ietf.org/mail-archive/web/idnet/current/msg00137.html
>
> learn that when delay>N then QoE is degraded, but when M<delay<N then QoE is
> satisfactory for the customers (please note that QoE cannot be measured
>                    Track: [Jerome]
> https://www.ietf.org/mail-archive/web/idnet/current/msg00141.html ; [Min-Suk
> Kim] https://www.ietf.org/mail-archive/web/idnet/current/msg00153.html
>
>                    Abstract:
>
>                             [Jerome] collect flow-level traffic metrics such
> as protocol information but also meta metrics such as distribution of packet
> sizes, inter-arrival times... Then use such information to label the traffic
> with the underlying application assuming that the granularity of
>                             [Min-Suk Kim] continuously collect packet data,
> then applying learning process for traffic classification with generating
> application using deep learning models such as CNN (convolutional neural
> network) and RNN (recurrent neural network). Data-set to apply into the
> models are generated by processing with features of information from flow in
> packet data.
>
>
>
>          2.6 TBD
>
>
>
> 3. Data Focus
>
>     3.1 Data attribute
>
>     3.2 Data format
>
>     3.3 TBD
>
>
>
> 4. Support Technologies
>
>                    Track:
> https://www.ietf.org/mail-archive/web/idnet/current/msg00146.html
>
>                    Abstract: A proper benchmarking framework comprises a set
> of reference procedures, methods, and models that can (or better *must*) be
> followed to assess the quality of an AI mechanism proposed to be applied to
> the network management/control area. Moreover, and much more specific to the
> IDNET topics, is the inclusion, dependency, or just the general relation of
> a standard format enforced to the data that is used (input) and produced
> (output) by the framework, so a kind of "data market" can arise without
> requiring to transform the data. The initial scope of input/output data
> would be the datasets, but also the new knowledge items that are stated as a
> result of applying the benchmarking procedures defined by the framework,
> which can be collected together to build a database of benchmark results, or
> just contrasted with other existing entries in the database to know the
> position of the solution just evaluated. This increases the usefulness of
> IDNET.
>
>
>
>     4.2 TBD
>
>

-- 
Prof. Stenio Fernandes
CIn/UFPE
http://www.steniofernandes.com

---

From: yanshen <[email-masked]>
Date: Tue, 15 Aug 2017 02:37:51 +0000

Hi Haoyu,

Agree. These two crucial cases in Network Management are what we are focusing on now. Since we plan to organize a dedicated session in NMRG, all the discussion will converge to the area of Network Management before Nov. 

Just expect Stenio's output few days later : )

Yansen

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Stenio Fernandes
> Sent: Tuesday, August 15, 2017 2:09 AM
> To: Haoyu song <[email-masked]>
> Cc: yanshen <[email-masked]>; [email-masked]
> Subject: Re: [Idnet] Summary 20170814 & IDN dedicated session call for case
> 
> Haoyu,
> 
> I'm working on the anomaly detection use case and will send it to the list this
> week.
> 
> Stenio
> 
> > Here is a summary and some index (2017.08.14). Till now, whatever the
> > case is supported or not, I tried to organize all the content and keep
> > the core part. It is still welcome to contribute and discuss.
> >
> >
> >
> >                    Abstract: Collect the history traffic data and
> > external data which may influence the traffic. Predict the traffic in
> >                    Abstract: Use multiple paths to distribute the
> > traffic flows. Adjust the percentages. Avoid congestion and ensure QoS.
> >
> >
> >
> > jitter, losses, etc) and measure QoE. Then use ML to understand what
> > is the relation between satisfactory QoE and the low-level metrics. As
> > an example learn that when delay>N then QoE is degraded, but when
> > M<delay<N then QoE is satisfactory for the customers (please note that
> > QoE cannot be measured directly over your network). This is useful to
> > understand how the network must be operated to provide satisfactory QoE.
> >
> >
> >
> > https://www.ietf.org/mail-archive/web/idnet/current/msg00141.html ;
> > [Min-Suk Kim]
> >                             [Jerome] collect flow-level traffic
> > metrics such as protocol information but also meta metrics such as
> > distribution of packet sizes, inter-arrival times... Then use such
> > information to label the traffic with the underlying application
> > assuming that the granularity of classification may vary (type of
> > application, exact application name,
> > version...)
> >
> > (convolutional neural
> > network) and RNN (recurrent neural network). Data-set to apply into
> > the models are generated by processing with features of information
> > from flow in packet data.
> >
> >
> >
> >          2.6 TBD
> >
> >
> >
> > 3. Data Focus
> >
> >     3.1 Data attribute
> >
> >     3.2 Data format
> >
> >     3.3 TBD
> >
> >
> >
> > 4. Support Technologies
> >
> >                    Abstract: A proper benchmarking framework
> comprises
> > a set of reference procedures, methods, and models that can (or better
> > *must*) be followed to assess the quality of an AI mechanism proposed
> > to be applied to the network management/control area. Moreover, and
> > much more specific to the IDNET topics, is the inclusion, dependency,
> > or just the general relation of a standard format enforced to the data
> > that is used (input) and produced
> > (output) by the framework, so a kind of "data market" can arise
> > without requiring to transform the data. The initial scope of
> > input/output data would be the datasets, but also the new knowledge
> > items that are stated as a result of applying the benchmarking
> > procedures defined by the framework, which can be collected together
> > to build a database of benchmark results, or just contrasted with
> > other existing entries in the database to know the position of the
> > solution just evaluated. This increases the usefulness of IDNET.
> >
> >
> >
> >     4.2 TBD
> >
> >

---

From: yanshen <[email-masked]>
Date: Tue, 15 Aug 2017 12:25:06 +0000

Hi Oscar,

There are quite numbers of schemes that related with NFV scenario.

Personally, this case can be concluded into “resource allocation”. Not only the VNF resource, but also other resource. Such as,
         * In Data Center Network, the link resource or bandwidth allocation.
         * In the gateway device, the bandwidth allocation of different applications / priorities / subscribers / ports / etc…
         * TBD

I think that the basic ideas of these cases are similar: real-time measuring -> training -> optimizing the key characters -> next iteration. The difference is that they focus on various parameters. So this class questions can become a typical case.

Just my two cents.

Regards,

Yansen

From: Oscar Mauricio Caicedo Rendon [mailto:[email-masked]]
Sent: Tuesday, August 15, 2017 1:24 AM
To: Haoyu song <[email-masked]>
Cc: yanshen <[email-masked]>; [email-masked]
Subject: Re: [Idnet] Summary 20170814 & IDN dedicated session call for case

Dear all,

Scaling mechanisms in NFV can leverage ML features. For instance, Reinforcement Learning could be used to tune policies of VNFs scaling (in memory, processing, number of instances, and so on), targeted to improve VNFs performance.

Oscar

On Mon, Aug 14, 2017 at 11:47 AM, Haoyu song <[email-masked]<mailto:[email-masked]>> wrote:
Yansen,

Haoyu

From: IDNET [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of yanshen
Sent: Sunday, August 13, 2017 8:36 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [Idnet] Summary 20170814 & IDN dedicated session call for case

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

_______________________________________________
IDNET mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idnet

--
Oscar Mauricio Caicedo Rendón
PhD Computer Science - Federal University of Rio Grande do Sul
Full Profesor - University of Cauca

________________________________

Hacia una Universidad comprometida con la Paz Territorial

---

From: Hesham ElBakoury <[email-masked]>
Date: Tue, 15 Aug 2017 12:48:47 +0000

Hi Yanshen,

In all of these use cases what problems we are trying to solve that have not been solved before or why existing solutions are not good enough ?

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: Haoyu song <[email-masked]>
Date: Tue, 15 Aug 2017 20:09:53 +0000

Obviously we are not going to apply ML on networking just for the sake of ML. In my opinion, the only reason to apply ML is that it’s proved to be a better (or only) way to solve a problem. We have seen a few successful examples such as ETA and root cause analysis, and ML approaches seem promising on solving some networking problems， but I also agree that we are still far from  the conclusion (or don’t even have clear understanding) that all these use cases can be better solved by ML. Then it is even farther to think about standardizing the related protocols and interfaces at the current stage. That said, this doesn’t prevent us from studying the possibilities. The most important thing is to develop more solid use cases to demonstrate that ML is not only useful but preferred. The current list is still too high level which doesn’t help me gain such confidence yet. Hopefully more detailed reports will come.

Haoyu

From: IDNET [mailto:[email-masked]] On Behalf Of Hesham ElBakoury
Sent: Tuesday, August 15, 2017 5:49 AM
To: yanshen <[email-masked]>; [email-masked]
Subject: Re: [Idnet] Summary 20170814 & IDN dedicated session call for case

Hi Yanshen,

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: Hesham ElBakoury <[email-masked]>
Date: Wed, 16 Aug 2017 02:21:44 +0000

I would think that the self-service revolution can provide useful use cases for ML/AI.

Hesham

From: Haoyu song
Sent: Tuesday, August 15, 2017 1:10 PM
To: Hesham ElBakoury; yanshen; [email-masked]
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Haoyu

Hi Yanshen,

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: yanshen <[email-masked]>
Date: Wed, 16 Aug 2017 14:25:53 +0000

Hi Hesham,

I think this question seems simple but really significant. Please forgive if the answer was redundant. And it is welcome to point out my lacks if you have some different views.

In general, the new method may bring potential benefits. There had been some discussions before in our mail list. Please search the keyword "benefit" and check the result. But what I want to describe here are the following several points.

Firstly, in essence, the AI method can solve the scalability problem in each use case but existing methods cannot. In other words, the existing methods can hardly cover the complexity along with the huge increment of service in the future by repeating the current solutions multi-times.  For example, if the cost (including time and economic) of configuring a VPN is C, so that it will cost at least 100*C for 100 VPNs (this is ideal, it is even worse in practice). Even though somebody may say the software programs can also provide the automatic solution for this problems, but the over-linear incremental complexity will easily breakthrough the upper cost limit of  that we can pay. However, since the AI method may solve the same question in yet another new way so that it may change the complexity from over-linear growth to logarithmic growth (or even lower). This is significant because the current methods and their improved version can hardly implement such a change. I think this is one of the core problems that AI solves. It builds up a new pattern that can bear the pressure from new services and new requirements, which the traditional method cannot support.

Secondly, the new method can increase the automation level of the network but not only provide an automatic program. The AI method can make more process automatic such as the decision and policy deploying. Some of them cannot be implemented by program. For example, if we plan to capture and modify a series of devices/networks configuration, how can we implement that? The devices may be from different venders. The topology may be different. Their interfaces may be different. This situation is hard to solve by "if-else-then" logic or very hard. However, if we can abstract the configuration and the policies with some unified expression, such as using standard 0-1 series or formatted packet, the consulting between various devices will be simplified a lot. Of course, the "if-else-then" logic can implement that but the core of problem is the complexity.

However, some of the cases should not be mentioned. The aim of IDNet is introducing AI to serve the Network but not Application. Therefore, any case, whose output implements only a new function or new application, may be not suitable (personal view). Because it might be hardly produced by existing method but it is only a new tool but does not serve the network. We should focus on the case, whose output produces a decision, a strategy, or a configuration relative with network/device, or the cases that directly serve the Network Management, especially improving the efficiency and decreasing the cost. They are valuable.  So the answer may not depend on whether the existing methods can or cannot, but depends on the AI can or cannot provide a new form.

BTW, the new AI method may perform "better" than existing solutions. But in my opinion, this "better" cannot become the sufficient condition to apply because it is often covered by the cost of deployment.

In a word, in any cases, the IDNet will provide a scalability solutions and increase the automation level that the existing method and automatic software programs cannot. And our work here is to explore the standardization points and define the process, the data format, the architecture and etc., which support the compatibility for the various function entities. So that make the AI method enlarge the power of the network.

Regards,

Yansen

From: Hesham ElBakoury
Sent: Tuesday, August 15, 2017 8:49 PM
To: yanshen <[email-masked]>; [email-masked]
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Hi Yanshen,

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: Hesham ElBakoury <[email-masked]>
Date: Wed, 16 Aug 2017 21:20:10 +0000

Hi Yansen,

Thanks for your detailed response!

There are also existing solutions which use AI/ML in (some of) these use cases. How our AI solutions will be better than existing AI solutions.
Let us take classification of encrypted traffic as an example. There are already different solutions that use AI/ML to classify the encrypted traffic.
What are the problems and shortcomings of these solutions  that should be addressed in new and better solutions which also use AI/ML ?

Hesham

From: yanshen
Sent: Wednesday, August 16, 2017 7:26 AM
To: Hesham ElBakoury
Cc: [email-masked]
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Hi Hesham,

Regards,

Yansen

From: Hesham ElBakoury

Hi Yanshen,

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: yanshen <[email-masked]>
Date: Thu, 17 Aug 2017 04:09:19 +0000

Hi Hesham,

Personal view. It is not important that whether the existing traffic recognizing methods are accurate or whether the problems are serious. We should focus on the previous and following steps. Whether we can abstract the data or process becoming a standard format? And how to transmit the result into the form that can be used in the following configuring process.

Yansen

From: Hesham ElBakoury
Sent: Thursday, August 17, 2017 5:20 AM
To: yanshen <[email-masked]>
Cc: [email-masked]
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Hi Yansen,

Hesham

From: yanshen
Sent: Wednesday, August 16, 2017 7:26 AM
To: Hesham ElBakoury
Cc: [email-masked]<mailto:[email-masked]>
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Hi Hesham,

Regards,

Yansen

From: Hesham ElBakoury
Sent: Tuesday, August 15, 2017 8:49 PM
To: yanshen <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: Summary 20170814 & IDN dedicated session call for case

Hi Yanshen,

Thanks

Hesham

Dear all,

Yansen

---------  Roadmap End  ---------

    2.2 QoS Management

         2.4 QoE Management

         2.6 TBD

3. Data Focus
    3.1 Data attribute
    3.2 Data format
    3.3 TBD

4. Support Technologies

    4.2 TBD

---

From: Albert Cabellos <[email-masked]>
Date: Fri, 18 Aug 2017 15:23:10 +0900

Hi Hesham

Complementing what Yanshen mentioned:

ML (and specifically neural-nets) provide a modelling technique that work
very well with complex and/or nonlinear scenarios.

Traditional modeling techniques are analytical (e.g., markov chain, graph
models, etc) and computational models (e.g., simulators).

Analytical models do not work well with nonlinear behaviors and typically
require strong assumptions about the underlying hardware.

Simulator do not scale well with complexity, complex systems require long
development and/or runtimes.

ML work very well with nonlinear behaviors and scale very well with
complexity and dimensionality. The main drawback of ML techniques is the
training phase, for this you require a data-set and computational power.
Once trained the neural-net is lightweight and fast.

This has important advantages in optimization (QoS, VNF embedding) and
prediction (traffic, etc) scenarios.

Albert

On Thu, Aug 17, 2017 at 1:09 PM, yanshen <[email-masked]> wrote:

> Hi Hesham,
>
>
>
> Personal view. It is not important that whether the existing traffic
> recognizing methods are accurate or whether the problems are serious. We
> should focus on the previous and following steps. Whether we can abstract
> the data or process becoming a standard format? And how to transmit the
> result into the form that can be used in the following configuring process.
>
>
>
> Yansen
>
>
>
> *From:* Hesham ElBakoury
> *Sent:* Thursday, August 17, 2017 5:20 AM
> *To:* yanshen <[email-masked]>
>
> *Cc:* [email-masked]
> *Subject:* RE: Summary 20170814 & IDN dedicated session call for case
>
>
>
> Hi Yansen,
>
>
>
> There are also existing solutions which use AI/ML in (some of) these use
> cases. How our AI solutions will be better than existing AI solutions.
>
> Let us take classification of encrypted traffic as an example. There are
> already different solutions that use AI/ML to classify the encrypted
> traffic.
>
> What are the problems and shortcomings of these solutions  that should be
> addressed in new and better solutions which also use AI/ML ?
>
>
>
> Hesham
>
>
>
> *From:* yanshen
> *Sent:* Wednesday, August 16, 2017 7:26 AM
> *To:* Hesham ElBakoury
> *Cc:* [email-masked]
> *Subject:* RE: Summary 20170814 & IDN dedicated session call for case
>
>
>
> Hi Hesham,
>
>
>
> I think this question seems simple but really significant. Please forgive
> if the answer was redundant. And it is welcome to point out my lacks if you
> have some different views.
>
>
>
> In general, the new method may bring potential benefits. There had been
> some discussions before in our mail list. Please search the keyword
> “benefit” and check the result. But what I want to describe here are the
> following several points.
>
>
>
> Firstly, in essence, the AI method can solve the scalability problem in
> each use case but existing methods cannot. In other words, the existing
> methods can hardly cover the complexity along with the huge increment of
> service in the future by repeating the current solutions multi-times.  For
> example, if the cost (including time and economic) of configuring a VPN is
> C, so that it will cost at least 100*C for 100 VPNs (this is ideal, it is
> even worse in practice). Even though somebody may say the software programs
> can also provide the automatic solution for this problems, but the
> over-linear incremental complexity will easily breakthrough the upper cost
> limit of  that we can pay. However, since the AI method may solve the same
> question in yet another new way so that it may change the complexity from
> over-linear growth to logarithmic growth (or even lower). This is
> significant because the current methods and their improved version can
> hardly implement such a change. I think this is one of the core problems
> that AI solves. It builds up a new pattern that can bear the pressure from
> new services and new requirements, which the traditional method cannot
> support.
>
>
>
> Secondly, the new method can increase the automation level of the network
> but not only provide an automatic program. The AI method can make more
> process automatic such as the decision and policy deploying. Some of them
> cannot be implemented by program. For example, if we plan to capture and
> modify a series of devices/networks configuration, how can we implement
> that? The devices may be from different venders. The topology may be
> different. Their interfaces may be different. This situation is hard to
> solve by “if-else-then” logic or very hard. However, if we can abstract the
> configuration and the policies with some unified expression, such as using
> standard 0-1 series or formatted packet, the consulting between various
> devices will be simplified a lot. Of course, the “if-else-then” logic can
> implement that but the core of problem is the complexity.
>
>
>
> However, some of the cases should not be mentioned. The aim of IDNet is
> introducing AI to serve the Network but not Application. Therefore, any
> case, whose output implements only a new function or new application, may
> be not suitable (personal view). Because it might be hardly produced by
> existing method but it is only a new tool but does not serve the network.
> We should focus on the case, whose output produces a decision, a strategy,
> or a configuration relative with network/device, or the cases that directly
> serve the Network Management, especially improving the efficiency and
> decreasing the cost. They are valuable.  So the answer may not depend on
> whether the existing methods can or cannot, but depends on the AI can or
> cannot provide a new form.
>
>
>
> BTW, the new AI method may perform “better” than existing solutions. But
> in my opinion, this “better” cannot become the sufficient condition to
> apply because it is often covered by the cost of deployment.
>
>
>
> In a word, in any cases, the IDNet will provide a scalability solutions
> and increase the automation level that the existing method and automatic
> software programs cannot. And our work here is to explore the
> standardization points and define the process, the data format, the
> architecture and etc., which support the compatibility for the various
> function entities. So that make the AI method enlarge the power of the
> network.
>
>
>
>
>
> Regards,
>
>
>
> Yansen
>
>
>
> *From:* Hesham ElBakoury
> *Sent:* Tuesday, August 15, 2017 8:49 PM
> *To:* yanshen <[email-masked]>; [email-masked]
> *Subject:* RE: Summary 20170814 & IDN dedicated session call for case
>
>
>
> Hi Yanshen,
>
>
>
> In all of these use cases what problems we are trying to solve that have
> not been solved before or why existing solutions are not good enough ?
>
>
>
> Thanks
>
>
>
> Hesham
>
>
>
> *From:* IDNET [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *yanshen

---

From: Stenio Fernandes <[email-masked]>
Date: Fri, 18 Aug 2017 13:14:17 -0400

Hi Yansen et al.

Here is my take on the anomaly detection use case (2.6). I know that I was
supposed to write just 'rough thoughts and requirements', but I decided to
write in advance regarding the upcoming tasks (sep/oct).

======================================================================

Definitions:

First, it is important to understand that the scientific literature allows
different names for anomaly detection, such as outlier detection, novelty
detection, noise detection, deviation detection, or mining exception. This
not so strict naming also brings a number of definitions for what could be
considered as an outlier or anomaly. A more general definition for an
outlier is an observation that is somewhat inconsistent when compared to
the remainder of the set of observations. In the specific case of a network
anomaly, such observation may cause network failures and performance
problems such as congestions and denial of services (DOS).

Anomalies can be generally classified into three categories, namely point,
contextual, and collective. Point anomaly is related to a single
observation whereas collective anomaly is related to a series of
observations. For instance, a SYN Flood attack can be considered a
collective anomaly since a single TCP SYN segment is valid and is not
considered a point anomaly. Contextual anomaly is the interpretation
whether the point or collective anomaly is, in fact, an anomaly given a
proper context.

Applications:

Abnormal behavior from packets or streams of packets (flows) requires
precise and, in some cases, quick detection so that the network could react
and take appropriate measures to mitigate its short and long term effects.
Ideally, the network must be intelligent enough to automatically learn what
is normal traffic so it could adapt to any abnormal traffic patterns,
including zero-day attacks. Of course, not all anomalies come from
intentional attacks. Abnormal behavior could also come from
misconfiguration or malfunctions in the network.

Data / Features:

Most techniques for anomaly detection use features extracted from transport
or network layer data, mainly due to the widespread adoption of
IPFIX/NetFlow on routers/switches. It is possible to create new variables
(a.k.a. Feature Engineering in the data mining/machine learning lingo) from
the raw data, such as packet or flow inter-arrival times. Depending on the
measurement process one could also include data from other Internet layers
to make the detection more accurate. One must be aware that scalability is
always a concern when dealing with massive amount of data.

The types and characteristics of the input data often limit the choices of
techniques that could be used for anomaly detection. Some techniques
require labeled data (i.e., using prior knowledge to identify an
observation as normal or abnormal), which in most cases requires enormous
processing efforts.

Techniques:

Methods for anomaly detection come from several fields and their
subdisciplines, such as Statistics, Machine Learning, Data Mining,
Information Theory, Spectral Theory, and the like. Therefore, there are a
number of techniques to handle (i.e., detection and/or removal) abnormal
observations. As the main scope and interest of IDNET are on techniques
that can learn from the incoming network traffic and events, the
behavioral-based anomaly detection ones seem the best fit.

Behavioral-based anomaly detection methods are usually classified as
supervised, semi-supervised, or unsupervised, depending on the availability
of labeled data for the training phase. Supervised and semi-supervised
learning require labeled data whereas unsupervised learning is able to work
with unlabeled data.

Unsupervised anomaly detection techniques create initially a region (e.g.,
a cluster in an n-dimensional hyperspace) that represents the limits of a
normal behavior so that any observation beyond those bounds is considered
an anomaly. They can easily (automatically) adapt to changes in the
incoming network traffic/events.

As far as we are concerned to recent systems for anomaly detection, there
is a clear trending on hybrid techniques (i.e., the combination of two or
more techniques) to overcome well-known limitations of each individual
class of techniques, such as low precision/recall, high processing
overhead, and the like. This means in general building a system with two or
more phases that combines supervised, semi-supervised, and unsupervised
learning in sequence.

The outputs of anomaly detection techniques can be scores and/or labels. Of
course, the objectives of the classification problem (i.e., either single
class or multiclass) define their outputs. Therefore, given the type of
classification problem, a number of methods can be applied, such as the
ones based on unsupervised clustering.

Challenges:

Current challenges for network anomaly detection includes i) dealing with
high dimensional data, class imbalance, and noise, ii) performing fast and
accurate feature engineering, iii) ensuring cluster homogeneity, iv)
lowering false alarm rate, and v) handling sequential, spatial, and graph
data simultaneously.

======================================================================

Cheers,

Stenio

On Mon, Aug 14, 2017 at 10:37 PM, yanshen <[email-masked]> wrote:

> Hi Haoyu,
>
> Agree. These two crucial cases in Network Management are what we are
> focusing on now. Since we plan to organize a dedicated session in NMRG, all
> the discussion will converge to the area of Network Management before Nov.
>
> list this