Subject: Re: [Idnet] Benefits of Introducing AI into Network

From: "Fang, Luyuan" <[email-masked]>
Date: Wed, 10 May 2017 06:26:26 +0000

I think what is driving all these points is scale. Without intelligence, very large scale networks simply become unmanageable, and cannot maintain service availability. Just adding work force would not help.
I would suggest to add

-          ability to manage network at scale (as 2nd point)

-          potential self-healing (after “predictive”)

Thx,
Luyuan

From: IDNET <[email-masked]> on behalf of Sheng Jiang <[email-masked]>
Date: Tuesday, May 9, 2017 at 7:53 PM
To: "[email-masked]" <[email-masked]>
Subject: [Idnet] Benefits of Introducing AI into Network

In ETSI NGP ISP, we have an Work Item for IDN (Intelligence-Defined Network). In one of the general sections, We briefly describes the benefits of introducing AI into network, as below.

-          Towards Fully Autonomic Network

-          Ability of Handling Complex Issues

-          More Adaptive and Flex

-          Predictive

-          Potential Self-Evolving Ability

-          Potential Decision Efficiency

If there are major benefits that we have not covered, we would like to include.

Regards,

Sheng

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 11 May 2017 13:35:47 +1200

On 10/05/2017 18:26, Fang, Luyuan wrote:
> I think what is driving all these points is scale. Without intelligence, very large scale networks simply become unmanageable, 

I am not sure that statement is always true. It is safe to say that without *distributed solutions*, very large scale networks become unmanageable, and AI technologies can be used to support distributed solutions.

Let's take a specific example: the CASM work in the IETF. One approach to massive scale support of address and prefix management is a massive centralised database and an old-fashioned (NETCONF/YANG style) mechanism to push allocations out into the network. Another approach is a large set of distributed autonomic prefix managers that request new space from a distributed address pool when necessary. The second one is definitely a target for machine learning, in order to set optimal parameters for the individual prefix managers.

> and cannot maintain service availability. Just adding work force would not help.

It will help where the problem can genuinely be tackled by "divide and conquer", for example where localised optimisation and localised repair can help. But when localised solutions are inadequate and traditional centralised solutions do not scale, AI can be the answer.

> -          potential self-healing (after “predictive”)

Agreed.

    Brian
> _______________________________________________
> IDNET mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idnet
>

---

From: Pedro Martinez-Julia <[email-masked]>
Date: Thu, 11 May 2017 11:11:56 +0900

On Thu, May 11, 2017 at 01:35:47PM +1200, Brian E Carpenter wrote:
> > I think what is driving all these points is scale. Without
> > intelligence, very large scale networks simply become unmanageable, 
> 
> I am not sure that statement is always true. It is safe to say that
> without *distributed solutions*, very large scale networks become
> unmanageable, and AI technologies can be used to support distributed
> solutions.

Distributed solutions are required when the "large scale" is about the
number of elements forming a network. However, they are required but not
enough for dealing with a "large scale" of heterogeneity and complexity
in scenarios.

For example:
- AI not required: Increase the bandwidth of all links by 10%.
- AI required: Increase the bandwidth of all links to meet the possible
  demands they can have in the near future according to the behavior of
  the elements they are connected to.

I think that with these simple statements we can justify the need for AI
methods in networking but, please, let me know your comments. Thank you
very much.

Regards,
Pedro

-- 
Pedro Martinez-Julia
Network Science and Convergence Device Technology Laboratory
Network System Research Institute
National Institute of Information and Communications Technology (NICT)
4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
Email: [email-masked]
---------------------------------------------------------
*** Entia non sunt multiplicanda praeter necessitatem ***

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 11 May 2017 14:25:08 +1200

>   the elements they are connected to.

I can agree with that in the general case, but in some cases a fairly simple
predictive algorithm could be used that scarcely deserves the title of AI.
It depends very much on the individual use case.

> methods in networking 

I definitely agree. We just have to be careful not to "sell" AI for
cases where it is too much.

Regards
   Brian

> but, please, let me know your comments. Thank you
> very much.
> 
> Regards,
> Pedro
>

---

From: yanshen <[email-masked]>
Date: Thu, 11 May 2017 03:48:43 +0000

Dear Luyuan,

>    From: IDNET [mailto:[email-masked]] On Behalf Of Fang, Luyuan
Sent: Wednesday, May 10, 2017 2:26 PM
To: Sheng Jiang <[email-masked]>; [email-masked]
Subject: Re: [Idnet] Benefits of Introducing AI into Network

>    ability to manage network at scale (as 2nd point)

Agreed. That is what I want to express in the email that AI will bring more powerful controlling force for the network so that it will improve the transmission at large scale.

>    potential self-healing (after “predictive”)

Self-healing is valuable but I have a question here. What is the object of “self”? Is it the network or devices? I have ever think about this. If you means the network can self-heal from a unusual state to a usual state (e.g. from few links congestion to global links load-balance ), I think that is the breakpoint which AI can implement. However, I also learn some thought of device self-healing, which means AI help devices recovering from abnormal state (e.g. device failure). I think this is not the points we should consider with because it sounds more related with Machine Automation. The data which should be captured and processed tends to non-network. That means we need to focus on workbook of devices and deal with the fault recovering which we are not professional. (I am not sure whether I have describe it clearly).

>    Luyuan

From: IDNET <[email-masked]<mailto:[email-masked]>> on behalf of Sheng Jiang <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, May 9, 2017 at 7:53 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [Idnet] Benefits of Introducing AI into Network

-          More Adaptive and Flex

-          Predictive

Regards,

Sheng

---

From: yanshen <[email-masked]>
Date: Thu, 11 May 2017 06:53:50 +0000

> -----Original Message-----
> From: IDNET [mailto:[email-masked]] On Behalf Of Pedro Martinez-Julia
> Sent: Thursday, May 11, 2017 10:12 AM
> To: Brian E Carpenter <[email-masked]>
> Cc: Fang, Luyuan <[email-masked]>; [email-masked]; Sheng Jiang
> <[email-masked]>
> Distributed solutions are required when the "large scale" is about the number of
> elements forming a network. However, they are required but not enough for
> dealing with a "large scale" of heterogeneity and complexity in scenarios.
> 
> For example:

Agreed with your "AI required". The AI cannot create new resource but it is really good at dealing with complex problems (e.g. resource allocation).

> I think that with these simple statements we can justify the need for AI methods
> in networking but, please, let me know your comments. Thank you very much.
> 
> Regards,
> Pedro
> 
> --
> Pedro Martinez-Julia
> Network Science and Convergence Device Technology Laboratory Network
> System Research Institute National Institute of Information and Communications
> Technology (NICT) 4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
> Email: [email-masked]

---

From: David Meyer <[email-masked]>
Date: Thu, 11 May 2017 05:06:06 -0400

Brian,

On Wed, May 10, 2017 at 9:35 PM, Brian E Carpenter <
[email-masked]> wrote:

> > I think what is driving all these points is scale. Without intelligence,
> very large scale networks simply become unmanageable,

> unmanageable, and AI technologies can be used to support distributed

Can you support this assertion? Further, what is an "AI technology", and
which ways can "AI technologies can be used to support distributed
solutions."?

>
> Let's take a specific example: the CASM work in the IETF. One approach to
> massive scale support of address and prefix management is a massive
> centralised database and an old-fashioned (NETCONF/YANG style) mechanism to
> push allocations out into the network. Another approach is a large set of
> distributed autonomic prefix managers that request new space from a
> distributed address pool when necessary. The second one is definitely a
> target for machine learning, in order to set optimal parameters for the
> individual prefix managers.
>

Phrases like "massive scale" don't seem to convey anything quantitative (or
even qualitative, for that matter) about the problem space, and as such
don't appear to be that useful (unless of course you can meaningfully
quantify what is meant by "massive scale"). In addition, what kinds of
machine learning would be applicable to the case of distributed autonomic
prefix managers that you describe above (simply stating that it is so isn't
that helpful)? For example, how would learning work, e.g., what kinds of
models and related hyper-parameters do you envision being appropriate, and
what kinds of data sets would be needed to efficiently train these models?
In addition, how would inference work, where inference here means whatever
you do with the trained model, including inference (computing the
posterior), prediction (MAP or MLE), regression, ...?

> It will help where the problem can genuinely be tackled by "divide and
> conquer", for example where localised optimisation and localised repair can
> help. But when localised solutions are inadequate and traditional
> centralised solutions do not scale, AI can be the answer.
>

What is AI in this context? In addition, why can AI (whatever that is) be
the answer to localized or centralized solutions that don't scale?  Again,
simply stating that it is so doesn't really help.

Thanks,

Dave

> > In ETSI NGP ISP, we have an Work Item for IDN (Intelligence-Defined
> Network). In one of the general sections, We briefly describes the benefits
> of introducing AI into network, as below.
> >
> >

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 May 2017 08:30:22 +1200

On 11/05/2017 21:06, David Meyer wrote:
> Can you support this assertion? Further, what is an "AI technology",

I'm not brave enough to try to answer that in a few words. But clearly
ML is an example. Some kind of inference engine is another. Also, specific
heuristic algorithms seem very suitable for use in systems of distributed
agents, because each agent needs to respond to local conditions.

> and

Either by installing heuristics in individual agents, or by allowing
agents to consult a central oracle when they need help ("AI as a service"
if you like). But there's a lot to work out.
 
> quantify what is meant by "massive scale"). 

Well, in this case think of an ISP with tens of millions of subscribers,
and work back from that to how many distributed agents might be involved.
For example, at least 1000 agents each handling address space delegation
to 10000 subscribers.

> In addition, what kinds of

Dunno. These are good questions.
 
> simply stating that it is so doesn't really help.

Learning from experience is the key benefit, I think. That's why I imagine
that the distributed agents could share a central ML oracle, which will
build up knowledge over the long term. To invent a heuristic out of thin
air: "An agent will experience a burst of address space requests at 19:00
in its local time zone. Therefore it should request extra space at 18:30."
That seems like something that an ML/inference engine could produce from
reports sent in by distributed agents.

Regards
   Brian

---

From: Javier Antich Romaguera <[email-masked]>
Date: Thu, 11 May 2017 22:35:20 +0000

>>Let's take a specific example: the CASM work in the IETF. One approach to massive scale support of address and prefix management is a massive centralised database and an old-fashioned (NETCONF/YANG…

I´m sorry, but what is the relationship between something being centralized or distributed and using or not a mechanism like Netconf using YANG models to communicate certain information (in this case prefixes or addresses)? They are entirely orthogonal. Wherever you do your processing (centralized or distributed) or your storage (centralized or distributed) you still need a mechanism to model your data (YANG or other) and a mechanism to encode and transport your data (Netconf or other). Not sure what exactly is old-fashioned on Netconf/YANG for this purposes discussion, and what is has to do with using AI or not.

Thanks
Javier.
______________________________________________
Javier Antich Romaguera
Product Line Manager. JUNOS Team.
Juniper Networks
email: [email-masked]<mailto:[email-masked]>
m. +34639218428
“Achieving the difficult, trying the impossible"

From: IDNET <[email-masked]<mailto:[email-masked]>> on behalf of Brian E Carpenter <[email-masked]<mailto:[email-masked]>>
Organization: University of Auckland
Date: Thursday, May 11, 2017 at 3:35 AM
To: "Fang, Luyuan" <[email-masked]<mailto:[email-masked]>>, Sheng Jiang <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idnet] Benefits of Introducing AI into Network

I would suggest to add

Agreed.

    Brian
Thx,
Luyuan
[email-masked]<mailto:[email-masked]>

_______________________________________________
IDNET mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idnet

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 May 2017 14:59:04 +1200

Hi,

On 12/05/2017 10:35, Javier Antich Romaguera wrote:
> I´m sorry, but what is the relationship between something being centralized or distributed and using or not a mechanism like Netconf using YANG models to communicate certain information (in this case prefixes or addresses)? They are entirely orthogonal. 

I don't believe they are orthogonal when you consider scaling and resilience simultaneously.
Of course the actual delivery of parameters to end systems can be performed using the same
tools like NETCONF (or RADIUS, or DHCPv6/PD) whether or not it's centralised. However, at
large scale, if the network partitions, you will not get resiliency from a centralised scheme.

> Wherever you do your processing (centralized or distributed) or your storage (centralized or distributed) you still need a mechanism to model your data (YANG or other) and a mechanism to encode and transport your data (Netconf or other). 

Yes, if that was unclear in what I wrote, I apologise. I mean to say that the mechanisms
used to achieve very large scale, resilience and optimization will be distribution of
functions supported by machine learning etc. But indeed the delivery of config to the end
systems will very likely not change.

> Not sure what exactly is old-fashioned on Netconf/YANG for this purposes discussion, and what is has to do with using AI or not.
 
The point is that Netconf (and CLI, Radius, DHCP or any other way of delivering config)
are all fundamentally equivalent, and my understanding is that on this list we are
looking for innovation.

   Brian