Subject: Re: [netmod] netmod-opstate-reqs and error option terms (rollback on error)

From: Robert Wilton <[email-masked]>
Date: Tue, 5 Jan 2016 15:04:53 +0000

Hi Gert,

Considering a rollback-on-error request that failed:

If you leave the configuration in intended config then what happens if a 
separate config request comes in from another client?  Should that 
request be processed in the context of the failed intended config, or 
the last successful config change, or must it wait for the original 
client to decide how to process the failure that occurred?

Also what would happen if the client disconnects from the server when a 
rollback-on-error request has failed, and the server is waiting for the 
client to tell it what to do?

I think that what you describe sounds more like a config request to a 
candidate configuration datastore.  I.e. write the config to candidate, 
and then commit.  If the commit fails, the configuration in the running 
datastore is rolled back but the configuration remains in the candidate 
datastore.  But note that in this scenario both the intended and applied 
configuration have both been reverted to the state before the commit 
operation was attempted.

Thanks,
Rob

On 28/12/2015 17:40, Gert Grammel wrote:
> Rob,
>
> From a client perspective there should be no difference in the 
> intended state behavior depending on error conditions, if 
> continue-on-error leaves the intended state, a rollback-on-error 
> should keep it as well. Moreover, a client reaction on a unsuccessful 
> application of intended state could be to a) re-try, b) retry with 
> continue-on-error or c) retry wit stop-on-error. Those actions would 
> not require any change of intended state.
> It shouldn't be the server to decide what the intended state is 
> supposed to be or hoe long it should last.
>
> Gert
>
>
> Sent from my Apple ][
>
> On 23 Dec 2015, at 22:25, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>> Hi Gert,
>>
>> Please see one comment inline ...
>>
>> On 23/12/2015 10:24, Gert Grammel wrote:
>>> Rob, Kent,
>>>
>>> Adding to Rob’s comments:
>>>
>>>
>>>
>>> From: netmod <[email-masked]> on behalf of Robert Wilton 
>>> <[email-masked] <mailto:[email-masked]>>
>>> Date: Wednesday 23 December 2015 09:28
>>> To: Kent Watsen <[email-masked]>, "[email-masked] 
>>> <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>>> Subject: Re: [netmod] netmod-opstate-reqs and error option terms 
>>> (rollback on error)
>>>
>>> Hi Kent,
>>>
>>> Yes, I think that we are in agreement.
>>>
>>> I've one further comment inline below ...
>>>
>>> On 23/12/2015 02:41, Kent Watsen wrote:
>>>
>>>     [As a contributor]
>>>
>>>     Hi Robert, I want to go back to Jason’s original questions.  I
>>>     think we’re aligned on this, but please check my answers below.
>>>
>>>     Quoting Jason’s original text now:
>>>
>>>
>>>         Is there some intention in the opstate requirements to add
>>>         some sort
>>>         of all-or-nothing behavior to transition (C)?
>>>
>>>     Yes
>>>
>>> +1 (if rollback-on-error has been requested)
>>>
>>>
>>>
>>>         i.e. if some part of
>>>         an edit fails during the transition from intended->applied
>>>         we should
>>>         "rollback" the other parts that may have already been applied ?
>>>
>>>     Yes
>>>
>>> +1 (if rollback-on-error has been requested)
>>>
>>>
>>>
>>>         Would we then remove it all from intended as well ?
>>>
>>>     IMO, yes.  This is more easily understood when thinking about
>>>     Synchronous Configuration Operations (defined in opstate-reqs),
>>>     but I believe that it equally applies to Asynchronous
>>>     Configuration Operations, so long as the client explicitly
>>>     ops-in for the behavior.
>>>
>>> IMO no. The intended config is imposed by the client to the server 
>>> and other than performing some syntax check, the server has no 
>>> choice to accept it as-is. If the server can’t apply the intended 
>>> config, obviously the mismatch between intended and applied config 
>>> needs to be notified. As a result, it is up to the client to decide 
>>> what to do. Actions can vary according to the situation naming a 
>>> few: 1) retry intended config, 2) retry intended config with 
>>> continue-on-error, 3) re-apply the previous intended config, …
>>> In other words:
>>>
>>>  1. the client shall not touch the applied config directly,
>>>  2. the server shall not touch the intended config,
>>>  3. it’s up to the server to align the intended with the applied
>>>     config in a way requested by the client (i.e. rollback-on-error, …)
>>>  4. Once applied, the server notifies the client about success or
>>>     failure to do so
>>>
>> I think that it cleaner just to fail and roll back the entire request 
>> (both intended and applied), i.e. effectively implementing default 
>> transaction semantics.  The client still has full control as to what 
>> to do after the failure has occurred.  I'm not sure how leaving it in 
>> intended config would pragmatically work with subsequent requests 
>> that may have been queued up.
>>
>> Thanks,
>> Rob
>>
>
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Gert Grammel <[email-masked]>
Date: Wed, 6 Jan 2016 16:03:18 +0000

Hi Rob,

See below ...

--Gert

From: Robert Wilton [mailto:[email-masked]]
Sent: 05 January 2016 16:05
To: Gert Grammel
Cc: [email-masked]
Subject: Re: [netmod] netmod-opstate-reqs and error option terms (rollback on error)

Hi Gert,

If you leave the configuration in intended config then what happens if a separate config request comes in from another client?  Should that request be processed in the context of the failed intended config, or the last successful config change, or must it wait for the original client to decide how to process the failure that occurred?
Gert> If a client is has the intention to update/change a config, its decision is based on the present state of the configuration when the decision is taken. Ideally the present configuration is in a state where intended == applied config, so there is stable ground upon which a change is applied. However there is also the case where intended != applied config and there are two reasons for that.

a)      a previous intended config is in the process of being applied

b)      a previous configuration failed due to an error
In case a) the following config just needs to wait, or if it can be applied without interfering with the previous intended config *may* be executed in parallel (up to the server to implement)
Case b) is just about the same as stop-on-error or continue-on-error. If there is an error in configuring a node and the new intended config uses stop-on-error, then it should not be executed. In contrast, a continue-on-error configuration can be applied independently.

Also what would happen if the client disconnects from the server when a rollback-on-error request has failed, and the server is waiting for the client to tell it what to do?
Gert> the previous (rolled-back) configuration is applied and the intended config remains different from the applied config. It will stay like that until the Client figured out what to do. E.g. one option is to re-apply the intended config with a continue-on-error semantic. If a second client, despite the server calamities, decides to change the config, it can try to do so in a continue-on-error mode as well.

I think that what you describe sounds more like a config request to a candidate configuration datastore.  I.e. write the config to candidate, and then commit.  If the commit fails, the configuration in the running datastore is rolled back but the configuration remains in the candidate datastore.  But note that in this scenario both the intended and applied configuration have both been reverted to the state before the commit operation was attempted.
Gert> bear with me but I don't parse the two sentences as they seem contradictory. Perhaps you could spend a few more words.
I am advocating for a case with a consistent behavior in case of failures in all execution models. Irrespective of stop-on-error, continue-on-error or rollback-on-error, the non-performing configuration is the diff between intended and applied configuration that can conveniently be reported by the server. In any of those cases, the server is rightfully in an errored state with respect to configuration and it is risky to attempt a new configuration unless the client resolves the misalignment. It appears to me that we have broadly 3 failure cases to deal with:

1)      The client did a poor job in figuring out which configuration to push on a server

2)      The server did a poor job in reporting its own state to the client (see cheating synchronous), which in turn can lead to 1)

3)      Something unexpected happens in between the client figuring out the intended config and the server applying it.
Case 1) can only be dealt by the Client. 2) is often dealt with re-applying the same intended config a bit later, hoping the server is in a better mood now (by experience it often works). If it still doesn't, it's again up to the client to figure out what's best now. Since the event in 3) may impact the state of the node quite a bit, also here it is the job of the client to figure out which config to apply based on the new state.

Thanks,
Rob

>From a client perspective there should be no difference in the intended state behavior depending on error conditions, if continue-on-error leaves the intended state, a rollback-on-error should keep it as well. Moreover, a client reaction on a unsuccessful application of intended state could be to a) re-try, b) retry with continue-on-error or c) retry wit stop-on-error. Those actions would not require any change of intended state.
It shouldn't be the server to decide what the intended state is supposed to be or hoe long it should last.

Gert

Sent from my Apple ][

On 23 Dec 2015, at 22:25, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:
Hi Gert,

Adding to Rob's comments:

From: netmod <[email-masked]<mailto:[email-masked]>> on behalf of Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Wednesday 23 December 2015 09:28
To: Kent Watsen <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] netmod-opstate-reqs and error option terms (rollback on error)

Hi Kent,

Hi Robert, I want to go back to Jason's original questions.  I think we're aligned on this, but please check my answers below.

Quoting Jason's original text now:

Is there some intention in the opstate requirements to add some sort
of all-or-nothing behavior to transition (C)?
Yes
+1 (if rollback-on-error has been requested)

i.e. if some part of
an edit fails during the transition from intended->applied we should
"rollback" the other parts that may have already been applied ?
Yes
+1 (if rollback-on-error has been requested)

Would we then remove it all from intended as well ?
IMO, yes.  This is more easily understood when thinking about Synchronous Configuration Operations (defined in opstate-reqs), but I believe that it equally applies to Asynchronous Configuration Operations, so long as the client explicitly ops-in for the behavior.
IMO no. The intended config is imposed by the client to the server and other than performing some syntax check, the server has no choice to accept it as-is. If the server can't apply the intended config, obviously the mismatch between intended and applied config needs to be notified. As a result, it is up to the client to decide what to do. Actions can vary according to the situation naming a few: 1) retry intended config, 2) retry intended config with continue-on-error, 3) re-apply the previous intended config, ...
In other words:

  1.  the client shall not touch the applied config directly,
  2.  the server shall not touch the intended config,
  3.  it's up to the server to align the intended with the applied config in a way requested by the client (i.e. rollback-on-error, ...)
  4.  Once applied, the server notifies the client about success or failure to do so
I think that it cleaner just to fail and roll back the entire request (both intended and applied), i.e. effectively implementing default transaction semantics.  The client still has full control as to what to do after the failure has occurred.  I'm not sure how leaving it in intended config would pragmatically work with subsequent requests that may have been queued up.

Thanks,
Rob

netmod mailing list

[email-masked]<mailto:[email-masked]>

---

From: Kent Watsen <[email-masked]>
Date: Wed, 6 Jan 2016 16:43:03 +0000

[As a contributor]

[KENT] or due to missing hardware, right?   Regardless, I don’t think this thread has bearing of the requirements draft.  Note that I removed the *-on-error terms from the Terminology section in -02 by instead rewording requirement 2-C, which was the only place they were being referenced before and the reference was merely suggestive (not normative).  Can we leave it to the solution drafts to sort out?

---

From: Robert Wilton <[email-masked]>
Date: Wed, 6 Jan 2016 16:48:34 +0000

Hi Kent,

On 06/01/2016 16:43, Kent Watsen wrote:
> [As a contributor]
>
> Gert> If a client is has the intention to update/change a config, its 
> decision is based on the present state of the configuration when the 
> decision is taken. Ideally the present configuration is in a state 
> where intended == applied config, so there is stable ground upon which 
> a change is applied. However there is also the case where intended != 
> applied config and there are two reasons for that.
>
> a)a previous intended config is in the process of being applied
>
> b)a previous configuration failed due to an error
>
> [KENT] or due to missing hardware, right?   Regardless, I don’t think 
> this thread has bearing of the requirements draft.  Note that I 
> removed the *-on-error terms from the Terminology section in -02 by 
> instead rewording requirement 2-C, which was the only place they were 
> being referenced before and the reference was merely suggestive (not 
> normative).  Can we leave it to the solution drafts to sort out?

Yes, deferring this issue to the solutions draft works for me.

I don't think that it will have any bearing on the discussion of the 
overall solution approach.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Wed, 6 Jan 2016 17:00:09 +0000

Kent,

I would agree that the discussion doesn’t affect the requirements draft in its current state. The solutions draft would be a better place probably.

Gert

From: Kent Watsen
Sent: 06 January 2016 17:43
To: Gert Grammel; Robert Wilton
Cc: [email-masked]
Subject: Re: [netmod] netmod-opstate-reqs and error option terms (rollback on error)

[As a contributor]

---

From: Gert Grammel <[email-masked]>
Date: Wed, 6 Jan 2016 17:08:02 +0000

Sorry, missed Kent’s remark:
[KENT] or due to missing hardware, right?

That’s one root cause, but I didn’t want to go into much details here. Missing Hardware is one issue, other issues are unconscious configuration requests from the client, faulty HW, bugs etc. Whatever the root cause, the result is intended != applied and Client+Server have to deal with it in a meaningful manner.

Gert

From: Robert Wilton [mailto:[email-masked]]
Sent: 06 January 2016 17:49
To: Kent Watsen; Gert Grammel
Cc: [email-masked]
Subject: Re: [netmod] netmod-opstate-reqs and error option terms (rollback on error)

Hi Kent,

I don't think that it will have any bearing on the discussion of the overall solution approach.

Thanks,
Rob