Subject: [wpkops] Taxonomy of Browser Behaviors - "Hard Fail", "Soft Fail" and "Reload Request"

From: "Ben Wilson" <[email-masked]>
Date: Tue, 29 Apr 2014 12:58:50 -0600

In working on the next version of the Certificate Processing document, I
have come across two different uses of "hard fail."   I am also concerned
that use of the phrase, "soft fail," might encounter similar problems.  Also
I've seen "Retry" or "Reload" messages,  which are hard fail, but with an
option to try loading again. 

I've seen "hard fail" used (1) when referring to a session that is stopped
because of certificate revocation, where the user is prevented from
proceeding, and also (2) when referring to intentional client behavior when
encountering other problems with the certificate that indicate it is not
trustworthy.  (I suppose it could also mean a crash or other unintentional
behavior because of a bug in code.)  With respect to (2), I have called this
a "fatal error" - A behavior in which the browser detects an abnormal
condition and halts (or technically cannot complete) session negotiation and
drops the connection or otherwise blocks the user from continuing (also
referred to as "hard fail")."  However, in Phill's paper on revocation
behavior, he uses "hard fail", too.  

I have used the term "bypassable error" instead of "soft fail", defined as
"behavior in which the browser detects an abnormal condition and asks the
user whether to proceed with (i.e. click-through to) the SSL/TLS
connection."  Is this the same as "soft fail"?  (I'm assuming that a
negative visual indicator or a "downgrade" of security indicators like
removal of the lock icon, removal of EV indication, etc., are not "soft
fail."  I hope that everyone agrees.)  

Any thoughts?  Does it matter what kind of "next step" is provided in the
dialogue presented to the user?  For instance, the distinction between hard
and soft fail might be as simple as whether the error window lacks or
contains buttons or links that allow the user to proceed toward making the
SSL/TLS connection.  

For "hard fail," here is what I've seen:

[Error Message]

Firefox - "Fix connection problems" 

Opera - "This webpage is not available"

Chrome - "This webpage is not available"

Internet Explorer - "IE cannot display the webpage"  "What you can try:
Diagnose Connection Problems"

Internet Explorer - "This page cannot be displayed.  Fix connection
problems"

In looking at soft fail / bypassable errors,  here are some of the buttons
provided for a variety of different conditions, such as invalid
certificates:

[Error Message]

Firefox - "Get me out of here" "Technical Details" "I understand the risks"

Safari - "Show Certificate" "Cancel" "Continue"

Internet Explorer - "Click here to close" ""Continue to this website (not
recommended)" "More Information"

Chrome - "Proceed anyway" or  "Back to safety"  and "Help me understand"

Opera - "Show Certificate"  "Continue Anyway" and "Cancel" or "Back to
Safety"

A third option I've noticed is a "reload request", which I think is
different than a bypassable error or hard fail.  Am I right?

Here are some messages:

Chrome - "More" or  "Reload"

Firefox - "Try again"

Thanks,

Ben

---

From: Wayne Thayer <[email-masked]>
Date: Tue, 29 Apr 2014 22:02:37 +0000

Ben,

In the context of revocation, I have a different concept of the terms "soft fail" and "hard fail" than what you describe below. I think of soft fail as a scenario where a browser checks OCSP, does not receive a response, and proceeds as if it had received a "good" response without any indication to the user.

Also, I think of revocation "hard fail" as the scenario you describe below as "soft fail" where the browser presents a blocking error that the user can then choose to bypass.

Thanks,

Wayne

From: wpkops [mailto:[email-masked]] On Behalf Of Ben Wilson
Sent: Tuesday, April 29, 2014 11:59 AM
To: [email-masked]
Subject: [wpkops] Taxonomy of Browser Behaviors - "Hard Fail", "Soft Fail" and "Reload Request"

In working on the next version of the Certificate Processing document, I have come across two different uses of "hard fail."   I am also concerned that use of the phrase, "soft fail," might encounter similar problems.  Also I've seen "Retry" or "Reload" messages,  which are hard fail, but with an option to try loading again.

I've seen "hard fail" used (1) when referring to a session that is stopped because of certificate revocation, where the user is prevented from proceeding, and also (2) when referring to intentional client behavior when encountering other problems with the certificate that indicate it is not trustworthy.  (I suppose it could also mean a crash or other unintentional behavior because of a bug in code.)  With respect to (2), I have called this a "fatal error" - A behavior in which the browser detects an abnormal condition and halts (or technically cannot complete) session negotiation and drops the connection or otherwise blocks the user from continuing (also referred to as "hard fail")."  However, in Phill's paper on revocation behavior, he uses "hard fail", too.

I have used the term "bypassable error" instead of "soft fail", defined as "behavior in which the browser detects an abnormal condition and asks the user whether to proceed with (i.e. click-through to) the SSL/TLS connection."  Is this the same as "soft fail"?  (I'm assuming that a negative visual indicator or a "downgrade" of security indicators like removal of the lock icon, removal of EV indication, etc., are not "soft fail."  I hope that everyone agrees.)

Any thoughts?  Does it matter what kind of "next step" is provided in the dialogue presented to the user?  For instance, the distinction between hard and soft fail might be as simple as whether the error window lacks or contains buttons or links that allow the user to proceed toward making the SSL/TLS connection.

[Error Message]
Firefox - "Fix connection problems"
Opera - "This webpage is not available"
Chrome - "This webpage is not available"
Internet Explorer - "IE cannot display the webpage"  "What you can try:  Diagnose Connection Problems"
Internet Explorer - "This page cannot be displayed.  Fix connection problems"

In looking at soft fail / bypassable errors,  here are some of the buttons provided for a variety of different conditions, such as invalid certificates:

[Error Message]

Firefox - "Get me out of here" "Technical Details" "I understand the risks"
Safari - "Show Certificate" "Cancel" "Continue"
Internet Explorer - "Click here to close" ""Continue to this website (not recommended)" "More Information"
Chrome - "Proceed anyway" or  "Back to safety"  and "Help me understand"

Opera - "Show Certificate"  "Continue Anyway" and "Cancel" or "Back to Safety"

A third option I've noticed is a "reload request", which I think is different than a bypassable error or hard fail.  Am I right?

Here are some messages:

Thanks,
Ben

---

From: Gervase Markham <[email-masked]>
Date: Wed, 30 Apr 2014 09:48:38 +0100

On 29/04/14 23:02, Wayne Thayer wrote:
> In the context of revocation, I have a different concept of the terms
> “soft fail” and “hard fail” than what you describe below. I think of
> soft fail as a scenario where a browser checks OCSP, does not receive a
> response, and proceeds as if it had received a “good” response without
> any indication to the user.
> 
> Also, I think of revocation “hard fail” as the scenario you describe
> below as “soft fail” where the browser presents a blocking error that
> the user can then choose to bypass.

...or does not allow a bypass. Both are "hard fail" - the term does not
distinguish.

As Wayne says, certainly in discussions of revocation, hard-vs-soft fail
is a very limited question of the behaviour of the browser when it does
not receive a response of any kind from the OCSP server. In soft fail,
it shows the site anyway. In hard fail, it does not.

I would advise not carrying this terminology over to other areas. It's
not very precise in other contexts.

Gerv

---

From: Michael Jenkins <[email-masked]>
Date: Wed, 30 Apr 2014 08:25:31 -0400

I have always used the term "hard fail" to mean the browser makes the
determination for the operator - if validation fails, the operator is given
no option to over-ride.

I use the term "fail open" when the browser interprets inability to obtain
revocation information as "not revoked". It's not really relevant that
locks don't appear if the desired page shows up in the main window
(especially if I put a picture of a closed lock on my main page, as some do
:(

I don't like the term "soft fail". If I did, I would use it for the case
where the determination is handed from the browser to the operator (the
get-me-out-of-here/i-understand-the-risks choice), which I don't consider a
failure, it's the way things are supposed to work. I realize that's
currently a somewhat academic interpretation, but it also leaves room for
the operator to decide how hard he wants the browser to try (how much time
to spend on this connection, how many mechanisms to invoke) and perhaps how
much information the operator wants to broadcast in order to make this
connection work, should alternative validation mechanisms be built into
browsers.

On Tue, Apr 29, 2014 at 2:58 PM, Ben Wilson <[email-masked]> wrote:

> have come across two different uses of “hard fail.”   I am also concerned
> that use of the phrase, “soft fail,” might encounter similar problems.
> Also I’ve seen “Retry” or “Reload” messages,  which are hard fail, but with
> an option to try loading again.
>
>
>
> I’ve seen “hard fail” used (1) when referring to a session that is stopped
> behavior because of a bug in code.)  With respect to (2), I have called
> this a “fatal error” - A behavior in which the browser detects an abnormal
> condition and halts (or technically cannot complete) session negotiation
> and drops the connection or otherwise blocks the user from continuing (also
> referred to as “hard fail”).”  However, in Phill’s paper on revocation
> behavior, he uses “hard fail”, too.
>
>
>
> I have used the term “bypassable error” instead of “soft fail”, defined as
> “behavior in which the browser detects an abnormal condition and asks the
> connection.”  Is this the same as “soft fail”?  (I’m assuming that a
> negative visual indicator or a “downgrade” of security indicators like
> removal of the lock icon, removal of EV indication, etc., are not “soft
> fail.”  I hope that everyone agrees.)
>
>
>
> Any thoughts?  Does it matter what kind of “next step” is provided in the
> For “hard fail,” here is what I’ve seen:
>
>
>
> [Error Message]
>
> Firefox - “Fix connection problems”
>
> Opera – “This webpage is not available”
>
> Chrome – “This webpage is not available”
>
> Internet Explorer – “IE cannot display the webpage”  “What you can try:
> Diagnose Connection Problems”
>
> Internet Explorer – “This page cannot be displayed.  Fix connection
> problems”
>
>
>
> Firefox – “Get me out of here” “Technical Details” “I understand the risks”
>
> Safari – “Show Certificate” “Cancel” “Continue”
>
> Internet Explorer – “Click here to close” ““Continue to this website (not
> recommended)” “More Information”
>
> Chrome - “Proceed anyway” or  “Back to safety”  and “Help me understand”
>
>
>
> Opera – “Show Certificate”  “Continue Anyway” and “Cancel” or “Back to
> Safety”
>
>
>
> A third option I’ve noticed is a “reload request”, which I think is
> Chrome - “More” or  “Reload”
>
> Firefox – “Try again”
>
>
>
> Thanks,
>
> Ben
>
> _______________________________________________
> wpkops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/wpkops
>
>

---

From: "Yngve N. Pettersen" <[email-masked]>
Date: Wed, 30 Apr 2014 16:43:26 +0200

Hi,

Actually, IMO "soft fail" does NOT include handing the question of whether  
to continue over to the user.

I believe "soft-fail" means that the client continues to load the document  
(what you call "fail open"), despite the detected problem condition(s),  
e.g. OCSP lookup failed. The client may or may not give a visual  
indication that there was a problem, and may or not treat the next  
connection to the server differently (e.g. Opera 12 will remove the  
padlock for the displayed document, and would not not reuse the affected  
SSL session, in case revocation checks failed).

What you mention as an example of "soft fail", displaying a warning  
message that the user have to decide how to handle, is actually the  
intermediate step between what is called "soft fail" and "hard fail", the  
detected problem is serious enough (e.g. server name mismatch, missing CA)  
that the client is unwilling to continue without permission from the user.

IOW, the four levels and actions are:

No detected problem: Continue, no indications to the user necessary
Low level problem: Continue, visual problem indications optional, aka  
"soft fail"
Serious problem: Stop connection, ask the user whether to continue
Critical problem: Close connection, don't allow the user to continue, aka  
"hard fail"

-- 
Sincerely,
Yngve N. Pettersen

Using Opera's mail client: http://www.opera.com/mail/

---

From: Rob Stradling <[email-masked]>
Date: Wed, 30 Apr 2014 15:58:46 +0100

On 30/04/14 15:43, Yngve N. Pettersen wrote:
<snip>
> Critical problem: Close connection, don't allow the user to continue,
> aka "hard fail"

FWIW, I tend to label these four levels as...
   No Fail
   Soft Fail
   Hard Fail
   Block Fail

But it's clear to me from this thread that my labels are too vague.  ;-)

-- 
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online