Subject: [netmod] whitespace stripping

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Jan 2015 11:25:57 +0100

Hi,

YANG has this peculiar rule for double-quoted strings:

   If the double-quoted string contains a line break followed by space
   or tab characters that are used to indent the text according to the
   layout in the YANG file, this leading whitespace is stripped from the
   string, up to and including the column of the double quote character,
   or to the first non-whitespace character, whichever occurs first.  In
   this process, a tab character is treated as 8 space characters.

As I recently found out, this kind of whitespace stripping is difficult to implement with some standard parser libraries because the parser has to be able to get the column of the opening double quote. And even if it is possible, it slows down parsing.

The existing practice shows such multi-line strings are only used in descriptions and references where whitespace stripping is not needed at all - e.g., for translation to YIN format such strings have to be re-formatted anyway. In all other cases it is IMO much better to use concatenation

  "..."
+ "..."

So I’d suggest to remove whitespace stripping in YANG 1.1, for the benefit of parsers and the spec itself.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Jan 2015 11:34:38 +0100

We are supposed to be done in March with YANG 1.1. This is not the
time to add new issues unless they are bug fixes.

I do not think this is a bug fix.

/js

On Fri, Jan 09, 2015 at 11:25:57AM +0100, Ladislav Lhotka wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Jan 2015 14:47:51 +0100

> On 09 Jan 2015, at 11:34, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Guess what: even pyang’s parser doesn’t implement this CLR correctly, and that tells you something - it doesn’t take into account that a tab is equivalent to eight spaces.

If some applications need this kind of trimming, they should do it themselves and not impose it on the parser. This fancy feature just adds extra complexity that’s not needed at all.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Jan 2015 06:37:45 -0800

> If some applications need this kind of trimming, they should do it themselves and not impose it on the parser. This fancy feature just adds extra complexity that’s not needed at all.

I don't see a reason to remove the trimming requirements.
It matters for a multi-line default-stmt value.
It is better to make the tools deal with the whitespace, rather
than the readers and writers.

> Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Jan 2015 15:51:17 +0100

> On 09 Jan 2015, at 15:37, Andy Bierman <[email-masked]> wrote:
> 

default "foo
         bar";

Or perhaps

when ". = 'foo
      bar'";

This gives me goosebumps!

> than the readers and writers.

Module readers and writers are not affected at all, they could continue doing the same as before.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Jan 2015 06:56:54 -0800

> Module readers and writers are not affected at all, they could continue doing the same as before.

This takes away a feature from YANG 1.0 so IMO it breaks backward compatibility.
I do not think it is too difficult to implement.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 10 Jan 2015 07:16:46 +0100

> It matters for a multi-line default-stmt value.

Actually, it is outright broken: I guess the meaning of a module should
not depend on the indentation of its statements, but it is not the case
here:

  default "foo\n\t\tbar"; # two spaces of indentation

gives a different default value than

    default "foo\n\t\tbar"; # four spaces of indentation

Isn't this a clear bug?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 10 Jan 2015 07:34:48 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> not depend on the indentation of its statements

Right.

> but it is not the case

No it is not correct - whitespace trimming only applies to leading
whitespace within the string, so in the case above there is no such
whitespace, and the two strings are the same.  Note that with the
whitespace rules we have, the following examples:

  default "foo
           bar";

and

       default "foo
                bar";

gives *the same* default value, whereas without whitespace trimming
they would be different - which we agreed would be bad.

Also note that if you don't like these rules, you can use single
quotes.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 10 Jan 2015 08:17:03 +0100

> On 10 Jan 2015, at 07:34, Martin Bjorklund <[email-masked]> wrote:
> 

Well, this really surprises me. One of the examples in sec. 6.1.3.1 says the following strings are equivalent:

         "first line
            second line"

     "first line\n" + "  second line"

This means that “\n” is the same as the literal newline. Why then isn’t “\t” the same as the literal tab?

What about

default “foo\n                bar”; ?

> they would be different - which we agreed would be bad.

I am not sure what’s worse. In any case, such things should be avoided like plague.

> Also note that if you don't like these rules, you can use single

That’s not the point. The trimming rules are relatively complex and still it is doubtful whether they really improve on anything. Parser writers may not get them right and this can lead to inconsistencies. A clear indication is that we actually have *two* bugs in pyang related to this: one is in the parsing code that you wrote and another is in the DSDL plugin that I wrote.

Lada