Clarify and test rules for newline suppression

Clarify and test rules for when to suppress a newline insertion
based on indentation.

Fixes #12554

Author: odersky

compiler/src/dotty/tools/dotc/parsing/Parsers.scala

@@ -172,7 +172,7 @@ object Parsers {
   class Parser(source: SourceFile)(using Context) extends ParserCommon(source) {
 
     val in: Scanner = new Scanner(source)
-    //in.debugTokenStream = true    // uncomment to see the token stream of the standard scanner, but not syntax highlighting
+    // in.debugTokenStream = true    // uncomment to see the token stream of the standard scanner, but not syntax highlighting
 
     /** This is the general parse entry point.
      *  Overridden by ScriptParser

compiler/src/dotty/tools/dotc/parsing/Scanners.scala

@@ -469,12 +469,6 @@ object Scanners {
         true
       }
 
-    def isContinuing(lastToken: Token) =
-      (openParensTokens.contains(token) || lastToken == RETURN)
-      && !pastBlankLine
-      && !migrateTo3
-      && !noindentSyntax
-
     /** The indentation width of the given offset. */
     def indentWidth(offset: Offset): IndentWidth =
       import IndentWidth.{Run, Conc}
@@ -565,6 +559,25 @@ object Scanners {
           handler(r)
         case _ =>
 
+      /** Is this line seen as a continuation of last line? We assume that
+       *   - last line ended in a token that can end a statement
+       *   - current line starts with a token that can start a statement
+       *   - current line does not start with a leading infix operator
+       *  The answer is different for Scala-2 and Scala-3.
+       *   - In Scala 2: Only `{` is treated as continuing, irrespective of indentation.
+       *     But this is in fact handled by Parser.argumentStart which skips a NEWLINE,
+       *     so we always assume false here.
+       *   - In Scala 3: Only indented statements are treated as continuing, as long as
+       *     they start with `(`, `[` or `{`, or the last statement ends in a `return`.
+       *   The Scala 2 rules apply under source `3.0-migration` or under `-no-indent`.
+       */
+      def isContinuing =
+        lastWidth < nextWidth
+        && (openParensTokens.contains(token) || lastToken == RETURN)
+        && !pastBlankLine
+        && !migrateTo3
+        && !noindentSyntax
+
       currentRegion match
         case r: Indented =>
           indentIsSignificant = indentSyntax
@@ -582,7 +595,7 @@ object Scanners {
          && canEndStatTokens.contains(lastToken)
          && canStartStatTokens.contains(token)
          && !isLeadingInfixOperator(nextWidth)
-         && !(lastWidth < nextWidth && isContinuing(lastToken))
+         && !isContinuing
       then
         insert(if (pastBlankLine) NEWLINES else NEWLINE, lineOffset)
       else if indentIsSignificant then

docs/_docs/reference/other-new-features/indentation.md

@@ -243,6 +243,38 @@ case 5 => print("V")
 println(".")
 ```
 
+### Using Indentation to Signal Statement Continuation
+
+Indentation is used in some situations to decide whether to insert a virtual semicolon between
+two consecutive lines or to treat them as one statement. Virtual semicolon insertion is
+suppressed if the second line is indented more relative to the first one, and either the second line
+starts with "`(`", "`[`", or "`{`" or the first line ends with `return`. Examples:
+
+```scala
+f(x + 1)
+  (2, 3)        // equivalent to  `f(x + 1)(2, 3)`
+
+g(x + 1)
+(2, 3)          // equivalent to  `g(x + 1); (2, 3)`
+
+h(x + 1)
+  {}            // equivalent to  `h(x + 1){}`
+
+i(x + 1)
+{}              // equivalent to  `i(x + 1); {}`
+
+if x < 0 then return
+  a + b         // equivalent to  `if x < 0 then return a + b`
+
+if x < 0 then return
+println(a + b)  // equivalent to  `if x < 0 then return; println(a + b)`
+```
+In Scala 2, a line starting with "`{`" always continues the function call on the preceding line,
+irrespective of indentation, whereas a virtual semicolon is inserted in all other cases.
+The Scala-2 behavior is retained under source `-no-indent` or `-source 3.0-migration`.
+
+
+
 ### The End Marker
 
 Indentation-based syntax has many advantages over other conventions. But one possible problem is that it makes it hard to discern when a large indentation region ends, since there is no specific token that delineates the end. Braces are not much better since a brace by itself also contains no information about what region is closed.
@@ -404,7 +436,7 @@ end IndentWidth
 
 ### Settings and Rewrites
 
-Significant indentation is enabled by default. It can be turned off by giving any of the options `-no-indent`, `-old-syntax` and `-language:Scala2`. If indentation is turned off, it is nevertheless checked that indentation conforms to the logical program structure as defined by braces. If that is not the case, the compiler issues a warning.
+Significant indentation is enabled by default. It can be turned off by giving any of the options `-no-indent`, `-old-syntax` and `-source 3.0-migration`. If indentation is turned off, it is nevertheless checked that indentation conforms to the logical program structure as defined by braces. If that is not the case, the compiler issues a warning.
 
 The Scala 3 compiler can rewrite source code to indented code and back.
 When invoked with options `-rewrite -indent` it will rewrite braces to

tests/neg/i12554a.scala

@@ -0,0 +1,32 @@
+object Test {
+  def f(s: String) = s
+
+  def g: (Int, Int) = {
+    f("Foo")
+      (1, 2)  // error, ok in Scala 2
+  }
+  def g2: (Int, Int) = {
+    f("Foo")
+    (1, 2)  // ok, ok in Scala 2
+  }
+
+  def h: Unit = {
+    f("Foo")
+      {}  // error, error in Scala 2
+  }
+
+  def i: Unit = {
+    f("Foo")
+    {}    // ok, error in Scala 2
+  }
+
+  def j: Int = {
+    return  // error, error in Scala 2
+    1 + 2
+  }
+
+  def k: Int = {
+    return   // ok, error in Scala 2
+      1 + 2
+  }
+}

tests/neg/i12554b.scala

@@ -0,0 +1,30 @@
+import language.`3.0-migration` // behavior should be the same as for Scala-2
+object Test {
+
+  def f(s: String) = s
+
+  def g: (Int, Int) = {
+    f("Foo")
+      (1, 2)  // ok
+  }
+
+  def h: Unit = {
+    f("Foo")
+      {}  // error
+  }
+
+  def i: Unit = {
+    f("Foo")
+    {}    // error
+  }
+
+  def j: Int = {
+    return  // error
+    1 + 2
+  }
+
+  def k: Int = {
+    return  // error
+      1 + 2
+  }
+}