-Yscala-release support: checking if classes come from stdlib based on jar name; better documentation

Author: prolativ

compiler/src/dotty/tools/dotc/core/Contexts.scala

@@ -493,7 +493,7 @@ object Contexts {
       import math.Ordered.orderingToOrdered
       val latestRelease = ScalaRelease.latest
       val specifiedRelease = scalaRelease
-      if ((specifiedRelease.majorVersion, specifiedRelease.minorVersion) < (latestRelease.majorVersion, latestRelease.minorVersion)) then
+      if specifiedRelease < latestRelease then
         // This is needed to make -Yscala-release a no-op when set to the latest release for unstable versions of the compiler
         // (which might have the tasty format version numbers set to higher values before they're decreased during a release)
         TastyVersion.fromStableScalaRelease(specifiedRelease.majorVersion, specifiedRelease.minorVersion)

compiler/src/dotty/tools/dotc/core/classfile/ClassfileParser.scala

@@ -969,9 +969,13 @@ class ClassfileParser(
             if !isTastyReadable then
               reportWrongTasty("its TASTy format cannot be read by the compiler", TastyVersion.compilerVersion)
             else
-              val isTastyCompatible =
-                fileTastyVersion.isCompatibleWith(ctx.tastyVersion) ||
-                classRoot.symbol.showFullName.startsWith("scala.") // References to stdlib are considered safe because we check the values of @since annotations
+              def isStdlibClass(cls: ClassDenotation): Boolean =
+                ctx.platform.classPath.findClassFile(cls.fullName.mangledString) match {
+                  case Some(entry: ZipArchive#Entry) =>
+                    entry.underlyingSource.map(_.name.startsWith("scala3-library_3-")).getOrElse(false)
+                  case _ => false
+                }
+              val isTastyCompatible = fileTastyVersion.isCompatibleWith(ctx.tastyVersion) || isStdlibClass(classRoot)
               if !isTastyCompatible then
                 reportWrongTasty(s"its TASTy format is not compatible with the one of the targeted Scala release (${ctx.scalaRelease.show})", ctx.tastyVersion)
 

docs/docs/reference/language-versions/binary-compatibility.md

@@ -0,0 +1,25 @@
+---
+layout: doc-page
+title: "Binary Compatibility"
+---
+
+Thanks to TASTy files, which are produced during compilation of Scala 3 sources, unlike Scala 2, Scala 3 is backward binary compatible between different minor versions (i.e. binary artifacts produced with Scala 3.x can be consumed by Scala 3.y programs as long as x <= y).
+There are however already some ongoing attempts to make Scala 3 forward binary compatible which means that, with some restrictions, it might be possible to compile Scala code with a newer version of the compiler and then use the produced binaries as dependencies for a project using an older compiler.
+
+Scala 3.1.2-RC1 adds an experimental `-Yscala-release <release-version>` compiler flag which makes the compiler produce TASTy files that should be possible to use by all Scala 3 compilers in version `<release-version>` or newer (this flag was inspired by how `-release` works for specifying the target version of JDK). More specifically this flag enforces emitting TASTy files in an older format ensuring that:
+* the code contains no references to parts of the standard library which were added to the API after `<release-version>` and would crash at runtime when a program is executed with the older version of the standard library on the classpath
+* no dependency found on the classpath during compilation (except for the standard library itself) contains TASTy files produced by a compiler newer than `<release-version>` (otherwise they could potentially leak such disallowed references to the standard library)
+If any of the checks above is not fulfilled or for any other reason older TASTy cannot be emitted (e.g. the code uses some new language features which cannot be expressed the the older format) the entire compilation fails (with errors reported for each of such issues).
+
+As this feature is experimental it does not have any special support in build tools yet (at least not in sbt 1.6.1 or lower).
+E.g. when a project gets compiled with Scala compiler `3.x1.y1` and `-Yscala-release 3.x2` option and then published using sbt
+then the standard library in version `3.x1.y1` gets added to the project's dependencies instead of `3.x2.y2`.
+When the dependencies are added to the classpath during compilation with Scala `3.x2.y2` the compiler will crash while trying to read TASTy files in the newer format.
+A currently known workaround is to modify the build definition of the dependent project by explicitly overriding the version of Scala standard library in dependencies, e.g.
+
+```scala
+dependencyOverrides ++= Seq(
+  scalaOrganization.value %% "scala3-library" % scalaVersion.value,
+  scalaOrganization.value %% "scala3-library_sjs1" % scalaVersion.value // for Scala.js projects
+)
+```

docs/docs/reference/language-versions/language-versions.md

@@ -0,0 +1,6 @@
+---
+layout: doc-page
+title: "Language Versions"
+---
+
+Additional information on interoperability and migration between Scala 2 and 3 can be found [here](https://docs.scala-lang.org/scala3/guides/migration/compatibility-intro.html).

docs/docs/reference/language-versions.md renamed to docs/docs/reference/language-versions/source-compatibility.md

@@ -1,10 +1,12 @@
 ---
 layout: doc-page
-title: "Language Versions"
+title: "Source Compatibility"
 movedTo: https://docs.scala-lang.org/scala3/reference/language-versions.html
 ---
 
-The default Scala language version currently supported by the Dotty compiler is [`3.0`](https://scala-lang.org/api/3.x/scala/runtime/stdLibPatches/language$$3/0$.html). There are also other language versions that can be specified instead:
+Scala 3 does NOT guarantee source compatibility between different minor language versions (e.g. some syntax valid in 3.x might get deprecated and then phased out in 3.y for y > x). There are also some syntax structures that were valid in Scala 2 but are not anymore in Scala 3. However the compiler provides a possibility to specify the desired version of syntax used in a particular file or globally for a run of the compiler to make migration between versions easier.
+
+The default Scala language syntax version currently supported by the Dotty compiler is [`3.0`](https://scala-lang.org/api/3.x/scala/runtime/stdLibPatches/language$$3/0$.html). There are also other language versions that can be specified instead:
 
 - [`3.0-migration`](https://scala-lang.org/api/3.x/scala/runtime/stdLibPatches/language$$3/0-migration$.html): Same as `3.0` but with a Scala 2 compatibility mode that helps moving Scala 2.13 sources over to Scala 3. In particular, it
 

docs/sidebar.yml

@@ -119,7 +119,11 @@ sidebar:
           - page: docs/reference/experimental/named-typeargs-spec.md
           - page: docs/reference/experimental/numeric-literals.md
       - page: docs/reference/syntax.md
-      - page: docs/reference/language-versions.md
+      - title: Language Versions
+        index: docs/reference/language-versions/language-versions.md
+        subsection:
+          - page: docs/reference/language-versions/source-compatibility.md
+          - page: docs/reference/language-versions/binary-compatibility.md
   - title: Contributing
     subsection:
       - page: docs/contributing/contribute-knowledge.md

sbt-community-build/src/dotty/communitybuild/sbtplugin/CommunityBuildPlugin.scala

@@ -53,7 +53,7 @@ object CommunityBuildPlugin extends AutoPlugin {
         val forwardCompatFilter: ModuleID => Boolean = if (isForwardCompatProject.value) (_.revision.contains("-forward-compat")) else (!_.revision.contains("-forward-compat"))
         val stdlibOverrides = Seq(
           scalaOrganization.value %% "scala3-library" % scalaVersion.value,
-          scalaOrganization.value %% "scala3-library_sjs1" % scalaVersion.value,
+          scalaOrganization.value %% "scala3-library_sjs1" % scalaVersion.value
         )
         CommunityBuildDependencies.allOverrides(sLog.value).filter(forwardCompatFilter) ++ stdlibOverrides
       } else Nil