Term.Resolved

[FlandiaYingman]

This is a draft PR that adds Term.Resolved to indicate that a term's symbol is resolved. This information can be used later in the lowering stage. For example, append .class to a ref to an overloaded module definition.

[FlandiaYingman]

I am facing some problems now:

• Now only Ref can expand to be appended with .class. But we should also do that to selections. (This is the reason that the current test cases fail.) However, this is particularly difficult for selections because there is already an existing symbol mechanism for selections. I tried a bit for a few hours, but I guess I am going into another refactorization hell 🤥...

[FlandiaYingman]

OK I managed to wrap selections with Term.Resolved. The next step is to remove the existing symbols in selections, and replace them with the symbols in Term.Resolved.

[FlandiaYingman]

Hello @CAG2Mark and @chengluyu, I need your help on this PR 🤧🥺, particularly regarding the lifter and the UCS/UPS normalization.

Summary on this PR

In this PR, we try to disambiguate Term.Ref's and Term.Sel's BlockMemberSymbol during resolution (disambiguation means to infer which overloaded definition does the term refer to in the specific context), by wrapping the terms that have to be disambiguated in a term Term.Resolved(t: Term, sym: DefinitionSymbol[?]). (DefinitionSymbol is the same as MemberSymbol, except that BlockMemberSymbol is not a DefinitionSymbol but a MemberSymbol).

A direct consequence is that, we now have enough information in the lowering/codegen stage to defer the process of adding the Sel(t, new Ident(".class") thing for a Ref or Sel referring to a overloaded term. Originally it was in the elaborator, and then with one of my previous PR it was moved to the resolver. Now in this PR, it will be moved to the JS builder, which is also the right place, as this was just a JS trick to compile overloaded definitions.

I changed Value.Ref to encode the disambiguation information (to be used later during code generation):

- case Ref(l: Local)
+ case Ref(l: Local, disamb: Opt[DefinitionSymbol[?]])

and Select to encode the disambiguated symbol information (we want to be as specific as possible):

- case class Select(qual: Path, name: Tree.Ident)(val symbol: Opt[FieldSymbol])
+ case class Select(qual: Path, name: Tree.Ident)(val symbol_SelectSymbol: Opt[DefinitionSymbol[?]])

Then, the JS builder automatically, based on the information encoded, decides whether or not to append the .class suffix to the generated code for a Ref or Select. The exact logic of appending .class is here:

    case Value.Ref(l, disamb) => l match
      case l: BlockMemberSymbol if {
        // TODO @Harry: refactor(ize) duplicated logic
        l.hasLiftedClass && 
        disamb.flatMap(d => d.asMod orElse d.asCls).exists(_ isnt ctx.builtins.Array)
      } =>
        doc"${getVar(l, l.toLoc)}.class"
      case _ =>
        getVar(l, r.toLoc)

    case s @ Select(qual, id) => 
      val dotClass = s.symbol_SelectSymbol match
        case S(ds) if {
          val v = ds.asBlkMember.exists(_.hasLiftedClass) && 
          (ds.asMod orElse ds.asCls).exists(_ isnt ctx.builtins.Array)
          v
        } => doc".class"
        case _ => doc""
      val name = id.name
      doc"${result(qual)}${
        if isValidFieldName(name)
        then doc".$name"
        else name.toIntOption match
          case S(index) => doc"[$index]"
          case N => doc"[${makeStringLiteral(name)}]"
      }${dotClass}"

What's the Problem

In the lifter and the UCS/UPS normalization, there is a bunch of customized logic of appending .class to a term. With this PR, they now append duplicate .class suffixes.

Also, seems like the compiler largely transforms the terms in the lifter and the UCS/UPS normalization process. Some encoded information is stripped out, but I couldn't find out the reason 😥.

UCS/UPS Normalization

I am taking EvenOddTree.mls as an example.

Looking at the resolved tree for the patterns:

:rt
pattern OddTree =
  A |
  Node(EvenTree, A, EvenTree) | Node(OddTree, A, OddTree) |
  Node(EvenTree, B, OddTree) | Node(OddTree, B, EvenTree)
pattern EvenTree =
  B |
  Node(EvenTree, A, OddTree) | Node(OddTree, A, EvenTree) |
  Node(EvenTree, B, EvenTree) | Node(OddTree, B, OddTree)
//│ Resolved tree:
//| ...
//│             left = Composition:
//│               polarity = true
//│               left = Constructor:
//│                 target = Resolved{sym=object:A,typ=object:A}:
//│                   t = Ref{sym=member:A} of member:A
//│                   sym = object:A
//│                 patternArguments = Nil
//│                 arguments = N
//│               right = Constructor:
//│                 target = Resolved{sym=term:class:Node.Node}:
//│                   t = Ref{sym=member:Node} of member:Node
//│                   sym = term:class:Node.Node

The references to A and Node are resolved correctly to object:A and term:class:Node.Node (which is the constructor of Node). But after lowering, the lowered tree becomes:

:lot
B is @compile EvenTree
//| Lowered tree:
//| ...
//│               path = Ref:
//│                 l = member:A
//│                 disamb = N
//| ...
//│               path = Select{sym=class:Node}:
//│                 qual = Ref:
//│                   l = member:Node
//│                   disamb = N
//│                 name = Ident of "class"

while we expect it to be

:lot
B is @compile EvenTree
//| Lowered tree:
//| ...
//│               path = Ref:
//│                 l = member:A
//│                 disamb = object:A
//| ...
//│               Ref:
//│                   l = member:Node
//│                   disamb = term:class:Node.Node

Would you provide some help on this case? Thanks Luyu 😉

Lifter

Taking lifter/ClassWithCompanion.mls as an example:

The C.empty in

fun foo(x) =
  class C(y) with
    fun get = [x, y]
  module C with
    val empty = C(123)
  C.empty

should be compiled to JS C.class.empty. The resolved tree indicates that it is correctly resolved that C actually refers to a module C:

//│         res = Resolved{sym=term:module:C.empty}:
//│           t = Sel{sym=member:empty}:
//│             prefix = Resolved{sym=module:C,typ=module:C}:
//│               t = Ref{sym=member:C} of member:C
//│               sym = module:C
//│             nme = Ident of "empty"
//│           sym = term:module:C.empty

Afte lowering, it is expected to be

//│         rhs = Select{sym=term:module:C.empty}:
//│           qual = Ref{disamb=module:C}:
//│             l = member:C
//│             disamb = S of module:C
//│           name = Ident of "empty"

However, if :lifter is specified, the inner reference magically loses its disambiguation information:

//│         rhs = Select{sym=term:module:C.empty}:
//│           qual = Ref:
//│             l = member:C
//│             disamb = N
//│           name = Ident of "empty"

Would you look into it? Thanks Mark 💓

[CAG2Mark]

Oh cool, something to do in tomorrow's conference when I am bored

[CAG2Mark]

You can try this commit @FlandiaYingman

CAG2Mark@f24b454

[chengluyu]

@FlandiaYingman 🥹 To be honest, I still don’t quite understand what you want to do. Let’s set a time on Discord to discuss it in person.

[FlandiaYingman]

Finally, now only 15 cases fail.

• Lifter (I applied Mark's patch, but a new problem arrives!)
• LLIR
• Handler
• WASM

[FlandiaYingman]

OK the reason why WASM failed is because I forgot to npm install. 🤡

[FlandiaYingman]

Now only 4 test cases fail!

• Lifter (asking Mark)
• Object Buffer

[FlandiaYingman]

Here I saw the symbol f is a (dummy) BlockMemberSymbol denoting a while loop label (new BlockMemberSymbol("while", Nil, false)).

Is it necessary for it to be a BlockMemberSymbol? It seems like there is no meaningful disambiguation for such symbol f like here, so my opinion is to change it to another symbol type.

[LPTK]

This symbol is not for the label, but for the inserted function, and FunDefn requires a BlockMemberSymbol. Are you suggesting changing the definition of FunDefn?

[FlandiaYingman]

Ideally we should, but I guess it'd be another PR

[LPTK]

This symbol is not for the label, but for the inserted function, and FunDefn requires a BlockMemberSymbol. Are you suggesting changing the definition of FunDefn?

[LPTK]

Ok, let's merge this now and address the rest in follow-up PRs. Nice progress towards cleaning up the logic!