More refactoring

Author: mgreter

Makefile.conf

@@ -51,15 +51,16 @@ HPPFILES = \
 	exceptions.hpp \
 	settings.hpp \
 	memory.hpp \
-	memory/config.hpp \
-	memory/allocator.hpp \
-	memory/shared_ptr.hpp \
-	memory/memory_pool.hpp \
+	memory_config.hpp \
+	memory_allocator.hpp \
+	memory_pool.hpp \
+	shared_ptr.hpp \
 	MurmurHash2.hpp \
 	eval.hpp \
 	extender.hpp \
 	extension.hpp \
 	file.hpp \
+	import.hpp \
 	compiler.hpp \
 	fn_meta.hpp \
 	fn_maps.hpp \
@@ -142,6 +143,7 @@ SOURCES = \
 	ast_sel_super.cpp \
 	ast_sel_weave.cpp \
 	ast_selectors.cpp \
+	memory_allocator.cpp \
 	modules.cpp \
 	constants.cpp \
 	compiler.cpp \
@@ -156,6 +158,7 @@ SOURCES = \
 	environment.cpp \
 	ast_fwd_decl.cpp \
 	file.cpp \
+	import.cpp \
 	string_utils.cpp \
 	logger.cpp \
 	strings.cpp \
@@ -205,9 +208,7 @@ SOURCES = \
 	source_state.cpp \
 	source_span.cpp \
 	exceptions.cpp \
-	memory/allocator.cpp \
-	memory/shared_ptr.cpp \
-	LUrlParser/LUrlParser.cpp \
+	shared_ptr.cpp \
 	unicode.cpp \
 	cencode.cpp
 

docs/dev-env-stacks.md

@@ -6,38 +6,113 @@ for implementers. It documents how variable stacks are implemented.
 ## Foreword
 
 LibSass uses an optimized stack approach similar to how C compilers have always
-done it, by using a growable stack where we can push and pop items of. Unfortunately
+done it, by using a growable stack where we can push and pop items. Unfortunately
 Sass has proven to be a bit more dynamic than static optimizers like, therefore we
 had to adopt the principle a little to accommodate the edge-cases due to this.
 
 There are three different kind of entities on the stack during runtime, namely
-variables, functions and mixins. Each has it's own dedicated stack to optimize
+variables, functions and mixins. Each has its own dedicated stack to optimize
 the lookups. In this doc we will often only cover one case, but it should be
-applicable to any other stack object (with some small differences).
+applicable to any other stack object (with some small differences). Variables
+are the most complicated ones, as functions and mixins can only be declared
+on the root scope, so loops or functions don't need to be considered for them.
 
 Also for regular sass code and style-rules we wouldn't need this setup, but
 it becomes essential to correctly support mixins and functions, since those
-can be called recursively. It is also vital for loops, like @for or @each.
+can be called recursively. It is also vital for loops, like `@for` or `@each`.
 
 ## Overview
 
 The whole process is split into two main phases. In order to correctly support
-@import we had to introduce the preloader phase, where all @use, @forward and
-@import rules are loaded first, before any evaluation happens. This ensures that
+`@import` we had to introduce the preloader phase, where all `@use`, `@forward` and
+`@import` rules are loaded first, before any evaluation happens. This ensures that
 we know all entities before the evaluation phase in order to correctly setup
-all stack frames.
+all stack frames before populating them.
+
+### Parser/EnvFrame phase
+
+During parsing every potential scope block creates an `EnvFrame`, which is
+stored at the corresponding ast-node (e.g. on a `StyleRule`). The `EnvFrame`
+is designed as a RAII stack object. It adds itself to the frame-stack on
+creation and removes itself once it goes out of scope. Additionally it
+creates an `EnvRefs` instance on the heap, which is later used by the
+compile/evaluation phase.
+
+### Compiler/EnvScope phase
+
+The `EnvScope` is similar to the `EnvFrame`, as it is also designed to be a RAII
+stack object. On creation it will increase the stack for each entity and update
+the corresponding offset pointers and reverts it when it goes out of scope.
+
+### EnvRefs heap object
+
+The `EnvRefs` object is the main object holding all the information about a
+block scope. It mainly holds three (flat) maps, one for every entity type.
+These maps are used to resolve a name to an integer offset. Whenever a new
+entity (e.g. variable assignment), the item is added to the map if it does
+not already exist there and the offset is simply increased (size of the map).
+
+### EnvRoot object
+
+The `EnvRoot` is the main object where entities are stored. It mainly holds
+two stack vectors for every entity type. The actual stack vector and an
+additional vector holding the previous stack size (or offset position).
+Further it holds on to all created `EnvRefs` and all built-in entities.
+
+### Frame offset pointers
+
+Every `EnvRefs` object get a unique number, increasing from 0 (which is
+reserved for the root scope block). The `EnvRoot` objects has a vector
+containing all `EnvRefs` which should correspond to that index offset.
 
 ## Basic example
 
 Let's assume we have the following scss code:
 
 ```scss
-$a: 1;
+$a: a;
 b {
-  $a: 2;
+  $a: b;
 }
 ```
 
+### Parsing phase
+
+The parser will first initialize the `EnvRoot` with the root `EnvRefs`.
+First it will parse the top variable assignment, which will create a new
+entry in the variable map of the `EnvRefs` heap object.
+
+It will then parse the `StyleRule` and create a `EnvFrame` (which will also
+create and register a new `EnvRefs` heap object). It will then parse the inner
+assignment and create a new entry in the new `EnvRefs` of the `StyleRule`.
+
+### Evaluation phase
+
+First the compiler will create an `EnvScope` stack object, which will increase
+the stack size of `EnvRoot` accordingly. In this case it will increase the size
+of `varStack` by one to accommodate the single local variable. Note that the
+variable object itself is still undefined at this point, but the slot on the
+stack exists now. The `EnvScope` will also remember that it has to decrease
+that stack vector by one, once it goes out of scope.
+
+On the assignment the compiler knows that variable `$a` can be found at the
+local offset `0` (as stored within the `EnvRefs` map). Now it only needs to
+add the current `EnvRefs` position on the `varStack` to get the absolute
+address of the requested variable.
+
+
+
+ when it sees `b`, it will
+create and assign a new `EnvFrame` with the `StyleRule`. Each `EnvRefs`
+will have one Variable `$a` with the offset `0`. On runtime the compiler will
+first evaluate the top assignment rule, thus assigning the string `a` to the
+variable at offset `0` at the current active scope (root). Then it evaluates
+the ruleset and creates a new `EnvScope`, which will push new instances onto
+the env-stack for all previously parsed entities (one variable in this case).
+
+We now have two variables on the actual env-scope with the inner still undefined.
+
+
 This will allocate two independent variables on the stack. For easier reference
 we can think of them as variable 0 and variable 1. So let's see what happens if
 we introduce some VariableExpressions:
@@ -59,7 +134,7 @@ As you may have guesses, the `a0` expression will reference variable 0 and the
 variable 0 again. Given this easy example this might seem overengineered, but
 let's see what happens if we introduce a loop:
 
-```
+```scss
 $a: 1;
 b {
   @for $x from 1 through 2 {

docs/dev-var-scoping.md

@@ -0,0 +1,57 @@
+# Sass variable scoping
+
+Variable scoping in Sass has some quirks which I'll try to
+explain in this document. This partially also applies to
+functions and mixins, as they are scope objects like variables
+too. But they can only be defined on the root scope, so their
+logic is not that complicated as variables.
+
+## Variable declarations in Sass
+
+Sass doesn't have explicit variable declarations. Variables
+are declared whenever an assignment to a variable happens. It
+is then local to the scope block the assignment was in. E.g.
+every ruleset (opened via `{` in scss) is a new scope block.
+
+On the surface this makes variable scoping quite simple, as
+declarations always happen with the definition of the variable.
+Under the hood it has some quirks, mainly in combination with loops.
+
+## Block scopes and (semi) transparent loops
+
+Consider the following example:
+
+```scss
+$a: 0;
+@for $i from 1 through 3 {
+  @debug $a;
+  $a: $i;
+}
+@debug $a
+```
+
+The first `@debug` calls should be easy to guess (0,1,2). But the
+last `@debug` is a bit trickier. In this case the `@for` loop is
+transparent and the assignment inside it is made to the outer
+variable on the root scope and will be reported as `3`.
+
+Now lets wrap the loop into a ruleset scope:
+
+```scss
+$b: 0;
+a {
+  @for $i from 1 through 3 {
+    @debug $b;
+    $b: $i;
+  }
+  @debug $b
+}
+```
+
+Again, the inner `@debug` calls are simply 0,1 and 2 and the
+tricky question is again what the last `@debug` call will yield?
+You might have guess it will report `3` again, but that is not
+correct, as Sass will report `0` here.
+
+Conclusion: Loops like `@for`, `@each` and `@while` are transparent
+only on the root-scope, but not on any inner scopes.