board generator documentation (#4989)

* update "more flash" doc

* doc: try to emphasize OOM knowledge

* fighting with markdown?

* it seems I can't emphasize text in link

* board generator doc: need reviewers

* markdown formatting, typos

* typos

* add links

* + last memory failure allocation + emphasis exception decoder

* more emphasis on exception decoder

* repeat the board generator introduction

* fixes

Author: d-a-v

doc/faq/a02-my-esp-crashes.rst

@@ -9,8 +9,8 @@ My ESP crashes running some code. How to troubleshoot it?
 -  `What is the Cause of Restart? <#what-is-the-cause-of-restart>`__
 -  `Exception <#exception>`__
 -  `Watchdog <#watchdog>`__
--  `Check Where the Code Crashes <#check-where-the-code-crashes>`__
--  `Other Causes for Crashes <#other-causes-for-crashes>`__
+-  `Exception Decoder <#exception-decoder>`__
+-  `Other Common Causes for Crashes <#other-causes-for-crashes>`__
 -  `If at the Wall, Enter an Issue
    Report <#if-at-the-wall-enter-an-issue-report>`__
 -  `Conclusion <#conclusion>`__
@@ -161,8 +161,8 @@ out before restart, you should be able to narrow down part of code
 firing the h/w wdt reset. If diagnosed application or library has debug
 option then switch it on to aid this troubleshooting.
 
-Check Where the Code Crashes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Exception Decoder
+~~~~~~~~~~~~~~~~~
 
 Decoding of ESP stack trace is now easy and available to everybody
 thanks to great `Arduino ESP8266/ESP32 Exception Stack Trace
@@ -183,21 +183,25 @@ If you don't have any code for troubleshooting, use the example below:
       Serial.println();
       Serial.println("Let's provoke the s/w wdt firing...");
       //
-      // wait for s/w wdt in infinite loop below
+      // provoke an OOM, will be recorded as the last occured one
+      char* out_of_memory_failure = (char*)malloc(1000000);
       //
+      // wait for s/w wdt in infinite loop below
       while(true);
       //
       Serial.println("This line will not ever print out");
     }
 
     void loop(){}
 
-Upload this code to your ESP (Ctrl+U) and start Serial Monitor
-(Ctrl+Shift+M). You should shortly see ESP restarting every couple of
-seconds and ``Soft WDT reset`` message together with stack trace showing
-up on each restart. Click the Autoscroll check-box on Serial Monitor to
-stop the messages scrolling up. Select and copy the stack trace, go to
-the *Tools* and open the *ESP Exception Decoder*.
+Enable the Out-Of-Memory (*OOM*) debug option (in the *Tools > Debug Level*
+menu), compile/flash/upload this code to your ESP (Ctrl+U) and start Serial
+Monitor (Ctrl+Shift+M).  You should shortly see ESP restarting every couple
+of seconds and ``Soft WDT reset`` message together with stack trace showing
+up on each restart.  Click the Autoscroll check-box on Serial Monitor to
+stop the messages scrolling up.  Select and copy the stack trace, including
+the ``last failed alloc call: ...`` line, go to the *Tools* and open the
+*ESP Exception Decoder*.
 
 .. figure:: pictures/a02-decode-stack-tace-1-2.png
    :alt: Decode the stack trace, steps 1 and 2
@@ -263,7 +267,7 @@ Asynchronous Callbacks
    can happen.
 
 Memory, memory, memory
-   Running out of heap is the most common cause for crashes. Because the build process for
+   Running out of heap is the **most common cause for crashes**. Because the build process for
    the ESP leaves out exceptions (they use memory), memory allocations that fail will do
    so silently. A typical example is when setting or concatenating a large String. If 
    allocation has failed internally, then the internal string copy can corrupt data, and 
@@ -287,6 +291,11 @@ Memory, memory, memory
    rely on exceptions for error handling, which is not available for the ESP, and in any
    case there is no access to the underlying code.
 
+   Instrumenting the code with the OOM debug option and calls to ``ESP.getFreeHeap()`` will
+   help the process of finding leaks. Now is time to re-read about the
+   `exception decoder <#exception-decoder>`__.
+
+
 *Some techniques for reducing memory usage*
 
    * Don't use const char * with literals. Instead, use const char[] PROGMEM. This is particularly true if you intend to, e.g.: embed html strings.

doc/faq/a05-board-generator.rst

@@ -0,0 +1,81 @@
+:orphan:
+
+Board generator
+---------------
+
+The board generator is a python script originally intended to ease the
+Arduino IDE's `boards.txt` configuration file about the multitude of
+available boards, especially when common parameters have to be updated for
+all of them.
+
+This script is also used to manage uncommon options that are currently not
+available in the IDE menu.
+
+-  `How can I run the script ? <#how-can-i-run-the-script>`__
+-  `What can I do with it ? <#what-can-i-do-with-it>`__
+-  `When do I need to update it ? <#when-do-i-need-to-mess-with-it>`__
+-  `Why is my pull-request failing continuous-integration ? <#why-is-my-pull-request-failing-continuous-integration>`__
+
+How can I run the script ?
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Python 2 needs to be installed on your system.
+
+The script is located in the ``tools`` subdirectory of the core's root installation.
+It needs to be run from the root directory,
+
+::
+
+    $ tools/boards.txt.py
+
+::
+
+    C:\...> tools\boards.txt.py
+    C:\...> python tools\boards.txt.py
+
+Running without parameters will show the command line help.  They are
+generally self-explanatory.
+
+
+What can I do with it ?
+~~~~~~~~~~~~~~~~~~~~~~~
+
+As of today you can:
+
+* in the IDE: change the default serial programming speed of any board
+
+* in the IDE: add new serial programming speed
+
+* increase available flash space by disabling floats in ``*printf`` functions
+
+* enable WPS which is now disabled by default (at the cost of a smaller heap by ~4KB)
+
+* change led pin ``LED_BUILTIN`` for the two generic boards
+
+* change the default lwIP version (1.4 or 2)
+
+
+When do I need to mess with it ?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The board generator is used to automate generation of configuration files
+when possible.  It needs to be edited for:
+
+* All information for specific boards.  This is the only place where a new
+  board (definition, description) can be updated or added to the existing
+  list.
+
+* Memory mapping for ldscripts (flash and spiffs size combinations)
+
+
+Why is my pull-request failing continuous-integration ?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The generator is able to update a number of files (see list in help), and
+global coherency can be checked by the continuous integration facilities.
+
+After a modification in the generator, it is **mandatory** to regenerate all
+files (option ``--allgen``) and add them in the pull-request.
+
+
+`FAQ list :back: <readme.rst>`__

doc/faq/readme.rst

@@ -43,10 +43,14 @@ entering an issue report, please perform initial troubleshooting.
 How can I get some extra KBs in flash ?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Using ``*printf()`` with floats is enabled by default.  Some KBs of flash can
-be saved by using the option ``--nofloat`` with the boards generator:
+* Using ``*printf()`` with floats is enabled by default.  Some KBs of flash can
+  be saved by using the option ``--nofloat`` with the boards generator:
 
-``./tools/boards.txt.py --nofloat --allgen``
+  ``./tools/boards.txt.py --nofloat --allgen``
+
+* Use the debug level option ``NoAssert-NDEBUG`` (in the Tools menu)
+
+`Read more <a05-board-generator.rst>`__.
 
 Why can't I use WPS ?
 ~~~~~~~~~~~~~~~~~~~~~
@@ -56,6 +60,8 @@ WPS (and lose 4KB of useable ram), use this boards generator option:
 
 ``./tools/boards.txt.py --allowWPS --allgen``
 
+`Read more <a05-board-generator.rst>`__.
+
 This Arduino library doesn't work on ESP. How do I make it work?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -139,3 +145,17 @@ The following lines are compatible with both lwIP versions:
     }
 
 Ref.  `#1923 <https://github.com/esp8266/Arduino/issues/1923>`__
+
+
+Why is there a board generator and what about it ?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The board generator is a python script originally intended to ease the
+Arduino IDE's `boards.txt` configuration file about the multitude of
+available boards, especially when common parameters have to be updated for
+all of them.
+
+This script is also used to manage uncommon options that are currently not
+available in the IDE menu.
+
+`Read more <a05-board-generator.rst>`__.