Fixes #3947 Adds Fake driver and fixes fluent tests (iteration-zero) (#4225)

* Consider width2 chars that are not IsBmp

* Apply same fix in WindowsDriver

* Explicitly use type of local variable

* Revert changes to WindowsDriver

* Assume we are running in a terminal that supports true color by default unless user explicitly forces 16

* Switch to SetAttribute and WriteConsole instead of WriteConsoleOutput for 16 color mode

* Fix some cursor issues (WIP)

* Remove concept of 'dirty rows' from v2 as its never actually used

* Remove damageRegion as it does nothing

* Make string builder to console writing simpler

* Radically simplify Write method

* Simplify conditional logic

* Simplify restoring cursor position

* Reference local variable for console buffer

* Reduce calls to ConsoleWrite by accumulating till attribute changes

* When resizing v2 16 color mode on windows, recreate the back buffer to match its size

* Fixes for VTS enabled

* Fix _lastSize never being assigned

* Fixes VTS for Force16Colors

* Fixes force16Colors in VTS

* Fixes escape sequences always echoing in non-VTS

* Force Force16Colors in non-VTS. It have a bug in adding a newline in the last line

* WIP Add base class for NetOutput

* Abstract away how we change attribute

* WIP - Make WindowsOutput use base class

* WIP working to fix set cursor position

* Remove commented out code

* Fixes legacy output mode

* Fixes size with no alt buffer supported on VTS and size restore after maximized.

* Fix set cursor which also fixes the broken surrogate pairs

* Add force parameter

* Fixes an issue that only happens with Windows Terminal when paste surrogate pairs by press Ctrl+V

* In Windows escape sequences must be sent during the lifetime of the console which is created in input handle

* Ensure flush the input buffer before reset the console

* Flush input buffer before reset console in v2win

* Fixes issue in v2net not being refreshing the menu bar at start

* Only force layout and draw on size changed.

* Fix v2net issue not draw first line by forcing set cursor position

* Set _lastCursorPosition nullable and remove bool force from set cursor position

* Remove force parameter

* Add v2 version of fake driver attribute

* Make direct replacement and wire up window resizing events

* Update casts to use V2 fake driver instead

* Adjust interfaces to expose less internals

* Fix not raising iteration event in v2

* WIP investigate what it takes to do resize and redraw using TextAlignment_Centered as example

* Sketch adding component factory

* Create relevant fake component factories

* Add window size monitor into factory

* Fake size monitor injecting

* Add helper for faking console resize in AutoInitShutdown tests

* Fix size setting in FakeDriverV2

* Switch to new method

* Fix IsLegacy becoming false when using blank constructor

* Fix for Ready not being raised when showing same top twice also fixes garbage collection issue if running millions of top levels

* Fix tests

* Remove auto init

* Restore conditional compilation stuff

* Restore 'if running unit tests' logic

* Check only for the output being specific classes for the suppression

* Fix ShadowView blowing up with index out of bounds error

* Fix resize in fluent tests

* Fix for people using Iteration call directly

* Fix more calls to iteration to use
        AutoInitShutdownAttribute.RunIteration ();

* Add comment

* Remove assumption that Run with prior view not disposed should throw

* Fix timings in Dialog_Opened_From_Another_Dialog

* Fix Zero_Buttons_Works

* Standardize and fix Button_IsDefault_True_Return_His_Index_On_Accepting

* Fix iteration counts on MessageBoxTests

* Fix WizartTests and DrawTests_Ruler

* Implement SendKeys into ConsoleDriverFacade

* Fix SendKeys in console driver facade such that FileDialogTests works
Fix when Clip is null in popover

* Add missing dispose call to test

* Fix support for Esc in facade SendKeys

* Fix AutocompleteTests

* Fix various tests

* Replace LayoutAndDraw with run iteration

* Fix draw issues

* fix draw order

* Fix run iteration calls

* Fix unit tests

* Fix SendKeys in facade.

* Manipulate upper and lower cases.

* Add IsValidInput method to the interface.

* Fix SendKeys scenario

* Fixes surrogate pairs in the label

* Make tests more sensible - they are testing draw functionality.  Callbacks do not need to happen in Iteration method

* Fix tests and harden cleanup in AutoInitShutdownAttribute v2 lifecycle dispose

* Delete extra create input call

* Fix mocks and order of exceptions thrown in Run when things are not initialized

* Revert use of `MapConsoleKeyInfoToKeyCode`

* Ignore casing as it is not what test is really about

* Clear application top and top levels before each auto init shutdown test

* Fix for unstable tests

* Restore actually working SendKeys code

* option to pass logger in fluent ctor

* restore ToArray

* Fix SendKeys method and add extension to unit test

* Leverage the EscSeqUtils.MapConsoleKeyInfo method to avoid duplicate code

* Remove unnecessary hack

* Using only KeyCode for rKeys

* Recover modifier keys in surrogate pairs

* Reformat

* Remove iteration limit for benchmarking in v2

* remove iteration delay to identify bugs

* Remove nudge to unique key and make Then run on UI thread

* fix fluid assertions

* Ensure UI operations all happen on UI thread

* Add explicit error for WaitIteration during an invoke

* Remove timeout added for debug

* Catch failing asserts better

* Fix screenshot

* Fix null ref

* Fix race condition in processing input

* Test fixing

* Standardize asserts

* Remove calls to layout and draw, remove pointless lock and enable reading Cancelled from Dialog even if it is disposed

* fix bad merge

* Make logs access threadsafe

* add extra wait to remove race between iteration end and assert

* Code cleanup

* Remove test for crash on access Cancelled after dispose as this is no longer a restriction

* Change resize console to run on UI thread - fixing race condition with redrawing

* Restore original frame rate after test

* Restore nudge to unique key

* Code Cleanup

* Fix for cascading failures when an assert fails in a specific test

* fix for bad merge

* Address PR feedback

* Move classes to seperate files and add xmldoc

* xml doc warnings

* More xml comments docs

* Fix spelling

---------

Co-authored-by: BDisp <bd.bdisp@gmail.com>

Author: tznind

Examples/UICatalog/Scenarios/SendKeys.cs

@@ -1,4 +1,4 @@
-using System;
+using System.Text;
 
 namespace UICatalog.Scenarios;
 
@@ -39,7 +39,7 @@ public override void Main ()
 
         txtResult.KeyDown += (s, e) =>
                              {
-                                 rKeys += (char)e.KeyCode;
+                                 rKeys += e.ToString ();
 
                                  if (!IsShift && e.IsShift)
                                  {
@@ -81,17 +81,15 @@ void ProcessInput ()
 
             foreach (char r in txtInput.Text)
             {
-                ConsoleKey ck = char.IsLetter (r)
-                                    ? (ConsoleKey)char.ToUpper (r)
-                                    : (ConsoleKey)r;
+                ConsoleKeyInfo consoleKeyInfo = EscSeqUtils.MapConsoleKeyInfo (new (r, ConsoleKey.None, false, false, false));
 
                 Application.Driver?.SendKeys (
-                                             r,
-                                             ck,
-                                             ckbShift.CheckedState == CheckState.Checked,
-                                             ckbAlt.CheckedState == CheckState.Checked,
-                                             ckbControl.CheckedState == CheckState.Checked
-                                            );
+                                              r,
+                                              consoleKeyInfo.Key,
+                                              ckbShift.CheckedState == CheckState.Checked || (consoleKeyInfo.Modifiers & ConsoleModifiers.Shift) != 0,
+                                              ckbAlt.CheckedState == CheckState.Checked || (consoleKeyInfo.Modifiers & ConsoleModifiers.Alt) != 0,
+                                              ckbControl.CheckedState == CheckState.Checked || (consoleKeyInfo.Modifiers & ConsoleModifiers.Control) != 0
+                                             );
             }
 
             lblShippedKeys.Text = rKeys;

Terminal.Gui/App/Application.Run.cs

@@ -460,7 +460,7 @@ internal static void LayoutAndDrawImpl (bool forceDraw = false)
     /// <summary>This event is raised on each iteration of the main loop.</summary>
     /// <remarks>See also <see cref="Timeout"/></remarks>
     public static event EventHandler<IterationEventArgs>? Iteration;
-
+    
     /// <summary>The <see cref="MainLoop"/> driver for the application</summary>
     /// <value>The main loop.</value>
     internal static MainLoop? MainLoop { get; set; }
@@ -618,4 +618,8 @@ public static void End (RunState runState)
 
         LayoutAndDraw (true);
     }
+    internal static void RaiseIteration ()
+    {
+        Iteration?.Invoke (null, new ());
+    }
 }

Terminal.Gui/App/Application.cs

@@ -51,6 +51,20 @@ public static partial class Application
     /// </summary>
     public static ITimedEvents? TimedEvents => ApplicationImpl.Instance?.TimedEvents;
 
+    /// <summary>
+    /// Maximum number of iterations of the main loop (and hence draws)
+    /// to allow to occur per second. Defaults to <see cref="DefaultMaximumIterationsPerSecond"/>> which is a 40ms sleep
+    /// after iteration (factoring in how long iteration took to run).
+    /// <remarks>Note that not every iteration draws (see <see cref="View.NeedsDraw"/>).
+    /// Only affects v2 drivers.</remarks>
+    /// </summary>
+    public static ushort MaximumIterationsPerSecond = DefaultMaximumIterationsPerSecond;
+
+    /// <summary>
+    /// Default value for <see cref="MaximumIterationsPerSecond"/>
+    /// </summary>
+    public const ushort DefaultMaximumIterationsPerSecond = 25;
+
     /// <summary>
     ///     Gets a string representation of the Application as rendered by <see cref="Driver"/>.
     /// </summary>

Terminal.Gui/Drivers/V2/ApplicationV2.cs

@@ -1,5 +1,6 @@
 #nullable enable
 using System.Collections.Concurrent;
+using System.ComponentModel;
 using System.Diagnostics;
 using System.Diagnostics.CodeAnalysis;
 using Microsoft.Extensions.Logging;
@@ -12,10 +13,7 @@ namespace Terminal.Gui.Drivers;
 /// </summary>
 public class ApplicationV2 : ApplicationImpl
 {
-    private readonly Func<INetInput> _netInputFactory;
-    private readonly Func<IConsoleOutput> _netOutputFactory;
-    private readonly Func<IWindowsInput> _winInputFactory;
-    private readonly Func<IConsoleOutput> _winOutputFactory;
+    private readonly IComponentFactory? _componentFactory;
     private IMainLoopCoordinator? _coordinator;
     private string? _driverName;
 
@@ -24,29 +22,20 @@ public class ApplicationV2 : ApplicationImpl
     /// <inheritdoc/>
     public override ITimedEvents TimedEvents => _timedEvents;
 
+    internal IMainLoopCoordinator? Coordinator => _coordinator;
+
     /// <summary>
     ///     Creates anew instance of the Application backend. The provided
     ///     factory methods will be used on Init calls to get things booted.
     /// </summary>
-    public ApplicationV2 () : this (
-                                    () => new NetInput (),
-                                    () => new NetOutput (),
-                                    () => new WindowsInput (),
-                                    () => new WindowsOutput ()
-                                   )
-    { }
-
-    internal ApplicationV2 (
-        Func<INetInput> netInputFactory,
-        Func<IConsoleOutput> netOutputFactory,
-        Func<IWindowsInput> winInputFactory,
-        Func<IConsoleOutput> winOutputFactory
-    )
+    public ApplicationV2 ()
+    {
+        IsLegacy = false;
+    }
+
+    internal ApplicationV2 (IComponentFactory componentFactory)
     {
-        _netInputFactory = netInputFactory;
-        _netOutputFactory = netOutputFactory;
-        _winInputFactory = winInputFactory;
-        _winOutputFactory = winOutputFactory;
+        _componentFactory = componentFactory;
         IsLegacy = false;
     }
 
@@ -92,8 +81,8 @@ private void CreateDriver (string? driverName)
     {
         PlatformID p = Environment.OSVersion.Platform;
 
-        bool definetlyWin = driverName?.Contains ("win") ?? false;
-        bool definetlyNet = driverName?.Contains ("net") ?? false;
+        bool definetlyWin = (driverName?.Contains ("win") ?? false )|| _componentFactory is IComponentFactory<WindowsConsole.InputRecord>;
+        bool definetlyNet = (driverName?.Contains ("net") ?? false ) || _componentFactory is IComponentFactory<ConsoleKeyInfo>;
 
         if (definetlyWin)
         {
@@ -125,27 +114,44 @@ private IMainLoopCoordinator CreateWindowsSubcomponents ()
         ConcurrentQueue<WindowsConsole.InputRecord> inputBuffer = new ();
         MainLoop<WindowsConsole.InputRecord> loop = new ();
 
-        return new MainLoopCoordinator<WindowsConsole.InputRecord> (
-                                                                    _timedEvents,
-                                                                    _winInputFactory,
+        IComponentFactory<WindowsConsole.InputRecord> cf;
+
+        if (_componentFactory != null)
+        {
+            cf = (IComponentFactory<WindowsConsole.InputRecord>)_componentFactory;
+        }
+        else
+        {
+            cf = new WindowsComponentFactory ();
+        }
+
+        return new MainLoopCoordinator<WindowsConsole.InputRecord> (_timedEvents,
                                                                     inputBuffer,
-                                                                    new WindowsInputProcessor (inputBuffer),
-                                                                    _winOutputFactory,
-                                                                    loop);
+                                                                    loop,
+                                                                    cf);
     }
 
     private IMainLoopCoordinator CreateNetSubcomponents ()
     {
         ConcurrentQueue<ConsoleKeyInfo> inputBuffer = new ();
         MainLoop<ConsoleKeyInfo> loop = new ();
 
+        IComponentFactory<ConsoleKeyInfo> cf;
+
+        if (_componentFactory != null)
+        {
+            cf = (IComponentFactory<ConsoleKeyInfo>)_componentFactory;
+        }
+        else
+        {
+            cf = new NetComponentFactory ();
+        }
+
         return new MainLoopCoordinator<ConsoleKeyInfo> (
                                                         _timedEvents,
-                                                        _netInputFactory,
                                                         inputBuffer,
-                                                        new NetInputProcessor (inputBuffer),
-                                                        _netOutputFactory,
-                                                        loop);
+                                                        loop,
+                                                        cf);
     }
 
     /// <inheritdoc/>
@@ -171,6 +177,12 @@ public override void Run (Toplevel view, Func<Exception, bool>? errorHandler = n
             throw new NotInitializedException (nameof (Run));
         }
 
+        if (Application.Driver == null)
+        {
+            // See Run_T_Init_Driver_Cleared_with_TestTopLevel_Throws
+            throw new  InvalidOperationException ("Driver was inexplicably null when trying to Run view");
+        }
+
         Application.Top = view;
 
         RunState rs = Application.Begin (view);
@@ -258,4 +270,4 @@ public override void LayoutAndDraw (bool forceDraw)
         Application.Top?.SetNeedsDraw();
         Application.Top?.SetNeedsLayout ();
     }
-}
+}

Terminal.Gui/Drivers/V2/ComponentFactory.cs

@@ -0,0 +1,26 @@
+#nullable enable
+using System.Collections.Concurrent;
+
+namespace Terminal.Gui.Drivers;
+
+/// <summary>
+/// Abstract base class implementation of <see cref="IComponentFactory{T}"/>
+/// </summary>
+/// <typeparam name="T"></typeparam>
+public abstract class ComponentFactory<T> : IComponentFactory<T>
+{
+    /// <inheritdoc />
+    public abstract IConsoleInput<T> CreateInput ();
+
+    /// <inheritdoc />
+    public abstract IInputProcessor CreateInputProcessor (ConcurrentQueue<T> inputBuffer);
+
+    /// <inheritdoc />
+    public virtual IWindowSizeMonitor CreateWindowSizeMonitor (IConsoleOutput consoleOutput, IOutputBuffer outputBuffer)
+    {
+        return new WindowSizeMonitor (consoleOutput, outputBuffer);
+    }
+
+    /// <inheritdoc />
+    public abstract IConsoleOutput CreateOutput ();
+}

Terminal.Gui/Drivers/V2/ConsoleDriverFacade.cs

@@ -14,6 +14,10 @@ internal class ConsoleDriverFacade<T> : IConsoleDriver, IConsoleDriverFacade
     public event EventHandler<SizeChangedEventArgs> SizeChanged;
 
     public IInputProcessor InputProcessor { get; }
+    public IOutputBuffer OutputBuffer => _outputBuffer;
+
+    public IWindowSizeMonitor WindowSizeMonitor { get; }
+
 
     public ConsoleDriverFacade (
         IInputProcessor inputProcessor,
@@ -36,7 +40,8 @@ IWindowSizeMonitor windowSizeMonitor
                                          MouseEvent?.Invoke (s, e);
                                      };
 
-        windowSizeMonitor.SizeChanging += (_, e) => SizeChanged?.Invoke (this, e);
+        WindowSizeMonitor = windowSizeMonitor;
+        windowSizeMonitor.SizeChanging += (_,e) => SizeChanged?.Invoke (this, e);
 
         CreateClipboard ();
     }
@@ -68,7 +73,7 @@ public Rectangle Screen
     {
         get
         {
-            if (ConsoleDriver.RunningUnitTests)
+            if (ConsoleDriver.RunningUnitTests && _output is WindowsOutput or NetOutput)
             {
                 // In unit tests, we don't have a real output, so we return an empty rectangle.
                 return Rectangle.Empty;
@@ -384,7 +389,15 @@ public Attribute MakeColor (in Color foreground, in Color background)
     /// <param name="ctrl">If <see langword="true"/> simulates the Ctrl key being pressed.</param>
     public void SendKeys (char keyChar, ConsoleKey key, bool shift, bool alt, bool ctrl)
     {
-        // TODO: implement
+        ConsoleKeyInfo consoleKeyInfo = new (keyChar, key, shift, alt, ctrl);
+
+        Key k = EscSeqUtils.MapKey (consoleKeyInfo);
+
+        if (InputProcessor.IsValidInput (k, out k))
+        {
+            InputProcessor.OnKeyDown (k);
+            InputProcessor.OnKeyUp (k);
+        }
     }
 
     /// <summary>

Terminal.Gui/Drivers/V2/IComponentFactory.cs

@@ -0,0 +1,50 @@
+#nullable enable
+using System.Collections.Concurrent;
+
+namespace Terminal.Gui.Drivers;
+
+/// <summary>
+/// Base untyped interface for <see cref="IComponentFactory{T}"/> for methods that are not templated on low level
+/// console input type.
+/// </summary>
+public interface IComponentFactory
+{
+    /// <summary>
+    /// Create the <see cref="IConsoleOutput"/> class for the current driver implementation i.e. the class responsible for
+    /// rendering <see cref="IOutputBuffer"/> into the console.
+    /// </summary>
+    /// <returns></returns>
+    IConsoleOutput CreateOutput ();
+}
+
+/// <summary>
+/// Creates driver specific subcomponent classes (<see cref="IConsoleInput{T}"/>, <see cref="IInputProcessor"/> etc) for a
+/// <see cref="IMainLoopCoordinator"/>.
+/// </summary>
+/// <typeparam name="T"></typeparam>
+public interface IComponentFactory<T> : IComponentFactory
+{
+    /// <summary>
+    /// Create <see cref="IConsoleInput{T}"/> class for the current driver implementation i.e. the class responsible for reading
+    /// user input from the console.
+    /// </summary>
+    /// <returns></returns>
+    IConsoleInput<T> CreateInput ();
+
+    /// <summary>
+    /// Creates the <see cref="InputProcessor{T}"/> class for the current driver implementation i.e. the class responsible for
+    /// translating raw console input into Terminal.Gui common event <see cref="Key"/> and <see cref="MouseEventArgs"/>.
+    /// </summary>
+    /// <param name="inputBuffer"></param>
+    /// <returns></returns>
+    IInputProcessor CreateInputProcessor (ConcurrentQueue<T> inputBuffer);
+
+    /// <summary>
+    /// Creates <see cref="IWindowSizeMonitor"/> class for the current driver implementation i.e. the class responsible for
+    /// reporting the current size of the terminal window.
+    /// </summary>
+    /// <param name="consoleOutput"></param>
+    /// <param name="outputBuffer"></param>
+    /// <returns></returns>
+    IWindowSizeMonitor CreateWindowSizeMonitor (IConsoleOutput consoleOutput, IOutputBuffer outputBuffer);
+}

Terminal.Gui/Drivers/V2/IConsoleDriverFacade.cs

@@ -10,5 +10,16 @@ public interface IConsoleDriverFacade
     ///     e.g. <see cref="ConsoleKeyInfo"/> into <see cref="Key"/> events
     ///     and detecting and processing ansi escape sequences.
     /// </summary>
-    public IInputProcessor InputProcessor { get; }
+    IInputProcessor InputProcessor { get; }
+
+    /// <summary>
+    ///     Describes the desired screen state. Data source for <see cref="IConsoleOutput"/>.
+    /// </summary>
+    IOutputBuffer OutputBuffer { get; }
+
+    /// <summary>
+    ///     Interface for classes responsible for reporting the current
+    ///     size of the terminal window.
+    /// </summary>
+    IWindowSizeMonitor WindowSizeMonitor { get; }
 }

Terminal.Gui/Drivers/V2/IInputProcessor.cs

@@ -58,4 +58,15 @@ public interface IInputProcessor
     /// </summary>
     /// <returns></returns>
     public IAnsiResponseParser GetParser ();
+
+    /// <summary>
+    ///     Handles surrogate pairs in the input stream.
+    /// </summary>
+    /// <param name="key">The key from input.</param>
+    /// <param name="result">Get the surrogate pair or the key.</param>
+    /// <returns>
+    ///     <see langword="true"/> if the result is a valid surrogate pair or a valid key, otherwise
+    ///     <see langword="false"/>.
+    /// </returns>
+    bool IsValidInput (Key key, out Key result);
 }

Terminal.Gui/Drivers/V2/IMainLoop.cs

@@ -48,7 +48,14 @@ public interface IMainLoop<T> : IDisposable
     /// <param name="inputBuffer"></param>
     /// <param name="inputProcessor"></param>
     /// <param name="consoleOutput"></param>
-    void Initialize (ITimedEvents timedEvents, ConcurrentQueue<T> inputBuffer, IInputProcessor inputProcessor, IConsoleOutput consoleOutput);
+    /// <param name="componentFactory"></param>
+    void Initialize (
+        ITimedEvents timedEvents,
+        ConcurrentQueue<T> inputBuffer,
+        IInputProcessor inputProcessor,
+        IConsoleOutput consoleOutput,
+        IComponentFactory<T> componentFactory
+    );
 
     /// <summary>
     ///     Perform a single iteration of the main loop then blocks for a fixed length