gopls/internal/filewatcher: read and process event in separate goroutine

To work around the potential deadlock in fsnotify, the file watcher had
to run dir watching in a separate go routine which make the file watcher
complicated and harder to reason about.

This CL change the solution by having two methods with clear
responsibility but still work around the issue:

The run() method is designed to be highly responsive, ensures the
fsnotify's events and errors are consumed as soon as possible.

A process() method is created to actually handles the time-consuming
event processing. Including event conversion, dir watching... As a
result, the processor is now processing events in sequence and there
is no need for cancelling the ongoing watch attempt.

To keep things simple, the input handler function is now separate as
event handler and error handler. The event handler is promised to be
called sequentially but the error handler maybe called concurrently.

For golang/go#74292

Change-Id: I58b06be2022e9a94b4770eb88abff82e94d65ed9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/715882
Auto-Submit: Hongxiang Jiang <hxjiang@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

Author: h9jiang

gopls/internal/cmd/mcp.go

@@ -106,12 +106,10 @@ func (m *headlessMCP) Run(ctx context.Context, args ...string) error {
 		}
 	}()
 
-	w, err := filewatcher.New(500*time.Millisecond, nil, func(events []protocol.FileEvent, err error) {
-		if err != nil {
-			log.Printf("watch error: %v", err)
-			return
-		}
-
+	errHandler := func(err error) {
+		log.Printf("watch error: %v", err)
+	}
+	w, err := filewatcher.New(500*time.Millisecond, nil, func(events []protocol.FileEvent) {
 		if len(events) == 0 {
 			return
 		}
@@ -127,7 +125,7 @@ func (m *headlessMCP) Run(ctx context.Context, args ...string) error {
 		case nonempty <- struct{}{}:
 		default:
 		}
-	})
+	}, errHandler)
 	if err != nil {
 		return err
 	}