[Runtime] Fix incorrect memory ordering in ConcurrentReadableArray/HashMap.

When reallocating storage, the storing the pointer to the new storage had insufficiently strong ordering. This could cause writers to check the reader count before storing the new storage pointer. If a reader then came in between that load and store, it would end up using the old storage pointer, while the writer could end up freeing it.

Also adjust the Concurrent.cpp tests to test with a variety of reader and writer counts. Counterintuitively, when freeing garbage is gated on there being zero readers, having a single reader will shake out problems that having lots of readers will not. We were testing with lots of readers in order to stress the code as much as possible, but this resulted in it being extremely rare for writers to ever see zero active readers.

rdar://69798617

Author: mikeash

include/swift/Runtime/Concurrent.h

@@ -527,7 +527,12 @@ template <class ElemTy> struct ConcurrentReadableArray {
 
       storage = newStorage;
       Capacity = newCapacity;
-      Elements.store(storage, std::memory_order_release);
+
+      // Use seq_cst here to ensure that the subsequent load of ReaderCount is
+      // ordered after this store. If ReaderCount is loaded first, then a new
+      // reader could come in between that load and this store, and then we
+      // could end up freeing the old storage pointer while it's still in use.
+      Elements.store(storage, std::memory_order_seq_cst);
     }
 
     new(&storage->data()[count]) ElemTy(elem);
@@ -866,7 +871,11 @@ struct ConcurrentReadableHashMap {
       FreeListNode::add(&FreeList, elements);
     }
 
-    Elements.store(newElements, std::memory_order_release);
+    // Use seq_cst here to ensure that the subsequent load of ReaderCount is
+    // ordered after this store. If ReaderCount is loaded first, then a new
+    // reader could come in between that load and this store, and then we
+    // could end up freeing the old elements pointer while it's still in use.
+    Elements.store(newElements, std::memory_order_seq_cst);
     return newElements;
   }
 
@@ -900,7 +909,11 @@ struct ConcurrentReadableHashMap {
       newIndices.storeIndexAt(nullptr, index, newI, std::memory_order_relaxed);
     }
 
-    Indices.store(newIndices.Value, std::memory_order_release);
+    // Use seq_cst here to ensure that the subsequent load of ReaderCount is
+    // ordered after this store. If ReaderCount is loaded first, then a new
+    // reader could come in between that load and this store, and then we
+    // could end up freeing the old indices pointer while it's still in use.
+    Indices.store(newIndices.Value, std::memory_order_seq_cst);
 
     if (auto *ptr = indices.pointer())
       FreeListNode::add(&FreeList, ptr);