Update docs to reflect distributed module

Reviewers: #tensorflow, #framework_ip_review_-_any_oss_or_third-party_code_use_has_been_approved, #documentation, jamiep, christiana

Reviewed By: #tensorflow, #framework_ip_review_-_any_oss_or_third-party_code_use_has_been_approved, #documentation, jamiep, christiana

Subscribers: christiana

Maniphest Tasks: T68097

Differential Revision: https://phabricator.sourcevertex.net/D73894

Author: Frederik Mellbye

tensorflow/compiler/plugin/poplar/docs/api.rst

@@ -92,17 +92,22 @@ For example, this will not work:
   :members: IPUMultiWorkerStrategy
   :special-members: __init__
 
-.. automodule:: tensorflow.python.ipu.horovod
+.. automodule:: tensorflow.python.ipu.distributed
   :members:
 
-.. automodule:: tensorflow.python.ipu.horovod.ipu_horovod_strategy
+.. automodule:: tensorflow.python.ipu.distributed.ipu_horovod_strategy
   :members: IPUHorovodStrategy
   :special-members: __init__
 
-.. automodule:: tensorflow.python.ipu.horovod.popdist_strategy
+.. automodule:: tensorflow.python.ipu.distributed.popdist_strategy
   :members: PopDistStrategy
   :special-members: __init__
 
+.. note::
+  Both :py:class:`tensorflow.python.ipu.distributed.popdist_strategy.PopDistStrategy`
+  and :py:class:`tensorflow.python.ipu.distributed.ipu_horovod_strategy.IPUHorovodStrategy`
+  are still available through the deprecated module `tensorflow.python.ipu.horovod`.
+
 .. Serving utilities
 
 .. automodule:: tensorflow.python.ipu.serving

tensorflow/compiler/plugin/poplar/docs/distributed_training.rst

@@ -10,7 +10,7 @@ communication over the host network, for example for broadcasting the
 initial values of variables from the first instance to the others.
 
 To perform distributed training on Pod systems, use
-:class:`~tensorflow.python.ipu.horovod.popdist_strategy.PopDistStrategy`,
+:class:`~tensorflow.python.ipu.distributed.popdist_strategy.PopDistStrategy`,
 which performs data-parallel synchronous training using multiple host processes.
 In this sense it is similar to
 `MultiWorkerMirroredStrategy <https://www.tensorflow.org/api_docs/python/tf/distribute/MultiWorkerMirroredStrategy>`_
@@ -20,7 +20,7 @@ network using Horovod.
 Collective operations (explicitly through a member function like ``reduce()`` or
 implicitly by using an optimizer under the strategy scope) will be performed
 directly on the IPU by using compiled communications with the GCL library
-over the IPU links and GW-Links. The
+over the IPU-Links and GW-Links. The
 ``PopDistStrategy`` is designed for use with PopDist and PopRun.
 Refer to the `PopDist and PopRun User Guide
 <https://docs.graphcore.ai/projects/poprun-user-guide/>`_ for more details.

tensorflow/python/ipu/distributed/ipu_horovod_strategy.py

@@ -16,7 +16,7 @@
 from tensorflow.python.distribute import reduce_util
 from tensorflow.python.distribute.cluster_resolver import cluster_resolver as cluster_resolver_lib
 from tensorflow.python.ipu import ipu_multi_worker_strategy
-from tensorflow.python.ipu.horovod import Sum, Average, size, allreduce, broadcast
+from tensorflow.python.ipu.distributed import Sum, Average, size, allreduce, broadcast
 from tensorflow.python.training import server_lib
 from tensorflow.python.util import deprecation