@@ -12,7 +12,7 @@ There are a few ways you can perform distributed training in
1212PyTorch with each method having their advantages in certain use cases:
1313
1414* `DistributedDataParallel (DDP) <#learn-ddp >`__
15- * `Fully Sharded Data Parallel (FSDP ) <#learn-fsdp >`__
15+ * `Fully Sharded Data Parallel (FSDP2 ) <#learn-fsdp >`__
1616* `Tensor Parallel (TP) <#learn-tp >`__
1717* `Device Mesh <#device-mesh >`__
1818* `Remote Procedure Call (RPC) distributed training <#learn-rpc >`__
@@ -60,28 +60,18 @@ Learn DDP
6060
6161.. _learn-fsdp :
6262
63- Learn FSDP
63+ Learn FSDP2
6464----------
6565
6666.. grid :: 3
6767
6868 .. grid-item-card :: :octicon:`file-code;1em`
69- Getting Started with FSDP
69+ Getting Started with FSDP2
7070 :link: https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html?utm_source=distr_landing&utm_medium=FSDP_getting_started
7171 :link-type: url
7272
7373 This tutorial demonstrates how you can perform distributed training
74- with FSDP on a MNIST dataset.
75- +++
76- :octicon: `code;1em ` Code
77-
78- .. grid-item-card :: :octicon:`file-code;1em`
79- FSDP Advanced
80- :link: https://pytorch.org/tutorials/intermediate/FSDP_advanced_tutorial.html?utm_source=distr_landing&utm_medium=FSDP_advanced
81- :link-type: url
82-
83- In this tutorial, you will learn how to fine-tune a HuggingFace (HF) T5
84- model with FSDP for text summarization.
74+ with FSDP2 on a transformer model
8575 +++
8676 :octicon: `code;1em ` Code
8777
@@ -196,7 +186,6 @@ Custom Extensions
196186 intermediate/ddp_tutorial
197187 intermediate/dist_tuto
198188 intermediate/FSDP_tutorial
199- intermediate/FSDP_advanced_tutorial
200189 intermediate/TCPStore_libuv_backend
201190 intermediate/TP_tutorial
202191 intermediate/pipelining_tutorial
0 commit comments