Skip to content

Commit 76d4920

Browse files
huydhnfacebook-github-bot
authored andcommitted
Update TorchBench commit to main (#145455)
Summary: I'm adding sam2 to TorchBench #2566, so, as part of that, I'm updating PyTorch CI to use latest TorchBench commit. The corresponding change from TorchBench is #2584 The main thing to call out that the newer transformers added by #2488 is regressing several models. This needs to be investigated further, and I pin the version to unblock this change. * `hf_Roberta_base` a new model added by #2279, not sure why it fails accuracy on A10G, but it works fine on A100 * `speech_transformer` failures are pre-existing trunk failures, i.e. https://github.com/pytorch/pytorch/actions/runs/13040114684/job/36380989702#step:22:2408 X-link: pytorch/pytorch#145455 Approved by: https://github.com/kit1980 Reviewed By: ZainRizvi Differential Revision: D69056903 fbshipit-source-id: 6b98c246b9dc8811257ec46d4821ba6ee3363dfe
1 parent 3e5a389 commit 76d4920

File tree

2 files changed

+3
-2
lines changed

2 files changed

+3
-2
lines changed

userbenchmark/dynamo/dynamobench/common.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ class CI(NamedTuple):
189189
"stable_diffusion_text_encoder",
190190
"timm_efficientdet",
191191
"timm_nfnet",
192-
"timm_regnet",
192+
"timm_resnest",
193193
"timm_vision_transformer",
194194
"timm_vovnet",
195195
"vgg16",

userbenchmark/dynamo/dynamobench/torchbench.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,14 +34,14 @@ tolerance:
3434
- vgg16
3535
- mobilenet_v3_large
3636
- nvidia_deeprecommender
37-
- timm_efficientdet
3837

3938
# These models need >1e-3 tolerance
4039
even_higher:
4140
- soft_actor_critic
4241
- tacotron2
4342
- yolov3
4443
- timm_efficientdet
44+
- timm_efficientnet
4545
- squeezenet1_1
4646

4747
higher_fp16:
@@ -66,6 +66,7 @@ tolerance:
6666

6767
require_larger_multiplier_for_smaller_tensor:
6868
- yolov3
69+
- timm_efficientnet
6970

7071
# These benchmarks took >600s on an i9-11900K CPU
7172
very_slow: &VERY_SLOW_MODELS

0 commit comments

Comments
 (0)