[Intel HPU] add example benchmark scripts for hpu #5304
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds benchmark scripts and utilities for Intel HPU testing in FastDeploy. The changes include server/client scripts for running benchmarks, log parsing tools for analyzing performance data, and configuration files for model deployment on HPU hardware.
Key Changes:
- Benchmark server and client scripts supporting both fixed-length and ShareGPT datasets
- Log parsing and visualization utilities for performance analysis
- Configuration and demo scripts for HPU deployment
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| benchmark_paddle_hpu_server.sh | Server startup script with HPU-specific environment configuration |
| benchmark_paddle_hpu_server_sharegpt.sh | Server script variant supporting ShareGPT dataset benchmarking |
| benchmark_paddle_hpu_cli.sh | Client benchmark script for fixed-length input/output tests |
| benchmark_paddle_hpu_cli_sharegpt.sh | Client benchmark script for ShareGPT dataset tests |
| parse_benchmark_logs.py | Log parser extracting metrics and profiling data into CSV |
| draw_benchmark_data.py | Visualization tool generating graphs from profile logs |
| offline_demo.py | Offline inference demo for HPU |
| run_offline_demo.sh | Shell script to run offline demo with HPU configuration |
| bench_gsm8k.py | GSM8K benchmark evaluation script |
| intel_hpu_serving_benchmark.md | Documentation for running HPU benchmarks |
| eb45-300b-a47b-32k-bf16.yaml | Model configuration for ERNIE-4.5-300B |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #5304 +/- ##
==========================================
Coverage ? 59.60%
==========================================
Files ? 324
Lines ? 39702
Branches ? 5973
==========================================
Hits ? 23666
Misses ? 14150
Partials ? 1886
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
zoooo0820
reviewed
Dec 1, 2025
examples/intel_hpu/test.jsonl
Outdated
| {"question": "Janet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?", "answer": "Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.\nShe makes 9 * 2 = $<<9*2=18>>18 every day at the farmer\u2019s market.\n#### 18"} | ||
| {"question": "A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?", "answer": "It takes 2/2=<<2/2=1>>1 bolt of white fiber\nSo the total amount of fabric is 2+1=<<2+1=3>>3 bolts of fabric\n#### 3"} | ||
| {"question": "Josh decides to try flipping a house. He buys a house for $80,000 and then puts in $50,000 in repairs. This increased the value of the house by 150%. How much profit did he make?", "answer": "The cost of the house and repairs came out to 80,000+50,000=$<<80000+50000=130000>>130,000\nHe increased the value of the house by 80,000*1.5=<<80000*1.5=120000>>120,000\nSo the new value of the house is 120,000+80,000=$<<120000+80000=200000>>200,000\nSo he made a profit of 200,000-130,000=$<<200000-130000=70000>>70,000\n#### 70000"} | ||
| {"question": "James decides to run 3 sprints 3 times a week. He runs 60 meters each sprint. How many total meters does he run a week?", "answer": "He sprints 3*3=<<3*3=9>>9 times\nSo he runs 9*60=<<9*60=540>>540 meters\n#### 540"} |
Contributor
Author
There was a problem hiding this comment.
源文件已经删了，因为repo里面已经有了
| ```bash | ||
| ./benchmark_paddle_hpu_cli.sh | ||
| ``` | ||
|
|
Collaborator
There was a problem hiding this comment.
这里个example的作用应该也是面向用户实际使用和复现测试结果的。这个文档是否可以再详细一些，介绍下不同脚本的使用场景，包括一些用法、脚本参数、启动环境变量的含义
zoooo0820
approved these changes
Dec 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
add benchmark scripts for hpu
Modifications
benchmark server and cli scripts for hpu with log-parsing scripts
Usage or Command
please refer to the README in this patch
Accuracy Tests
No accuracy impact with this updates
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.conducted by local tests
releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.