Issue/170：GPTQ算子的CPU平台重构 #181
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xgqdut2016
force-pushed
the
issue/170
branch
3 times, most recently
from
38f7ad4 to
c7f8aa6
Compare
xgqdut2016
force-pushed
the
issue/170
branch
2 times, most recently
from
318d48e to
2c512d5
Compare
xgqdut2016
force-pushed
the
issue/170
branch
2 times, most recently
from
4a4fad1 to
d605520
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
对于a = torch.randn,b=1e-3 * torch.randn也能通过测试，但是由于CPU获取scale,zero涉及过多矩阵计算，测试非常慢，目前对于7B矩阵的测试，当group_size=-1的时候，需要3h，CUDA平台适配的是marlin，目前也能通过测试，国产芯片是arm架构，不支持immintrin.h，必须注释掉CPU平台和immintrin.h相关的函数和头文件才能编译成功
CUDA的性能如下图所示：