Skip to content

Apriel 2 #397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Apriel 2 #397

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
7e17715
Refactor Apriel2 configuration and preprocessing architecture
tscholak Nov 27, 2025
4496e2a
Fix cache validation test to properly test both empty and corrupted c…
tscholak Nov 27, 2025
c2c17e7
Fix Apriel2 config and converter issues
tscholak Nov 27, 2025
98a5d25
Clean up Apriel2 converters with stratified inheritance
tscholak Nov 27, 2025
c4a7709
Add Llava-to-Apriel2 HuggingFace converter with comprehensive tests
tscholak Nov 28, 2025
f3992bf
Separate model conversion from surgery for Apriel2
tscholak Nov 28, 2025
935f595
Replace legacy converters with expression-based plan system
tscholak Nov 28, 2025
c95b899
Add DIL conversion, stochastic mixer support, and fix tree collapsing
tscholak Nov 29, 2025
255be1b
Add streaming I/O for memory-efficient weight conversion
tscholak Nov 29, 2025
10a4f38
Refactor conversion into modular subpackage with source-agnostic conv…
tscholak Nov 30, 2025
31513b2
Add gated_delta_net mixer to stochastic supernet example
tscholak Nov 30, 2025
b9bd43a
Add surgery chains, Apriel2 source format, and clean up docstrings
tscholak Dec 1, 2025
3eb8bfb
Merge remote-tracking branch 'origin/main' into tscholak/apriel2-conv…
tscholak Dec 1, 2025
e135f00
Rename patch_convolution to embeddings for consistency with Fast-LLM
tscholak Dec 1, 2025
da3786b
Fix vision encoder numerical equivalence and add comprehensive test s…
tscholak Dec 2, 2025
bd321bd
Fix Apriel2 converter weight paths after external model refactor
tscholak Dec 3, 2025
1eb07a2
Merge origin/main into tscholak/apriel2-conversion
tscholak Dec 3, 2025
249250b
Add 2D rotary embedding equivalence tests for FastLLM vs Pixtral
tscholak Dec 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions fast_llm/models/gpt/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from fast_llm.layers.common.peft.config import PeftConfig
from fast_llm.layers.language_model.config import LanguageModelConfig, MultiTokenPredictionConfig
from fast_llm.models.gpt.conversion.config import (
Apriel2CheckpointFormat,
Apriel2TextCheckpointFormat,
AprielHybridSSMCheckpointFormat,
AutoGPTHuggingfaceCheckpointFormat,
DiffusionDreamCheckpointFormat,
Expand Down Expand Up @@ -112,7 +112,7 @@ class GPTModelConfig(FastLLMModelConfig):
DiffusionDreamCheckpointFormat,
DiffusionLlamaCheckpointFormat,
AprielHybridSSMCheckpointFormat,
Apriel2CheckpointFormat,
Apriel2TextCheckpointFormat,
)

@classmethod
Expand Down
Loading
Loading