fix(coreml): remove broken --optimize-ane flag from generate script#3632
Open
AlanConstantino wants to merge 1 commit intoggml-org:masterfrom
Open
fix(coreml): remove broken --optimize-ane flag from generate script#3632AlanConstantino wants to merge 1 commit intoggml-org:masterfrom
AlanConstantino wants to merge 1 commit intoggml-org:masterfrom
Conversation
The --optimize-ane flag in convert-whisper-to-coreml.py is explicitly marked as 'currently broken' in its help text. When enabled, it produces CoreML models that load successfully but crash with SIGSEGV during inference. The ANE optimization code (WhisperANE classes) hasn't been maintained to work with: - Newer model architectures (e.g., large-v3-turbo with 128 mel bins) - Recent coremltools versions that generate iOS 18+ specific ops Removing this flag allows generate-coreml-model.sh to produce working CoreML models that run correctly on Apple Silicon via Metal/ANE without the broken optimization path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
generate-coreml-model.shscript currently passes--optimize-ane Trueto the conversion script, but this flag is explicitly marked as broken:Problem
When
--optimize-ane Trueis used, the generated CoreML models:Core ML model loadedThe ANE optimization code (
WhisperANEclasses) hasn't been maintained to work with:Ios18.scaledDotProductAttention, etc.)Solution
Remove the
--optimize-ane Trueflag from the shell script, allowing it to use the working default (False).Added a comment explaining why ANE optimization is disabled for future maintainers.
Testing
Tested on M4 Mac mini with large-v3-turbo model:
COREML = 1Related