Any chance this would run on an iphone 16 pro?
#1
by
prajwal-ai
- opened
Curious what the ram usage is like, and how much performance degrades.
It needs 13GB of RAM, and more for the context, so no, since the iPhone total memory is half that. It would run on a 24GB Mac, considering the OS needs about 6GB, so that would leave you sufficient room for long context. Performance degrades in quants. I found it best in the q6-hi on this model, even q5-hi does fairly well.
The q4 is the lowest you can go on gpt-oss before it gets unusable.