Runs great on the M2 96GB
#1
by
jac-jim
- opened
Running on battery, here are the speeds:
llama_perf_sampler_print: sampling time = 71.98 ms / 1130 runs ( 0.06 ms per token, 15697.93 tokens per second)
llama_perf_context_print: load time = 29767.26 ms
llama_perf_context_print: prompt eval time = 340.96 ms / 40 tokens ( 8.52 ms per token, 117.32 tokens per second)
llama_perf_context_print: eval time = 20637.74 ms / 1089 runs ( 18.95 ms per token, 52.77 tokens per second)
llama_perf_context_print: total time = 55587.54 ms / 1129 tokens
llama_perf_context_print: graphs reused = 1084
Interrupted by user
Wow! I can't wait to use this at work on Monday! :)