Differences

This shows you the differences between two versions of the page.

--- tips:llm [2025/12/20 17:49] – sscipioni
+++ tips:llm [2026/02/15 08:29] (current) – sscipioni
@@ Line 23: / Line 23: @@
 | A100 40GB | 156.7 | 45.2 | |
 | M3 Max 128GB | 34.8 | 4.2 | |
-| Strix Halo 128GB | | 5.1 | 85.02 |
+| Strix Halo 128GB ollama | | 5.1 | 85.02 |
+| Strix Halo 128GB llama.cpp | |  | 90 |
 | RTX 3060 | | | 131.76 |
+ROCM
+^ model                  ^ capabilities             ^ size     ^ context  ^ quantization                                                                      ^ eval rate [token/s]  ^ prompt eval rate [token/s]  ^
+| llama3.2  | completion tools | "3.2B"   | 131072    | "Q4_K_M" | 52.78 | 1957.30 |
+| qwen-strixhalo  | completion tools | "30.5B"   | 262144    | "Q4_K_M" | 53.54 | 1056.37 |
+| qwen3-coder  | completion tools | "30.5B"   | 262144    | "Q4_K_M" | 52.10 | 776.55 |
+| qwen3:30b-a3b  | completion tools thinking | "30.5B"   | 262144    | "Q4_K_M" | 50.19 | 803.06 |
+| gpt-oss:20b  | completion tools thinking | "20.9B"   | 131072    | "MXFP4" | 45.37 | 519.90 |
+| glm-4.7-flash  | completion tools thinking | "29.9B"   | 202752    | "Q4_K_M" | 41.54 | 470.09 |
+| qwen3:8b  | completion tools thinking | "8.2B"   | 40960    | "Q4_K_M" | 32.68 | 890.98 |
+| qwen3-coder-next  | completion tools | "79.7B"   | 262144    | "Q4_K_M" | 33.06 | 380.21 |
+| qwen2.5-coder:14b-instruct-q4_K_M  | completion tools insert | "14.8B"   | 32768    | "Q4_K_M" | 17.25 | 527.74 |
+VULKAN
 ^ model                  ^ capabilities             ^ size     ^ context  ^ quantization                                                                      ^ eval rate [token/s]  ^ prompt eval rate [token/s]  ^
-| llama3.2               | completion tools         | "3.2B"   | 131072   | "Q4_K_M"                                                                          | 88.14                | 715.43                      |
+| qwen3-coder  | completion tools | "30.5B"   | 262144    | "Q4_K_M" | 54.03 | 805.43 |
-| ministral-3:14b        | completion vision tools  | "13.9B"  | 262144   | "Q4_K_M"                                                                          | 23.78                | 302.07                      |
+| llama3.2  | completion tools | "3.2B"   | 131072    | "Q4_K_M" | 52.54 | 1838.82 |
-| qwen3-coder:30b        | completion tools         | "30.5B"  | 262144   | "Q4_K_M"                                                                          | 73.75                | 72.41                       |
+| gpt-oss:20b  | completion tools thinking | "20.9B"   | 131072    | "MXFP4" | 43.36 | 475.60 |
-| llama3:70b | completion   | "70.6B"  | 8192     | "Q4" | 5.55 | 9.72 |
-| llava  | completion vision | "7B"   | 32768    | "Q4"  | 49.92                | 207.27                      |
-| deepseek-coder-v2:16b  | completion insert        | "15.7B"  | 163840   | "Q4"                                                                            | 84.44                | 111.71                      |
+ollama model
-| bjoernb/qwen3-coder-30b-1m:latest  | completion tools | "30.5B"   | 1048576    | "Q4_K_M" | 74.23 | 94.84 |
+<code>
-| freehuntx/qwen3-coder:8b  | completion tools | "8.2B"   | 40960    | "Q4_K_M" | 37.97 | 565.68 |
+FROM qwen3-coder
-| networkjohnny/deepseek-coder-v2-lite-base-q4_k_m-gguf:latest  | completion tools | "3.2B"   | 131072    | "Q4_K_M" | 86.02 | 1124.53 |
-| phi4-mini  | completion tools | "3.8B"   | 131072    | "Q4_K_M" | 72.24 | 31.37 |
+# STRIX HALO AGENTIC TUNING
-| qwen2.5:7b  | completion tools | "7.6B"   | 32768    | "Q4_K_M" | 42.98 | 153.34 |
+PARAMETER num_ctx 128000
-| llama3.3:70b-instruct-q4_K_M  | completion tools | "70.6B"   | 131072    | "Q4_K_M" | 5.06 | 15.50 |
+PARAMETER num_batch 1024
+PARAMETER num_predict 4096
+SYSTEM """
+You are a Strix Halo Optimized Coding Agent.
+Always use asynchronous patterns and favor memory-efficient algorithms.
+"""
+</code>

Galileo Labs

User Tools

Site Tools

Differences

Page Tools