How does your machine compare to others running Ollama?

Results - llama2:7b

Tokens / Sec Load Time OS GPU CPU RAM OBM Score Ollama Version Reported Count
149 1 LinuxMint 21.2 1 x NVIDIA Corporation Device 2684 with 23.98828125GB Intel Gen Intel® Core™ i9-13900F with 32 cores at 0 129GB 1121 0.1.20 1
124 2 Ubuntu 22.04.3 LTS 1 x NVIDIA Corporation GA102 [GeForce RTX 3090] with 24GB AMD Ryzen 7 3700X 8-Core Processor with 16 cores at 3.6 32GB 929 0.1.18 1
116 9 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 80GB Intel Xeon® with 12 cores at 2.2 171GB 869 0.1.17 1
116 1 Microsoft Windows 11 Pro 10.0.22631 1 x NVIDIA NVIDIA GeForce RTX 4090 with nullGB Intel Core™ i9-14900KF with 32 cores at 3.2 GB 869 0.2.1 3
113 2 Microsoft Windows 10 Pro 10.0.19045 1 x NVIDIA NVIDIA GeForce RTX 3090 with nullGB AMD Ryzen 9 5950X 16-Core Processor with 32 cores at 3.4 GB 848 0.1.31 2
111 9 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 12 cores at 2.2 85GB 834 0.1.17 1
101 5 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation GV100GL [Tesla V100 SXM2 16GB] with 16GB Intel Xeon® with 2 cores at 2.3 13GB 755 0.1.17 1
95 2 Ubuntu 22.04.3 LTS 1 x Red Hat, Inc. Virtio GPU with 0.00390625GB, 1 x NVIDIA Corporation TU102GL [Quadro RTX 6000/8000] with 24GB AMD EPYC-Rome Processor with 14 cores at 0 46GB 711 0.0.18 1
94 4 Microsoft Windows 11 Home 10.0.22631 1 x NVIDIA NVIDIA GeForce RTX 3090 with nullGB Intel Gen Intel® Core™ i9-12900K with 24 cores at 3.2 GB 702 0.1.25 1
90 3 Microsoft Windows 11 Pro 10.0.22631 1 x NVIDIA NVIDIA GeForce RTX 3090 with nullGB Intel Gen Intel® Core™ i7-11700F with 16 cores at 2.5 GB 674 0.1.30 1
83 2 Ubuntu 20.04.5 LTS 1 x Amazon.com, Inc. Device 1111 with 0.00390625GB, 1 x NVIDIA Corporation Device A10G with 24GB AMD EPYC 7R32 with 16 cores at 0 64GB 624 0.1.17 1
69 4 Microsoft Windows Server 2022 Datacenter 10.0.20348 1 x (Standard display types) Microsoft Basic Display Adapter with nullGB, 1 x NVIDIA NVIDIA RTX A4000 with nullGB Intel Xeon® Gold 5315Y with 8 cores at 3.2 GB 516 0.1.25 1
63 6 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 472 0.1.18 1
62 1 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 469 0.1.24 1
62 2 Ubuntu 22.04.4 LTS 1 x NVIDIA Corporation GA104 [GeForce RTX 3070 Ti] with 8GB AMD Ryzen 5 5600X 6-Core Processor with 12 cores at 3.7 32GB 468 0.1.33 1
62 1 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 463 0.1.17 1
59 3 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 443 0.1.21 2
59 8 Fedora Linux 40.20240604.0 1 x Advanced Micro Devices, Inc. AMD Radeon Pro W6800 with 29.984375GB AMD Ryzen 3 3300X 4-Core Processor with 8 cores at 3.8 16GB 440 0.1.40 1
57 6 macOS 14.2.1 1 x Apple Apple M1 Max with 65536GB Apple M1 Max with 10 cores at 2.4 65536GB 431 0.1.32 1
56 1 macOS 14.2.1 1 x Apple Apple M1 Max with 32GB Apple M1 Max with 10 cores at 2.4 32GB 419 0.1.17 1
48 6 Ubuntu 20.04.5 LTS 1 x Amazon.com, Inc. Device 1111 with 0.00390625GB, 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® Platinum 8259CL with 4 cores at 2.5 16GB 360 0.1.17 2
45 3 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 16 cores at 2.2 14GB 340 0.1.18 1
45 3 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 32 cores at 2.2 121GB 336 0.1.18 1
44 15 Ubuntu 20.04.5 LTS 2 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 24 cores at 2.2 171GB 333 0.1.17 1
44 4 Microsoft Windows Server 2022 Datacenter 10.0.20348 1 x Microsoft Microsoft Remote Display Adapter with nullGB, 1 x NVIDIA NVIDIA Tesla T4 with nullGB Intel Xeon® with 32 cores at 2.2 GB 330 0.1.25 1
43 13 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 2 cores at 2.2 13GB 326 0.1.18 2
43 4 Microsoft Windows Server 2022 Datacenter 10.0.20348 1 x Microsoft Microsoft Remote Display Adapter with nullGB, 1 x NVIDIA NVIDIA Tesla T4 with nullGB Intel Xeon® with 16 cores at 2.2 GB 320 0.1.25 1
39 14 Ubuntu 20.04.5 LTS 2 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 4 cores at 2.2 26GB 292 0.1.17 1
33 24 Ubuntu 20.04.5 LTS 4 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 16 cores at 2.2 105GB 249 0.1.17 1
30 3 Microsoft Windows 11 Pro 10.0.22631 1 x NVIDIA NVIDIA GeForce RTX 3050 with nullGB AMD Ryzen 9 3900X 12-Core Processor with 24 cores at 3.8 GB 225 0.1.25 1
28 3 macOS 14.1.2 1 x Apple Apple M1 Pro with 16GB Apple M1 Pro with 8 cores at 2.4 16GB 212 0.1.22 1
7 4 Microsoft Windows 11 Home Single Language 10.0.26100 1 x NVIDIA NVIDIA GeForce RTX 3050 Laptop GPU with nullGB, 1 x Advanced Micro Devices, Inc. AMD Radeon(TM) Graphics with nullGB AMD Ryzen 7 6800HS Creator Edition with 16 cores at 3.2 GB 52 0.5.11 1
7 5 Microsoft Windows 11 Home 10.0.22621 1 x Advanced Micro Devices, Inc. AMD Radeon(TM) Graphics with nullGB, 1 x NVIDIA NVIDIA GeForce RTX 2050 with nullGB AMD Ryzen 5 7535HS with Radeon Graphics with 12 cores at 3.3 GB 51 0.1.25 2
6 3 Microsoft Windows 11 Home Single Language 10.0.26100 1 x NVIDIA NVIDIA GeForce RTX 3050 Laptop GPU with nullGB, 1 x Advanced Micro Devices, Inc. AMD Radeon(TM) Graphics with nullGB AMD Ryzen 7 6800HS Creator Edition with 16 cores at 3.2 GB 42 0.5.5 2
2 6 Microsoft Windows 11 Pro 10.0.22631 1 x Intel Corporation Intel(R) Iris(R) Plus Graphics with nullGB, 1 x NVIDIA NVIDIA GeForce GTX 1650 with Max-Q Design with nullGB, 2 x DisplayLink DisplayLink USB Device with nullGB Intel Core™ i7-1065G7 with 8 cores at 1.3 GB 14 0.1.28 1

Results - llama2:13b

Tokens / Sec Load Time OS GPU CPU RAM OBM Score Ollama Version Reported Count
90 0 LinuxMint 21.2 1 x NVIDIA Corporation Device 2684 with 23.98828125GB Intel Gen Intel® Core™ i9-13900F with 32 cores at 0 129GB 4641 0.1.20 1
83 11 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 80GB Intel Xeon® with 12 cores at 2.2 171GB 4260 0.1.17 1
78 3 Ubuntu 22.04.3 LTS 1 x NVIDIA Corporation GA102 [GeForce RTX 3090] with 24GB AMD Ryzen 7 3700X 8-Core Processor with 16 cores at 3.6 32GB 4014 0.1.18 1
77 11 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 12 cores at 2.2 85GB 3949 0.1.17 1
65 18 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation GV100GL [Tesla V100 SXM2 16GB] with 16GB Intel Xeon® with 2 cores at 2.3 13GB 3322 0.1.17 1
57 3 Ubuntu 22.04.3 LTS 1 x Red Hat, Inc. Virtio GPU with 0.00390625GB, 1 x NVIDIA Corporation TU102GL [Quadro RTX 6000/8000] with 24GB AMD EPYC-Rome Processor with 14 cores at 0 46GB 2910 0.0.18 1
50 3 Ubuntu 20.04.5 LTS 1 x Amazon.com, Inc. Device 1111 with 0.00390625GB, 1 x NVIDIA Corporation Device A10G with 24GB AMD EPYC 7R32 with 16 cores at 0 64GB 2560 0.1.17 1
36 1 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 1862 0.1.18 1
36 11 macOS 14.2.1 1 x Apple Apple M1 Max with 65536GB Apple M1 Max with 10 cores at 2.4 65536GB 1855 0.1.32 1
36 12 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 1850 0.1.24 1
36 12 Fedora Linux 40.20240604.0 1 x Advanced Micro Devices, Inc. AMD Radeon Pro W6800 with 29.984375GB AMD Ryzen 3 3300X 4-Core Processor with 8 cores at 3.8 16GB 1826 0.1.40 1
35 12 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 1820 0.1.17 1
35 11 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 1788 0.1.21 2
34 18 Ubuntu 20.04.5 LTS 2 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 24 cores at 2.2 171GB 1766 0.1.17 1
32 1 macOS 14.2.1 1 x Apple Apple M1 Max with 32GB Apple M1 Max with 10 cores at 2.4 32GB 1648 0.1.17 1
27 7 Ubuntu 20.04.5 LTS 1 x Amazon.com, Inc. Device 1111 with 0.00390625GB, 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® Platinum 8259CL with 4 cores at 2.5 16GB 1386 0.1.17 2
26 27 Ubuntu 20.04.5 LTS 4 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 16 cores at 2.2 105GB 1361 0.1.17 1
26 16 Ubuntu 20.04.5 LTS 2 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 4 cores at 2.2 26GB 1350 0.1.17 1
25 5 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 32 cores at 2.2 121GB 1261 0.1.18 1
24 16 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 16 cores at 2.2 14GB 1258 0.1.18 1
23 27 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 2 cores at 2.2 13GB 1195 0.1.18 2
15 13 macOS 14.1.2 1 x Apple Apple M1 Pro with 16GB Apple M1 Pro with 8 cores at 2.4 16GB 760 0.1.22 1
6 3 Ubuntu 22.04.4 LTS 1 x NVIDIA Corporation GA104 [GeForce RTX 3070 Ti] with 8GB AMD Ryzen 5 5600X 6-Core Processor with 12 cores at 3.7 32GB 305 0.1.33 1

Results - llama2:70b

Tokens / Sec Load Time OS GPU CPU RAM OBM Score Ollama Version Reported Count
25 21 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 80GB Intel Xeon® with 12 cores at 2.2 171GB 2438 0.1.17 1
14 27 Ubuntu 20.04.5 LTS 2 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 24 cores at 2.2 171GB 1425 0.1.17 1
10 37 Ubuntu 20.04.5 LTS 4 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 16 cores at 2.2 105GB 999 0.1.17 1
8 57 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 806 0.1.18 1
8 60 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 805 0.1.24 1
8 57 macOS 14.2.1 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 800 0.1.21 2
8 58 macOS 14.1.2 1 x Apple Apple M1 Max with 64GB Apple M1 Max with 10 cores at 2.4 64GB 790 0.1.17 1
8 59 macOS 14.2.1 1 x Apple Apple M1 Max with 65536GB Apple M1 Max with 10 cores at 2.4 65536GB 780 0.1.32 1
2 19 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation Device A100 with 40GB Intel Xeon® with 12 cores at 2.2 85GB 200 0.1.17 1
1 7 Ubuntu 20.04.5 LTS 1 x Amazon.com, Inc. Device 1111 with 0.00390625GB, 1 x NVIDIA Corporation Device A10G with 24GB AMD EPYC 7R32 with 16 cores at 0 64GB 145 0.1.17 1
1 65 LinuxMint 21.2 1 x NVIDIA Corporation Device 2684 with 23.98828125GB Intel Gen Intel® Core™ i9-13900F with 32 cores at 0 129GB 127 0.1.20 1
1 10 Ubuntu 20.04.5 LTS 1 x NVIDIA Corporation TU104GL [Tesla T4] with 15GB Intel Xeon® with 32 cores at 2.2 121GB 122 0.1.18 1

Try it on your machine. Be nice if it has a problem. Report any issues here: https://github.com/technovangelist/obm

Download for Windows Download for Linux Download for Intel Mac Download for Apple Silicon Mac

After downloading, you may need to make it executable, move it to your path. You may want to rename it to something like obm.