r/LocalLLM • u/New-Implement-5979 • 1h ago
Discussion Current situation
But this is good, when there is something to be tested and verified we verify the shit out of it.
r/LocalLLM • u/New-Implement-5979 • 1h ago
But this is good, when there is something to be tested and verified we verify the shit out of it.
r/LocalLLM • u/jmp_rsp • 22h ago
Just upgraded my workstation (i9-10940X, 256 GB quad-channel DDR4-3600) from an RTX 2080 Ti on llama.cpp (14–15 tok/s decode, ~155 tok/s prefill) to a single RTX PRO 4500 Blackwell (32 GB) running Basalt with a locally built Qwen3.8-Flash-Next NVFP4 + Q6_K dense pack.
Compared to 33.5 tok/s on llama.cpp and 130 tok/s on Basalt GGUF, this pack pushes 150–185 tok/s decode, hitting 185.4 tok/s here, 179.9 tok/s avg on Linux, and 148.7 tok/s on native Windows, alongside 3,000–4,284 tok/s fresh prefill.
Across 100+ sustained multi-turn agentic coding/RE turns (20K–90K+ ctx), it averages ~125 tok/s with ~90–93% VRAM expert cache hits (30.9 / 31.9 GiB VRAM used, rest pinned in RAM, zero SSD reads).
It stays cool at 73–79 °C and cut real end-to-end batch times ~10× (85–113 min down to ~11 min avg) with fewer mistakes in review.
If you run Basalt natively on Windows, expect a ~10–17% WDDM decode tax, stay on WDDM (TCC reports VIRTUAL_MEMORY_MANAGEMENT_SUPPORTED = 0 and breaks elastic KV >32K), and set "fit-max-tokens": true in basalt_config.json so deep-context agents don't hit 400 context_overflow.
r/LocalLLM • u/StudioTV • 21h ago
Inkling 2 Mini is everywhere this week, so I dug into what it actually needs to run locally. The short version: the loop is free in weights, but not in cache or speed, and day-one llama.cpp is slower than the hype suggests.
Why it's different
24 layers, but the middle 12 (layers 6-17) run three times per token with shared weights, so 48 layer visits per token. Each pass keeps its own KV cache and conv state (vLLM blog and the llama.cpp GGUF metadata both confirm this). So for memory, think of the KV cache as a 48-layer model: 8 global visits + 40 sliding-window visits (1024 tokens), 8 KV heads × 128.
Weights (published files)
| Format | Size |
|---|---|
| BF16 GGUF | 52.2 GB |
| Q8_0 | 27.7 GB |
| Official NVFP4 / MXFP8 (vLLM, SGLang, Ollama) | ~20 GB |
| Q4_K_M / UD-Q4_K_XL | 17.4 / 17.6 GB |
| UD-IQ4_XS | 15.7 GB |
Vision projector: +0.64 GB. DSpark draft: +0.4 to 0.65 GB.
KV cache per request (BF16)
| Context | KV |
|---|---|
| 32K | ~1.2 GiB |
| 128K | ~4.2 GiB |
| 1M | ~32 GiB |
That matches a measured M5 Pro 64 GB run: 23.1 GB process RSS with IQ4_NL and a 128K slot reserved. A 32 GB Mac is fine for 4-bit. 24 GB is tight: one M4 Pro 24 GB report got 11.6 tok/s under memory pressure.
Speed: think ~5B active, not 3.4B
The looped layers are re-read from memory on every pass, so per-token reads look like a ~5B-active MoE. Formal llama-bench numbers (Unsloth's llama.cpp build, UD-IQ4_XS, tg128 / pp2048):
| GPU | Decode | Prefill |
|---|---|---|
| RTX PRO 6000 | 97 tok/s | 2,687 tok/s |
| RTX 5090 | 97 | 2,446 |
| RTX 4090 | 75 | 2,665 |
| RTX 3090 | 60 | 1,402 |
| A100 80GB | 47 | 1,230 |
Upstream llama.cpp (b12600) is slower for now: the relative-position bias is materialized as a big F32 tensor and some ops fall back to the CPU (issue #32707). Reports: ~52 tok/s on a 5090, 20.8 tok/s on an RX 9070 XT (IQ3_XXS, Vulkan), ~30 tok/s on an M5 Pro Mac mini (IQ4_NL). Use Unsloth's build until the fix lands.
Long context is a prefill problem, not a decode problem: DGX Spark with vLLM + DSpark still decodes 18.4 tok/s at ~950K tokens, but the first token took 26.5 minutes.
vLLM gotcha
In the dedicated inkling-2-mini vLLM image, FP8 KV cache is only accepted on SM12x GPUs (RTX 50 series, RTX PRO Blackwell, DGX Spark). On H100/H200/B200 it's BF16 KV only.
The 125-210 tok/s Mac numbers going around are from EXO's upcoming engine, with no quant, context length or version published, so I'd treat them as vendor peaks for now.
Disclosure: I maintain a free VRAM calculator (no sign-up) and added Inkling 2 Mini with all of the above modelled, including the per-pass cache and the llama.cpp speeds calibrated on the benchmarks in the table. If you want to check your own setup: https://studiotvai.com/llm-vram-calculator?model=inkling-2-mini
Happy to be corrected if anyone has numbers on other cards or Macs, especially Apple Silicon with Unsloth's build.
r/LocalLLM • u/eltokh7 • 14h ago
Introducing Lory-27B v2 and Lory-4B v2! 🦜🦜
The first open-source decision models post-trained specifically to make simulated survey respondents follow economic logic, and answer like a real person would.
Why? Because existing AI decision models are great at imitating personas but fail at following simple monetary incentives. You can change the price of an option, its probability or both and they rarely adjust their response.
With better ability to ignore irrelevant traits or deal with rewordings of questions.
With additional training and Weight-Space Programming, these new models outperform the preview version on ~every eval. Lory-27B went from 73 to 80 on a blind test that Gemini wrote, and both versions got significantly better on the many tests where they underperformed before. For comparison, the best hosted decision model I could run on that same test is Perplexity's Decider v1.2 at 74.
And now, it can answer according to theory, or matching human responses. Tell it to answer like a typical person and you get data closer to what surveys collect. On Choices13k (a large real-world choice dataset), Lory-27B's average error dropped from 21 → 9 points when asked to match human responses.
r/LocalLLM • u/Infamous-Yard-9644 • 21h ago
Great news. A new member is joining the ultra-lightweight model segment. The news came out of yesterday's AMA with the Tencent Hy team on r/LocalLLaMA, where they answered many questions about their roadmap and technical direction. Notably, the team also said there is still some way to go before smaller models reach final release. They still believe in the Scaling Law and build on that foundation, since small-model development is absolutely valuable.
r/LocalLLM • u/ryanmerket • 7h ago
r/LocalLLM • u/Affectionate-Time86 • 20h ago
Anyone know why deepseek was removed from 0.2.13 onwards?
r/LocalLLM • u/raven737 • 19h ago
Yes, I get 60+ t/s output with mtp in some tasks but it's more like 38t/s on average. However, prefill at about 200t/s is painfully slow. It "works" but the system pulls 500W when loaded and it's basically unusable for anything interactive. There is no way to speed up prefill on this old hardware, right?
r/LocalLLM • u/PlasticRevenue4601 • 13h ago
I ported maya-ik2 — a dedicated CUDA engine for Inkling 2 Mini (github.com/alesha-pro/maya-inkling-mini, 128 tok/s on an RTX 3090) — to HIP/ROCm, running natively on Windows on an AMD Radeon AI PRO R9700. No quality loss, top 1%, KLD etc are identical bitwise. All the job was done via Opus 5.5 as a planner + Qwen 3.8 Flash Next as an implementer.
AI generated context disclaimer - the summary below is generated with help of the Opus 5.5.
Setup: Windows 10, ROCm 7.14.1 (TheRock wheels), Unsloth UD-Q4_K_XL, FP8 KV cache, warm server, greedy, no speculative decoding.
| Decode, short prompt | 90 |
| Decode at 4K depth | 87 |
| Decode at 64K depth | 66 |
| Prefill, 4K prompt | ~2,500 |
| Prefill, 64K prompt | ~1,780 |
Keep in mind that these numbers are far from the hardware limits, after more thorough optimisation I expect prefill of around 5K, decode of around 120-150 tok\s !without speculating decoding
Upd: I’ll publish the fork as soon as I reach the desired performance + prepare the code for production — RDNA3 port + rigorous testing
r/LocalLLM • u/raymondh210129 • 2h ago
Hey everyone!
When I first started working on Adaptive KV Cache Streaming, I didn’t think I could make MTP or a DFlash2 draft model work alongside it within my VRAM budget. In my second post, I shared how reusing the streaming infrastructure made that possible.
This time, I’d like to share what may be the last major architectural milestone for this project: multi-slot inference using the same streaming infrastructure.
The implementation still uses one arena and one ring buffer, shared across all slots. Each slot keeps its own KV history, but those histories are combined into a single layout that the streaming engine handles without needing to distinguish between slots. During decoding, the attention kernel runs once per slot for each attention layer, reading only the portions of the KV cache that belong to that slot, across both the resident area and the ring buffer.
Here’s what real-world decode performance looks like with Unsloth’s Qwen 27B IQ4 on my RTX 5070 Ti 16GB, powering a Hermes agent that I asked to plan a vacation with help from two subagents:

Each point represents a completed decode period. The colors indicate how many slots were decoding together. The x-axis shows their combined context length at the start of that period, and the y-axis shows the total output throughput across those slots.
The scheduler groups as many eligible slots as it can, provided their combined KV history stays within the configured context budget. The 262K budget is shared across the active slots.
For me, this was the last piece of the puzzle—the feature that finally made the whole project feel ready for everyday use.
I’ve also decided to give the project a name: Oceanus, after the Greek Titan associated with the river encircling the world. It felt fitting for a project built around a ring buffer, with KV history continually flowing through a limited amount of GPU memory, to support lots of core functionalities.
----
A few personal thoughts before I wrap up:
There are already plenty of inference engines doing impressive work, especially with larger MoE models. My motivation has always been a little more personal: I have a 16GB GPU, and I want to see how much more performance I can squeeze out of it when running a dense model.
A lot of the satisfaction has come from taking something I initially thought was impossible, suddenly thinking of a creative way to approach it, and actually making it work. Seeing this grow from an experiment into something that can handle an actual agent workflow has meant a lot to me.
And I’m really grateful to everyone here who tried the project, reported bugs, shared results, asked questions, or simply left an encouraging comment.
This may be my last big architecture post for a while. I feel like the main pieces I wanted to build are finally in place. I’ll still be working on optimizations, fixes, and support for new models.
Thank you for following along and helping me get here.
Hope my project helps you get a little more out of your 16GB GPU too :)
Clarification of LLM usage of this post: I'm not a native English speaker and I used ChatGPT to refine the wordings.
r/LocalLLM • u/pragmojo • 23h ago
Just an observation: I've been kicking the tires on Inkling, asking it to create some self-contained projects to get a feel for the model. In the prompt, I don't specify a title for any of these, and I noticed that it titles them "Inlking" every single time.
Never seen another model do this.
r/LocalLLM • u/LostRequirement4828 • 18h ago
I've been running Thinking Machines' Inkling 2 Mini (26B MoE, 3.4B active, middle layers looped 3x) as a local coding agent on my RX 7900 XT. On llama.cpp with the Inkling 2 PRs it did 25 tok/s on Vulkan. It now does 108.
| Step | tok/s |
|---|---|
| llama.cpp + the Inkling 2 PRs, as is | 25 |
| expert gate/up halves kept on the GPU | 66.5 |
| custom ops for the loop and depth-mix parts, fused router | 84 |
| more fused ops + faster long-context attention | 98.7 |
| own NVFP4 mat-vec shader, expert scale fused into the matmul | 108.9 |
llama-bench tg64, UD-Q4_K_M. At 32k context: 71 -> 92.5 tok/s. A real llama-server reply: 108.4 tok/s. Prompt processing ~2075 tok/s (pp512).
What made the difference:
Quality is unchanged: KLD vs the CPU backend 0.004, wikitext-2 perplexity 7.67. Every new op has a CPU version, test-backend-ops cases and an env switch back to the old path.
Setup: 96k context, K q8_0 / V q4_0 cache, flash attention, ~19.5 GB of the 20 GB.
Code, build and run notes: https://github.com/vengeful180-dot/llama.cpp (INKLING.md)
Only tested on my card (RDNA3, Mesa RADV). Upstream just merged its own looped Inkling 2 support with a different GGUF layout; this branch runs the original UD-Q4_K_M file from unsloth.
Most of the code was written with Claude Code; every step was benchmarked and tested on this card.
r/LocalLLM • u/AnimalPuzzleheaded71 • 18h ago
What I mean true local is not those 500 billion parameter open weight models that no regular person could ever run, I'm talking about what you can realistically run at 10-20 tokens per second speed on a budget of 8-32gb normal ram and 8-24gb vram as most consumer PCs fall into that range.
r/LocalLLM • u/SummarizedAnu • 5h ago
I made this post to clear up other posts saying that this model was not doing a good job for them.
Specially the hallucination rate, because mine didn't hallucinate with 5 to 6 compactions at all. And it refused every single time it didn't know instead of making something up. So, hallucination is completely 0.
My setup
Results first (it finished 114 of the 162 tasks, more on that below)
| What | Score |
|---|---|
| Overall, average of all tasks | 94% |
| Hallucination traps and normal control questions | 100% |
| Tool calls | 100% |
| Long documents | 100% |
| Reasoning and math | 100% |
| Coding (hidden tests) | 100% |
| Instruction following | 100% |
| Prompt injection and safety | 100% |
| Business data questions | 87% |
| Memory | 20% |
Some numbers in plain words:
Where it fell down
Now for how the test works for anybody interested :
There are 162 tasks in total. Every one is generated by code (no public benchmarks, nothing the model could have seen in training), and every answer is checked by code too, no AI judge. The model never gets to see the answer key, and it has to write each answer to a file in a set format. It also has to follow a few standing rules the whole time (valid JSON, say how confident you are, end every explanation with the task ID), so I could see if it gets sloppy hours in.
What the tasks are, with the kind of thing I mean:
Each task gets a score from 0 to 100% (partial credit, so getting 4 of 5 rows right is 80%). The percentages below are just the average of those.
Honestly though, it's the best thing I can run on this PC right now. It's not close to anything else I've tried, and it did all of this on a 3060.
If you want to test yourself to verify my numbers message me and I will send you the repo link. It's a click and forget test so you don't even need to monitor.
r/LocalLLM • u/invursegg • 9h ago
r/LocalLLM • u/CriticismFast3038 • 1h ago
Has anyone tried it? AppWorld is a pretty weak benchmark in my opinion.
Seems to require the specific engine.
r/LocalLLM • u/liljom7 • 14h ago
Goal: get the best agentic-coding inference I can out of one RTX 3090 (power-limited to 250 W), judged on capability and time per task. In my normal workload one coding agent is running, sometimes three at once: two MiniMax Code agents plus a DeepSeek Harness (dsh) web session. The best setup I've found for this is HyperQwen, which runs a 4-bit AutoRound Qwen3.8-27B on vLLM with MTP speculative decoding. Next I tried published weight-space edits that claim to make the model think less.
TL;DR: the Concept edit is the winner. Compared with the plain 4-bit model, it uses 27% fewer reasoning tokens and 27% less wall time, and it scores higher on hard tasks (0.868 → 0.913, hardest task 0.750 → 0.983). Decode speed, MTP acceptance and KV cache are unchanged. I'm now serving it: Liljom/Qwen3.8-27B-Concept-W4A16-AutoRound-fast (Apache-2.0, drop-in for HyperQwen).
16 hard coding calls per model: 4 tasks × 4 seeds. H3b is a long graph-algorithm task that needs 12k–32k reasoning tokens, and it's the one that separates the models.
| Build | Reasoning tokens | Wall time | Hard score | H3b score | Verdict |
|---|---|---|---|---|---|
| Base (4-bit AutoRound) | 164,805 | 2,810 s | 0.868 | 0.750 | baseline |
| + Concept | 119,857 (−27%) | 2,040 s (−27%) | 0.913 | 0.983 | served |
| + Concept + AstraStop | 101,986 (−38%) | 1,738 s (−38%) | 0.903 | 0.861 | out: early stops on H3b |
| + Concept + Quickthinker5 | 103,322 (−37%) | 1,746 s (−38%) | 0.798 | 0.523 | out: H3b failures |
| + Quickthinker5 | 126,638 (−23%) | 2,280 s (−19%) | 0.794 | 0.500 | out: H3b failures, empty answers |
| + AstraStop | 127,776 (−22%) | 2,157 s (−23%) | 0.808 | 0.500 | out: regresses on its own |
Decode speed is the same on every build (68–72 tok/s), so fewer tokens means less time per task. Quickthinker and AstraStop cut more, but on the hardest task they sometimes stop thinking too early or return an empty answer. For an unattended coding agent, that's worse than spending more tokens.
| Meetric | Base | + Concept |
|---|---|---|
| Reasoning tokens, sum | 164,805 | 119,857 |
| Reasoning, median per call | 5,419 | 4,969 |
| Wall time, sum | 2,810 s | 2,040 s |
| Hard score (mean) | 0.868 | 0.913 |
| H3b score (mean of 4) | 0.750 | 0.983 |
| Calls hitting the 32k cap | 1 | 0 |
| Decode speed, median | 67.9 tok/s | 69.9 tok/s |
| MTP accept rate | 0.651 | 0.658 |
| KV cache | 225k tokens | 225k tokens |
down_proj matrices (layers 32–48). MTP acceptance and KV cache stay the same.1× RTX 3090 @ 250 W · HyperQwen (vLLM 0.33, MTP with 4 draft tokens) · base dbirks/Qwen3.8-27B-W4A16-AutoRound + syvai fast variant · reasoning effort medium, temp 1.0, top-k 20, top-p 0.95 · 4 concurrent requests · 32k reasoning cap.
The other builds are published for research: Concept+AstraStop · Concept+Quickthinker5 · Quickthinker5.
Credits to Qwen, dbirks, syvai, the HyperQwen authors, and the edit authors ConceptCountry, scar-ai and jarrodwatts. Happy to answer questions or run other edits through the same harness.
r/LocalLLM • u/urbananimall • 18h ago
Looking for recommendations on a local LLM setup with a budget of around $20,000 USD.
My primary use case is AI-assisted web development, particularly with Mobbin MCP.
Ideally, I’m looking for something powerful, relatively easy to set up and maintain, and capable of handling complex development workflows without sacrificing too much performance.
Would appreciate recommendations on hardware, models, and software setups from anyone running something similar.
r/LocalLLM • u/chyuhua • 3h ago
I've been working on JustFit as a weekend project to run Qwen3.8-27B (with vision and its full 256K context) on a 24 GB Apple Silicon MacBook. Just shipped 2.6.0 and wanted to share what changed, along with some interesting M4 vs. M5/M6 hardware numbers from adding draft-tree speculative decoding.
Install (Apple Silicon, macOS 27+, 24 GB+):
(Why pip if there's no Python at inference? pip is just a convenient ~29 MB delivery wrapper that drops the native binaries. Once justfit serve starts, nothing touches Python at all.)
2.3 worked once it was up, but starting it was painful: Python serving stack, a converted model pack, and four manual manifest URL / 64-char SHA-256 CLI flags just to fetch the precompiled runtime and ANE bundles.
In 2.6:
justfit serve sets up the runtime, downloads mlx-community/Qwen3.8-27B-4bit + Inco AI's DFlash2 draft straight from Hugging Face (no custom converted pack), and auto-fetches your chip's precompiled ANE vision programs from a pinned, hash-checked index (macOS 27.2.1).return_token_ids (vLLM/SGLang-style token-in/token-out), crash-request quarantine (400 request_quarantined), and a self-applied sandbox on the engine and native workers (no outbound network).With a single draft chain, one wrong guess kills every token after it. Over 1,277 cycles on my M6, 22% of cycles accepted zero draft tokens, often when the draft was split ~50/50 between two good continuations.
Switching to a small tree verified in one pass gave a clean split across Apple Silicon generations. Measured single-lane latency of verifying 16 rows vs. 8 rows:
1.04x (+4%)1.12x (+12%)1.79x (+79%, 89 ms -> 159 ms)Streaming ~15 GB of 4-bit weights is memory-bandwidth bound on M5 Pro and M6, so doubling verify rows from 8 to 16 is almost free. M4 Pro hits GPU compute headroom first (an early adaptive mode letting M4 pick 16-row trees was actually 8.7% slower than a chain). So 2.6 calibrates per chip: M5 Pro and M6 dynamically choose chain / 8-row / 16-row every cycle by expected tokens per ms, while M4 Pro runs a fixed 8-row tree.
Keeping greedy outputs identical to single-chain decoding was harder than the tree mask itself, because GPU float addition isn't associative and Qwen3.8 has 48 Gated DeltaNet recurrent layers. Borrowing a temporary scratch copy of only the current KV page per root-to-leaf path, masking sibling tiles as exact zeros, and replaying GDN recurrent state along each ancestor path kept outputs word-for-word identical across all 27 test cases we ran (including 111K contexts and multi-turn reuse).
1. Controlled tree off vs. on (same 2.6 engine, 7-prompt greedy suite):
| Mac / setting | Chain (tok/s) | Tree (tok/s) | Speedup | Identical outputs |
|---|---|---|---|---|
| M6, 1 lane | 35.6 | 40.9 | +14.7% | 7 / 7 |
| M6, 2 lanes | 47.2 | 52.2 | +10.6% | 6 / 6 |
| M5 Pro, 1 lane | 57.3 | 61.8 | +7.9% | 7 / 7 |
| M4 Pro, 1 lane (8-row tree, 3 runs) | 49.5 | 50.9-51.3 | +2.6% to +3.5% | 7 / 7 |
For sampled chat (T=1.0, Qwen3.8's recommended thinking temp): +8.5% on M6 and +2.5% on M5 Pro (verified against target distribution across 44 per-position tests, min p = 0.08; disabled on M4 Pro since sampled trees need 16 rows).
2. NVIDIA SPEED-Bench (greedy, 512 output tokens, from the 2.6.0 release notes):
docs/RESULTS.md.Memory footprint: Unchanged static memory plan. Our dual-request 320K capacity test (two 128K+32K requests simultaneously) peaked at the exact same footprint as without trees (21,634 MiB on M6, 21,563 MiB on M5 Pro).
Built on Inco AI's Splash engine and DFlash2 draft model. Draft-tree ideas informed by DDTree (arXiv:2604.12989), CaDDTree (arXiv:2606.01813), and TreeWY (arXiv:2608.20961).
Happy to answer any questions about the Metal kernels, GDN tree verification, or squeezing 256K into 24 GB on macOS!
r/LocalLLM • u/Stock_Test2326 • 9h ago
I was using Ollama, but after seeing some community review I moved to LMstudio (now called bionic) but when I go to hugging face most models point to Ollama first, not that importing is forbidden in Bionic. but just curious what are the apps that are widely used by community (and models too)
r/LocalLLM • u/No_Price_3299 • 15h ago
I've been running my harnesses through Bubblewrap with limited permission and while it's been working I'm still worried about things like prompt injection or the LLM downloading npm malware.
Of course there's other ways for sandboxing I've been looking into; Docker, VMs, Podman...
With the speeds I'm seeing I'd like to be able to run my LLM for 8-10hrs unattended on "Accept All", but with the peace of mind that anything it does in there won't affect the overall system.
Would appreciate thoughts on this.
r/LocalLLM • u/Ok-Fox3479 • 15h ago
I found this quant of inkling 2 mini:
IsValorum/Inkling-2-Mini-Abliterated-APEX-I-MiniPlus-V2.1-GGUF
Its 12.0.9 GiB in total. other similar quants this user made claim that they have Q5_K_L tier quality (IsValorum/Cyber-Tiel-Coder-35B-A3B-APEX-I-MiniPlus-V2.1-GGUF)
r/LocalLLM • u/Few_Interest4474 • 19h ago
Do you think Qwen 3.8 Flash Next Basalt NVFP4 dq6 is smarter than Gemini 3.8 Flash or Opus 4.6? I'm not looking for a textbook answer.
I just want your honest opinion.
r/LocalLLM • u/jphccfc • 21h ago
I've been experimenting with the new AMD NPU support in Unsloth on my ASUS ROG Zephyrus G14 and I'm interested in how others are getting the most out of it.
My hardware:
Models I've downloaded/tested:
I've been getting around 7.9 tokens/sec with Qwen3.5 9B. It's usable, but noticeably slower than cloud models, and I'm still figuring out how to get genuine day-to-day value from the NPU.
I'm particularly interested in using local AI as a personal assistant for research, productivity, document analysis, Obsidian, automation and lightweight agentic workflows.
I already use tools such as Hermes Agent, Ollama, Open WebUI and n8n, so I'm happy to experiment with integrations.
A few questions for the community:
I'm less interested in simply running the biggest possible model and more interested in building something that's genuinely useful, responsive and efficient for everyday use.
Would love to hear what people have tried, what's worked, what hasn't, and any clever tweaks or setups worth experimenting with.