I'm using local models. Why pay somebody else or hand them my data?
Sometimes you need to search for something and it's impossible because of SEO, however you word it. A LLM won't necessarily give you a useful answer, but it'll at least take your query at face value, and usually tell you some context around your question that'll make web search easier, should you decide to look further.
Sometimes you need to troubleshoot something unobvious, and using a local LLM is the most straightforward option.
Using a LLM in scripts adds a semantic layer to whatever you're trying to automate: you can process a large number of small files in a way that's hard to script, as it depends on what's inside.
Some put together a LLM, a speech-to-text model, a text-to-speech model and function calling to make an assistant that can do something you tell it without touching your computer. Sounds like plenty of work to make it work together, but I may try that later.
Some use RAG to query large amounts of information. I think it's a hopeless struggle, and the real solution is an architecture other than a variation of Transformer/SSM: it should address real-time learning, long-term memory and agency properly.
Some use LLMs as editor-integrated coding assistants. Never tried anything like that yet (I do ask coding questions sometimes though), but I'm going to at some point. The 8B version of LLaMA 3 should be good and quick enough.
I thought MoEs had to be loaded entirely in the (V)RAM and the inference speedup was because you only need to use a fraction of layers to compute the next token (but the choice of layers can be different for each token, so you need them all ready; or keep moving data between the disk <-> RAM <-> VRAM and get reduced performance).
Otherwise we have no idea. We have some models of physics, none perfectly describing our universe. We don't know the structure of space, or the structure of time.
Even if we did: what would it mean for a line or a plane to exist? There could be equivalent descriptions of our universe, some including those as objects and some only as emergent properties.
I don't usually edit PDFs on my phone. On the PC, I use pdftk+qpdf+img2pdf+ocrmypdf (all command-line apps). Some of those can be found in the default Termux repos once you install the terminal emulator; some, perhaps, could be compiled and used as well.
Some updates might be restricted to certain architectures, Android versions &c. Some could be beta versions. Or your repositories simply need to be synchronised.
If it isn't the latter, check the following settings: "Include incompatible versions", "Include anti-feature apps" and "Unstable updates".
Sure, they're just typically from the same industry with similar perspectives, similar blind spots and similar affinity for rants on topics X, Y and Z.
Some get annoyed by this after a while so ignoring comments is a valid choice if you feel like that.
Downloading there is straightforward: look at network requests, redownload svg's of individual pages with wget and reassemble those into a pdf. I did that today and the resulting quality wasn't exactly low - though I didn't examine it too closely. Readability was perfect.
Probably could be automated, but I'm not bothered enough to do so yet.
Wizard-Vicuna-30B-Uncensored works pretty well for most purposes. It feels the smartest of all I've tried. Even when it hallucinates, it gives enough to refine the google query on some obscure topic. As usual, hallucinations are also easily counteracted by light non-argumentative gaslighting.
It isn't very new though. What's the current SOTA for universal models of similar size? (both foundation and chat-tuned)
I'm using local models. Why pay somebody else or hand them my data?