Space cadet was my jam!
Wolfam alpha actually has an LLM api so your local models can call its factual database for information when doing calculations through tool calling. I thought you might find that cool. Its a shame there is no open alternative to WA they know their dataset is one of a kind and worth its weight in gold. Maybe ond day a hero will leak it 🤪
Models running on gguf should all work with your gpu assuming its set up correctly and properly loaded into the vram. It shouldnt matter if its qwen or mistral or gemma or llama or llava or stable diffusion. Maybe the engine you are using isnt properly configured to use your arc card so its all just running on your regular ram which limits things? Idk.
Intel arc gpu might work with kobold and vulcan without any extra technical setup. Its not as deep in the rabbit hole as you may think, a lot of work was put in to making one click executables with nice guis that the average person can work with..
Models
Find a bartowlski made quantized gguf of the model you want to use. Q4_km is recommended average quant to try first. Try to make sure it all can fit within your card size wise for speed. Shouldnt be a big problem for you with 20gb vram to play with. Hugging face gives the size in gb next to each quant.
Start small with like high quant of qwen 3 8b. Then a gemma 12b, then work your way up to a medium quant of deephermes 24b.
Thinking models are better at math and logical problem solving. But you need to know how to communicate and work with llms to get good results no matter what. Ask it to break down a problem you already solved and test it for comprehension.
kobold engine
Download kobold.cpp, execute it like a regular program and adjust settings in graphical interface that pops up. Or make a startup script with flags.
For input processing library, see if Vulcan processing works with Intel arc. Make sure flash attention is enabled too. Offload all layers of the model I make note of exactly how many layers each model has during startup and specify it but it should figure it out smartly even if not.
You can use discord in your web browser with some privacy addons like fingerprint and user gent spoofing to help restrict how much gets leaked to discord. If you install it as an app that runs in background you better believe they're collecting more data and metrics.
Any device someone ask my help with figuring out. Its rarely the appliance that pisses me off and more the blatant learned helplessness and fundimental inability for fellow adults to rub two braincells together on figuring out a new thing or to troubleshoot a simple problem. A lifetime of being the techie fixer bitch slave constantly delegated the responsibility of figuring out everyones crap for them has left me jaded to the average persons mental capacity and basic logical application abilities.
I would receommend you read over the work of the person who finetuned a mistral model on many us army field guides to understand what fine tuning on a lot of books to bake in knowledge looks like.
If you are a newbie just learning how this technology works I would suggest trying to get RAG working with a small model and one or two books converted to a big text file just to see how it works. Because its cheap/free t9 just do some tool calling and fill up a models context.
Once you have a little more experience and if you are financially well off to the point 1-2 thousand dollars to train a model is who-cares whatever play money to you then go for finetuning.
It is indeed possible! The nerd speak for what you want to do is 'finetune training with a dataset' the dataset being your books. Its a non-trivial task that takes setup and money to pay a training provider to use their compute. There are no gaurentees it will come out the way you want on first bake either.
A soft version of this thats the big talk right now is RAG which is essentially a way for your llm to call and reference an external dataset to recall information into its active context. Its a useful tool worth looking into much easier and cheaper than model training but while your model can recall information with RAG it won't really be able to build an internal understanding of that information within its abstraction space. Like being able to recall a piece of information vs internally understanding the concepts its trying to convey. RAG is for wrote memorization, training is for deeper abstraction space mapping
If you were running amd GPU theres some versions of llama.cpp engine you can compile with rocm compat. If your ever tempted to run a huge model with partial offloaded CPU/ram inferencing you can set the program to run with highest program niceness priority which believe it or not pushes up the token speed slightly
You're welcome. Also, whats your gpu and are you using cublas (nvidia) or vulcan(universal amd+nvidia) or something else for gpu postprocessing?
Enable flash attention if you havent already
True! Most browsers don't have native gemini protocol support. However a web proxy like the ones I shared allow you to get gemini support no matter the web browser. Gemtext is a simplified version of markdown which means its not too hard to convert from gemtext to html/webpage. So, by scraping information from bloated websites, formatting it into the simple gemtext format markdown, then mirroring it back as a simple web/html page, it works together nicely to re-render bloated sites on simple devices using gemini as a formatting medium technology. You don't really need to understand gemini protocol to use newswaffle + portal.mozz.us proxy in your regular web browser
Its called that because you have a new appreciation for your life after coming back from that alive... "Thank God I didn't drop to my death!"
"Hey, how's it going"
stares blankly at you like a deer looking at headlights for 15.50 seconds with an uncomfortable silence.
"Good."
walks away
Ill have you know I run a dual side business of selling ball-bouncing-in-polygon software as NFTs as well as counting the r-'s in various spellings of strawberry for the private defense sector...
during the time I was born TVs were small square boxes powered by glass tubes and turny knobs. I want to say 480p but tbh if you were using a junky 10 inch display at the turn of the century on satallite it was closer to like 240p. The jump from square 480p to widescreen 720/1080 was an actual graphical revolution for most people in a very big way, especially for watching movies that were shot in wide. In terms of games 1080p is both where 16:9 took off and the point where realistic looking graphics meet acceptable resolution for like skin pours and godrays shit like that. GTA5, TLOU and RDR are the examples that come to mind from the AAA 1080p era and their original states still probably hold up today.
When the 4k stuff finally came around and it was advertised as the next revolution I was excited man. However compared to going from 480 to 1080 it wasn't a huge change tbh. It seems once you're already rendering skin detail and individual blades of grass, or simulating atmospheric condition godrays, there isn't much more that can be drastically improved just by throwing a billion more polygons at a mesh and upscaling textures. The compute power and storage space required to get these minimal detail gains also starts escalating hard. Its such bullshit that modern AAA games are like 80gb minimum with half of that probably being 4k textures.
I will say that im like the opposite of a graphics snob and slightly proud of it so my opinions on 4k and stuff are biased. Im happy with 1080p as a compromise between graphical quality and compute/disk space required. Ive never played a 1080p at maximum graphics and wanted for more. Im not a competitive esports player, im not a rich tech bro who can but the newest upgraded gpu and 500tb of storage. I don't need my games to look hyperrealistic. I play games for the fun gameplay and the novel experiences they provide. Some of the best games I've ever played look like shit and can be played on a potato. Most of the games I found boring were AAA beautiful open worlds that were as wide and pretty as an ocean but gameplay wise it was as deep as a dried up puddle. I hopped off the graphics train a very long time ago, so take my cloud yelling with a grain of salt.
"I use Arch bt-"
"ITS SHiTE!"
"...excuse me?"
" YOUR BLOODY ROLLING RELEASE DISTRO IS FUCKING RAW. HOW MANY TIMES HAVE YOU RECOOKED IT AFTER A DEPENDENCY PACKAGE BROKE?"
"B-bhut chef... Its a rolling release bleeding distro that expects users to compile with the help of a wik-"
"I ASKED HOW MANY TIMES YOU HAD TO RECOMPILE IT THIS YEAR YOU FUCKING DONKEY"
"5 times sir."
"FIVE FUCKING TIMES??? JESUS CHRIST DID I ASK FOR CONSTANT MAINTENANCE WITH A SIDE OF COMPUTER PROGRAMS IN BETWEEN? IF I WANTED A RAW OPERATING SYSTEM I WOULD HAVE BECOME A FLAGSMAN INSTEAD OF A CHEF AND ASKED FOR A DISH OF "GENTOO". COOK ME A REAL OPERATING SYSTEM."
Ken Cheng is a great satirist and probably knows thats not how it works anymore. Most model makers stopped feeding random internet user garbage into training data years ago and instead started using collections of synthetic training data + hiring freelance 'trainers' for training data and RLHF.
Oh dont worry your comments are still getting scraped by the usual data collection groups for the usual ad selling and big brother bs. But these shitty AI poisoning ideas I see floating around on lemmy practically achieve little more than feel good circle jerking by people who dont really understand the science of machine learning models or the realities of their training data/usage in 2025. The only thing these poor people are poisoning is their own neural networks from hyper focusing defiance and rage on a new technology they can't stop or change in any meaningful way. Not that I blame them really tech bros and business runners are insufferable greedy pricks who have no respect for the humanities who think a computer generating an image is the same as human made art. Also its bs that big companies like meta/openAI got away with violating copyright protections to train their models without even a slap on the wrist. Thank goodness theres now global competition and models made from completely public domain data.
Some games just aren't meant for you and thats okay. For example I spent a few hours playing civ enough to understand the experience it offers. I did not enjoy a single moment of its gameplay or strategy layers at any point. Apparently its a good enough game for many people to put hundreds/thousands of hours into and buy again every few years+dlc. I just didn't pick up what it was putting down.
I have no issue with remakes themselves. Games are a kind of art, and good art should be kept alive for the next generations to enjoy. The problem to me is:
- the only thing big studios now want to put out remakes/remasters of the backlog they already made because its a safe and easy cash grab. One of the top comments about there being 7 skyrims and 2 oblivions before ES6 is soo real man. Its like all the people who founded the companies who were responsible for creative novel design/story that gave big titles their soul in the 2000s no longer exist in the industry except a few indie devs. Now all big game companies are just run by business associates without a shred of humanity outsourcing everything for a quick buck.
- Graphics have plateud from late 2010s and onward. Remastered and remaked stuff made a lot more since for the ps2/xbox and backwards, with the ps3/x360 1080p resolution it made a little less sense but I could still understand them porting like TLOU to ps4 at 4k or whatever. But now were remastering games that came out 5 years ago at 4k and trying to sell it as some huge graphical overhaul worth the asking price. Maybe im insane or old but my eyes can barely tell the difference between 1080p and 4k, going from 4k to 8k is like the same picture with slightly different shaders.
How do you know?!?! This is one of the most laser precise call out to my childhood ive ever seen in an internet comment.