Minor update to this escapade. I updated Strata a few days ago and at some point between OP and now, the token speed basically doubled. I'm averaging about high 70s low 80s tok/s. In busts, it can hit in the 100-120 tokens per second range.
AMD cards may not be perfect, but it seems there's more to squeeze out of them!
XFEDI
ActivityPubRemote profile
vandorb12
No longer teaching, but still studying cybersecurity. Stuck behind the helpdesk ticketing system.
I sometimes help out with WRCCDC. Likely needs a nap and favorite snacks.
West coast thoughts and tangents abound.
Certified failed gifted child.
Posts
Getting a local Chinese model to do upcoming election research and summarizing seems... cathartic.
I got my stances on things... It's the people that mostly fly under the radar like judges and schoolboards that I have a hard time with.
Dig up the dirt. Tell the LLM my issue topics, Summarize and make a pretty markdown report.