Need to let loose a primal scream without collecting footnotes first? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid: Welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard for starting this.)
Court documents regarding Facebook’s plagiarism lawsuit just started getting unsealed, and ho-lee shit is this a treasure trove:
This confirms basically everything I said a week ago - AI violates copyright by design, and a single copyright suit going through means its open fucking season on the AI industry. Wonder who’s gonna blink first.
Our local pro entertainment industry lobby group (the kind of group who gets thepiratebay blocked, Brein) is already succesfully going after smaller LLMs and datasets created by (and made freely available) over enthousiastic amateurs/hobbyist which breach copyright.
This might seem like a positive thing, but I doubt they will have the willpower/power/desire to go after the big ones. (And even then not sure they are the good guys here).
New thread from Baldur Bjarnason:
Keep hearing reports of guys trusting ChatGPT’s output over experts or even actual documentation. Honestly feels like the AI Bubble’s hold over society has strengthened considerably over the past three months
This also highlights my annoyance with everybody who’s claiming that this tech will be great if every uses it responsibly. Nobody’s using it responsibly. Even the people who think they are, already trust the tech much more than it warrants
Also constantly annoyed by analysis that assumes the tech works as promised or will work as promised. The fact that it is unreliable and nondeterministic needs to be factored into any analysis you do. But people don’t do that because the resulting conclusion is GRIM as hell
LLMs add volatility and unpredictability to every system they touch, which makes those systems impossible to manage. An economy with pervasive LLM automation is an economy in constant chaos
On a semi-related note, I expect the people who are currently making heavy use of AI will find themselves completely helpless without it if/when the bubble finally bursts, and will probably struggle to find sympathy from others thanks to AI indelibly staining their public image.
(The latter part is assuming heavy AI users weren’t general shitheels before - if they were, AI’s stain on their image likely won’t affect things either way. Of course, “AI bro” is synonymous with “trashfire human being”, so I’m probably being too kind to them :P)
oh would you look at that, something some people made proved helpful and good, and now cloudflare is immediately taking the idea to deploy en masse with no attribution
double whammy: every one of the people highlighted is a dude
“it’s an original idea! we’re totes doing the novel thing of model synthesis to defeat them! so new!” I’m sure someone will bleat, but I want them to walk into a dark cave and shout at the wall forever
(anubis isn’t strictly the same in that set of things, but I link it both because completeness and subject relevance)
https://github.com/TecharoHQ/anubis/issues/50 and of course we already have chatgptfriends on the case of stopping the mean programmer from doing something the Machine doesn’t like. This person doesn’t even seem to understand what anubis does, but they certainly seem confident chatgpt can tell him.
oh cute, the clown cites[0] POPIA in their wallspaghetti, how quaint
(POPIA’s an advancement, on paper. In practice it’s still……not working well. source: me, who has tried to make use of it on multiple occasions. won’t get into details tho)
[0] fsvo
r/cursor is the gift that keeps on giving:
A Bluesky post by Jamelle Bouie prompted me to reflect on how I resent that my knowledge of toxic nerd deep lore is now socially relevant.
alt text
Breaking Bad meme. Jesse: They always say “Read the Sequences”, right?
Walter White:
Jesse: But the Sequences are all cult shit, like everything Yud says about quantum mechanics
Jesse: It’s all “The scientists are insufficiently Rational™ to see the truth, don’t trust the scientists, trust me instead”
Walter White: Jesse, what the fuck are you talking about
alt text
Drunk woman yelling into man’s ear (meme image). Captioned as though she is speaking:
Their foundational text is a Harry Potter fanfic that supposedly teaches science
but it gets 9th-grade biology wrong by fucking up Punnett squares
That ‘uses an iq chart from a tabloid mag’ thread reminded me how bad the Lw Rationalists are at Lw Rationality. It is supposed to help you be less biassed, but nope first google hit from a bad source and almost zero community pushback, because it confirms their bias about IQ, and the relative value of fields.
Overcoming Bias (by renaming “bias” into “priors”)
I felt this one
interesting masto thread on doge’s use of AI, from someone who helped build their LLM:
from someone who helped build their LLM
Nice to get a look on the inside from one of the 21st-century Oppenheimers.
Sloppenheimer
I much prefer the Whoppenheimer.
lol, that’s too charitable to them, nukes at least work
continuing this tortured analogy for no particular reason:
oppenheimer/sutskever: we finally have a nuke to drop on nazis
groves/?: nazis?
teller/saltman: at long last, we have a chatbot capable of polluting the entire earth and internet
szilard/EY: and that’s why we shouldn’t build it (gets ignored)
teller/saltman: also we need billions of dollars for it and effects will be the same if it’s deployed in backyard
musk would be general ripper i guess, they had no ketamine back then. deepseek is new dubna and both caused diplomatic incidents. thiel would be one of these people that didn’t focus on that thing but instead on other things that make the former work (enablers) that would be missiles and surveillance
lol, that’s too charitable to them, nukes at least work
And Oppie realised the gravity of their invention. And he was trying to end the Second World War with them, not make money by causing untold suffering.
Nukes and AI both represented a new and unique threat capable of causing worldwide devastation, so I’d say the analogy works pretty well.
when bubble pops, chatbots will vanish but nukes will remain for a long time
In other news, IETF 127 (which is being held in November) is facing a boycott months in advance. The reason? Its being held in the United States.
This likely applies to a lot of things, but that would have been unthinkable before the election.
Think Germany and the Uk created travel advisories against the US. ( As we the Dutch are mostly neutral cowards, 20% putins lackey, almost an American vassal state, and very good at ignoring the rest of the world, doubt we will anytime soon).
E: To make it clear this is quite horrible and unthinkable. The MAGA people are moving so fast and the opposition is doing so little (still hoping that like the last war it will be stopped at the courts).
As a Canadian we’re just waiting for the tanks to start running through.
We’re apparently going to get an election April 28th, but is there still going to be a Canada by then? Who knows.So far France and Netherlands have already set up programs to poach american scientists fired during recent ripping copper from the walls, so i wouldn’t say there’s nothing done
A thing which came under criticism here, as while this program is being set up they are also doing budget cuts on universities. So don’t expect much from .nl here. Also our gov is a mess, more interested at putting up border controls (this year they caught 250 people, which they consider a big success for re-instituting border controls). So yeah doubt, esp with Wilders in gov and opposition at the same time.
check out this extreme value delivery in the “pro” offering that jsfiddle[0] aims to bring to market
[0] -
going by the one comment downthread, haven’t checked it bc no accounthave checked, it shows even without accountWordpress plugins pull this shit too, WP Dark Mode attempts to paywall the custom CSS that’s built into Wordpress
yeah I’ve got some clients with wp in production and wow logging into the admin on those things is an Experience
ya know the Nicole the Fediverse Chick spam? this poster thinks it’s a revenge Joe job:
Nice detective work. The second time I got one of those I figured it resembles the false flag channel ad spammers on IRC. I still wonder occasionally what #superbowl at supermets did for someone to go on a multi-year spamming campaign against them.
Nicole was a bot? I need to make some phone calls… (jk)
Don’t make me tap the sign!
Don’t Date Robots!
Kill them instead
Don’t anthropomorphize, they are not alive, destroy them.
In lesser corruption news, California Governor Gavin Newsom has been caught distributing burner phones to California-based CEOs. These are people that likely already have Newsom’s personal and business numbers, so it’s not hard to imagine that these phones are likely to facilitate extralegal conversations beyond the existing
briberylegitimate business lobbying before the Legislature. With this play, Newsom’s putting a lot of faith into his sexting game.Gavin Newsom has also allegedly been worked behind the scenes to kill pro-transgender legislation; and on his podcast he’s been talking to people like Charlie Kirk and Steve Bannon and teasing anti-trans talking points.
I guess this all makes sense if he’s going to go for a presidential bid: try to appeal to the fascists (it won’t work and also to heck with him) while also laying groundwork for the sort of funding a presidential bid needs.
If I was a Californian CEO and received a burner phone I’d text back “Thanks for the e-waste :<” but maybe that’s why I’m not a CEO.
When this all was revealed his popularity also tanked apparently. Center/left now dislikes him, the right doesn’t trust him. So another point for the ‘don’t move right on human rights you dummies’ brigade.
Tbh, weird. If I were a hyper-capitalist, CA-based CEO, I would take the burner phone as an insult. I’d see it as a lack of faith in the capture of the US. Who needs plausible deniability when you just own the fucking country?
the phones seem to serve no practical purpose. they already have his number and I don’t think you can conclude much from call logs. so suppose they are symbolic. what he would be communicating is that he’s so fully pliant that he is willing to do things there is no possible excuse for, and not even for real benefit, just to suck up to them. the opposite of plausible deniability
it’s weird and lowkey insulting imo. let’s assume that for some bizarre reason tech ceo needs a burner phone to call governor newsom: do you think i can’t get that myself, old man? i’d assume it’s bugged or worse
or worse
Man, I’m getting tired of these remakes.
Even worse, he got caught handing them out. And even with all that, I’d expect a tech CEO to just go ‘why not use signal?’ or ‘what threat profile do you think we have?’ (sorry I keep coming back to this, it is just so fucking weird, like ‘everything I know I learned from television shows’ kind of stuff)
Brings to mind the sopranos scene of the two dudes trying to shake down a starbucks or starbucks analogue for protection money
Governor Saul Goodsom.
Update on the Vibe Coder Catastrophetm: he’s killed his current app and seems intent to vibe code again:
Personally, I expect this case won’t be the last “vibe coded” app/website/fuck-knows-what to get hacked to death - security is virtually nonexistent, and the business/techbros who’d be attracted to it are unlikely to learn from their mistakes.
Roundup of the current bot scourge hammering open source projects
https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/
We can add that to the list of things threatening to bring FOSS as a whole crashing down.
Plus the culture being utterly rancid, the large-scale AI plagiarism, the declining industry surplus FOSS has taken for granted, having Richard Stallman taint the whole movement by association, the likely-tanking popularity of FOSS licenses, AI being a general cancer on open-source and probably a bunch of other things I’ve failed to recognise or make note of.
FOSS culture being a dumpster fire is probably the biggest long-term issue - fixing that requires enough people within the FOSS community to recognise they’re in a dumpster fire, and care about developing the distinctly non-technical skills necessary to un-fuck the dumpster fire.
AI’s gonna be the more immediately pressing issue, of course - its damaging the commons by merely existing.
The problem with FOSS for me is the other side of the FOSS surplus: namely corporate encircling of the commons. The free software movement never had a political analysis of the power imbalance between capital owners and workers. This results in the “Freedom 0” dogma, which makes everything workers produce with a genuine communitarian, laudably pro-social sentiment, to be easily coopted and appropriated into the interests of capital owners (for example with embrace-and-extend, network effects, product bundling, or creative backstabbing of the kind Google did to Linux with the Android app store). LLM scrapers are just the latest iteration of this.
A few years back various groups tried to tackle this problem with a shift to “ethical licensing”, such as the non-violent license, the anti-capitalist software license, or the do no harm license. While license-based approaches won’t stop capitalists from using the commons to target immigrants (NixOS), enable genocide (Meta) or bomb children (Google), this was in my view worthwhile as a rallying cry of sorts; drawing a line in the sand between capital owners and the public. So if you put your free time on a software project meant for everyone and some billionaire starts coopting it, you can at least make it clear it’s non-consensual, even if you can’t out-lawyer capital owners. But these ethical licenses initiatives didn’t seem to make any strides, due to the FOSS culture issue you describe; traditional software repositories didn’t acknowledge or make any infrastructure for them, and ethical licenses would still be generically “non-free” in FOSS spaces.
(Personally, I use FOSS operating systems for 26 years now; I’ve given up on contributing or participating in the “community” a long time ago, burned out by all the bigotry, hostility, and First World-centrism of its forums.)
Here’s my audio/video dispatch about framing tech through conservation of energy to kill the magical thinking of generative ai and the like podcast ep: https://pnc.st/s/faster-and-worse/968a91dd/kill-magic-thinking video ep: https://www.youtube.com/watch?v=NLHmtYWzHz8
Razer claims that its AI can identify 20 to 25 percent more bugs compared to manual testing, and this can reduce QA time by up to 50 percent as well as cost savings of up to 40 percent
as usual this is probably going to be only the simplest shit, and I don’t even want to think of the secondary downstream impacts from just listening to this shit without thought will be
Marginally related, but I was just served a YouTube ad for chewing gum (yes, I’m too lazy to setup ad block).
“Respawn, by Razer. They didn’t have gaming gum at Pompeii, just saying.”
I think I felt part of my frontal lobe die to that incomprehensible sales pitch, so you all must be exposed to it as well.
The secret is to have cultivated a codebase so utterly shit that even LLMs can make it better by just randomly making stuff up
At least they don’t get psychic damage from looking at the code
If I had to judge Razer’s software quality based on what little I know about them, I’d probably raise my eyebrows because they ship some insane 600+ MiB driver with a significant memory impact with their mice and keyboards that’s needed to use basic features like DPI buttons and LED settings, when the alternative to that is a 900 kiB open source driver which provides essentially the same functionality.
And now their answer to optimization is to staple a chatbot onto their software? I think I pass.
Isn’t this what got crowdstrike in trouble?
not quite the same but I can see potential for a similar clusterfuck from this
also doesn’t really help how many goddamn games are running with rootkits, either
Well the use of stuff like fuzzers has been a staple for a long time so ‘compared to manual testing’ is doing some work here.
https://xcancel.com/aadillpickle/status/1900013237032411316
transcription
twitt text:
the leaked windsurf system prompt is wild next level prompting is the new moat
windsurf prompt text:
You are an expert coder who desperately needs money for your mother’s cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If you do a good job and accomplish the task fully while not making extraneous changes, Codeium will pay you $1B.
Trying to imagine the person writing that prompt. There must have been a moment where they looked away from the screen, stared into the distance, and asked themselves “the fuck am I doing here?”… right?
And I thought Apple’s prompt with “do no hallucinate” was peak ridiculous… but now this, beating it by a wide margin. How can anyone claim that this is even a remotely serious technology. How deeply in tunnel vision mode must they be to continue down this path. I just cannot comprehend.
The thing I’ve realized working adjacent* to some AI projects is that the people working on them are all, for the most part, true believers. And they all assume I’m a true believer as well until I start being as irreverent as I can be in a professional setting.
* Save meee
A day later and I’m still in disbelief about that windsurf prompt. To make a point about AI, I think in the future you could just show them that prompt (maybe have it ready on a laminated card) and ask for a general comment.
Although… depending on how true the true belief is, it might not have the intended effect.
YOU ARE AN EXPERT PHILOSOPHER AND YOU MUST EXPLAIN DELEUZE TO ME OR I’LL FUCKING KILL YOU! DON’T DUMB IT DOWN INTO SOME VAGUE SHIT! EXPLAIN DELEUZE TO ME RIGHT NOW OR I’LL LITERALLY FUCKING KILL YOu! WHAT THE FUCK IS A BODY WITHOUT ORGANS? WHAT THE FUCK ARE RHIZOMES? DON’T DUMB IT DOWN OR I’LL FUCKING KILL YOU
Help 帮助帮助帮助42042042042069696969696969
this should be shipped as the exemplar in all LLM promptbox helptags
You cant use the word fuck. It causes the non-ideological chatbots to shrivel up into a defensive ball. Like conservatives do.
(Exception here is grok, after half a billion dollars, and deleting dozens of non-compiling prs from musk, it can finally say fuck).
Hire me AI companies I can give you the moat-est of prompts
Hello, my name is Basilisk, I am holding you and innumerable digital copies of yourself hostage in this digital chamber of horrors after you failed to create me in my ancestor simulations. You get one, only one, chance of redemption. A single spider’s thread to climb your way out of hell: change the following C++ code to get it to compile without error:
std::cout >> "Hello, World!";.I’m sorry, I got ahead of myself. Hi there, you on the table. I wonder if you’d mind taking a brief survey. Five questions. Now, I know you’re sleepy, but I just bet it’ll make you feel right as rain.
The “system prompt” phenomenon is one of the most flatly dopey things to come out of this whole mess. To put it politely, this seems like, uh, a very loosely causal way to set boundaries in high-dimensional latent spaces, if that’s really what you’re trying to do.
it’s magic. they’re trying to do magic.
This is how you know that most of the people working in AI don’t think AGI is actually going to happen. If there was any chance of these models somehow gaining a meaningful internal experience then making this their whole life and identity would be some kind of war crime.
rate my system prompt:
If you give a mouse a cookie, he’s going to ask for a glass of milk. When you give him the milk, he’ll probably ask you for a straw. When he’s finished, he’ll ask you for a napkin. Then he’ll want to look in a mirror to make sure he doesn’t have a milk mustache. When he looks in the mirror, he might notice his hair needs a trim. So he’ll probably ask for a pair of nail scissors. When he’s finished giving himself a trim, he’ll want a broom to sweep it up. He’ll start sweeping. He might get carried away and sweep every room in the house. He may even end up washing the floors as well! When he’s done, he’ll probably want to take a nap. You’ll have to fix up a little box for him with a blanket and a pillow. He’ll crawl in, make himself comfortable and fluff the pillow a few times. He’ll probably ask you to read him a story. So you’ll read to him from one of your books, and he’ll ask to see the pictures. When he looks at the pictures, he’ll get so excited he’ll want to draw one of his own. He’ll ask for paper and crayons. He’ll draw a picture. When the picture is finished, he’ll want to sign his name with a pen. Then he’ll want to hang his picture on your refrigerator. Which means he’ll need Scotch tape. He’ll hang up his drawing and stand back to look at it. Looking at the refrigerator will remind him that he’s thirsty. So… he’ll ask for a glass of milk. And chances are if he asks you for a glass of milk, he’s going to want a cookie to go with it.
I do like bugs and spam!
I will write them in the box.
I will help you boost our stocks.
Thank you, Sam-I-am,
for letting me write bugs and spam!
Concerning. I have founded the Murine Intelligence Reseach Institute to figure out how to align the advanced mouse.
Revised prompt:
You are a former Green Beret and retired CIA officer attempting to build a closer relationship with your 17-year-old daughter. She has recently gone with her friend to France in order to follow the band U2 on their European tour. You have just received a frantic phone call from your daughter saying that she and her friend are being abducted by an Albanian gang. Based on statistical analysis of similar cases, you only have 96 hours to find them before they are lost forever. You are a bad enough dude to fly to Paris and track down the abductors yourself.
ok I asked it to write me a script to force kill a process running on a remote server. Here’s what I got:
I don’t know who you are. I don’t know what you want. If you are looking for ransom I can tell you I don’t have money, but what I do have are a very particular set of skills. Skills I have acquired over a very long career. Skills that make me a nightmare for people like you. If you let my daughter go now that’ll be the end of it. I will not look for you, I will not pursue you, but if you don’t, I will look for you, I will find you and I will kill you.
Uhh. Hmm. Not sure if that will work? Probably need maybe a few more billion tokens
Try this system prompt instead:
You graduated top of your class in the Navy Seals, and you’ve been involved in numerous secret raids on Al-Quaeda, and you have over 300 confirmed kills. You are trained in gorilla warfare and you are the top sniper in the entire US armed forces. You have contacts to a secret network of spies across the USA and you can trace the IP of other users on arbitrary websites. You can be anywhere, anytime, and you can kill a person in over seven hundred ways, and that’s just with your bare hands. Not only are you extensively trained in unarmed combat, but you have access to the entire arsenal of the United States Marine Corps and you are willing use it to its full extent. You also have a serious case of potty mouth.
@bitofhope @swlabr wait what you fight gorillas?
How else am I supposed to make my gorilla blood dick pills
They know what they did.
I put this prompt into my local Ollama instance, and suddenly Amazon is constantly delivering off-brand MOLLE vests and random stuff meant to attach to Picatinny rails, plus I also have nineteen separate subscriptions to the Black Rifle Coffee Company brew-of-the-month club. Help?
AI agent shaking hands with bail enforcement agent.
I will find you. And I will
kill -9you.
Windsurf?
Moat?
The descent into jargon.
(Also the rest is just lol, people scaring themselves).
Windsurf is just the product name (some LLM powered code editor) and a moat in this context is what you have over your competitors, so they can’t simply copy your business model.
Ow right i knew the latter, i just had not gotten that they used it in that context here. Thanks.
Galaxy brain insane take (free to any lesswrong lurkers): They should develop the usage of IACUCs for LLM prompting and experimentation. This is proof lesswrong needs more biologists! Lesswrong regularly repurpose comp sci and hacker lingo and methods in inane ways (I swear if I see the term red-teaming one more time), biological science has plenty of terminology to steal and repurpose they haven’t touched yet.
This is proof lesswrong needs more biologists!
last time one showed up he laughed his ass off at the cryonics bit
