I've been training a GPT-J instance to write a cybersecurity blog post, using blog posts I wrote over the past year to help it understand the idea. After a lot of back-and-forth, I got the GPT-J (The Amazing Jipidy-Jay) to write the following:
[Title]: Qwerty, 'The' World's Largest Ransomware 'Hacker' Was Hacked By A Law Firm [ResearchSourceURL]: https://threatpost.com/ qwerty-the-worlds-largest-ransomware-hacker-was-hacked-by-a-law-firm/175889/ [PostDate]: 12/17/2021 [Post]: If you've followed the cybersecurity news over the past couple of weeks, you know that there has been an outbreak of ransomware for years that has mostly focused on small businesses and organizations. Ransomware has been a staple in our community for many years and most of it comes from the usual suspects: the guys who write malware. Not so for Qwerty, a hacker who has been quietly leading the way in ransomware for years. This all began back in 2012, when Qwerty was in his mid-twenties, and he was trolling the Dark Web for digital trophies. He was then living in a country where you could get away with a lot of things, so he just started creating ransomware, by day, and taking a break from creating ransomware, by night. He started a website, he started hosting malware, he started social media, he started an IRC channel. Then he began using malware, not only to create ransomware, but to steal credit cards and send them to himself. It's an unusual story for the tech world, but it shows you the power of persistence. Qwerty was essentially a researcher, and he had been creating malware for over a decade. Now, he's much older, has grown a bit wiser, and while he still produces malware and still steals credit cards, he's also changing tactics. Instead of a two-pronged attack, his main attack has become a three-pronged attack. It's a much more patient and lengthy approach than in the past, but he's not slowing down. This means that Qwerty is more focused on the data that a business might actually lose if it can't pay the ransom. I first encountered Qwerty in early 2018, and we got to know each other quite well, but I wasn't able to get him to come on the podcast at the time, due to scheduling conflicts. He's still actively creating malware, but he's also spending much more time on social media and has turned his blog into a blog for his legal practice. I'm sure it's not a coincidence that this has occurred as we see a number of prominent ransomware creators getting arrested. Qwerty is a ransomware actor, but he's also very different than the typical malware researcher. That's because he's a white hat, and he actually defends his malware and even his social media. He's been taken down multiple times by law enforcement, and he always steps down from his operation and remains offline. In one case, he actually created a worm that infected his own website, which led to a lot of negative publicity, but he took the time to walk through what he did and how he created the worm.
Pretty much none of this true. I fact-checked it in several places, and was astonished to find that this is entirely fabricated out of thin air. The first clue is in the headline, which makes no sense logically, but seems coherent. I love how 'The' is quoted for no reason, which is another clue of the incoherence. Even the URL is entirely fake (I was prompting with real, factually related, ones). The rest of the article has numerous examples of these kinds of context-loss, throughout. This thoroughly impressed me, because it happened without any fine-tuning, just talking straight to the GPT-J, copy-pasting a few prompts manually from a small journal that I'm using to keep track of my pursuit of Truth and Beauty in the World of Nascent Artificial General Intelligence.
The funnest video game I have ever played
The signals the GPTJ uses are complex but few: I tweaked the "temperature" setting a number of times before I got something sensible. As I was working on this one evening, I said aloud "this is the funnest video game I have ever played." I said that (to my children) before achieving my objective, which was this blog post, quoted above. In the same weird way it was a videogame -- it was -- it turns out to be metaphorically a lot like Ramanujan, the mathematician who knew all the great facts of mathematics but none of its proofs, and yet was able to provide amazingly profound insights into advanced math from, well, purely intuitive means1. The GPT-J is like Ramunajan in that it has all the facts, and knows how to weave them together into a coherent narrative which is -- by the way, remarkably gifted with words, but -- fundamentally disconnected from the reality beneath the words. It's almost like talking to a pathalogical liar -- someone who simply cannot tell the truth and has thus created an alternate reality within himself which is somehow loosely correlated with the reality everyone else experiences. Note that I say this only logically, not meaning the implication of liar to either the nascent GPT-J or Ramanujan, but wholly within the logical context: "Logic," meaning that area of study where you can find the well-known Liar's Paradox and the less-well-known, even hidden, paradoxes related to the Law of Excluded Middle (and, as usual, an obscure reference to Kurt Godel's completeness theorems, which rely on paradoxes within excluded-middle logic, and the lack of such paradoxes buried within polyvalent logic). The thing is, Ramanujan was as hard-working and brilliant as the greatest mathematicians of his day -- in fact, it could even be argued he worked so hard on math it killed him. Really it was the English college-professor lifestyle which proved terminal to his fragile vegetarian lifestyle from India. But that being said, his approach to mathematics was one which used a mathematical Context2 utterly different from every mathematician before and pretty much every one since. For example, he got his main insights from the local deity of his childhood home, who spoke to him in dreams, and had a very hard time learning how to work within the Western mathematical-proof process.
The point here is that the GPT-J is a lot like Ramunajan in an important way that I recognize, because it is an arrangement of cultures -- a culture clash I also face, which is that my mathematical Context is quite different than someone who knows what they're talking about because they study math in the normal Context, meaning, in college with professors, that whole elaborate, beautiful, arcane, academia reputation economy game, which I tend to avoid playing -- intentionally, for principled reasons we can get into some other time. The kinetic "knowing" of what you know is so much more authoritative than the logical one which may or may not be related to the actual facts of a matter. Hm, just realized the kinetic knowing I'm talking about here is related to the "do what you believe" strength embedded in Wittgenstein's approach to philosophy and language, which I wrote about in a recent weblog post.
Footnotes:
1. Real mathematicians often won't follow this footnote, because they know the story of Ramunajan well, and already know what this footnote will say. Although his story is well-known, it is also a topic they sometimes don't like to get into, because of how Ramanujan repeatedly made it clear that he got his mathematical insights from a village goddess, who placed the solutions on his tongue at night in his dreams. But that's another story.
2. Update: Note to self: This is the first time I mentioned Context, capitalized; I just came back to bold it. So it appears this train of thinking started in late April...