At the middle of November, I stumbled on an article describing the GPT-2 Model, a form of machine learning for predictive text. To clarify, it’s a text-generating AI system that can be used to create coherent text. What’s more, this AI system can be trained from specific texts to generate a specific type of response.
The go-to example of this “training” was often for poetry, or the works of William Shakespeare. Sure, it’s cool, but it’s not quite what I wanted to do. For the next few weeks, I kept this system in the back of my mind. I wanted to do something with it, I just didn’t know what.
Fast forward a few weeks to the beginning of December. I was listening to the new episode of The Adventure Zone when I noticed something I gave little thought to before. Each episode of TAZ: Amnesty had a transcript. Wait, Transcripts!
Every episode of The Adventure Zone (TAZ): Amnesty has it’s own transcripts!
Now I knew what I wanted to generate. An episode of The Adventure Zone: Amnesty based on the texts available. I couldn’t wait, this was going to be fun!
Be sure to read to read the full script!
Warning! Mild TAZ: Amnesty Spoilers Ahead
The Plan & Process
The first thing I had to do, was learn how to use the GPT-2 Model. Having literally no experience in Python, machine learning or the GPT-2 model meant I had to start with a beginner’s guide. With a little trial and error, I was off. The first steps of installing Python and a GPT-2 package were fairly straightforward as long as you make sure you install the right versions of tensorflow.
The next step was to create the training data. This was by far the most tedious and labor intensive time of the entire process. All 36 episodes had to be copied from a PDF File, then carefully formatted so the computer can understand the input. I won’t bore you with the details, but there is one important things to mention with the training data. I removed all episode titles, and almost (more on that later) all advertisements from the transcripts. The reason for this is I wanted the truest representation of the actual podcast itself, without advertisements muddying the water.
The third step was the easiest for me: training! In this case, Training is how the computer will know what kind of text I’m looking to be generated. I set my computer to start training based on the script I created, and let it run for a weekend. It probably could have gotten done much faster for many others with a current graphics card, but for this amount of data, running it off of the CPU was just fine.
The Generation process is a bit interesting. With the guide I was using, the newly created model can only create about a paragraph of text at a time, which lead to some problems. The first is that the generator would often get confused on the scope of the conversation. While this lead to some incredibly fun whiplash-inducing subject changes, it also lead to some… well, garbage.
A lot of what was generated was completely unusable, so I had to generate multiple samples for each, and pick from the ones that made the most sense. There were a few repeated lines, or sentences that needed to be re-fluffed into something more substantial.
So does that mean that everything is 100% cherry-picked generation samples? Well, no. It’s more like 99% generated. There were a few times when the generated text forgot who it was talking to, or what the subject was about, and in those small cases, I did make a few small edits. There were also a few spelling mistakes or missing spaces that were also corrected.
What struck me
While there were quite a few results that were a bit silly or nonsensical, there were some things that absolutely floored me. This really is a powerful tool, and with enough training data and computational resources, it could accomplish a lot more than my weekend experiment.
For those of you unfamiliar with TAZ: Amnesty, the cast is compromised the three McElroy brothers, Justin, Travis, Griffin, and their father, Clint. Justin plays a character named Duck, Travis plays Aubrey, and Clint plays Ned. Griffin is their Game Master, providing voices for all of the other characters and setting up the scenes. While a lot of the game is “in-character” where the cast is talking as the character their playing, there is also quite a bit of side chatter as well.
The first thing that I found incredible, is that with surprising accuracy, the generated text would know which character was whose. When Justin talked, it was about what his character Duck would do. When griffin would say something like “What does Aubrey do”, Travis responds. When Ned and Aubrey were having a discussion, they had some quick asides, where Travis and Clint said something before rolling right back into character.
The second part is how each character has their own unique voice that shows through. As an example, there is one character in the series nicknamed “Mama”. All of the characters, and cast call her “Mama”, with the exception of Clint’s character Ned. Ned calls her by her actual name “Madeline”. In the generated responses, the computer was able to contextually figure this out. It could tell that Ned calls her Madeline, but Ned’s player Clint calls her Mama! That’s absolutely incredible!
A final remark about a mistake I made. As I mentioned above, I did attempt to remove all of the advertisements in the training data, however, I failed. In The Adventure Zone, Griffin always introduces the advertisements in the middle of the episode with a “Hey everybody. It’s Griffin. Your Game Master…” and so on. Well I must have left one or two of them in there inadvertently, because GPT-2 generated some! They weren’t very coherent, but they sure did make me laugh!
There were some moments that were generated that were all to brilliant in their own way, that I couldn’t let go to waste without sharing with the world. So here we go, in the wise words of an AI Travis McElroy:
Continuing the Robert-Chicken-levels of horseshit.A.I. Generated Travis McElroy
These goofy results
Lots of Fish
[Ned]: Did I tell you what I do?
[Aubrey]: Yeah, let‘s see, what do you do?
[Ned]: Um, I uh, I live on fish. Lots of fish. Lots of fish.
[Duck]: Lots of fish.
[Griffin]: He says…
[Vincent]: Um, I live… let me try to clarify that for ya. I don‘t do live through brands. That‘s for sauces.
[Duck]: Yeah, a lot of the time, we‘re not in the market for a new grill material.A lot of the time, we‘re
[Griffin]: To continue the tradition of introducing new voices to the Adventure Zone, we asked Justin to introduce the ladies here with his song, “Hey Hey, Muffin Man.”
[Griffin]: Constance writes…
[Concerned Female]: I write a lot of songs for the ladies
[Griffin]: You all are standing in the foyer of the Cryptonomica, and Vinyl‘s there,and he‘s kind of reading a magazine. As you all enter, he makes a face. And he sort of nods at you as you come in, and he says…
[Vinyl]: So uh, how, uh, are you… how have you changed?
[Griffin]: He laughs, and he says…
[Vinyl]: Well, I mean, I moved. And that didn’t exactly go like, the desired route.Um, but um, I mean, I’m actually a little bit closer to home these days.
[Aubrey]: That sounds dope.
[Vinyl]: It seems like a little bit.
[Aubrey]: It kind of sounded like a big conspiracy to bring me here. But, uh, it actually sounded like a good idea.
[Vinyl]: And you… you seem like a nice guy. A nice candidate.
[Duck]: Y’know, as long as we let her get close, it‘s… it will be too late for us.
[Justin]: Uh, I start reaching towards the tree, and then I realize I’m just wanting to grab the tree. And then I think, ―Oh, fuck. I’m gonna store the tree‘s nuts. I’m gonna grab the tree‘s nuts.‖
[Travis]: [laughs] How fun would that be?
[Minerva]: Who‘s that?
[Duck]: I‘m… I am so glad to see you again.
[Griffin]: She looks a little… older. A little bit thinner. She shakes her head, and she says…
[Minerva]: Duck Newton?
[Justin]: No. Not yet.
[Duck]: No. I will talk to you when—
[Minerva]: You‘re not—Duck: No, I can‘t, I’m not—
[Duck]: I’ll talk to—I’ll talk to you when we are in range. Okay.
[Minerva]: Okay. Thank you.
[Minerva]: When we are in range. Okay. I’ll see you all in a couple minutes.
[Minerva]: One moment, I want to check in on my friend, Duck Newton.
[Justin]: As he leaves the office, Minerva turns to Minerva. [laughs]
[Griffin]: [laughs] She says…
[Minerva]: You‘re open.
[Justin]: [laughs] Minerva goes—quickly?
[Griffin]: She takes a step closer.
[Minerva]: Duck Newton… It‘s been awhile.
[Travis]: I do think, though, that we need to set up a few things. Like, first of all, do, because I don‘t want this to be some one off, where she‘s like, ―Oh, you were in the car with me when that goat man attacked.‖ Um, but more importantly, we need to set up some sort of, y’know, regular in-fiction narrative that is using the flyin‘ around the NPC‘s estates, and establishing…
[Travis]: [inhales sharply] That that is the narrative thread that connects all of this.
[Griffin]: I like that. I like the narrative thread that connects all these different couple months of development that have just come to a head here, and y’know, we‘re all a little bit caught off guard by it. But I feel like we‘re all a little bit caught off guard, too,Ned, when we…
[Clint]: Well, when did this narrative thing even go…
[Griffin]: This is six months after the finale?
[Clint]: No, I think we‘re in the middle of the final battle.
[Clint]: And so, there‘s been some reflection and study. I mean, I’ve talked to a lot of players, and they have been like, ―Ned, there is so much more to play for than just destruction and death. There‘s so much more to play for, and playing a character that could be like, a cosmic dragon, or a magical horse, or a space squid, or a deep,deeper, cosmic character, is an absolute honor.
[Griffin]: I like that. It‘s like—it‘s a nod to that sort of nostalgia. I would imagination that like, in those early days, where you’re just randomly clicking through a door, and you come up for air only to be like, ―Oh,hell yeah, yeah, yeah. More. More. More.‖
[Griffin]: That has brought Ned back into the office, where Aubrey and Duck are hacking away at some wiring, and you see Aubrey
[Travis]: I do think, though, that we need to clear this thing out, y’know? Ned has been there for like,three decades. I don’t know how old he‘s been in those three decades.
[Travis]: Y’know, he used to ride his bike to the store.
[Griffin]: Um, yeah, okay. You all are in front of the Cryptonomica. It is, uh, as the owner of this awesome shop, uh, Mr. Crypt, has it secured in place with tape.It is secure in, y’know, all the secure things. I think all of Ned‘s stuff is secure in His wisdom. Uh, he‘s not gonna, y’know, shoot it into the ground and expect it to stay in the ground.
[Mama]: Can I come in?
[Mama]: I just—
[Aubrey]: [softly] Come on.
[Mama]: I’m just saying—
[Aubrey]: Come on.
[Mama]: I’m saying, we got a… we got a client that‘s… that is… uhh—
[Ned]: What? What do you want, Barclay?
[Aubrey]: What would you like, Ned?
[Mama]: I just—
[Aubrey]: Come on.
[Mama]: I just need… I just need you… come on. Come on and let me know what you want.What would you like, Duck?
[Duck]: We got a client. We got an international criminal organization that‘s been going around for a while,and they are after… we said bank, I heard about this. We got a client.
[Aubrey]: Russian mob.
[Mama]: Russian mob. That sounds—
[Aubrey]: Russian mob.
[Mama]: Russian mob. That sounds like something I would associate… that sounds like something I would associate with.
[Duck]: A real, I would associate it with something shady.
[Aubrey]: I‘ll take care of—
[Mama]: You gotta come back, Duck. You gotta come back to me. Don‘t make a mess of things.We got a client. A very shady client.
[Duck]: What would you like, Barclay?
[Barclay]: Is there anywhere in particular where we can observe him, or y’know, anything that might be interesting, or you can, y’know, map it out? I mean, I guess, we got a few leads,but I mean, maybe you can find something else.
[Duck]: You can find something else. I’d love to see you, Barclay.
[Barclay]: Yeah. Wherever he is, I’d love to be with him.
Minister of the Ghost
[Travis]: Aubrey opens the door.
[Griffin]: Papa says…
[Papa]: Okay, Ned. What is so important?
[Griffin]: And Mama‘s face lights up as Ned explains. And as they hear this, the crowd at the Cryptonomica has dwindled to just you, Mama, and Dr. Harris Bonkers.And the four of you walk into your respective places at the table together.And Dr. Harris Bonkers says…
[Musician]: So, um… I—
[Aubrey]: No, it‘s fine.
[Griffin]: Uh, Dr. Harris Bonkers says…
[Dr. Harris Bonkers]: I am… I am Minister of the Ghost?
[Aubrey]: No, I know—
[Bonkers]: That‘s not your… that‘s not your—
[Aubrey]: No, it‘s not—it‘s right here.
[Travis]: Okay. Uh, so, it is a six?
[Griffin]: It is a six. That is to be expected.
[Travis]: Oh, because I have three plus weird.
[Griffin]: Yeah, so it‘s an eight.
[Travis]: An eight! That‘s a mixed success.
[Griffin]: Uh, okay. With a twelve on—
[Travis]: [quietly] I can still do pretty much everything on this thing.
[Aubrey]:[V for asteroid]
[Agent Stern]: The cybernetic spider. I‘m looking for Anya, theweb cat.
[Griffin]: She says…
[Agent Stern]: The cybernetic spider was looking for you.
[Agent Stern]: He was always looking for you.
[Aubrey]: Uh, funny, how
[Aubrey]: Uh, my name is Aubrey, and I’m a goat.
[Duck]: And your head hurts too, right?
[Aubrey]: Yes, I have a very large, very pretty, very fragile looking head, andI’m injured.
[Duck]: You can talk?
[Aubrey]: I’m injured, and you can understand me.
[Justin]: Duck‘s a goat.
[Travis]: That‘s canon.
[Justin]: Duck a goat.
[Travis]: It‘s a classic.
[Justin]: This is a classic 90s television.
[Griffin]: Yeah, this is the 80ies. We actually also used to do a Trump episode, but I don‘t remember ifthat‘s still on. Somebody should tell us.
[Clint]: Should I grab the rope?
[Griffin]: No, don‘t grab the rope.
[Justin]: Or do I…
[Griffin]: No, don‘t touch the rope.
[Clint]: This is a body of water. This is Swimming with Bears.
[Griffin]: This is Swimming with Bears.
[Travis]: Swimming with Bears.
[Justin]: Swimming with Bears.
[Clint]: With Bears.
[Justin]: With Bears and Lights.
[Clint]: With Bears and Lights.
[Griffin]: There is a soft moonlight up above you, and you see Sheriff Zeke Owens standing in the Cryptonomica parking lot,aspiring a needle through the stillness.And you are insane.Do I have time?
[Griffin]: He says…
[Zeke]: Now, I admit, I don‘t… I don‘t have much time to teach you howto shut the fuck up, outlaw.
[Justin]: Thanks for the [expletive deleted].
[Griffin]: And he looks super pissed.
[Justin]: [imitating a bad voice] You [expletive deleted].
[Griffin]: Oh, hooray for instant credibility.
[Justin]: Thanks for the [expletive deleted].
A Better Ad
[Griffin]: Hey, everybody. This is Griffin McElroy, your best friend, your dungeon master, and your vacation spot, now is like, understandably tight. I brought this up in passing, but like, I forgot like, how much you alleviated up for this episode. I actually ended up just editing it really fast, because I want to make it really good, and I want it to be like, episode 25, episode 26, whatever. But like, I haven‘t forgotten how much you all loved the climactic shootout in the second half of the last arc, and I was eternally in gratitude for your love of this show.So, one of the rewards for our first listener is gratis in-fiction downloads of the second half of The Adventure Zone, which is getting ready to be like, released. And now, it is, finally, Episode 26, part of the second arc, with the finale coming in like, November. So gratis in-fiction downloads of like, the first half of the second season, and gratis digital downloads of the finale.The concept is still the same, but I am so excited about how it’s gonna turn out,and I got a bonus episode already in the opener for good behavior.