In this notebook we will try to use the new GPT-Neo to generate endless quests for a game using dummy data

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

First, let's get the model loaded and see how it works

try:
    from transformers import pipeline, set_seed
except ModuleNotFoundError:
    !pip install git+https://github.com/huggingface/transformers.git
    from transformers import pipeline, set_seed

GPT-2

GPT-2

GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.

More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to i but not the future tokens.

This way, the model learns an inner representation of the English language that can then be used to extract features useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a prompt.

generator = pipeline('text-generation', model='gpt2')




set_seed(42)
text = generator("I would like to have some pasta for dinner", min_length=200)[0]['generated_text']
print(text)
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
I would like to have some pasta for dinner today.  I think it was our night before, right?  We were in the middle of watching the Super Bowl  before my friend went into a bad mood.  She left the

GPT-Neo

EleutherAI/gpt-neo-1.3B

GPT-Neo 1.3B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. GPT-Neo refers to the class of models, while 1.3B represents the number of parameters of this particular pre-trained model.

neo = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B')





set_seed(42)
text = neo("I would like to have some pasta for dinner",do_sample=True, min_length=100)[0]['generated_text']
print(text)
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
I would like to have some pasta for dinner next night, and I was wondering what you guys think about adding some mushrooms as well. I would rather have the mushroom and the pasta together so I am thinking it sounds like a good combination. I have

Let's try to generate quests using the Pokemon dataset

You can get the dataset from here

pokemons = pd.read_csv('https://gist.githubusercontent.com/armgilles/194bcff35001e7eb53a2a8b441e8b2c6/raw/92200bc0a673d5ce2110aaad4544ed6c4010f687/pokemon.csv')
pokemons[:3]
# Name Type 1 Type 2 Total HP Attack Defense Sp. Atk Sp. Def Speed Generation Legendary
0 1 Bulbasaur Grass Poison 318 45 49 49 65 65 45 1 False
1 2 Ivysaur Grass Poison 405 60 62 63 80 80 60 1 False
2 3 Venusaur Grass Poison 525 80 82 83 100 100 80 1 False
sample = neo("Bulbasaur",do_sample=True, min_length=100,pad_token_id=50256)[0]['generated_text']
print(sample)
Bulbasaur's popularity as a playable character in both the Nintendo 3DS and Wii U versions of the game was boosted with the appearance of a new, fully colored, 3DS XL version, in which the blue/gray color scheme returns with a
def generate_quest(model,pokemon,min_length=100):
    quest_body = np.random.choice([
        '{pokemon} has been spawned in {location} go and {action}',
        'Prepare yourself for going to {location} and {action} {pokemon}',
        '{action} few {pokemon}, you can usually find them in the {location} area'
    ])
    elements = {
        'location' : np.random.choice(['kyoto','asakusa','tokyo','hiroshima','nagasaki']),
        'action' : np.random.choice(['catch','kill','save']),
        'pokemon': pokemon
    }
    random_quest_body = quest_body.format(**elements)

    return model(random_quest_body, do_sample=True,min_length=min_length,pad_token_id=50256)[0]['generated_text']

Let's pick up a random pokemon name and generate a random quest

rand_pokemon = pokemons.sample(1)['Name'].values[0]
print('Pokemon:',rand_pokemon,'\n')

generated_quest = generate_quest(model=neo,pokemon=rand_pokemon,min_length=200)
print(generated_quest)
Pokemon: Hydreigon 

Hydreigon has been spawned in asakusa go and catch a look at its official website www.yoshizuma-hikari-haikai-haikai.co.jp or you could click here and look at the
for _ in range(10):
    rand_pokemon = pokemons.sample(1)['Name'].values[0]
    print('---'*10)
    print('Pokemon:',rand_pokemon,'\n')

    generated_quest = generate_quest(model=neo,pokemon=rand_pokemon,min_length=200)
    print(generated_quest)
    print('---'*10,'\n')
------------------------------
Pokemon: Magmar 

catch few Magmar, you can usually find them in the nagasaki area of the west end of the town. you can check the map of the nagasaki area to get there.

Downtown Nagasaki:

The town
------------------------------ 

------------------------------
Pokemon: Bibarel 

Bibarel has been spawned in kyoto go and kill the other and this is a very good reason.

It is important to note that bibarel originated in the ube state and there are no longer in ube, only
------------------------------ 

------------------------------
Pokemon: Corsola 

Corsola has been spawned in hiroshima go and catch up with the newest news. The new news of 2019 was officially announced by the group's leader Shigeo Takahashi by way of Twitter. The members of Corsola made
------------------------------ 

------------------------------
Pokemon: Sealeo 

Sealeo has been spawned in kyoto go and catch up on what we could do about tatata kyoto lol.

So with all the hype about the war of the yuji ken and the kenm
------------------------------ 

------------------------------
Pokemon: Palpitoad 

Prepare yourself for going to nagasaki and kill Palpitoad. This might seem boring, but it will be something you'll remember for a long time. This is a game by the creators of Tomb Raider and it's only the last I
------------------------------ 

------------------------------
Pokemon: Swellow 

save few Swellow, you can usually find them in the nagasaki area of Tokyo, the most popular restaurant here with their dishes are the sushi, which is amazing, there are also many other restaurants that serve food and offer different dishes including some
------------------------------ 

------------------------------
Pokemon: DarmanitanZen Mode 

kill few DarmanitanZen Mode, you can usually find them in the kyoto area
- In L.2 you can find "Toxic Shaft" and "Toxic Wind" along with the main DarmanitanZen Mode,
------------------------------ 

------------------------------
Pokemon: Octillery 

kill few Octillery, you can usually find them in the tokyo area. I always went to the main shrine to get some food and water, but every time I went there, the shrine was closed. I think she probably closed the shrine
------------------------------ 

------------------------------
Pokemon: Mandibuzz 

Mandibuzz has been spawned in tokyo go and save the world with this wonderful, and slightly addictive platforming platform game.

After going to prison and getting sentenced to death, a young man called Mitsuyoshi Nagase comes home
------------------------------ 

------------------------------
Pokemon: Slakoth 

Prepare yourself for going to asakusa and save Slakoth and his crew.

The final act starts here for me…

You can save those of you who want to. I will say this: your loss and my gain
------------------------------ 

Conclusions

  • Those GPT models are awesome
  • The Neo version takes way more time than version 2
  • I think this might be a nice way to get inspiration for quest creation after more iterations and improvements in the quest body