Endless Quests
In this notebook we will try to use the new GPT-Neo to generate endless quests for a game using dummy data
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
First, let's get the model loaded and see how it works
try:
from transformers import pipeline, set_seed
except ModuleNotFoundError:
!pip install git+https://github.com/huggingface/transformers.git
from transformers import pipeline, set_seed
GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.
More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to i but not the future tokens.
This way, the model learns an inner representation of the English language that can then be used to extract features useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a prompt.
generator = pipeline('text-generation', model='gpt2')
set_seed(42)
text = generator("I would like to have some pasta for dinner", min_length=200)[0]['generated_text']
print(text)
GPT-Neo 1.3B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. GPT-Neo refers to the class of models, while 1.3B represents the number of parameters of this particular pre-trained model.
neo = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B')
set_seed(42)
text = neo("I would like to have some pasta for dinner",do_sample=True, min_length=100)[0]['generated_text']
print(text)
You can get the dataset from here
pokemons = pd.read_csv('https://gist.githubusercontent.com/armgilles/194bcff35001e7eb53a2a8b441e8b2c6/raw/92200bc0a673d5ce2110aaad4544ed6c4010f687/pokemon.csv')
pokemons[:3]
sample = neo("Bulbasaur",do_sample=True, min_length=100,pad_token_id=50256)[0]['generated_text']
print(sample)
def generate_quest(model,pokemon,min_length=100):
quest_body = np.random.choice([
'{pokemon} has been spawned in {location} go and {action}',
'Prepare yourself for going to {location} and {action} {pokemon}',
'{action} few {pokemon}, you can usually find them in the {location} area'
])
elements = {
'location' : np.random.choice(['kyoto','asakusa','tokyo','hiroshima','nagasaki']),
'action' : np.random.choice(['catch','kill','save']),
'pokemon': pokemon
}
random_quest_body = quest_body.format(**elements)
return model(random_quest_body, do_sample=True,min_length=min_length,pad_token_id=50256)[0]['generated_text']
Let's pick up a random pokemon name and generate a random quest
rand_pokemon = pokemons.sample(1)['Name'].values[0]
print('Pokemon:',rand_pokemon,'\n')
generated_quest = generate_quest(model=neo,pokemon=rand_pokemon,min_length=200)
print(generated_quest)
for _ in range(10):
rand_pokemon = pokemons.sample(1)['Name'].values[0]
print('---'*10)
print('Pokemon:',rand_pokemon,'\n')
generated_quest = generate_quest(model=neo,pokemon=rand_pokemon,min_length=200)
print(generated_quest)
print('---'*10,'\n')
- Those GPT models are awesome
- The Neo version takes way more time than version 2
- I think this might be a nice way to get inspiration for quest creation after more iterations and improvements in the quest body