Why my generative model is depressive?
I was experimenting with GPT-2 and discovered the unknow territory of text generation.
Introduction
In last year there was a craze about GAN (generative adversarial network), especially in the field of NLP natural learning processing. We have seen open-source projects and publicly available both model and data to experiment with new AI technologies. This is where we enter the world of AI which can talk like humans and generate stories, poems, and all kinds of text data in multiple languages. If you fine-tune your model, you can produce amazing work specific to your field, this is where I come in with experimental studies on NLP.
Experimentation
I was experimenting with GPT-2 and other Facebook text generative models, but I didn’t know how to control the parameters, so I wrote the first line and adjusted few parameters to get a depressive completely new paragraph.
The paragraph below is AI generative text with only the first line as a guide.
“Ahhhh…. How do I start? Today, my neighbor took his life by hanging himself. His name was David. We had a very close relationship. He had just turned 18 years old. He was a student at the University of Utah. David had been coming over for the past two years. I saw him a couple of times a week for the last two weeks. I think I saw him the last time on Friday. We had talked for about 30 minutes and I just said, “Hey, David.” I said, “What’s up?” He said, “I’m going to kill myself….”
At first, I was amazed to see the results but then, things got even worst when I wanted to write a short story on the same parameters. This generative model created a short story about his mother and his relationship with his neighbor. I am still improving my model as I write this post, but this unknown territory made me think how the future of AI will be highly influenced by what type of data we train these models on. Even the best performing model will have some biases against certain groups of people, maybe our little bot will be as depressive as any human being.
Learning Resources
If you are new to the world of Machine learning and have experienced with python then your go-to book should be Approaching (Almost) Any Machine Learning Problem.
For a free copy click here.
Do follow the author @abhi1thakur, I am a big fan of his work.
Deep Learning and GAN
- If you want to learn GAN, you can buy the book Deep Learning for Coders with fastai and PyTorch: AI Applications Without a PhD. This book has helped me a lot with deep learning especially the math part. I will recommend this book to anyone who has little knowledge about machine learning and python.
- If you are interested in experimenting with models and have some knowledge about transformers, I will suggest you check out Hugging Face.
- For more information, you can check out FacebookAI and OpenAI.
You can follow me on twitter @1abidaliawan.
