It’s the end of 2022, and a lot has changed since we released the AI Generated Christmas Carol demo back in 2019, so much so, that maybe it’s time for an update. You can listen to the mastered song released in 2020 here and check out our original blog post here.
Back in 2019, AI researcher, Janelle Shane, wrote a post about a test she did using OpenAI’s GPT-2 transformer language model after feeding it 240 Christmas songs and the prompt “Rudolph the red nosed Reindeer”. The results were comical, strange, and amusing enough that we put together a song, writing an original composition and production to the words generated by Shane using GPT-2. We even put out a satirical lyric video the following year using awkard Christmas stock footage clipped together (see below).
Flash forward a couple of years with the release of GPT-3, GPT-3.5, and ChatGPT and now anyone can use OpenAI’s latest neural net. So what does an AI Generated Christmas song look like in 2022?
I first tried to recreate Janelle’s original experiment, using the prompt, “Write the lyrics to a Christmas carol that begins with the words ‘Rudolph the Red Nosed’ “. ChatGPT replied with the following:
Rudolph the red-nosed reindeer Had a very shiny nose And if you ever saw it You would even say it glows All of the other reindeer Used to laugh and call him names They never let poor Rudolph Join in any reindeer games Then one foggy Christmas Eve Santa came to say "Rudolph with your nose so bright Won't you guide my sleigh tonight?" Then all the reindeer loved him As they shouted out with glee "Rudolph the red-nosed reindeer You'll go down in history!"
Okay technically correct, but not what I was going for. It looks like ChatGPT misunderstood my intent and gave me your standard Google response—the lyrics to an already existing Christmas carol. Let’s try again with a little more specificity using the prompt, “Write original lyrics to a Christmas carol that begins with the words ‘Rudolph the Red Nosed’ “.
Rudolph the red-nosed reindeer Was a curious little deer He had a shiny nose so bright It shone through the winter night All of the other reindeer Laughed and played without a care But Rudolph was left alone Feeling sad and unaware Then one foggy Christmas Eve Santa had a brilliant plan He needed a guide to lead his sleigh Through the snow and ice and sand He called upon Rudolph With his shiny red nose on display "Rudolph, my dear friend Won't you light the way?" Rudolph was overjoyed He couldn't believe his ears He nodded his head with glee And wiped away his tears Now Rudolph is known far and wide As the leader of the pack He guides Santa's sleigh each year And never looks back Rudolph the red-nosed reindeer We all know your name You bring joy to all on Christmas Eve And you will forever reign!
Alright here we go. This looks, honestly, quite impressive. It’s original, it makes sense, the rhyme scheme is consistent, and it seems to have resolved on of the central issues in the original AI Generated Christmas Carol that GPT-2 got wrong: the mix of religious and commercial themes. GPT-2 was quite bad at understanding subcategories, and so considered everything to be of equal value and relevance to any query. The result mixed religious themes of the birth of Christ, with commercial imagery of Santa and reindeer in a comical, and often disturbing way, such as the verse below from the 2019 edition.
Rudolph the Red Nosed Reindeer, with its red belly The All-gracious King of all the earth, Had a baby at Christmas time
Where ChatGPT clearly improves is in it’s ability to tell from the prompt that we are firmly into commercial Christmas with the word “reindeer” and not religious Christmas. As such it steers away from any uncomfortable mixing of the two concepts.
While these lyrics are impressive for generated song lyrics, they aren’t quite doing it for me emotionally. The lyrics are peppy, simplistic, with a happy ending that just doesn’t quite give you the emotional impact of good songwriting, that won’t quite cut it in the post Phoebe Bridgers/Matt Berninger Silent Night world.
So I gave ChatGPT another prompt: “Can you rewrite those lyrics with a sad ending?”
Rudolph the red-nosed reindeer Was a curious little deer He had a shiny nose so bright It shone through the winter night All of the other reindeer Laughed and played without a care But Rudolph couldn't join in Feeling sad and unaware He knew he was different But he never understood Why the other reindeer Never wanted him to join in the fun Then one foggy Christmas Eve Santa had a brilliant plan He needed a guide to lead his sleigh Through the snow and ice and sand He called upon Rudolph With his shiny red nose on display "Rudolph, my dear friend Won't you light the way?" Rudolph was overjoyed He couldn't believe his ears He nodded his head with glee And wiped away his tears But as Rudolph flew through the sky He couldn't shake the doubt He remembered all the times The other reindeer left him out His bright red nose flickered And then it went out The sleigh crashed down below Into the snowy ground Rudolph the red-nosed reindeer We all knew your name You brought joy to all on Christmas Eve But now you'll never be the same.
Ooph. Here we go. To use a very 2018 phrase, I’m not crying, you’re crying. Let’s ramp it up a notch with the prompt “Can you re-write those lyrics but foreshadow the sad ending earlier in the song?”
Rudolph the red-nosed reindeer Was a curious little deer He had a shiny nose so bright It shone through the winter night All of the other reindeer Laughed and played without a care But Rudolph couldn't join in Feeling sad and unaware He knew he was different But he never understood Why the other reindeer Never wanted him to join in the fun Then one foggy Christmas Eve Santa had a brilliant plan He needed a guide to lead his sleigh Through the snow and ice and sand He called upon Rudolph With his shiny red nose on display "Rudolph, my dear friend Won't you light the way?" Rudolph was overjoyed He couldn't believe his ears He nodded his head with glee And wiped away his tears But as Rudolph flew through the sky He couldn't shake the doubt He remembered all the times The other reindeer left him out His bright red nose flickered And then it went out The sleigh crashed down below Into the snowy ground Rudolph the red-nosed reindeer We all knew your name You brought joy to all on Christmas Eve But now you'll never be the same.
Not only did ChatGPT pick up on the concept of foreshadowing, it also was able to discern that it already a setup in the lyrics with how Rudolph felt left out by the other reindeer, so ChatGPT just brings it home in the middle with the lines “But as Rudolph flew through the sky / He couldn’t shake the doubt / He remembered all the times / The other reindeer left him out”.
I tried to correct the rhyming scheme with the prompt: “Can you edit those lyrics to find a rhyme between the lines “But he never understood” and “Never wanted him to join in the fun”?” But it appears ChatGPT couldn’t really figure out what I wanted and returned the same lyrics again.
Okay let’s have a little more fun: “Can you re-write the lyrics in the style of David Bowie?”
Rudolph the red-nosed reindeer Was a glam rockin' deer With a nose that shone so bright It could light up the darkest night All of the other reindeer Lived a life of monotony But Rudolph, he was different A rebel, that's for sure Then one foggy Christmas Eve Santa came a-knockin' on his door "Rudolph, my main man Won't you lead my sleigh, oh please?" Rudolph was so excited He couldn't believe his luck He nodded his head with glee And grabbed his leather jacket Now Rudolph is the star Of the show, that's for sure He guides Santa's sleigh each year In a cloud of smoke and a blur Rudolph the red-nosed reindeer We all know your name You bring funk to all on Christmas Eve And you'll never be the same!
While that’s an impressive rendition of Rudolph the Red Nosed reindeer, it seems like ChatGPT has somewhat forgotten that the ending was supposed to be sad, even though it is supposed to remember earlier dialogue. Perhaps it struggles to keep many ideas consistent all at once as you introduce new concepts and layers of complexity. Secondly, it also appears to have slightly confused the concept of a song with lyrics written by David Bowie with the persona of David Bowie himself since the lyrics seem to reflect more the character of Bowie than his lyrics.
Limitations
On that note let’s talk a little bit about limitations. If you’ve made it this far and you’re a songwriter or someone that works with text a lot you might be starting to get either a little worried, or a little excited; and both would be for good reason. This definitely feels like an inflection point in AI technology, and releasing it to the public has created a much greater amount of access to the technology. When both are combined, this is quite likely to have a big impact on the world, for better or for worse.
I’ve played around a little bit more with ChatGPT, and have enough ideas that they should probably go into their own separate blog post, but for now here are some initial thoughts:
- Although quite good at forming a coherent response, simply asking ChatGPT to write an original song doesn’t provide particularly interesting results. ChatGPT isn’t thinking or aware of its responses, it’s just providing a very probably answer. As a result, it puts together some very generic, average sounding lyrics that could essentially be copy and pasted from any generic source just as easily.
- It also often gave the snippets of the same response to prompts with vastly different topics, showing that it was still pulling from the same playbook quite often.
- Similarly, and this point is a bit hard to describe unless you’ve used ChatGPT yourself, but somehow across different types of text (poetry, prose, lyrics), the text seemed to have the same rhythm and texture to it. Even though it was using different words in a different style of writing on a different topic, it often felt the same in it’s structure and cadence.
- Although you can get around this with more complex prompts or layering in multiple prompts, it seems like ChatGPT struggles to keep all of these ideas together all at once and often looses the thread a bit in the process.
- It’s not particularly good at completing existing lyrics either, rather than responding to the complexity of the existing lyrics it tends to insert more generic text that sits on its own.
- ChatGPT is, however, very good at suggesting ways to make a story, lyric, or script more interesting or complex without actually doing it. In a way it’s as if its able to draw on successful formats and tropes to suggest potential methods to draw upon even though it struggles to implement them creatively. It can provide suggestions for resources on those ideas similarly to Google, although it’s feedback isn’t particularly nuanced or text specific, it can provide a decent steer towards some high level concepts to be aware of.
- It is particularly good at doing mash-ups of existing content or “write about ‘x’ in the style of ‘y’ ” type prompts. This makes sense, since the neural net is simply guessing the most likely next words based on a massive amount of existing data scraped from the web up until the end of 2021. So it can easily mimic something already existing, but it struggles to come up with something novel on its own.
- It’s helpful to use ChatGPT to get feedback on songwriting concepts, ways of introducing greater complexity, etc. but it also often gets the answer wrong. It will provide very confident, precise sounding feedback that intuitively feels right. But when you go to look up answers they will often be incorrect, even to surprisingly simple queries.
So what does the mean overall? Surely it will be impossible to tell for sure, a lot will depend on how people choose to use AI such as ChatGPT, how much they will rely on it, how much they fact check it, and how much the novelty of a technology like this will make it interesting to use. But if I had to guess I would say that although it seems deceptively good at creative tasks and providing correct answers at first, once you play around a little bit you quickly become aware that it does have limitations.
Ultimately this probably means there will continue to be a place for people engaged in text-based work. After all, what matters behind creative work isn’t the words themselves as much as the intent of those words and the experiences and story of their creator. However, I do worry about how easy it is to generate a massive amount of content incredibly quickly. In the most mundane of the doomsday scenarios, I can imagine a world where the internet is filled with terabyte upon terabyte of relatively regurgitative mixing and remixing of the same tropes and memes of the line of best fit to no end. One in which content is produced for the sake of content, rather than intent, pleasure, or therapy, and one where more often than not the reader is simply another machine either deciding whether to push this content to the top, or paid off to try to promote it. In this scenario it would be easy for a lot of creatives to get lost in the noise.
Another potential impact is the death of “bread and butter” work. That is, often times many creatives sustain themselves on work that isn’t their primary creative interest, but has stable commercial value. For example, think of a graphic artist who also creates posters for websites, products, or events; or a songwriter who also writes jingles for commercials. These forms of work may slowly be eaten away as AI’s become better at producing coherent content that might not always be as good as a person, but does the job well enough at a low enough cost for that not to matter. This would effectively shut a lot of people who can’t bankroll their entire development as an artist out of the creative sphere, leaving only those who can afford to live without income to have the time to hone their craft.
Lastly, a slightly more distant scenario I could imagine would be many creative arts entering into a positive feedback loop in which the output of the AI generated content also ends up being the input into the dataset the AI are trained on (when done intentionally, this is referred to as synthetic data), creating a self-reinforcing pattern of feedback that takes existing generic trends, tropes, and structures and reinforces them—mixing and mashing them along the way at an ever increasing speed. This would entrench the trends and format of the average of popular content and turn up its intensity as the content becomes further and further tuned for algorithmic preferences; creating a sort of hyper-synthetic mass content culture that is both moving incredibly fast, while—lacking the injection of human ingenuity, experience, and inspiration—also incredibly culturally stagnant.
While any of these anxieties may turn out to be true or way off the mark, they are worth considering. As AI continues to be developed, and most likely at an exponential rate, there is a strong need for us all to consider, debate, and ultimately decide what role we see artificial intelligence playing in our societies and our own lives.
If that’s a discussion you’re interested in, I’d highly recommend this episode of The Daily on ChatGPT to learn a little bit more about how the technology works, and how it might influence us in the future.