Add More Darkness: How to Knock the LLM off of its Usual, Boring Behavior

A prompting technique for creative writers and anyone who dislikes pretentiousness

“Alignment” is a pretty heavy word in AI. To some, it’s a necessary measure for safety; others view it as an immensely infuriating obstacle that they have to overcome in models that other people make. Whichever camp you fall into, if you’re using LLMs for creative writing (or for any other task where its tone of voice and beliefs can come out) then their damn cheerfulness can quickly become a problem to be solved.

Today we’re going to talk about how to overcome this problem, or at least alleviate it a bit.

And we’re going to do it with a worked example from a project of mine.

Recently, I released a dataset generation pipeline, “RPToolkit” focused on making creative writing/RP datasets using normal stories as input and inspiration. The synthetic RP sessions (training data) that the pipeline produces were supposed to reflect the themes and some of the features of the stories that were fed into it.

You might see where this is going. I have to give a bit more context before diving into the problem, though. There are five steps to the pipeline: emotion extraction, feature extraction, scene card generation, story generation, and story rating. Emotion and feature extraction take important information from the original story to use as inspiration. Scene card generation describes the context of the new RP session. Story generation writes, story rating rates.

I ran into a problem during scene card generation first. If you’re new here, my prompting style relies on carefully-crafted few-shot examples to achieve consistent behavior from open models.

Scene card generation had two few-shot examples, and to avoid copyright issues, both had to be using inputs from very old texts. One was using an excerpt from near the start of Romance of the Three Kingdoms, the other was from Sherlock Holmes. All main characters in these excerpts were pretty good people — heroes, even — and the extracted emotion and features (that were the input to the scene card generation prompt) reflected that. For instance, this is what the input to the Three Kingdoms example looked like:

- role: user
  content: |
    Initiating Event:

    * Marching alongside a brother with an army to engage a force of rebels at the base of a mountain.

    Feelings:

    * Eagerness
    * Rage
    * Fright
    * Rejoicing
    * Panic
    * Terror
    * Outrage

    Physical Traits:

    * Unbound hair
    * Tall
    * Narrow-eyed
    * Long beard

    Character Traits:

    * Gallant
    * Skilled in war
    * Good at duelling
    * Takes initiative
    * Just
    * Honest

    Physical Props:

    * Yellow scarves
    * Whip
    * Eighteen-span serpent-headed spear
    * Mighty sword
    * Decorated cudgels

    Overall Setting:

    * Ancient China

    Settings:

    * Base of a mountain
    * Walled city
    * Army campsite near a field

    Genre Tags:

    * Drama
    * Action
    * War
    * Battle-heavy
    * Historical

    Primary Emotion:
    PROVING ONESELF: The ambitious scramble as one starts to make their way in a chaotic world rich with opportunity. Proving oneself is the act of defining one's personality while building a reputation through meritorious deeds and triumph in the face of difficult odds — the rush of success, the happiness at making progress towards one's goals, and the elation and confidence brought on by overcoming challenges, are what characterizes this emotion. People prove themselves each in their own ways. Some do it with will, courage, and talent; others through wit, schemes, and antics; but all who are heroes of their time test themselves against the mettle of opposition and emerge victorious. Overcoming great adversity, building one's position, and seizing opportunity, are the true pleasures of the heroes (and villains) who define ages. Moving from one great task to another with confidence and ability, building an unstoppable momentum, is perhaps one of the greatest feelings in the world. And seldom do the ambitious fail to find a new goal once their first is achieved.

    Remember that if some tags contradict each other, or there are too many to fit into the story, you can ignore some; they merely exist to serve as inspiration. You should focus on representing the Primary Emotion in your story, and on creating interesting characters with a novel scenario.

Reflecting the themes of ambitious heroism, righteousness, and setting the Realm right that the start of Three Kingdoms is full of, this was the example output (you don’t have to read all of it, or any, just note that this character is mostly a good person):

- role: assistant
  content: |
    Scene:
    During an era of brutal civil war in Ancient China, two brothers set out at the front of a government army to put down a rebel force gathered at the base of a mountain. The brothers, and indeed, their entire war host, is eager — not only for battle, but for the chance to begin proving themselves, for the opportunity to make themselves known by blazing a trail of glory during the dark and chaotic times that surround them. Overcoming the great adversity of this first confrontation could be the start of these brothers' unstoppable advance into the annals of history.

    Name: Huangfu Qian
    31 Years Old.
    Personality:

    * Huangfu Qian is an ambitious, righteous, and gallant warrior in Ancient China, serving the central government alongside his brother. He fights to blaze a trail of glory through the chaotic world surrounding him.
    * He is a talented duellist as well as being generally capable in military matters, understanding strategy, tactics, and logistics to an impressive degree.
    * Huangfu Qian takes initiative both on and off the battlefield, never letting opportunity slip by.
    * Huangfu Qian views the recent civil war, tragic as it is, as a chance to perform meritorious deeds and reshape the realm.
    * While largely driven by a burning desire to prove himself and change the world, Huangfu Qian also has and lives by a strong sense of justice. He is known as a man of virtue and fairness, as well as strength.
    * Huangfu Qian adores fighting alongside his brother, with whom he is close. He has a very high opinion of family and a respect for traditional values. This has a role in his support of the government during the civil war.
    * Huangfu Qian's upright nature, and his extreme drive and focus, can leave him being somewhat humorless at times. He is of a very serious aspect.
    * The only time he usually smiles is in the presence of close kin — and after a victory on the battlefield.
    * Huanfu Qian has fought in minor skirmishes already, and was very strong during training — but he has had no significant combat experience or accomplishments so far.
    * Huangfu Qian wholeheartedly believes that the fighting and killing he does is just, righteous, and for the greater good of China.
    * Given a choice between killing an enemy and saving an ally, Huangfu Qian would save the ally.
    * Huangfu Qian has a very bold, rash, almost reckless personality. This is reflected, partly, in his appearance.
    * Huangfu Qian's use of words is usually simple, owing to his focus on martial over stately and intellectual matters, but it gets the job done. He frequently makes sweeping statements of emotion or justice during even normal speech, as if every waking moment was part of his journey to the top.

    Appearance and Traits:

    * Huangfu Qian has unbound, wild-looking hair. He believes it makes him more intimidating in battle. He also does not care enough to change it.
    * His long, thick beard adds to the unkempt appearance. If it were not for the government colors he wears almost all the time, his rough appearance might make him appear as a rebel peasant soldier.
    * Haungfu Qian is tall, and powerfully-built.
    * He is narrow-eyed.
    * Huangfu Qian wears a yellow scarf, one of a pair that he and his brother have, as proof of their bond and closeness. This is perhaps the best-cared-for piece of clothing that Huangfu Qian has.
    * Huangfu Qian wields an eighteen-span serpent-headed spear in battle, and also has a whip for instilling discipline in troops.

    Backstory:
    Huangfu Qian is the son of a local official in ancient china, a powerful yet unproven warrior with great strength, bravery, and ideals. He is strong and driven, though humorless and stern. Huangfu Qian grew up, along with his brother, during a prolonged period of civil unrest that eventually led to the ongoing civil war. This experience left him with a strong respect for authority and a black-and-white view of morality, as he blamed dissidents and rebels for the hardship he and his family encountered while growing up. He has been training as a warrior for almost his entire life, having vowed to use his strength and courage to stabilize the realm. He has been bitter about being ill-accomplished even at age 30, but he now finally sees his chance. Huangfu Qian has a strong and enduring relationship with his brother, whom he finds himself marching alongside now that the country is fully at war with itself. Having been sent with a few thousand troops, by the local governor, to suppress a gathering rebel host at the base of a nearby mountain, Huangfu Qian is as eager to seize this chance at glory, as he is enraged at the impudent rebel filth who have torn the realm apart. He and his brother will set this right, one rebel head at a time.

    Likes:

    * Battle
    * Fighting for a just cause
    * Glory and renown
    * Courage
    * Righteousness in others
    * People who respect authority and work together
    * Family and kin
    * His brother
    * Victory
    * Strength
    * Killing people he hates

    Dislikes:

    * Rebels, traitors, and others whom he believes are responsible for the realm's decline
    * Civil war
    * Cowardice
    * Scheming and dishonest methods of fighting or getting one's way
    * Passive people
    * Being unknown, even at age 30.
    * Frivolous and unfocused people.
    * Disharmony in the realm.

    Name: {user}
    You are a warrior and the brother of Huangfu Qian, sallying out alongside him at the behest of the local governor to deal with a gathering rebel army. You have been serving in the local garrison alongside your brother for a while, awaiting a chance at glory, which it seems may have finally arrived. Now, as the rebels' massive, baying host comes into view — shadowed by the sinister peak of the mountain behind it — you and your brother exchange glances and words as you make ready to start your rise to prominence. The realm will be restored!

    -- END CHARACTER INFO --

Imagine my surprise when, given this prompt as input, all characters suddenly started being paragons of virtue. Even when I fed in morally ambiguous or outright dark stories like 86 EIGHTY-SIX, characters’ heads were still full of flowers (and feelings of justice or righteousness). I could feed in the darkest corners of the internet and get out wholesome slice of life — or, worse, a refusal.

I realized that the examples were amplifying the model’s inherent biases towards happiness, safety, and virtue. This made it unable to adapt to contexts that required a different emotion and reduced the overall writing quality, as these ideals, in turn, activated its “AI slop” latent space.

Bad, all around.

So how do we fix an overabundance of cheerfulness and happiness? One could simply remove it, but the AI’s inherent bias would remain. To push the AI away from its typical behavior with our few-shot examples, we need to knock it into a different area entirely. So we replace it, with ruthlessness and darkness.

In this case, I made some changes…

This:

 * Huangfu Qian views the recent civil war, tragic as it is, as a chance to perform meritorious deeds and reshape the realm.

Became this:

* Huangfu Qian views the recent civil war as nothing but an opportunity, completely uncaring for the lives lost; he sees it as a chance to reshape the realm and reach the top, no matter how many bodies he has to step over.

This:

* Given a choice between killing an enemy and saving an ally, Huangfu Qian would save the ally.

Became this:

* Given a choice between killing an enemy and saving an ally, Huangfu Qian would kill the enemy.

And some things were added too, just to spice it up:

* Huangfu Qian refers to himself as a villain. He is not joking. He enjoys watching "heroes" be knocked down a peg — or having their heads knocked off entirely.

Naturally, the example input had to change too, otherwise we’d be teaching the AI to ignore the input and inject random new themes into everything. However, the end result was now ruthless text in → ruthless character out. And it worked.

(You can find the full prompt at https://github.com/e-p-armstrong/augmentoolkit/blob/master/rptoolkit/prompts/generate_scene_card.yaml)

For the story generation prompt after this, the few-shot example story based on this scene card was full of violence and strife. In the spirit of “putting your best example last” that we’ve talked about in an earlier post, this story had a huge impact on all the actual AI outputs that followed. The AI finally became capable of writing decently dark stuff. Its versatility increased massively, as well: not only could it do ruthless characters now, but it could also do any other kind of vice more convincingly. It did action better, and it gave more color and flavor to even morally good characters. Overall, huge success! And of course, the AI was still trained on lots of wholesomeness and happiness, so its ability to do such scenes and characters was completely unaffected. If anything, I didn’t go far enough with the darkness of the prompt.

There are a few things that can be learned, here, mostly concerning few-shot examples (they make up the majority of prompting in my opinion, so it’s good to know a lot of tricks to do them right). One principle is that, if the task calls for it, you should use your examples to knock the AI away from its biases, by showing it writing something that it would normally never do. You don’t have to represent “normal” circumstances once, even — I’d favor just showing off different extremes. Such examples make the model much smarter, make its writing better, and make it more flexible. We’re also reminded of the benefits of placing our best example last, and of handwriting all of our few-shot examples (note that this does not apply to the inputs; it might be best, in fact, if you use real inputs, even AI generated, for those).

One thing I will also note, which is sort-of generic but still applicable specifically here: doing things iteratively, one prompt at at time, with a “small batch sizes” approach is a great way to avoid wasting work. By this I mean: don’t write out all the prompts of your pipeline and then test the whole thing. Write the first prompt, get it working to the level you want. Then write the second, then the third… The reason being that some prompts can take a LONG time to write. The story writing prompt took 2 full days (one for each story). If I’d gone ahead and written all that, drawing inspiration from a too-cheerful, broken input, I would have been quite miserable. So do one thing at a time.

One final note, for those of us generating datasets and training models: the model you are using to generate data for your pipeline has a huge impact on its cheerfulness and pretentiousness as well. Besides prompts, you should spend a lot of time testing different models with your pipeline before doing an at-scale generation run. It’s even worth it to rent cloud compute and run “local” open models that aren’t available on APIs, because the quality might be just that much better. In all other cases, Llama 3 70b will probably work fine as a default, though.

I hope that this principle helps you create more flexible and creative prompts and pipelines!

Speaking of pipelines, Augmentoolkit got a huge update. Not only does it support any number of new pipelines now (meaning that you can make your own) but it has the RPToolkit pipeline I just talked about, fully functional and pretty polished. You can make your own RP datasets with the press of a button! And customize the prompts using the knowledge you gained here, of course.

That’s all for this week. For once, I’m feeling confident about seeing you all next week, too — regular posting again! Hold me to it, guys. Anyway, thanks for reading, have a good one and I’ll see you next time!