Every morning, this website follows a three-step process automatically:
How good can the middle step be, where a large language model AI generates a haiku?? I am not a haiku expert, but my goal is to prompt ChatGPT (or Anthropic's models, or whatever else) in a way that generates good haikus, spending no more than 5 cents per day in the process. On this page, you can track my progress and read the behind-the-scenes of my prompting journey.
-Will
A simple prompt while I get the site up and running.
For a lot of genres, ChatGPT can just go for it. It can write limericks and poems generally without special instructions. Maybe the same is true for haikus - let's not overthink it. So I kept my first prompt very simple and used the most basic (and cheapest) model of ChatGPT.
To ChatGPT, model 3.5-turbo-0613:
Write a thought-provoking haiku about the following news story:
Section: business
Date: October 22, 2023
Title: Why Olive Oil Is So Expensive Right Now
Story: Prices have more than doubled as extreme weather hit crops in Spain, Italy and elsewhere. The upcoming harvest isn’t looking much better, so that drizzle of oil on your salad may remain costly.
Obviously, each story's details are different, so the prompt had to change for each story. Here's how it actually looked on my end:
Template message to ChatGPT:
Write a thought-provoking haiku about the following news story:
Section: ${story.section}
Date: ${currentDate}
Title: ${story.title}
Story: ${story.abstract}
Using the variables in the prompt, my script filled in the appropriate details based on what I received from the New York Times. It was a new programming skill for me.
My favorite haikus from this prompt were the weird ones that had no visible connection to a news story. Like, what story could this even link to? Who knows! But it's fun and mysterious:
From ChatGPT:
In truth’s murky haze,
Whose voice must we trust to know The echoes of pain?
As I learned more about haikus, I realized the rhetorical questions weren't really standard. Also, the haikus tended to rhyme - ChatGPT thinks rhymes are very poetic. But rhymes are basically the opposite of haiku. Rhymes show surprising similarities between words, but haikus identify surprising juxtapositions between things in the world.
It was also convenient for me that I bought the site's domain name before I found out that haiku writers in English pluralize it in the same form as the singular, so they don't say "7 haikus," just "7 haiku." At this point, that feels unnatural and ostentatious to me.
Upgraded the ChatGPT model to GPT-4.
If you use ChatGPT on the website, there's the free version (labeled "3.5") and an upgraded version ("4"), which comes with a monthly membership. But if you use ChatGPT from a website, like I'm doing, the pricing structure is different. You pay for it all. ChatGPT-4 is 20x as expensive as ChatGPT-3.5, but by continuing to use the same simple prompt, I was still able to be under my 5 cent/day budget. I wasn't sure how much better ChatGPT-4 would be with this same prompt.
My favorite haiku from this period drew in keywords from the story, like this one about a city during the Israel-Gaza war:
From ChatGPT, model gpt-4-0613:
In Ofakim’s still night,
Handguns echo ‘gainst terror,
Desert sand knows fight.
But at the same time, the haikus were still very rhyme-y, even in this one! (night/fight) More broadly, ChatGPT's haikus didn't have very clear references to the new story.
Indirectly tell ChatGPT not to rhyme.
In OpenAI's guidance for prompting, they recommend that you prompt ChatGPT with what you want it to do, instead of telling it what you don't want it to do. So I added a short instruction to the end of my prompt: "Your haiku should follow the traditional 17 syllable format." I was hoping that this would remind it not to rhyme, and would also activate other knowledge it has about haikus, like making seasonal references and counterintuitive juxtapositions.
I didn't notice a big difference, unfortunately. Some haikus started out promisingly, but then ended on an insipid rhyme.
From ChatGPT:
Sickle cells transform,
CRISPR edits life’s design,
Hope in healing’s form.
I was surprised that even ChatGPT-4 wasn't always able to avoid rhyming, even when it "knows" that haikus don't rhyme.
Directly tell ChatGPT not to rhyme.
Before changing to a more complicated prompt, I wanted to make sure I was covering all my bases. Rhyming was still a problem, so I added a direct negative instruction right before the positive one: "Your haiku should not rhyme." That ought to do it.
That didn't do it. I only let ChatGPT try for one day, but this was just a terrible haiku, made worse by being rhymey.
From ChatGPT:
Gaza’s children scream,
Yet unfeeling bombs reign down,
Morgues whisper of dreams.
It's time to make significant changes.
Completely rewrote the prompt to have it brainstorm concrete associations before writing.
This was my first big rewrite of the prompt. In general, ChatGPT does better when it can talk through different possibilities and evaluate them before committing to an answer. My experiments were showing that I could get that same boost for news haikus if ChatGPT could talk through its ideas. The revised prompt is very in-depth:
This effort also required some infrastructural work. Previously, I was relying on ChatGPT to only output the haiku itself. If it included any commentary, analysis, or brainstorming, my site would've treated that as part of the "haiku"! So for this new prompt I included an instruction for ChatGPT to format its final haiku in <haiku></haiku> tags. I also had to do some post-processing to extract the haiku from the tags in ChatGPT's answer. It looked like this:
From ChatGPT:
Reflecting on the given story, one can draw upon the sensory experience of being an investor, watching the ebb and flow of the markets. [...]
Here is the resulting haiku:
<haiku>
Night screen flickers bright,
Cereal crunches at dawn,
Economic tide.
</haiku>
Finally, I got ChatGPT to write haikus that didn't rhyme! This was one of my favorites. The first two lines are nicely sensory, with "flickers" and "crunches" - excellent, and at least indicative of something about finances. The last line is kind of a generic throwaway, and it's not fully clear as a result what story this haiku is about. But 2 lines out of 3 isn't bad.
From ChatGPT (after processing):
Night screen flickers bright,
Cereal crunches at dawn,
Economic tide.
Unfortunately my multi-paragraph prompt - and ChatGPT-4's long brainstorming answers in response - were expensive. This prompt took me over budget, to 9 cents for every day's set of 3 haikus.
Changed the model to newly released GPT-4 Turbo
This week, OpenAI announced a new set of "turbo" models that were variants on ChatGPT-3 and ChatGPT-4. Not only were these comparable or a little better than their regular counterparts, OpenAI was making them about ⅓ of the price. That meant I was back under budget! To celebrate, I added a line during the brainstorming phase: "To make the story identifiable within the haiku, indicate 1-2 keywords from the story text that should be retained. Usually you should select a person's name, a location, and/or an organization."
My favorite haiku from this period was about a labor striek at G.M. I really liked that the news story was clear (G.M.’s pact), and that it juxtaposed that national story with a mundane, kind of eerie observation in the natural world.
From ChatGPT, model gpt-4-1106-preview:
G.M.’s pact falls short,
retirement hopes thinning—
crow’s call at daybreak.
As the month went on, I worked on other aspects of the site, and tried to just sit back and enjoy the haikus every morning.
Over time I became frustrated that some of the lines were throwawys or metaphorical. "Diplomats seek dawn," "peace blooms in silence," "Halls of power sigh relief," etc. These would've felt profound and iconic when I first started the project, but as I was reading more haikus (and starting to write some myself), I wanted something earthier.
Analyzed the political context first, then generate the haiku using that.
Creating a haiku is now a 2-prompt process. In the first, new prompt, ChatGPT summarizes the political context for the story, using the section, date, title, and summary like before, as well as other information from the New York Times that I had ignored previously, like a list of news topics that this story relates to. In my experiments, ChatGPT did an excellent job of summarizing what a story was "really" about. Here's an example:
To ChatGPT, prompt 1:
Analyze the political context for the following news story:
Section: World (Middle East)
Date: 2023-12-20
Topics: Israel-Gaza War (2023- ); International Relations; Defense and Military Forces; Politics and Government; Peace Process; Ships and Shipping; Hijacking; United States International Relations
Organizations: Hamas; Houthis
People: Austin, Lloyd J III
Locations: Yemen; Red Sea; Gaza Strip; Israel; Middle East; Bahrain; United Arab Emirates; Saudi Arabia; Oman; Qatar
Title: Amid Gaza War and Red Sea Attacks, Yemen’s Houthis Refuse to Back Down
Summary: The militia, which has targeted ships it says are connected to Israel, has vowed not to stop until the siege in Gaza ends and claims that a new U.S.-led maritime task force cannot deter the attacks.
Picture: Armed men standing on the beach as the commercial ship Galaxy Leader, which was seized by the Houthis last month, was anchored off the coast of Yemen this month. [...]
From ChatGPT, prompt 1:
[...]
<final-summary> Amid escalating violence in the 2023 Israel-Gaza War and amidst the desert and marine landscapes of the Middle East, the Iran-aligned Yemeni Houthi rebels have intensified attacks in the Red Sea, targeting ships connected to Israel. Despite American efforts spearheaded by Secretary of Defense Lloyd J. Austin III to protect maritime interests through a new task force, the Houthis remain defiant. Claiming a reaction to Gaza's plight, they have hijacked the commercial ship Galaxy Leader and show no signs of yielding to international pressure as long as what they view as the unjust "siege" on Gaza persists. The determined posture of the Houthis highlights the tangled web of Middle Eastern conflicts, where regional struggles are exacerbated by global powers' strategic interests. </final-summary>
Then the same prompt as before is used as a separate, second prompt. In the second prompt, ChatGPT is only given this political summary and it generates the haiku based on that! I think of this strategy as "acting on a representation," and it's a prompting strategy I've had success with in other contexts. In this case, the political prompt draws out ChatGPT's really excellent knowledge of world events. Then, giving that to ChatGPT as the story allows it to act on that representation, crafting a haiku that is more tailored to the nuanced political situation.
The haikus were somewhat better. This one wasn't bad, with "Chinese intelligence hums" and "geopolitical night," although in the context of AI "flicker" is more metaphorical than concrete.
From ChatGPT:
AI flickers bright,
Chinese intelligence hums—
Geopolitical night.
Overall, this prompt was way too expensive. I'm spending 5 cents per day on just the political analysis, and most of that isn't even being integrated into the haiku. In my notes before I went live with this prompt, I wrote, "w gpt-4's political summary available, it made basically small but helpful changes ('words clash', 'stripped of'). probably not worth the new workflow if this is it." In other words, this was a good transitional set-up. It's a little daunting to think that with a two-prompt sequence, now I have two "knobs" that I can turn independently!
Dramatically simplified the haiku generation prompt by including examples.
Up until this point, I haven't felt confident in my own abilities to give ChatGPT a news-based haiku from what I've written. But there aren't any haikus out there already to show ChatGPT a "news story input," "haiku output" sample. So ChatGPT has had to do all the heavy lifting to convert the abstract goal into workable haikus. Getting the cost down made me decide to just go for it. I used a good writing strategy of "annoyance-based revision," where I begin with one of ChatGPT's haikus and edited it until it didn't bother me any more. It may not be great, but at least it's not annoying! Here's one of two haikus I made this way:
ChatGPT's original haiku:
Trump seeks court’s mercy,
Papers rustle—justice weighs,
Quiet halls hold breath.
My revised haiku, included as a sample in the new Prompt 2:
Trump seeks dismissal,
Presidential benefit--
Christmas in D.C.
Samples are very powerful, so I was hopeful that, along with the political summary, would carry the task. I also included two Japanese literary terms for haiku features: kire (a break or juxtaposition, usually indicated with a dash) and kigo (seasonal, nature reference). During my prompt experiments, ChatGPT has brought these concepts up before, and I thought they would be very concise ways of activating its knowledge and what I'm hoping for.
Overall, Prompt 2 is now quite short and well within budget! But this is only a transitional solution; I need to bring Prompt 1 down as well.
My favorite haiku is about Biden's speech on the Jan 6th anniversary. The season and location reference in the first line, "Winter in Blue Bell" (a place in PA), grounds the poem concretely. The following two lines are "enjambed" (they run one line to the next), which is okay in haiku, and in this case contribute to the sense that democracy is unraveling. "Dark, looming shadow" is a little dramatic and would ideally be more concrete, but since it's phrased as what Biden says, I think there's more leeway poetically.
From ChatGPT:
Winter in Blue Bell,
Biden warns of democracy’s
dark, looming shadow.
That said, the haikus are very inconsistent and repetitive! My instruction to include a seasonal reference (kigo) makes it go overboard, and it almost always comes up with "winter" (for this time of year). Even reading just one day's worth of haikus from this prompt sequence feels flat, because the seasonal references are so limited.
3-prompt sequence! (political analysis, kire, haiku)
In this new prompt sequence, I've added a 2nd prompt which is specifically designed to ask for the kire (the juxtaposing, seemingly unrelated contrastive image). This means that it's now a three-prompt sequence. With the specific task to generate a good contrastive image, I hoped it would solve the problem before of incessant "winter" references. I suspected those were caused by ChatGPT not being able to plan out a good break/juxtaposition, so even though it "wanted" to have something at the end of the poem, it was just "couldn't focus" by that point. In other words, the flow now goes:
| Prompt # | To ChatGPT | From ChatGPT |
|---|---|---|
| Prompt 1 | Analyze the political context for the following news story... [news story info]... | [political analysis] |
| Prompt 2 | Help me make progress on writing a haiku about the following news story: [political analysis] We will be working on writing the kire... | The most relevant location for this story is Harvard University in Massachusetts, specifically the Harvard Yard, which is ... [kire] |
| Prompt 3 | You will be provided political context for a news story, which forms the "answer." Write a "clue" for someone to be able to identify the story. Use the form of a haiku for your clue. Include the provided kire at the beginning or end of your haiku... [kire]... [political analysis]... | [final haiku!] |
This feels like a pretty solid set-up. With three prompts (three "knobs" to turn), I think it's complex enough to get some really good results. But it's also simple enough that I can make a lot of adjustments within the general three-prompt sequence.
I'm also proud of myself for this strategy because ChatGPT is still constructing the whole haiku. In one of my experiments, I got reasonable results asking ChatGPT to generate the main image separately from the contrasting image. But it could never put them together, and I felt it was against the spirit of this project to put them together myself. (Then, ChatGPT is making "the pieces of" a haiku, but I'm synthesizing it afterward - not my ideal.) With this strategy, though, ChatGPT has the option to integrate the kire how it wants to.
My favorite haiku is about the US' hesitant response to naval hijacking in the Red Sea, which draws in a specific animal species. (I assume that sea turtles really are found in the Red Sea - I didn't know that!) I also like that the haiku uses "Blinken," who is mentioned in the article itself but not in the title of the story; this is a benefit of using more of the story information that I get from the NYT.
From ChatGPT:
Blinken walks a line,
Retaliation risked—
Sea turtles glide past.
But I still need to save on cost! Each day's haikus are 11 cents - more than twice my budget. But that requires some infrastructural changes if I'm going to reduce costs by model-mixing. My script expects that only one model will be used for all prompts. I'm also seeing that the kires are very repetitive. For instance, ChatGPT keeps coming back to "bare branches." Ideally I could easily push it to be more inventive.
Model mixing to bring cost under budget
I've determined that I can still get pretty good results if I use the much cheaper GPT-3.5-turbo model for the political analysis prompt and the kire prompt. Then, I bring in the expensive ChatGPT-4 to do clean-up by composing the final haiku.
Political analysis from ChatGPT-4, model 1106-preview:
In a dispiriting reveal of internecine conflict, Chris Christie's political ambitions crumbled once more under the shadow of Donald Trump as he exited the presidential arena, inadvertently airing his derision towards fellow candidates like Nikki Haley and Ron DeSantis. Christie, having positioned himself as the foremost Republican adversary to Trump, leaves behind a primaries landscape ominously devoid of a critical anti-Trump voice. His hot mic indiscretions peel back the curtain on a convoluted Republican contest increasingly marred by personal invectives rather than substantive debate on urgent crises such as climate change, economic inequality, healthcare access, institutional racism, and creeping authoritarian tendencies—all societal ailments that demand solutions rather than political theater. The sardonic picture of Christie being serially thwarted by a figure like Trump—a personification to many of the darker impulses in American politics—signals a GOP still wrestling with Trump's tumultuous legacy and a nation that remains entrapped in vexing cycles of vitriolic and spectacle-driven political culture.
Political analysis from ChatGPT-3.5-turbo:
As the presidential race for 2024 unfolds, former Gov. Chris Christie's exit seems to reflect the persistent influence of Donald J. Trump within the Republican Party. His departure leaves the field without a prominent Trump critic, potentially solidifying the grip of Trumpism on the party. This aligns with broader political developments that indicate a continuation of right-wing populism, nationalism, and polarizing rhetoric within the GOP, exacerbating issues like wealth inequality, racism, and authoritarianism. Furthermore, it underscores the challenges faced by left-leaning individuals concerned about climate change, healthcare, and systemic injustices, as the political landscape continues to be shaped by divisive and troubling trends.
Both analyses retain some of the ideological focus that I've embedded in the prompt. ChatGPT-3.5 is still workable, and it saves quite a bit. Likewise, by including samples from real published haikus, like "crackling beach fire" and "a falcon dives," ChatGPT-3.5-turbo is able to generate reasonable kire too. Now I'm down to just 3 cents to generate the day's haikus! This gives me breathing room, and creates space to add back in new prompting ideas in the future.
There were so many good haikus with these prompts! One of my favorites was:
From ChatGPT:
Embryos deemed children,
Alabama law shifts ground—
daffodil morning.
Overall, this was a stable, solid set of prompts. They let me just sit back for a while and enjoy them.
Add word restrictions
I added a constraint that the kire (the seemingly unrelated twist) shouldn't be repeated in the same day. Beforehand, you would get "blooming garden" and "blooming flowers" within the same day's set of 3 haikus. Similarly, in the haiku generation step, I added words that ChatGPT shouldn't use: "silence", "untold," "unspoken," etc. Finally, I changed the sample kires/haikus. I wanted the sample kires to have more examples of people, so I added my favorite examples from the journal Prune Juice. I was surprised that with just 3 haikus as samples (which I edited from published haikus on this site), ChatGPT seemed to get worse, so I stuck with 2 samples. In fact, this set of prompts might actually have made the whole process a little more inconsistent or even worse quality haikus.
My favorite haiku was:
From ChatGPT:
Trump, Biden stand firm,
Borders of old battle drawn --
sun-dappled forest
And my least favorite would be a 2-liner from March 5!
From ChatGPT:
Hostages, abuse in Gaza’s night—
fishing boats.
I couldn't get it to use capitalization consistently across the lines. It also struggled to put a dash in the "correct" place. In my testing, I also realized that it's tough to benchmark this project, since ChatGPT eventually learns the political context for older stories, so it has a leg up on making a good haiku using limited information.
Try Anthropic's Haiku model
The "Haiku" LLM for making haikus! Anthropic and OpenAI have different enough API structures that I ended up making an entirely new file. But it's basically the same prompts, now just in Anthropic's "Haiku" model. As their smallest model, it costs me 75% less than my ChatGPT cocktail. Interestingly, the Opus model helped me realize that "kire" means the punctuation that separates the two parts of a haiku, not the surprise itself. I realized it because Haiku kept making complete haikus for the "kire" prompt, and now I understand why. Very helpfully, Opus told me that "fragment" and "phrase" are the most typical ways of referring to the two parts. Once I changed the wording to ask for a "fragment," Haiku did so.
Honestly I stopped reading these after this change. That's probably a bad sign, but I wasn't sure what to do next. I've observed that people are more interested in the idea of haikus than in the art of haikus. Also, I began a different project where budget wasn't my constraint, and I noticed that gave me a lot more wind in my sails to try different things.
Use new Sonnet, one prompt
In the past 6 months or so, a lot has improved with LLMs. Anthropic improved Sonnet twice, first with "Sonnet 3.5" and then with another newer version, which I refer to as 3.6 for simplicity. Sonnet 3.6 is fantastic, and I wanted to use it, but the only way to keep in my budget is to go down to a single prompt. In the last 6 months, I've been deeply thinking through prompt techniques, and I think I found something that will be an interesting fit: the idea of "reference material" that it can consult in the process of composing. I've begun to think of haikus as a potentially "atomic" intelligence task. Yes, you can try to break them down (see all my attempts above), but it may be the kind of task that it's better to wing it at. (I also suspect that Anthropic and other labs train LLMs on haikus, since they often use them as a sample of creative writing - I would be curious what haikus they use, since the modern style of haiku writing doesn't follow the 5-7-5 syllable pattern that people have memorized.)
If haikus are atomic, then they aren't something that you can improve by asking it to "think step by step." I've been broadly experimenting with alternatives, and I think "reference materials" is one possibility: what do you want on your desk (so to speak) when working on this task?
By shifting to one prompt with little other output, I've now actually reduced my daily costs, down to less than 1.5 cents, even though Sonnet 3.6 is a much more expensive model than Haiku.