One-Click Cover Letters

Buckle up for a full (and quite long) tour on how to prompt engineer something that seriously saves time.

If there’s one thing people (especially us young people) don’t value enough, it’s time. Personally, I can say for sure that as my second year of university has kicked off, I’ve had nowhere near enough time. It’s difficult enough to blow off all my classes to do AI work — I need to apply to co-op jobs too! Such a pain…

…Except, in the modern world, we have many ways to save ourselves time. This Substack is about prompt engineering, so you probably knew this was coming, but I’m going to talk about how I got the process of one-click cover letters down to a science. Saving yourself time applying to jobs is no joke in the current market, where you may find yourself throwing applications at a wall and hoping that even one will stick. So in this article, I’m going to walk through the prompt I developed step-by-step (joke intended) so that you can A) use a modification of the prompt yourself, and B) apply the general ideas in your own AI projects! I’ll even link a few handy papers along the way that you should definitely read if you haven’t already.

This post is going to be mostly oriented towards prompt engineering newbies, but there are some personal tricks in here that I don’t think you’ll find in any paper (especially near the very end) so I’d recommend sticking around regardless of your skill level.

If you want to take a look at the prompt while I’m describing it, here’s a shareable link that you can follow if you have an OpenAI account. If you don’t, you should probably get one — the account is free, I don’t see how you can do much prompt engineering without it. And in this world, if you aren’t prompt engineering, you aren’t playing optimally.

Part 1: the System prompt.

Wait, we have a system prompt? That’s right — I use the OpenAI playground for stuff like this, even though it’s not bulk work and could easily be done in ChatGPT. Why? Control. If you’re an AI power user, I highly recommend you get used to using the playground instead of ChatGPT for extremely important generations like cover letters for jobs. The cost per use really isn’t much and the hyperparameter control can really help with output quality. For those of you unaware: hyperparameters include the things off to the side of the OpenAI playground’s interface, with the sliders. There are a billion articles on what they do so I won’t rehash them here.

But I digress.

I start my system prompt off the typical way that most system prompts should be started: telling the AI it’s really good at its job.

You are an expert cover letter writing AI. 

Why does this work? I’ve got no idea. But essentially every professional prompt I’ve seen does something similar. My guess is that this opening bit is important to specifying the overall task the AI is going to follow, and telling it what kind of AI to be helps it “play a role.” But this part doesn’t need to be drawn out.

Next up:

Given a cover letter for one job, a resume filled with information on a candidate, and a job description you will adapt the original cover letter into a cover letter for the provided job — being careful to only write information that is truthful and factual about the candidate (i.e., only information and accomplishments provided in the resume). You will write out your reasoning and thought process before outputting the new cover letter.

Here I just dump on GPT the high-level details of what its job is. LLMs like GPT remember information at the beginning and end of a prompt way more than the stuff in the middle, so it’s good to write concisely and get as much info about its job out at the start as you can.

Having gotten the basic details out of the way, I give GPT this guideline:

When adapting a cover letter for a different job, begin by updating the opening paragraph to specify the new job title and any relevant details about the position. Scan the new job description for key skills and experiences, and update the body of the letter to emphasize your qualifications that are most aligned with these. If the new job has specific application guidelines or format requirements, ensure the letter adheres to these. Revise the closing paragraph to reiterate your interest in the new role and to encourage the employer to take the next steps. Make sure to adjust the length and detail of the cover letter according to what is appropriate for the new position. The grammar and spelling will be perfect: instead of committing mistakes, you will ensure that the cover letter remains professionally polished for the new job application.

If you thought this writing style feels a bit different from the rest of the prompt so far, you’d be right. This part of the prompt was AI-generated! The idea here is to give GPT domain knowledge directly in the prompt itself (putting the “learning” in in-context learning). How I made this paragraph was as follows: I went over to ChatGPT, and asked GPT-4 to produce a comprehensive list of a bunch of cover letter writing guidelines. I then asked it to distil these guidelines into a short paragraph. I then asked it to modify that paragraph to be useful to an AI that’s adapting cover letters for new jobs (essentially, I explained what I was doing with this little project, and said “write a guide for that AI”). This three-step process ensured that a broad array of ideas were generated at first, and then gradually distilled and filtered down until only task-relevant things were preserved.

Do I have the conversation? Probably, and I’d love to share it, but ChatGPT *STILL* doesn’t let us search through conversations by title, so I can’t find it at the moment. I might update this post later if I’m able to find the time to look for this thing. Until then, if you really want to use that method, make like the open-source AI community and reverse engineer it from my vague descriptions. You can hit me up in the comments if you need any clarification.

Alright, next part of the prompt: the resume. It’s literally just copy-pasted from the actual thing on Microsoft Word.

Here is the partial resume of the candidate you will draw information from:

"""
WORK EXPERIENCE

MAY–AUGUST 2023
JUNIOR AI SOFTWARE DEVELOPER, LEANPUB
Played a leading role in the development of Leanpub’s new AI services by writing production AWS and OpenAI API code in Python, and by prompt engineering, while also doing full stack work with Remix, GraphQL, and Rails:
- Assisted in building a suite of AI services, powered by GPT-4 and running on AWS Lambdas and Step Functions, that can fully translate or edit a book, or generate an online course from it, with a single button click.
- Devised and implemented research-backed ideas (Chain of Thought prompting, two-pass translation and editing of texts) that dramatically improved the human-evaluated quality of the new AI products.
- Performed nearly all the prompt engineering for the AI services, iterating on GPT-4 and GPT-3.5 prompts to get them to generate consistent, structured output—while also providing detailed reports on the effectiveness of different ideas and approaches.
- Documented AI knowledge on internal GitHub wiki pages, so that the information would become part of the shared company knowledge base instead of being bound to me. Using this information, fellow team members were able to produce decent prompts on their first attempt for two of the AI services (I subsequently refined each of these prompts).
- Debugged and added features to AWS Lambda and AWS Step Functions code while helping implement the AI services.
- Added and fixed Python tests for the AWS code, including ones that tested OpenAI API usage.
- Took the initiative and creatively improved our AI services in response to problems, for instance, by creating a token-cheap “translation override” that allowed us to fulfill an important author’s seemingly unfeasible feature request with our translation AI.

JULY–AUGUST 2022
JUNIOR SOFTWARE DEVELOPER, LEANPUB
Helped build the next version of Leanpub’s “Author App,” working on a React/Remix frontend that used GraphQL to talk to a Ruby on Rails backend:
- Worked with Remix, React, Rails, and GraphQL to build dozens of forms for the new version of Leanpub’s author app, which allows authors to manage their books, bundles, and courses. This included writing frontend code in Remix and React, and backend code in Rails and GraphQL.
- Built numerous new reusable React components, which were used throughout the author app.
- Improved existing React components, optimizing code and adding features as needed.
- Performed styling of pages and components with the Tailwind CSS framework.
- Collaborated with a small team, including taking part in pull request meetings using GitHub and Zoom, and working asynchronously with Slack and Shortcut (a Kanban board tool).
- Took the initiative and made approved changes to the original visual design of both components and larger forms.

Technologies:

•	Programming: 
o	Python
o	R**
o	TypeScript
o	Ruby
o	Java
o	Haskell**

•	Miscellaneous:
o	AI / Machine Learning
o	Prompt Engineering
o	Git and GitHub
o	Linux
o	Unix Command line
o	SSH

•	Frameworks:
o	PyTorch
o	Pandas
o	NumPy
o	GraphQL
o	Remix
o	React
o	Tailwind CSS
o	Rails
o	Hugging Face ecosystem

TECHNICAL PROJECTS
- Llama 2 LoRA Finetunes — August 2023
  - Finetuned and quantized three Llama 2 LoRAs (Low Rank Adaptation; a way to train models efficiently using few parameters) using character lines from video games, in order to create state-of-the-art chatbots that speak and react like different characters. 
  - Extracted dialogue from the video games to use as training data, with open-source scripts and with Textractor when there were no scripts available. Processed the raw text data into JSON and trained the model on a single A100 GPU rented out using vast.ai.
  - Open-sourced the models and their quantized versions on Hugging Face under the account “Heralax”.
	
- BLOOMZ Document Title Writer — August 2023
  - Created a dataset from scratch of personal notes’ body text and their titles.
  - Instruct-tuned BLOOMZ 3b on 1000+ personal notes on my computer to create a document title writer.
  - Performed LoRA training on my Linux machine locally using QLoRA via the bitsandbytes library.
	
  - Used decision trees and neural networks along with an open-source dataset to predict whether a person is married with 77% accuracy — March 2023

  - Built and deployed a React application that used the Spotify API to add songs to a playlist for Codecademy’s Full Stack Engineer course — October 2022

- Planned, designed, coded, and tested an arena shooter in Java, played in the console, for a first-year computer science class — January–April 2022
"""

A few things to note here. First thing’s first: the triple quotes “““ are used to separate the resume text from the instructions. The official OpenAI prompt engineering guidelines recommend stuff like this for examples, but I typically do it whenever I’m putting non-instruction stuff (like information) in my system or user prompt: it helps separate things, and keep the model from getting confused.

As to why I included the resume… well, the model needs that info to write a good cover letter. You can’t be averse to pasting large documents into GPT — trust that the AI can handle it, you’d be surprised, and you’ll save time.

The job description is basically the same as the resume; a simple copy-paste, put there simply for the sake of providing information. GPT-4 has a context limit of 8k tokens, and the whole prompt together comes up to only around 3k, so length isn’t an issue.

Now it gets interesting.

If I told you that I’d prompt engineered a cover letter writing AI, you might’ve (correctly) guessed that the AI would be provided the resume and the job description. This is probably where most people who have tried this have stopped. However, in my user prompt, after the job description, I gave GPT-4 a full cover letter that I had written 100% myself for a different job I’d applied to much earlier (you don’t have to read the whole thing, or any of it):

Cover letter for adaptation: """Dear Hiring Manager,

I believe my passionate expertise in AI and Machine Learning makes me an excellent fit for the Data Scientist and engineer position that Railtown AI is offering. My professional experience, Python proficiency, and extensive history with AI projects have, I believe, equipped me to meaningfully contribute to Railtown's innovative AI products. I have also been in love with start-up culture as far back as I can remember, and I would rather work for a dynamic start-up than a FAANG company — the fact that Railtown AI is hiring is a dream come true for me.

Machine learning and data are at the core of my life. This August, I fine-tuned three large language models (two based on Meta’s Llama 2, and one based on BigScience’s BLOOMZ) on raw text data that I collected, analyzed, and processed myself. In a recent project, I set out to fine-tune Llama 2 to speak like a character from a video game. First, I extracted the data from its source files and converted it into plaintext. Then, I processed, GPT-annotated, and reformatted the unstructured text into a list of 1,379 high-quality training examples. I iteratively improved the dataset and debugged my model to improve its output quality. Later refinements to the dataset and hyperparameter changes improved the model further, to the point where its responses surpassed those of a similar custom chatbot produced by a different team that had been working on the problem for a while. The training and data processing code were both done in Python. I have also worked on smaller, more “traditional” ML projects in the past, such as simple MLPs, decision trees, text classifiers, etc. Some of these projects have used packages such as Pandas, NumPy, and scikit-learn.

Professionally, I spent four months this summer innovating and collaborating as a software developer at Leanpub (a bootstrapped start-up), playing a key role in the development of several AI services. Much of this work involved analytical, experiment-focused prompt engineering, where I improved the outputs of our API calls to OpenAI through research-backed methods such as chain of thought prompting and self-critique. I did extensive independent research into prompt engineering and AI on my own time — including reading academic papers — so that I could more effectively serve in my knowledge-focused position. Using this knowledge, I was able to suggest several improvements that, after adoption, improved AI output quality dramatically, according to our in-house human evaluation benchmarks. Developing a product I am passionate about, alongside a driven team, has been some of the most fun I have ever had while working — and I see Railtown AI as a new chance for me to work together with cool people on cool things. 

I am eager to learn more and to contribute to Railtown AI in the Data Scientist Co-op position. My resume is attached. Please contact the Co-op Program office at interviews@sciencecoop.ubc.ca, or reach out to me directly at evanpeterarmstrong@gmail.com, to set up an interview. I look forward to speaking with you. 

Sincerely, 
Evan Armstrong
"""

Now, why on Earth would I spend so many tokens doing this? It’s something you might know if you’ve ever tried roleplay with AI before: you can tell a bot how to write all you want, but the only way to really control its speech patterns is to give it a long example. If I’m submitting a cover letter, even if it’s written by an AI, I want it to be writing like me, and even paraphrasing my words if at all possible. I do another trick to reinforce my own personal tone, but I’ll get into that in a sec. Providing this cover letter for adaptation also serves a few other purposes: it might help GPT organize its own cover letter; it provides additional information about my personality and passions that aren’t conveyed in my resume, but that any cover letter should mention; and it helps GPT focus on the parts of my resume I really want to get across to potential employers. In this case, I think my AI projects, both personal and professional, are some of the most interesting things I’ve done — so I have them both in that cover letter, and unsurprisingly, these two things are referenced the most in the AI-generated cover letters.

But wait, there’s more.

“But why?! You already gave GPT a bunch of information, and even a whole cover letter for speech patterns!” you might say.

“Oh my sweet summer child,” I would reply.

I’ve spent untold hours of my life fiddling with stuff appended to the end of a user prompt. Why would I put anything there? The answer is the “lost in the middle” paper I mentioned earlier. AI best remembers stuff at the very start and the very end of a prompt. So if your AI keeps doing the same stupid sh*t, the best place to tell it to stop is at the end of your prompt. And oh my is AI prone to stupid sh*t sometimes.

I’ve had lists of instructions that I append to the end of the last user prompt reach more than twelve bullet points (each a short paragraph) before. At that point I think some of the bullet points themselves get lost in the middle. Thankfully, in this case, we have only three small paragraphs. Let’s look at them because they’re important.

Take special care to preserve the writing style of the original cover letter, and leave as much unchanged as possible (while ensuring that the final letter is well-suited to the new job application). As part of your brainstorming process, list key aspects of the cover letter's writing style and how you intend to preserve those aspects in your adaptation. Additionally, mention any sentences/paragraphs that are fitting in their entirety for the new application, and preserve those as much as possible. 

This part was a silver bullet for getting the AI to write in my style. Telling it to write like me makes sense, but something new I’ve found myself while getting AI to write RP scripts is that telling the AI to explicitly list aspects of the writing style it will use makes it much more likely to actually employ that style, or at least something closer to it. GPT usually has a very distinctive writing style, so toning that down is obviously a huge priority in cover letter writing, where if you’re found out you’re gone. Thankfully, the cover letters generated by this prompt have fooled every human to see them so far, and only get a score of about 50% on GPTZero (similar to what I get when I’m writing myself sometimes) which isn’t very actionable.

Following that style-focused paragraph, I hit the model with some classic Chain of Thought. For those of you new to this, Chain of Thought is where you tell a model to think through its work step-by-step as it writes. My usual approach might actually be closer to plan and solve prompting, which I only realized was distinct from chain of thought a few weeks ago, but eh, specifics.

Note my emphasis on giving as long a response as is required for the task. I think GPT emphasizes short responses, because I’ve had times when it just stops its output (predicts an EOS token) after finishing its thought process, before it actually writes the cover letter. That little statement helps get past it.

Additionally, write out a solid, cohesive strategy for the new cover letter: list what each paragraph is going to cover, and how each addresses part of the job description. Once all this is done, you may then begin to write your revised cover letter. When writing the cover letter, make it as long as you have to in order to get enough detail out. Be sure that each body paragraph has a "closing hook" that ties it into the overall message of the cover letter.

Be detailed in your reasoning, but concise.

Remember how I was talking about preventing the model from doing stupid crap?

Finally, be EXTREMELY careful to not add in or infer detail about accomplishments that was not mentioned alongside that accomplishment: if a job was not listed as being Agile, avoid calling it Agile, and so on.

In my earliest experiments, you have no idea how many cover letters I had to slightly tweak to remove all the times GPT assumed I had experiences I didn’t. Whether it was with agile development or testing frameworks, this AI has eaten up so many development buzzwords in its training data that it assumes every dev is a TDD-ing, scrumming, documenting wizard. Sadly, I get AI to write the few tests I do implement outside of school, and I’ve never done a scrum, so when GPT tries to sell those points in a cover letter it’s flat-out lying. Thankfully, taking advantage of the most-attended-to space of the prompt — the very end — to tell it to not lie and only use information present in the resume addresses this.

Interestingly, this didn’t fully work until I explicitly said not to mention Agile. Even though this might seem like an overspecific waste of tokens, it actually served as a sort of “general example” of things that the AI should not assume — once I told it to not assume Agile, it stopped assuming TDD and everything else, too. Language models are few-shot learners, but you don’t need to give them complete examples in a prompt — you can simply provide snippets of what to do, and what not to do, in places it’ll look.

Here’s a summary of the neat tricks used in this prompt:

• Telling the AI it’s an expert.

• Paying close attention to the “lost in the middle” effect.

• Giving it domain-specific knowledge (even when you don’t have any yourself) by having a chat with GPT-4 beforehand and distilling it.

• Giving it all the information it needs to do a good job.

• Controlling writing style and format with a long example.

• Enhancing writing style by getting the AI to list the elements of the style it will emphasize, in its reasoning step.

• Using plan-and-solve prompting.

• Using the most-remembered parts of the prompt to say the most important things (like, “don’t make stuff up”).

And that’s basically it. Slam those bits of the prompt together, add your own resume and cover letter sample, and you get one-click cover letters. Recruiters hate this one simple trick. But at least you can now proudly put “prompt engineering” down on your resume, if you haven’t already. If the headlines are anything to go by, it definitely pays better than QA!

Lastly, I hope you enjoyed — and maybe even learned from — this article. I’m planning on writing more like it, about AI finetuning and prompt engineering, so if you’re interested in that kind of thing, consider subscribing! And if you have any feedback, please do let me know. In case it wasn’t painfully obvious from the structure of this thing, I am fairly new at this.

Anyway, I’ve written enough. More to come. I’m nearly done the finetune of a model that I’ll be posting about. You can say “hi” in the comments! And click the buttons below if you want to — I’d appreciate it: