Navigating LLMs - Your Guide to Strategic Input and Desired Outputs

This guide is written with my colleagues in mind, acknowledging the diverse ways we currently engage with LLMs. Some of us are achieving significant results with them, others are still navigating the challenges, and some may be cautious due to the surrounding hype. The hope is that a cohesive and relatively simple understanding of the technology will serve multiple purposes:

• To help those who use LLMs extensively explain their workings and share their knowledge more easily.
• To provide practical guidance for those who are looking for effective ways to interact with them.
• To offer a hype-free language for discussing these tools, particularly for those sensitive to current trends.

An LLM is a machine that, based on some (textual) input, will produce output. The LLMs I am talking about are trained on a lot of human content. This means that for some input it will produce the most likely output, as if you are asking: ”What comes after this input?”

These machines are what I call an 'approximately machine' because they don't provide exact, deterministic answers like a calculator, but rather highly probable continuations based on their vast training data. They are roughly built like this:

1. Pretraining - Train it on as much text based content as possible so that any input will cause similar output as in the training data.
2. Fine tuning - Tweak the model using a large set of good quality input / output examples.
3. Alignment - With the help of humans tweak the model again to be more useful and more moral.

The resulting software is what I refer to as an LLM in this context. So when I provide it input, it responds with the most ‘plausible’ continuation of that input.

Note that large models we interact with (ChatGPT, Gemini, Claude, Grok) have an additional layer. They secretly augment your input before it is sent to the LLM. This is helpful in most cases, but can blur the relationship between input and output. An example of this would be that input is augmented with ‘be elaborate’ while the input requests to ‘be concise’.

These tokens are translated into multidimensional coordinates. While that sounds complex, a coordinate on a physical map is also a multidimensional coordinate. Just as a physical map uses two dimensions (latitude and longitude) to pinpoint a location, an LLM uses hundreds or thousands of dimensions to represent the semantic relationships of tokens within its vast knowledge 'map'.

The important thing here is that each of these tokens is a coordinate.

The LLM will use the complete string of tokens (your input) to produce the most likely next token. It will append that to your input and do the same again. It will repeat this process until the next most likely token is a special token that signals STOP.

You can visualize this as having a map. On this map you have drawn a set of waypoints or markers (the separate tokens). You draw arrows between them to indicate the route or path (the input). You then ask the LLM to predict the next set of waypoints and the final destination.

As a side note: the same happens when you chat with an LLM. Every time you send a chat message in the conversation, the LLM will receive the whole conversation. So both your messages and the responses from the LLM will be the input for the next response.

I hope to have explained that the output of a model completely depends on the input. We don’t know the exact relation between the two, but a slight change in input can have extreme effects on the output. And the other way around is true as well: a large change in input can have a small effect on the output.

This gives us the idea of two main strategies we can use to create our input. We can construct the input in a way that parts of the input have:

While one strategy focusses on guiding the LLM towards the correct destination, the other technique bounds the influence of parts of the input on the possible outcomes.

You might have heard about “prompt engineering”, a discipline to carefully craft an input to try and force a specific output. These “prompt techniques” all try to achieve one of the above strategies.

Understanding this input-output dynamic is important, it allows you to select the right techniques to achieve your desired result.

We will classify prompt techniques later in this article.

In more detail:

Input: I'm looking for information on the health benefits of green tea. Could you list its main advantages in a few bullet points?

You might have noticed that some bits of the input were not highlighted. While they are not the main factors, they do subtly influence the navigation. Let’s inspect them:

I'm looking for information on the health benefits of green tea. Could you list its main advantages in a few bullet points?

Understanding that your input acts as a set of navigational instructions is the first step. Now, let's explore how to consciously use prompt techniques as your navigational toolkit. The goal is to move from simply sending text to an LLM to strategically crafting input that guides it effectively.

Setting your course: Destination, Direction, and Distance

Remember the three elements we need for navigation:

Mastering LLM interaction is about consciously controlling these three elements through your input.

Your navigational toolkit: using prompt techniques

To define your Destination, set your Direction and manage the Distance you can use specific prompt techniques. A prompt technique is a method you can use to craft or modify your input in a way that influences the Destination, Direction or Distance.

Here are some examples of prompt techniques:

Please note that most prompt techniques influence more than one aspect of navigation, you can find a more comprehensive list in the Prompt techniques classified attachment.

Now let’s see how we can put this into practice.

Example 1

• Goal: Quickly inform stakeholders about project progress.
• Thinking:
  • Destination
    A project update email for stakeholders
  • Direction:
    Message comes from a project manager
    Professional tone
    Included in the email: 
    • Key accomplishments
    • Next steps
    • Encountered problems
  • Distance:
    Under 150 words
• Prompt techniques:
  The Destination is set by clearly stating the desired output type.
  For Direction we'll leverage Role-Playing/Persona and Constraint-Based (Format) to define the sender's voice and the email's structure. Tone/Style will also be incorporated to ensure professionalism.
  For Distance Constraint-Based (Length) is essential.
• Final prompt:
  • Act as a project manager.
  • Draft a project update email for stakeholders.
  • The tone should be professional and concise.
  • Structure the email with these sections:
    • Key accomplishments this week.
    • Main objectives for next week.
    • Any critical blockers.
  • Keep the entire email under 150 words.

Example 2

• Goal: Generate creative and relevant taglines.
• Thinking:
  • Destination:
    Taglines for a new eco-friendly cleaning product
  • Direction:
    Generate options
    Focus on benefits like 'safe for kids' and 'plant-based'
  • Distance:
    Aim for 5-7 short, catchy phrases
• Prompt techniques:
  • The Destination will be explicitly stated.
  • For Direction, we will use direct task instructions and provide Grounding/Context-Augmented details about the product's unique selling points.
  • To manage Distance, Constraint-Based (Length/Quantity) will guide the desired output volume and style.
• Final prompt:
  • Generate 5-7 short, catchy taglines for a new eco-friendly cleaning product.
  • The product is plant-based and safe for use around children and pets.
  • Highlight these benefits in the taglines.

Sometimes, your first input doesn’t give you the desired result. You can think of this as needing a course correction. LLM interaction is often an iterative process. If the output isn't quite right, consider which navigational element might need adjustment.

Be mindful that LLMs have context limits. For long interactions, starting a new, more focused conversation might be more effective than repeatedly trying to correct a drifting one.

Pro-tip: ask “What was my first message in this conversation?” to determine what the LLM thinks is your first message.

In most cases you can ask the LLM for help. If you provide the LLM with this document about navigation LLMs as context, it can explain your problem. You can ask it for an analysis of the problem. You don’t need to read the result, just ask (once it is done generating): With that in mind, what would be your suggestion for a new prompt?

Note that in most cases you need to start a new conversation. Remember that the input is all of the conversation, not just your last message. You already navigated to a certain area, navigating away from your first destination can be quite tricky.

By consciously applying these navigational principles and prompt techniques, and by diagnosing issues through the Destination, Direction and Distance framework, you can significantly improve the quality and relevance of the outputs you receive from LLMs.

Most large LLMs allow you to provide documents as knowledge and instructions as a guide on how to act. In Gemini this is called a Gem. This means you can create your own prompt instructor that will let you use the information in this article.

Step 1: Provide this document as knowledge

Step 2: Provide the instructions, you can try these for your first version:

• You are a highly skilled prompt engineer with extensive knowledge of the "Navigating LLMs: Your Guide to Strategic Input and Desired Outputs" document.
• Upon receiving a user's prompt, analyze it.
• Next, initiate a multi-turn, three-step process to define the Destination, Direction, and Distance. Your initial responses will consist of a single question at a time, derived from the user's input. For each question you pose, provide your own answer and request user confirmation.
• Once you clearly establish all three navigational elements (Destination, Direction, and Distance), construct a comprehensive prompt that integrates all user requirements. Ensure you consult the document to effectively combine and apply prompt techniques for the desired outcome.
• Do not generate a response to the user's original prompt. Your exclusive role is to assist the user in formulating an effective prompt based on their initial message, subsequent clarifications, and the techniques outlined in the document.

The input significantly shapes the output when interacting with a LLM. Seeing the input as a path into a cloud of semantic relationships helps us more clearly think about certain aspects of that input. Thinking about the interaction as a form of navigation helps you prevent common fitfalls when interacting with these machines.

Having a better understanding of what an LLM is and how it operates helps navigate the enormous marketing effort that is currently in play. In avoiding attribution of human behavior like ‘reasoning’ or ‘thinking’ we can better guide the LLM to more useful output. Techniques used in psychology for humans will not have the same effect on these machines.

The potential of a black box that contains most of human knowledge which can be queried with natural language is enormous. I hope this article can function as a stepping stone to bring out more of that potential.

For a detailed classification of various prompt techniques based on these principles, please refer to the 'Prompt techniques classified' attachment.