[LLM] 1. Prompt Engineering Basics #1

 

This post heavily relies on this lecture:

개발자를 위한 ChatGPT 프롬프트 엔지니어링

Two types of LLMs

1. Base LLM
   • Predicts next word based on text training data
2. Instruction Tuned LLM
   • Tries to follow instructions
   • Fine-tune on instructions and good attempts at following those instructions
   • Often further refined using RLHF technique
     • RLHF: Reinforcement Learning with Human
   • Trained to be Helpful, Honest, and Harmless
   • Thus, likely to be less toxic than Base LLM
   • Recommended to be used for practical usages

Guidelines for Prompting

• First Principle: Write clear and specific instructions
  • Clear prompt doesn't mean short prompt
  • Detailed tactics:
    1. Use delimiters to clearly indicate distinct parts of the input
       • Delimiters can be anything like: ```, """, < >, <tag> </tag>, ---, etc
       • Delimiters can also avoid prompt injections
         • prompt injection: if a user is allowed to add some input into your prompt, they might give kind of conflicting instructions to the model that might make it follow the user's instructions rather than doing what you wanted it to do
         • In other words, model can successfully distinguish the input part and the instruction part to avoid confusions
    2.  Ask for structured output
       • for example: JSON, HTML
    3. Ask the model to check whether conditions are satisfied
       • Example: If the text does not contain a sequence of instructions, then simply write "No steps provided".
    4. "Few-shot" prompting
• Second Principle: Give the model time to THINK
  • Detailed tactics:
    1. Specify the steps required to complete a task
    2. Instruct the model to work out its own solution before rushing to a conclusion
       • When we simply provide sample answer and ask if that is correct, the model might skim read it and simply say that it is correct without fully thinking about the answer 
       • Therefore, we should first ask to draw its own solution first and then make the model compare its own and the sample answer

Model limitations: Hallucinations

• Even though large language models are exposed to a vast amount of knowledge during its training process, it has not perfectly memorized the information it have seen and so it doesn't know the boundary of its knowledge very well.
• Thus, those models are likely to make statements that sound plausible but are not true.
• How to reduce hallucinations:
  • First ask the model to find any relevant quotes from the text and then ask it to use those quotes to answer the question

 

---

If you need inspiration, don't do it.
 -Elon Musk-