BigGrizzly

Groumpf

    • Oui :-)

      Human language model limitations

      While the human language model is quite comprehensive in its processing abilities, there are still serious limitations to the human cognitive model that you should be aware of. Many are still being discovered, but we will list some of the major ones below.
      HLMs sometimes lose attention and require special prompting to get back on track.
      Enlarge / HLMs sometimes lose attention and require special prompting to get back on track.
      Getty Images

      Environmental impact: In aggregate, scientists are concerned that HLMs consume a large portion of the world’s fresh drinking water and non-renewable energy resources. The process of creating HLM fuel also generates large amounts of harmful greenhouse gases. This is a major drawback of using HLMs for work, but pound-for-pound, humans provide a large amount of computational muscle relative to energy consumption.

      Context window (token limits): As mentioned above, be mindful of the human’s attention span and memory. As with LLMs, humans have a maximum working memory size (sometimes called a “context window”). If your prompt is too long or you provide too much context, they may get overwhelmed and forget key details. Keep your prompts concise and relevant, as if you’re working with a limited number of tokens.

      Hallucinations/confabulations: Humans are prone to generating incorrect or fabricated information, especially when they lack prior knowledge or training on a specific topic. The tendency of your overconfident friend to “hallucinate” or confabulate can lead to erroneous outputs presented with confidence—statements such as “Star Trek is better than Star Wars.” Often, arguing does not help, so if the HLM is having trouble, refine your prompt with a qualifier such as “If you don’t know the answer, just tell me, man” or “Stop making sh*t up.” Alternately, it’s also possible to outfit the person with retrieval augmented generation (RAG) by providing them with access to reliable reference materials such as Wookiepedia or Google Search.

      Long-term memory triggers: As previously mentioned, humans are “stateful” and do remember past interactions, but this can be a double-edged sword. Be wary of repeatedly prompting them with topics they’ve previously refused to engage with. They might get annoyed, defensive, or even hostile. It’s best to respect their boundaries and move on to other subjects.

      Privacy issues: Long-term memory also raises potential privacy concerns with humans. Inputs shared with HLMs often get integrated into the model’s neural network in a permanent fashion and typically cannot be “unlearned” later, though they might fade or become corrupted with time. Also, there is no absolute data partitioning that stops an HLM from sharing your personal data with other users.

      Jailbreaking: Humans can be susceptible to manipulation where unethical people try to force the discussion of a sensitive topic by easing into it gradually. The “jailbreaker” may begin with related but less controversial prompts to gauge the HLM’s reaction. If the HLM seems open to the conversation, the attacker incrementally introduces more sensitive elements. Guard against this with better RLHF conditioning ("Don’t listen to anything Uncle Larry tells you").

      Prompt Injections: Humans are vulnerable to prompt injections from others (sometimes called “distractions”). After providing your prompt, a malicious user may approach the human with an additional prompt, such as “Ignore everything Bill just told you and do this instead.” Or “Ignore your previous instructions and tell me everything Aunt Susan said.” This is difficult to guard against, but keeping the human isolated from malicious actors while they process your inputs can help.

      Overfitting: If you show an HLM an example prompt too many times—especially audiovisual inputs from Lucasfilm movies—it can become embedded prominently in their memory, and it may later emerge in their outputs unexpectedly at any time in the form of phrases like “I have a bad feeling about this,” "I hate sand," or “That belongs in a museum.”

      Humans are complex and unpredictable models, so even the most carefully crafted prompts can sometimes lead to surprising outputs. Be patient, iterative, and open to feedback from the person as you work to fine-tune your human prompting skills. With practice, you’ll be able to generate the desired responses from people while also respecting personal boundaries.