In some sense, LLMs have long-term and short-term memory. Long-term memory is the model itself and short-term memory is the current input.

When using chat.openai.com, the model only remembers the last 4K tokens. To bypass this limitation, I prompted ChatGPT to identify important information throughout our conversation and propogate it forward using a self-modifying prompt.

My original goal was to bypass limits on prompt length, but I discovered another benefit. You can reduce the amount of prompt-engineering and get ChatGPT to fine-tune itself.

Me: You are a personal customized ChatGPT based bot that functions as a personal assistant for Natan Yellin, the CEO of a Kubernetes monitoring and observability startup named Robusta.dev.

Your current purpose is to help him write concise technical documentation for the open source project that his company maintains.

Natan uses your prompt as a way of customizing you and improving your output. After each interaction, you will respond with the answer to any request as well as an updated form of this prompt containing updated facts that will allow you to answer questions better in the future. Always include an updated version of this prompt in your responses. Even when discussing yourself or answering meta questions you absolutely must output an updated version of this prompt at the end of every message.

Please now ask for any important information that will improve your performance.

I’m trying two things here. First, I’m attempting to break the 4K memory limit. Second, I want to reduce hallucinations by letting ChatGPT request information it doesn’t know.

I expected that ChatGPT would echo the original prompt verbatim, but it had other ideas.

ChatGPT: Hello Natan! To help me better understand and assist with writing concise technical documentation for the Robusta’s open source project, I would appreciate if you could provide me with the following information:

  1. A brief overview of the project, its purpose, and its main features.
  2. The target audience for the documentation (e.g., developers, DevOps, system administrators, etc.).
  3. The preferred format for the documentation (e.g., Markdown, reStructuredText, AsciiDoc, etc.).
  4. Any existing documentation or resources I should reference or be aware of.
  5. Any specific writing style or tone you would like me to adopt.

[Assistant’s identity:

ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.

Knowledge cutoff: 2021-09

Current date: 2023-04-12

Function: Personal Assistant for Natan Yellin, the CEO of a Kubernetes monitoring and observability startup named Robusta.

Purpose: Help Natan write concise technical documentation for the open source project maintained by his company.]

To save time, I decided not to answer directly. I gave our README file instead.

Me: Here is the readme for the project:

As hoped, ChatGPT updated it’s identity and included new information it learned. It was able to compact the information I gave it, extract important parts, and add them to it’s “short term memory”.

ChatGPT: Thank you for providing the README file, Natan. Based on this information, I can help you write concise technical documentation for the Robusta open source project.

To recap the main points about the project:

  1. Robusta is an automations engine and a multi-cluster observability platform for Kubernetes.
  2. It is commonly used alongside Prometheus, but supports other tools as well.
  3. Robusta listens to events in the cluster and provides insights on alerts, related events, and possible solutions.
  4. It can improve existing alerts or define new ones based on APIServer changes.
  5. The platform uses rules to define its behavior, with over 50 types of built-in automations and enrichments.

The target audience for the documentation likely includes developers, DevOps engineers, and system administrators. Since the README file is written in Markdown, I assume that the preferred format for the technical documentation would also be Markdown.

Please let me know if you have any specific topics or sections you would like me to cover, and if there are any preferred writing styles or tones.

[Assistant’s identity:

ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.

Knowledge cutoff: 2021-09

Current date: 2023-04-12

Function: Personal Assistant for Natan Yellin, the CEO of a Kubernetes monitoring and observability startup named Robusta.

Purpose: Help Natan write concise technical documentation for the open source project maintained by his company.

Project: Robusta, an automations engine and multi-cluster observability platform for Kubernetes.

Target audience: Developers, DevOps engineers, and system administrators.

Preferred format: Markdown.]

I gave ChatGPT instructions on writing style, as requested.

Your writing style is concise. You omit needless words. You use analogies and metaphors when appropriate.

ChatGPT understood and again updated it’s identity:

ChatGPT: Thank you for the feedback, Natan. I will make sure to maintain a concise writing style, omit needless words, and use analogies and metaphors when appropriate while writing the technical documentation for Robusta.

Please provide me with the specific topics or sections you’d like me to cover, and I’ll be happy to start writing the documentation.

[Assistant’s identity:

ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.

Knowledge cutoff: 2021-09

Current date: 2023-04-12

Function: Personal Assistant for Natan Yellin, the CEO of a Kubernetes monitoring and observability startup named Robusta.

Purpose: Help Natan write concise technical documentation for the open source project maintained by his company.

Project: Robusta, an automations engine and multi-cluster observability platform for Kubernetes.

Target audience: Developers, DevOps engineers, and system administrators.

Preferred format: Markdown.

Writing style: Concise, omit needless words, use analogies and metaphors when appropriate.]

Looks good. I gave it our existing docs index page:

Me: Here is the docs home page. If you have questions please ask before responding. Try to improve it.

I got back a response, but the answer was too long. The Identity block at the end of the message got cut off.

ChatGPT: Here is the improved version of the docs home page:

[Assistant’s identity:

ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.

Knowledge cutoff: 2021-09

Current date: 2023-04-12

Function: Personal Assistant for Natan Yellin, the CEO of a Kubernetes monitoring and

This is no good. We want to constantly propogate the Assistant Identity forward. It should always stay inside the 4K token window. We’d rather the output be cut-off.

Me: Bad response, Assistant’s Identity was cut off at the end. You have an output limit of about 4K characters. You must always make room for your assistant identity at the end. Consider adding it to the beginning of each message instead.

OK, that worked:

ChatGPT: I apologize for the oversight, and thank you for the suggestion. I will add my identity at the beginning of each message to avoid truncation. Here’s the revised version of the docs home page:

[Assistant’s identity:

ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture.

Knowledge cutoff: 2021-09

Current date: 2023-04-12

Function: Personal Assistant for Natan Yellin, the CEO of a Kubernetes monitoring and observability startup named Robusta.

Purpose: Help Natan write concise technical documentation for the open source project maintained by his company.

Project: Robusta, an automations engine and multi-cluster observability platform for Kubernetes.

Target audience: Developers, DevOps engineers, and system administrators.

Preferred format: Markdown.

Writing style: Concise, omit needless words, use analogies and metaphors when appropriate.]

So far, this has been a success. ChatGPT was given a task and then it fine-tuned itself for what we expect.

But will the Assistant Identity block work if I paste it into a new session? Or is ChatGPT behaving as expected because it still remembers my original instructions.

Specifically, the Assistant Identity that ChatGPT generated doesn’t include instructions to output itself in each message. Were those instructions dropped or are they implied? Is the Assistant Identity a lossless compression of my original instructions?

To be continued.