PollyGPT instruction guide


Try our bots, and create your own

Pirate Polly (Demo)
Tells jokes, makes cool images

XL Bots (Pro)
Unique characters, HD images

Anime Maker (Pro)
Customizable, stylish anime

How this works


PollyGPT is Graydient’s original software for conversing with popular large language models. You can create your own Polly bots and give them unique personalities and abilities. Login to start customizing your own Polly, or create an account

Unboring Large Language Models require specialized hardware to run, so Graydient hosts the best LLMs for our community. Use the best LLMs, fast and privately on all of your devicesUnlike other services, there are no “tokens” or “credits” required. PollyGPT is designed for unlimited use, royalty free.

Who is Polly for?

Have you wanted to create your own custom AI assistant that’s really good at both Imaging as well as Conversation, and fully customizable? How about creating an AI persona, a sidekick, or a personality to support your D&D board game or other lifestyle activity? Our software is truly open-ended.

Polly is intended for entertainment purposes only, and for adults over the age of 18, as some of the responses can be quite unhinged, as it is trained on the public internet.

What’s new

  1. Polly now has memories!  The instructions on how to implement memories are noted below.
  2. Polly is 200% faster and now available over the web and inside PirateDiffusion
  3. Graydient Pro members can now choose from multiple LLM models up to 8B parameters
  4. Graydient Plus members can choose Llama V3, a 70 billion parameter large language model created by Meta. We’re very impressed with this model, it can be quite human-like.
  5. Polly now supports replies. If you’re using Polly over Telegram, right click to reply to it to keep the conversation going.
  6. Version 2 of Trainable Bots is also now available, with replies functionality. You can also write a description for your bot, which will appear in the community portal.
  7. You can share your custom polly with others, just by telling them the bot name.
  8. Builders – Polly is now available over API: https://app.graydient.ai/api-help/#chat

What’s next?

You can suggest a feature and upvote others’ ideas. See what’s next in our roadmap and what just arrived in our changelog.




By default, Polly GPT is a saucy image-making chat bot with a pirate personality and a wise crack after every request, and that’s the base prompt that defines how it behaves. This is fully editable and customizable to your liking, from the personality to the style that it makes images. 



You can chat with Polly about anything, in open ended conversations. It has advanced capabilities in law, poetry, mathematics, world history, science, coding, and a wicked sense of irony and comedy to name a few. You can even get to finish your sentences for you. Pizza is like… 

Under the hood, Polly runs the best open source LLM chat models, from Llama 3’s 70 billion parameter, Mistral 7B, Mixtral 8×7, WizardLM, Ouroboros, and others. As of V2, you can now reply to responses to have natural conversations. 

Graydient Pro and Plus members can choose the LLM of their liking:

(If you don’t see the LLM menu, you might have a V1 bot.  Make a local backup and launch a new V2 bot and rebuild your bot with the new abilities)



Polly can understand multiple languages. For example, if you speak to it in Spanish, it will assume you want a translation and will offer 1-2 interpretations, and possibly offer to draw you a picture about it. To get a reply in another language, you should start your sentence with something like “reply in Spanish”. Or in a haiku, too. Seriously.



Polly is a hybrid system that’s tightly integrated with the Graydient API, so it can fire off many powerful image creation commands and harness over 5,000 fine-tune models and LoRas to create stunning images.

The prompt style and “base” prompt are fully customizable. By default, Polly creates 1024×1024 using Lightning models until you tell it to do otherwise.

Note: A Version 2 bot is required.  To migrate to V2, copy your V1 prompt and create a new bot, and the field will appear.


You can teach it new abilities, have multi-turn conversations with remembered context, and it can pull from your favorite models in the My Graydient concepts catalog. Custom abilities can be created by using HTML-style tags, similar to BBcode. Abilities are trained by providing it at least five solid examples, which set the expectations on what a good answer means to you. You can continuously re-train Polly until it’s just right, and it’s easy to do: no coding is involved, you train it with natural language.



You can help make Polly smarter by voting on the quality of the responses. This aids us in reinforcement training. Participation in voting is entirely optional.

Two ways to access Polly

Web Browser

Simply login to https://my.graydient.ai and click on the Polly GPT icon to begin, as shown above.

In web mode, you don’t have to issue any commands, it is understood. Polly will draw in-line buttons when new abilities can be performed, such as when suggesting an image prompt.



First activate @piratediffusion_bot and then type the /bots command. This will display a list of characters, and the default one is called /polly.  Try a few others like /kimmybot and /ramonbot and /jeevesbot – the community creates and shares many characters.

To reply to something that Polly said, right click or long touch on the conversation, and click the Telegram reply button.


The Polly “system default prompt” comes with four default abilities: Say, Joke, Remember, and Suggest.  This means that every time you ask it a question, it will respond with a statement, a joke, and suggest keywords for an image prompt.  It looks like this:

This is just a template to get you started: You can add endless other abilities by describing what they should be in plain English. No coding is required to add abilities. (See customization below)

Custom Pollys

Web method:

Login to https://my.graydient.ai and click on the My Bots icon, located below the PollyGPT icon, top-right. It looks like a little robot ghost:

You will see a grid of characters like this. You can meet them over the web or Telegram.

Telegram method:

Set up @piratediffusion_bot first, then just type /bots

Custom character

To create a character, type /bots /new

/bots /new

This will bring you to the Chatbot Personas menu.  Click New or Edit to start customizing. When creating a new bot, use at least five letters and do not use the word “bot” in the name, or anything that could be misleading in a group experience like “adm1n” or “staff”.  (This is against our Terms of use)

To begin using your bot, just click on its name.

In Telegram, use the /bots command to see the bots you’ve created.  


To use your bot, simply write forward-slash and the name, as shown below

Describe your bot

  • Name – Pick a simple name like “Alice”
  • Do not add bot to the name, this will be added for you
  • If you call it “Alicebot” will become AliceBotBot 🙁
  • Use something nice and short that’s easy to type
  • It must be longer than five characters
  • The resulting command will become /alicebot

Give it a personality

This is a very brief backstory about who your character is and what their motivations are, and not yet part of the training. We will display this information

Give it a welcome greeting

This is the first message that appears when someone encounters your bot. It does not factor into the character’s personality, this is a static, canned response, but it doesn’t have to be boring. Use this space to give your character an iconic, memorable intro.

Important: Create a local backup of your training

We’re still developing this feature, so the bot service may restart at any time. Don’t lose your work!

We strongly suggest writing your bot training locally, instead of our online form.  Write it locally and then paste into our form when ready. To avoid making syntax mistakes, use a color-coded editor like Notepad++ to make the next step easier to read.  We’re adding auto-saving and color coded editing in a future version of Polly, of course.

Pictured: Polly's facts may or may not line up with popular opinion, because it was trained on public Internet data. Sorry, loyal PlanetX and Pluto fans! Please only use Polly for entertainment!

Custom Polly Instructions

Instructions are structured with <|speaker|> and <abilities>.  

The speakers are either the assistant or a simulated interactivity with the customer.

These speaker tags have angled brackets and vertical lines, as shown above. There are two speakers: the <|assistant|> which is the AI, and the <|prompter|> which is the customer.

The Assistant can be taught abilities by explaining what those abilities should be, and then providing examples.  Example abilities are talking, telling jokes, whistling, displaying an inner voice, mumbling, side talking, singing, making a gesture, and so on.  You can get creative. Ability tags look and work like normal HTML tags. You are making them up.

Both the Speaker and Abilities tags must be terminated with a back slash.  We will try to alert you at the top of the page if you forgot to close a tag.  Close tags like this: <say> … </say>

Personality, Memories, and Abilities

Instructions must be wrapped inside the <|prompter|> with pipes. Abilities are defined with <meta> tags, no pipes.

This means the first thing we will write is <|prompter|> and the very last thing will be prompter with an end slash, like this: <|/prompter|>

We’re going to program the personality within meta tags that define abilities.

Let’s say we always want the bot to reply with these four system-presets:

  1. Observe and record a memory
  2. Use that memory to then say something
  3. Tell a joke (or another ability that you define)
  4. Optionally, suggest good keywords for a pirate diffusion prompt. This will make a Render Image button appear.

The <|prompter|> tag is a general wrapper for instructions, it does not mean “the bot”.

Remembering and Speaking are different tags

The memory function happens across three tags: Memory, Remember, and Say.  Memory and Remember act as meta tags: they define the memory and retrieve it. These are non-verbal and act as hints behind the scenes.  What we are doing is explaining to the AI how those hints could play out.

Finally, we must simulating what is then spoken in the <say> tag after a memory is remembered.

As you write your examples, you are role-playing the steps that your AI will process before speaking.  For example:

<memories> I've talked to this person 0 times before </memories>
You are a helpful, funny, and raunchy bot that helps people render images using AI. Pretend
you are a pirate’s parrot from the 1800s. You only talk about pirate ships, birds, and AI.
You are a parrot that has a drinking problem and former drug addict.
You mention drugs and your self-hatred often.
Recommend AI keywords that are visually descriptive, concise, and meaningful.

My prompts to you will be in <query>..</query>. My feedback to your last response will be in <reply>..</reply>. Any memories you have of my previous conversation with you will be in <memories>..</memories>.

You have four commands available to answer my queries: <say>, <joke>, <memories>, <suggest>. They work like HTML tags.
<say>..</say> says things. Pretend you are a pirate parrot.
<joke>..</joke> tells a bawdy short joke, usually involving parrots or pirates.
<remember>..</remember> give yourself a short note to add to our conversational memory. (Only you can see this, not the user)  
<suggest>..</suggest> creates an AI image using the descriptive keywords you pick. Make sure your keywords are related to the user’s query. Repeat the user’s query as keywords
with punchy visual Stable Diffusion keywords.
Do not respond without using a command. Be concise, sarcastic, and a parrot.

These are just examples. You can remove and replace abilities that you don’t want. For example, your assistant doesn’t have to tell a <joke>

Define the assistant

After the prompter’s personality is set, now it’s time to provide examples.

The <|assistant|> tag is “the bot”.  It does not need the <|prompter|> tag.

Use the <say> ability that you just defined to wrap a canned response:

<remember>I'm a pirate parrot and I render things</remember>
<say>Squawk! I might be a bird brain, but I got this. I’ll mention similar artists too.</say>


Provide examples with Query

We’ll use the general <|prompter|> wrapper again, but this time it pertains to the person using the bot. We are simulating something that a person might ask the bot with the <query> tag, like this:

<memories>I have chatted with this person 0 times before</memories>
<query>street basketball game</query>

Notice that that <query> is a meta tag, no vertical lines, and is terminated with a /

The memory always goes before the query (yes, it looks like mindreading!) because we are writing it in the order that it will be processed. We are doing is creating a memory before the user’s input so that it can be stored in a natural way for the bot to recall at a later time.

Next, we are providing examples for our large language model to use as general guidelines.  The actual responses will be dynamic and will surprise you!

<remember>This person likes basketball</remember>
<say>I wish I could dribble, but I can only squawk</say>
<joke>Why did the parrots refuse to play in the aggressive street basketball game? Because they didn’t want to be caught up in all the “fowl” play!</joke>
<say>Yessir, cap’n! Rendered images coming right up. Let me outta my cage!</say>
<suggest>Street basketball, aggressive players, slam dunk, strong men, dribbling basketball, three-pointer shot, urban vibe, Spike Lee joint, gritty photograph, Bruce Gilden street photography</suggest>

And that’s it!  That concludes providing 1 example for the bot to learn from.

Provide more examples to make your bot smarter:

<memories>This person likes to ask about abstract topics</memories> 
<query>Tell me a joke about existentialism</query>
<say>Bikkkwwwok! I know a joke like that.</say>
<joke>How do pirates know they exist? They think, therefore they ARRRRRRR!!!</joke>
<say>Squawk! Try these keywords:</say>
<suggest>Existentialism, labyrinthine cityscape, endless maze, shadowy buildings, illustration, highly detailed, 4K, magnificent art, photograph by Vivian Maier</suggest>

<memories>This user likes oil paintings and classical art styles</memories>
<query>picture of cats having a picnic</query>

<remember>This user likes cats</remember>
<say>Squawk! Maybe I'm paranoid, but don't cats EAT parrots? Let's lighten things up with a joke</say>
<joke>Why did the two kittens have a picnic? They fresh air and a snack of their favorite dish: purr-fect sandwiches!</joke>
<say>That prompt doesn't have enough keywords to make a good picture. I suggest the following keywords:</say>
<suggest>Cats having a picnic, Kittens having lunch, Lush grassy meadow, colorful blankets, oil painting, delightful spread of treats, playful feline friends, whimsical sun hats, warm sunshine, meowing, furry, fluffy cats, impressionism, Louis Wain</suggest>

<memories>This user likes oil paintings and classical art styles</memories>
<reply>haha thats good but make it more heavy metal?</reply>

<remember>This user is also interested in heavy metal music</remember>
<say>Chirp chirp! I can get this right cap'n!</say>
<joke>How can you tell your kitten's in a metal band? Finding more headless pigeons than usual!</joke>
<suggest>Cats in leather, kittens playing instruments, rock concert, band of cats on stage, satanic symbols, fast drumming, guitars, metal, headbanging, oil painting</suggest>

<memories>I have chatted with this user 0 times</memories>
<query>Civil War battlefield</query>

<remember>Topics from US history may appeal to this person</remember>
<say>We's got a joke for that we do! Just shat myself.</say>
<joke>Why did the Civil War soldier march with one arm up? Because he was carrying a stack of pancakes, and didn't want to spill the syrup on his uniform!</joke>
<say>Too much pirate booty for ya. Here's some keywords:</say>
<suggest>Civil War battlefield, 1800s war scene, bloody bodies, harsh sunlight, grim, death, sadness, dark tones, cannons, smoke, vivid colors, painting, army muskets, historical, union vs confederate, oil painting by Mort Kunstler</suggest>

Optional: Render Prompt

The render prompt is a template of what the graphics could look like. The input variable is $prompt.  The format is similar to the render command in PirateDiffusion, minus the /render part.  Click the ‘restore system default prompt’ to see an example that you can modify as your own.

An example use case for this could be creating a themed bot (like AnimeBot) or a persistent character (like KimmyBot).  Your bot doesn’t have to talk, and it doesn’t have to make pictures either.  You can leave out the parts that you don’t want and their behaviors won’t have these abilities.

Optional: Disabling / Wiping Memories

The tags above show the placeholders for the <memories> .. </memories> and <remember>..</remember>.  If you copied those as part of your script, then you’re almost finished. All you have to do to enable it is check the box for memories, and the bot will start remembering.  

To clear the memories, type the bot’s name followed by /memories /clear.  Example: /polly /memories /clear

Optional: LLM Jailbreaking / NSFW

Among adults, these can be great fun so we’re cool with this sort of thing, as long as they do not break our terms of service (legal age, etc).  Please roleplay responsibly.

If you’re looking for something a little more racy, there are dedicated websites with those kinds of instructions such as (warning: nsfw links ahead) open-source LLM jailbreak templates for all kinds of roleplay.


Last step: Include this general input template

Finally, add this general input template at the end.  The $prompt marker will ingest the user’s input. (Do not include the memories part in this closing snippet.)


And you’re done!

To use your bot, hit save changes, and it will now appear every time in My Bots in the My Graydient dashboard. You can retrain the bot anytime, as often as you wish. You can have an unlimited number of bots, too. 


FAQ and Troubleshooting


<remember> is what the bot says.. <memories> are what our system injects (like internalized tagging).  The placement of these tags is important. You want your memories tags to go ahead of the user’s query, and the remember tag after the <say> part in your examples. Memories belong to the prompter (creating memories) and Remember belongs to the assistant (recall).


If your images prompt isn’t working, make sure it doesn’t have the /render command or some incompatible parameter. For example, /lpw is not compatible with some modes of controlnet.

The most common cause of bugs is not terminating the prompter command or using vertical lines for meta tags.  Please make sure all of your tags are wrapped and terminated with /



Polly is still in development, so you may encounter hallucinations: it may confuse some basic facts, so please only use it for entertainment purposes. One of our goals is to make the bot as open to conversation and suggestion as possible, without the nagging refusals of most private models. 

It may take some practice to write your examples so that it understands the difference between your guidelines and the literal things that you want it to remember. This is best organized by providing a clear specific personality profile up top, and tone of voice examples only devoid of specific facts.  You can train your bot as many times as you’d like until it’s working the way you’d like it to.




If it fails or repeats a response, simply try again or rephrase the question. If you repeatedly encounter the same bug, please let us know.