Use role prompting with IBM watsonx and Granite

In this tutorial, we will follow step-by-step instructions to perform a prompt engineering technique called role prompting. We will use an IBM® Granite™ Model to assign personas for nuanced model outputs.

Role prompting is a prompt engineering technique that instructs an artificial intelligence (AI) model to take on a specific role or persona when generating a response. This technique can be used to guide the model's tone, style and behavior, which can lead to more engaging outputs.

Prompt engineering is about optimizing the model input, so it responds with appropriate, meaningful answers. Zero-shot and few-shot prompting are two popular techniques used to converse with large language models (LLMs). LLMs have a natural ability to perform natural language processing (NLP) tasks due to their ability to process and interpret human language. The language capabilities of AI models are valuable for tasks ranging from chatbot conversations and multiagent interactions to open-ended creative writing. 

Generative AI (gen AI) gets more personal when an LLM is instructed to act as a specific persona to fulfill a role’s specific needs. The AI’s responses can be more accurate and relevant when first prompted with an assigned role. AI models leverage huge datasets so an assigned role can be anything from a teacher, historical figure, salesperson or others, bound only by what one’s imagination can think of. This ability is what makes role prompting, also referred to as persona prompting, such a powerful technique. An AI model’s adaptability makes it a master of disguise, able to generate responses tailored to a user or system’s specific needs.

Role prompting can be used to give a chatbot a persona to better interact with users or an AI agent to better interact with other agents. If you’re familiar with prompt templates, you may have already seen role prompting in action. For example, many agentic frameworks use role-playing agents to complete tasks and collaborate effectively. ChatDev uses a role-prompting technique called a self-attention mechanism. This mechanism clearly defines the agent’s role that acts as a guideline for its generated outputs.

To follow this tutorial, you need an IBM Cloud® account to create a watsonx.ai™ project.

While you can choose from several tools, this tutorial walks you through how to set up an IBM account to use a Jupyter Notebook. Jupyter Notebooks are widely used within data science to combine code, text, images and data visualizations to formulate a well-formed analysis.

1. Log in to watsonx.ai Runtime using your IBM Cloud account.
2. Create a watsonx.ai project.

                 Take note of the project ID in project > Manage > General > Project ID.
                 You’ll need this ID for this tutorial.

 3.  Create a Jupyter Notebook.

This step opens a notebook environment where you can copy the code from this tutorial to perform zero-shot classification on your own. Alternatively, you can download this notebook to your local system and upload it to your watsonx.ai project as an asset. This Jupyter Notebook is available on GitHub.

In this step, you associate your project with the watsonx.ai service.

1. Create a watsonx.ai Runtime instance (choose the Lite plan, which is a free instance).

2. Generate an API Key in watsonx.ai.

3. Associate the watsonx.ai Runtime to the project that you created in watsonx.ai.

We'll need some libraries and modules for this tutorial. Make sure to import the following, and if they're not installed, a quick pip installation will resolve the problem.

Run the following to input and save your watsonx.ai Runtime API key and project ID:

Next, we'll set up IBM's Granite-3.1-8B-Instruct to perform role prompting.

To give a simple and fun example of role prompting, let’s ask the model to take on the persona of a famous figure, William Shakespeare. We will ask the model, with its newly assigned role, to rewrite some song lyrics in the style of Shakespeare’s famous sonnets.

The code block below sets up and defines the prompt that the model will respond to. Feel free to change the song to whatever you’d like but note that Granite was not trained on copyrighted content. If you pick a song that is not a part of the public domain, for example “Espresso” by Sabrina Carpenter, you’ll have to paste in the lyrics with the prompt as well.   
 
For a prompt that just works, this example will use the classic children’s lullaby “Twinkle, Twinkle, Little Star” by Jane Taylor (1806). Run the code block to define your user prompt along with some example input text.

Once the prompt is defined, we can run the next code block to allow the model to generate an output. Let's hope it's up to our Shakesperian standards.

Here is the model’s output with readable formatting:

"Twinkle, twinkle, little star, how I wonder what you are; 
Up above the world so high, like a diamond in the sky. 
In night's velvet cloak, you shine so bright, 
Guiding sailors through the darkest night. 
 
Your radiant beams, a celestial art, 
Illuminate the canvas of the heart. 
In silent whispers, secrets you share, 
Of galaxies far and constellations rare. 
 
Yet, mortal eyes can never grasp your form, 
Beyond our realm, in realms beyond the storm. 
O, star! O, star! Your light does pierce my soul, 
A cosmic dance, a timeless role. 
 
In dreams, I chase you, in slumber's hold, 
Your mystery, like an enchanted scroll. 
 
But wakefulness claims me, daylight steals your guise, 
Leaving behind but a memory of your surprise.
 
Thus, twinkling star, in slumber's peaceful reign, 
I bid you farewell, till morning brings again."

The model was able to produce a response to the role-based prompt to rewrite the song as a Shakespearian sonnet. The model was able to abide by the rules and style expected from its role prompt. *Note that because of the creative nature of the prompt, your output might look different than this example.

In this next example, we compare a straight-forward system prompt to a role-based system prompt. Let’s say a veterinarian’s office has recently implemented a virtual assistant on their webpage. To provide the best customer support, this office wants their pet owners to feel heard and supported even in their virtual interactions, a relatable goal to many businesses. A visitor might ask a question such as, “My pet cat has been sneezing a lot lately and is licking her paws what should I do?” In this scenario the model has not been assigned a role in its prompt. We’re just using the model out of the box with no input guidance.

The model responds accordingly with advice and information, however there isn’t a personal touch and isn’t much different from what you’d see on a search engine results page. The model’s output is raw and lacking creativity. This solution might be acceptable but it doesn’t set this veterinarian offices’ virtual assistant apart from the rest. Let’s try the same question again, this time assigning it a role as a “compassionate, professional and experienced veterinarian.”

The language in the model’s response is more humanized because it speaks to an emotional awareness of the context that the straight-forward system prompt lacked. The model was able to accomplish this tone while also providing a complete and relevant answer proving that it is a more nuanced response. This type of human interaction with artificial intelligence is a way to meet subjective expectations within organizations and applications.  

If you are a developer or business looking to add more personalization and meaningful interactions in your gen AI applications, consider understanding how role prompting can make an impact. Most modern language models are capable of role prompting. Some basic models might not grasp the nuances of the role or maintain consistency in their responses, while others might be fine-tuned to respond in a certain way. Foundation models such as IBM’s Granite series are trained on large amounts of enterprise-specific data that boosts the models’ ability to take on roles to produce tailored responses based on business needs.

Role prompting encourages the model to perform constantly given its expectations from its assigned persona. We performed a simple example by assigning the LLM with the role of a historical figure in our prompt to turn song lyrics into a sonnet. Next, we compared the output of a nonrole prompted model versus a role prompted model for chatbot responses. We concluded by addressing that the response provided by role prompting is more nuanced and supportive in its language, providing elevated customer care.