InstructLab opensource CLI v0.21.0

Getting started: https://github.com/instructlab

Tech pattern: https://gen-ai-platform.tech-patterns.techzone.ibm.com/use-cases/ai_engineering/instructlab-hands-on/

Product documentation: https://docs.instructlab.ai/

Usage and purpose:

This is the InstructLab opensource CLI that can be used to experiment and learn about synth data for model tuning. It should not be used for customer demos or POCs.

This also contains the watsonx.ai flows engine for connecting to https://watzen.ibm.stepzen.com/

InstructLab is a model-agnostic open source AI project that facilitates contributions to Large Language Models (LLMs).

Both Merlinite and Granite models are downloaded onto this RHEL 9.2 VSI from https://huggingface.co/instructlab

Recommended minimum 8/32, but optimal with 16/64 CPU/MEM

Access your resources:

Once you receive an email stating your reservation is ready, click on the reservation ID in your email to open the reservation. Additionally, you will receive an IBM Cloud Account invite as a separate email inviting you to the Cloud Account where your reservation is hosted.

Example Ready Email

Example Cloud Account Invitation

Once you have completed the above, navigate to your reservation, scroll to the bottom, and retrieve the SSH Key attached to the bottom of your reservation.

Example SSH Key Download

Once you have downloaded the key, just open two SSH terminal sessions, one to serve the model, and the other to chat:

ssh -i [key.pem] itzuser@[ipaddress] -p2223;

Note: You may get an error that the [key.pem] is "too open".

Please make [key.pem] more restrictive, i.e. "chmod 400 [key.pem] and try again.

#in first window

sudo bash;

cd /instructlab;

source venv/bin/activate;

ilab model serve --model-path /root/.cache/instructlab/models/granite-7b-lab-Q4_K_M.gguf

#or

ilab model serve --model-path /root/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf

#in second window

sudo bash;

cd /instructlab;

source venv/bin/activate;

ilab model chat

For watsonx.ai flows engine, run the CLI as the itzuser (not root).

Note: GPU requiring capabilities are not available.