🌦️ Work with any model

The GenericModel class makes it possible to test and fine-tune the models which are not directly available via the BaseModel class. Apart from the base class, we can use classes mentioned below to load the models for memory-efficient computations:

Class Name	Description
GenericModel	Loads the normal version of the model
GenericInt8Model	Loads the model ready to fine-tune in INT8 precision
GenericLoraModel	Loads the model ready to fine-tune using LoRA technique
GenericLoraInt8Model	Loads the model ready to fine-tune using LoRA technique in INT8 precsion
GenericLoraKbitModel	Loads the model ready to fine-tune using LoRA technique in INT4 precision

To initialize the model, simply run the following 2 commands:

from xturing.models import GenericModel

model_path = 'aleksickx/llama-7b-hf'

model = GenericLoraModel(model_path)

The 'model_path' can be a locally saved model and/or any model available on the HuggingFace's Model Hub.

To fine-tune the model on a dataset, we will use the default configuration for the fine-tuning.

model.finetune(dataset=dataset)

In order to see how to load a pre-defined dataset, go here, and to see how to generate a dataset, refer this page.

Let's test our fine-tuned model, and make some inference.

output = model.generate(texts=["Why LLM models are becoming so important?"])

We can print the output variable to see the results.

Next, we need to save our fine-tuned model using the .save() method. We will send the path of the directory as parameter to the method to save the fine-tuned model.

model.save('/path/to/a/directory/')

We can also see our model(s) in action with a beautiful UI by launchung the playground locally.

from xturing.ui.playground import Playground

Playground().launch()