Welcome to PEFT-Factory documentation!

PEFT-Factory is a unified framework for efficient fine-tuning of over 100 language models. It provides a simple and consistent interface.

Check out the Usage section for further information, including how to Installation the project.

Parameter-Efficient Fine-Tuning made easy

PEFT-Factory is a fork of LLaMa-Factory ❤️, upgraded with easy to use PEFT interface, support for HuggingFace PEFT methods and datasets for benchmarking PEFT.

Supported PEFT methods

PEFT methods table

PEFT method name	Support
LoRA (including variants)	✅ 🦙
OFT	✅ 🦙
Prefix Tuning	✅ 🤗
Prompt Tuning	✅ 🤗
P-Tuning	✅ 🤗
P-Tuning v2	✅ 🤗
MPT	✅ 🤗
IA3	✅ 🤗
LNTuning	✅ 🤗
Bottleneck Adapter	✅ 🤖
Parallel Adapter	✅ 🤖
SeqBottleneck Adapter	✅ 🤖
SVFT	✅ ⚙️
BitFit	✅ ⚙️

Note

This project is under active development.

Contents

• Home
• Usage
  • Quickstart
    • Create some variables for envsubst
    • Use the template
    • Run the factory
  • Installation
    • Using pip
    • From Source
      • Clone the repository
      • Build the wheel package
      • Install with pip
  • Get data and methods
  • Run training
    • From Command Line
    • Using web UI
• Contributing
  • PEFT Providers Support
    • Fork the repository
    • Create new branch
    • Update the constants file
    • Make a pull request
  • Adding PEFT Methods
    • Templates
      • config.py
      • model.py
  • Adding Datasets