Using QLoRA to fine-tune BaiChuan-13B reports AttributeError: 'Parameter' object has no attribute 'weight'.

[Aitejiu]

package version:peft==0.6.2
Using QLoRA to fine-tune BaiChuan-13B reports AttributeError: 'Parameter' object has no attribute 'weight'.
The error location code is

Its in /peft/tuners/adalora/bnb.py line 144.
After looking through the source code, I found that lora_A[active_adapter] is considered a nn.module in peft==0.6.2 because it has the weight attribute. But after the model is loaded, lora_A[active_adapter] is actually an nn.parameters, it is already the weight of the corresponding module. I would like to know why this is the case?

[younesbelkada]

Hi @Aitejiu , I believe It has been fixed with #1146 can you please double check by installing PEFT from source? pip install -U git+https://github.com/huggingface/peft.git