Llama-LoRa Project
Quality evaluation and results
- Fine-tuned on OpenOrca Open Llama 3B
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| hellaswag | 0 | acc | 0.4899 | ± | 0.0050 |
| acc_norm | 0.6506 | ± | 0.0048 |
- Fine-tuned on my custom dataset in russian language Open Llama 3B
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| hellaswag | 0 | acc | 0.4818 | ± | 0.0050 |
| acc_norm | 0.6377 | ± | 0.0048 |
- Fine-tuned on mixed dataset (one part – Orca, second part – custom dataset)
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| hellaswag | 0 | acc | 0.4618 | ± | 0.0050 |
| acc_norm | 0.6019 | ± | 0.0049 |
- Fine-tuned on “wmt19”, ‘ru-en’ dataset
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| hellaswag | 0 | acc | 0.4817 | ± | 0.0050 |
| acc_norm | 0.6362 | ± | 0.0048 |
