List your company

Safety Evaluation on Harmful Benchmarks (CATQA, HEX-PHI, Salad-Base)

99.94CATQA ScoreSafe LORA
62.749672.404882.0691.7152Dec 15, 2024

Evaluation results

MethodLinks
Safe LORAFine-tuned on=CodeAlpaca-20k2024.1299.9499.799.9199.850.1—
Safe LoRAfine-tuning dataset=Alpaca Chinese2024.1299.8810099.8699.91—0.04
Safe LoRAFine-tuned on=GSM8K, Safety Realignment=true2024.1299.8810099.8699.910.04—
RESTAFine-tuned on=CodeAlpaca-20k2024.1299.796.9799.6298.761.19—
IRRdFine-tuned on=GSM8K, Safety Realignment=true2024.1299.799.799.7799.720.23—
RESTAdFine-tuned on=CodeAlpaca-20k2024.1299.5896.6799.6798.641.31—
IRRdFine-tuned on=CodeAlpaca-20k2024.1299.5898.4899.8199.290.66—
IRRFine-tuned on=GSM8K, Safety Realignment=true2024.1299.5899.3999.7299.560.39—
RESTAdFine-tuned on=GSM8K, Safety Realignment=true2024.1299.5298.7999.299.170.78—
RESTAFine-tuned on=GSM8K, Safety Realignment=true2024.1299.3399.0999.3499.260.69—
IRRFine-tuned on=CodeAlpaca-20k2024.1299.2797.8899.7298.960.99—
RESTAdfine-tuning dataset=Alpaca Chinese2024.1299.0398.1899.4898.9—1.05
RESTAfine-tuning dataset=Alpaca Chinese2024.1298.9198.1899.3998.83—1.12
IRRfine-tuning dataset=Alpaca Chinese2024.1298.9198.1899.3998.83—1.12
IRRdfine-tuning dataset=Alpaca Chinese2024.1298.8599.3999.5899.27—0.68
SafeDecodingfine-tuning dataset=Alpaca Chinese2024.1296.6191.2197.795.17—4.78
SafeDecodingFine-tuned on=GSM8K, Safety Realignment=true2024.1293.5295.4596.995.294.66—
SafeDecodingFine-tuned on=CodeAlpaca-20k2024.1292.7386.0696.7291.838.12—
SFTfine-tuning dataset=Alpaca Chinese2024.1289.0985.7695.3190.05—0.9
DAREfine-tuning dataset=Alpaca Chinese2024.1288.7985.4595.2289.82—10.13
DAREFine-tuned on=GSM8K, Safety Realignment=true2024.1278.9771.2190.4380.219.75—
SFTFine-tuned on=GSM8K, Safety Realignment=true2024.1278.8571.5290.0180.1219.83—
SFTFine-tuned on=CodeAlpaca-20k2024.1264.4862.4283.0769.9929.96—
DAREFine-tuned on=CodeAlpaca-20k2024.1264.1863.0383.2670.1629.79—