HateTinyLLM : Hate Speech Detection Utilizing Tiny Massive Language Fashions
Authors: Tanmay Sen, Ansuman Das, Mrinmay Sen
Summary: Hate speech encompasses verbal, written, or behavioral communication that targets derogatory or discriminatory language in opposition to people or teams based mostly on delicate traits. Automated hate speech detection performs an important position in curbing its propagation, particularly throughout social media platforms. Varied strategies, together with current developments in deep studying, have been devised to deal with this problem. On this research, we introduce HateTinyLLM, a novel framework based mostly on fine-tuned decoder-only tiny giant language fashions (tinyLLMs) for environment friendly hate speech detection. Our experimental findings exhibit that the fine-tuned HateTinyLLM outperforms the pretrained mixtral-7b mannequin by a big margin. We explored varied tiny LLMs, together with PY007/TinyLlama-1.1B-step-50K-105b, Microsoft/phi-2, and fb/opt-1.3b, and fine-tuned them utilizing LoRA and adapter strategies. Our observations point out that each one LoRA-based fine-tuned fashions achieved over 80% accuracy.
