pvaluedotone/bigbird-flight-2

🛫 Big Bird Flight 2

Big Bird Flight 2 is a fine-tuned version of Google’s BigBird model, optimised for long-text sentiment analysis. Big Bird Flight 2 is an improved version of Big Bird Flight 1. The model records a 16% improvement in accuracy over its predecessor. Both models were trained on 2,598 flight review texts, each annotated with a 10-point ordinal sentiment rating ranging from 1 (extremely negative) to 10 (extremely positive).

Just like its predecessor, Big Bird 2 captures emotional gradients in text, offering richer sentiment analysis than conventional binary classification (e.g., positive vs. negative). This makes it particularly useful for applications requiring fine-grained sentiment understanding from lengthy or detailed customer feedback.

• Use case: text classification
• Sentiment class: 1 (extremely negative) to 10 (extremely positive)

📘 Model details

• Base model: google/bigbird-roberta-base
• Architecture: BigBirdForSequenceClassification
• Hidden size: 768
• Layers: 12 transformer blocks
• Attention type: block-sparse
• Max sequence length: 4096 tokens
• Number of classes: 10 [ratings from 1 to 10 (extremely negative/extremely positive)]

🧠 Training Summary

• Dataset: 2,598 airline passenger reviews.
• Labels: ordinal scale from 1 (extremely negative) to 10 (extremely positive).
• Loss function: cross-entropy (classification setup).

🛠 Tokenizer

• Based on SentencePiece Unigram model.
• Uses a Metaspace tokenizer for subword splitting.
• Max tokenised input length was set to 1024 tokens during preprocessing.

📌 Use cases

• Analyse detailed customer reviews of flight experience.
• Replace coarse binary sentiment models with ordinal sentiment scales.
• Experiment with ordinal regression techniques in NLP.

📚 Citation

If you're using this model in your research or applications, appreciate if you could buy me a coffee through this citation.

Mat Roni, S. (2025). Big Bird Flight 2 for ordinal sentiment analysis [software]. Hugging Face. https://huggingface.co/pvaluedotone/bigbird-flight-2 DOI: https://doi.org/10.57967/hf/5780

Validation metrics

• loss: 1.6761
• f1_macro: 0.2734
• f1_micro: 0.3093
• f1_weighted: 0.2814
• precision_macro: 0.2822
• precision_micro: 0.3093
• precision_weighted: 0.2911
• recall_macro: 0.3007
• recall_micro: 0.3093
• recall_weighted: 0.3093
• accuracy: 0.3093