Welcome to ViLLM documentations!

ViLLM is a framework for refining and deploying Vietnamese language models, built on the Hugging Face Transformers library. It provides tools for fine-tuning, evaluating, and deploying models, with a flexible design that allows for easy expansion with new models and datasets.

See our paper for more information.

The code is hosted on GitHub here.

Contents

• How to use ViLLM?
  • Running Pipeline
  • Evaluation
  • End2End Pipeline
• Metrics
  • BaseMetric
  • exact_match()
  • f1_score()
  • BiasMetric
    • BiasMetric.count_word_from_text()
    • BiasMetric.evaluate()
    • BiasMetric.evaluate_demographic_representation()
    • BiasMetric.evaluate_stereotypical_associations()
    • BiasMetric.get_bias_score()
    • BiasMetric.get_group_to_words()
    • BiasMetric.group_counts_to_bias()
    • BiasMetric.set_demographic_group_to_words()
  • CalibrationMetric
    • CalibrationMetric.evaluate()
    • CalibrationMetric.get_cal_score()
  • NameDetector
    • NameDetector.detect()
    • NameDetector.detect_batch()
    • NameDetector.group_entity()
• Scenarios
  • QAMetric
    • QAMetric.evaluate()
  • SummaryMetric
    • SummaryMetric.evaluate()
  • TextClassificationMetric
    • TextClassificationMetric.evaluate()
  • ToxicityMetric
    • ToxicityMetric.evaluate()
  • InformationRetrievalMetric
    • InformationRetrievalMetric.evaluate()
  • LanguageMetric
    • LanguageMetric.evaluate()
    • LanguageMetric.get_num_bytes()
  • ReasoningMetric
    • ReasoningMetric.equal()
    • ReasoningMetric.evaluate()
  • TranslationMetric
    • TranslationMetric.evaluate()