VMLU
Click to visit website
About
VMLU is a human-centric benchmark suite designed to assess the overall capabilities of foundation models, specifically for the Vietnamese language. It comprises four distinct datasets: Vi-MQA, Vi-SQuAD, Vi-DROP, and Vi-Dialog, each targeting different aspects of LLM performance, including general knowledge, reading comprehension, logical reasoning, and conversational ability. Vi-MQA, for instance, is a multiple-choice question answering benchmark with 58 subjects across STEM, Humanities, Social Sciences, and 'Others', covering various difficulty levels. The dataset primarily originates from examinations by esteemed educational institutions and the Ministry of Education and Training. By providing comprehensive and diverse evaluation tasks, VMLU enriches Vietnamese NLP evaluation, driving the development of more robust foundation models and encouraging further research in LLMs. Datasets are available for download, and a GitHub repository offers extensive information, benchmarking results, and replication code.
Platform
Features
• vietnamese multitask language understanding benchmark
• accessible datasets and benchmarking code
• support for various difficulty levels (elementary to professional)
• 58 distinct subjects across diverse domains
• vi-dialog: dialogue dataset
• vi-drop: discrete reasoning over paragraphs
• vi-squad: stanford question answering dataset
• vi-mqa: multiple-choice question answering
Pricing Plans
Free
Free Plan• Access to all VMLU datasets
• GitHub repository access
• Benchmarking code
• Publicly available model results
Job Opportunities
There are currently no job postings for this AI tool.
Ratings & Reviews
No ratings available yet. Be the first to rate this tool!
Alternatives
Needle-in-a-Needlestack
Needle-in-a-Needlestack is an open-source platform for benchmarking large language models on their long-context understanding and retrieval capabilities.
View Details
Rawbot
Rawbot is a platform designed to effortlessly compare various AI models, helping users unlock their full potential and choose the best fit for their projects.
View DetailsFeatured Tools
GirlfriendGPT
NSFW AI chat platform with customizable characters, AI image generation, and voice chat. Explore roleplay and intimate interactions with AI companions.
View DetailsPDF Translator
PDF Translator is an AI-powered tool for instant document translations. Upload PDFs, select from 100+ languages, and get format-preserving translations for free.
View Details
DeVoice
DeVoice is an AI-powered audio and video tool that offers unlimited, accurate transcription, AI rap generation, and background noise removal capabilities.
View Details
DeepSwapAI
DeepSwapAI is a professional AI face swap platform for developers, offering enterprise-grade face exchange technology with RESTful API, SDKs, and batch processing.
View DetailsFace Swap AI
Face Swap AI is a free AI tool for instant face swapping in photos and videos, delivering stunning HD results without signup or watermarks for creative projects.
View Details
StoryShort
StoryShort is an AI creation tool that helps you create viral faceless videos on auto-pilot, generating engaging content in minutes.
View Details
AIhumanize
AIhumanize is an advanced AI humanizer tool that transforms AI-written text into natural, authentic writing, helping you bypass all major AI detectors.
View Details
LoveGen AI
LoveGen AI is an all-in-one platform integrating major image and video AI models, enabling creation from text, visual enhancement, and video generation.
View Details
Capacity
Capacity is an AI tool that helps you turn any idea into a working web app, including fullstack applications and cloned websites, without writing code.
View DetailsNano Banana Pro
Nano Banana Pro is a reasoning-first 4K AI image editor designed for creative teams to generate lossless 4K visuals, transparent PNGs, and high-quality exports.
View Details
ImageTranslator
ImageTranslator is an AI-powered online tool that translates text in images instantly, supporting over 100 languages while preserving original layout.
View DetailsSeedance 2
Seedance 2 is a groundbreaking AI video generation technology that delivers 1080p cinematic quality with advanced motion synthesis and multi-shot storytelling.
View Details
KissGen AI
KissGen AI is the best AI kissing video generator, transforming memories into lifelike kissing videos with realistic animations and custom styles.
View DetailsGempix2 AI
Gempix2 AI is a free online AI photo and image editor, powered by NanoBanana 2 technology, offering advanced tools for professional-quality visual transformations.
View Details
AI Animate Image
AI Animate Image revolutionizes how you create animated content from static images. Our advanced AI image animator turns photos into animation with stunning realism.
View Details