VMLU
Click to visit website
About
VMLU is a human-centric benchmark suite designed to assess the overall capabilities of foundation models, specifically for the Vietnamese language. It comprises four distinct datasets: Vi-MQA, Vi-SQuAD, Vi-DROP, and Vi-Dialog, each targeting different aspects of LLM performance, including general knowledge, reading comprehension, logical reasoning, and conversational ability. Vi-MQA, for instance, is a multiple-choice question answering benchmark with 58 subjects across STEM, Humanities, Social Sciences, and 'Others', covering various difficulty levels. The dataset primarily originates from examinations by esteemed educational institutions and the Ministry of Education and Training. By providing comprehensive and diverse evaluation tasks, VMLU enriches Vietnamese NLP evaluation, driving the development of more robust foundation models and encouraging further research in LLMs. Datasets are available for download, and a GitHub repository offers extensive information, benchmarking results, and replication code.
Platform
Features
• vietnamese multitask language understanding benchmark
• accessible datasets and benchmarking code
• support for various difficulty levels (elementary to professional)
• 58 distinct subjects across diverse domains
• vi-dialog: dialogue dataset
• vi-drop: discrete reasoning over paragraphs
• vi-squad: stanford question answering dataset
• vi-mqa: multiple-choice question answering
Pricing Plans
Free
Free Plan• Access to all VMLU datasets
• GitHub repository access
• Benchmarking code
• Publicly available model results
Job Opportunities
There are currently no job postings for this AI tool.
Ratings & Reviews
No ratings available yet. Be the first to rate this tool!
Alternatives
Needle-in-a-Needlestack
Needle-in-a-Needlestack is an open-source platform for benchmarking large language models on their long-context understanding and retrieval capabilities.
View DetailsFeatured Tools
GirlfriendGPT
NSFW AI chat platform with customizable characters, AI image generation, and voice chat. Explore roleplay and intimate interactions with AI companions.
View Details
xMates AI
xMates AI is a next-generation AI chat app powered by large language models, offering human-like interactions and roleplaying with customizable AI characters.
View Details
AI Song Maker
AI Song Maker is an AI music generator that helps users create songs effortlessly. Compose tracks, generate AI songs, and enjoy royalty-free music creation with ease.
View Details
Wan 2.5
Wan 2.5 is a revolutionary native multimodal video generation platform. It features synchronized A/V output, 1080p HD cinematic quality, and precision image editing.
View Detailsnexos.ai
nexos.ai is an all-in-one AI platform for enterprises, enabling secure, organization-wide AI adoption, policy setting, and oversight for tech leaders.
View Details
Sora 2 AI
Sora 2 AI is the next generation AI video generator, creating more realistic, controllable, and immersive videos that understand the laws of physics.
View DetailsYamiTools
YamiTools is an innovative AI platform that helps content creators and businesses generate text, images, and code effortlessly, enhancing productivity and creativity.
View Details