XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval
Mohammad Abdullah Matin Khan, Mehwish Bari, Xuan Long, Weishi Wang, Md Rizwan Parvez, Shafiq Joty
Abstract
Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, Shafiq Joty. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
Topics & Concepts
Computer scienceBenchmark (surveying)Machine translationCode (set theory)Translation (biology)Scale (ratio)Code generationNatural language processingArtificial intelligenceParallel computingProgramming languageComputer architectureOperating systemQuantum mechanicsSet (abstract data type)PhysicsGeodesyGeneBiochemistryMessenger RNAKey (lock)GeographyChemistryNatural Language Processing Techniques