LEPISZCZE is an open-source benchmark and a continuous-submission leaderboard, concentrating public Polish datasets (existing and new) in specific tasks. Integrating datasets and tasks with model performance and efficiency allows academia and industry to gauge performance on tasks of interest quickly. Finally, it intends to foster constructive competition and innovation by bringing together and promoting previously disparate resources.
Augustyniak, L.; Tagowski, K.; Sawczyn, A.; Janiak, D.; Bartusiak, R.; Szymczak, A.; Janz, A.; Szymański, P.; Wątroba, M.; Morzy, M.; Kajdanowicz, T.; Piasecki, M. This Is the Way: Designing and Compiling LEPISZCZE, a Comprehensive NLP Benchmark for Polish. In Advances in Neural Information Processing Systems; Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A., Eds.; Curran Associates, Inc., 2022; Vol. 35, pp 21805–21818.
@inproceedings{augustyniak2023lepiszcze,
author = {Augustyniak, Lukasz and Tagowski, Kamil and Sawczyn, Albert and Janiak, Denis and Bartusiak, Roman and Szymczak, Adrian and Janz, Arkadiusz and Szyma\'{n}ski, Piotr and W\k{a}troba, Marcin and Morzy, Miko\l aj and Kajdanowicz, Tomasz and Piasecki, Maciej},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {21805--21818},
publisher = {Curran Associates, Inc.},
title = {This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/890b206ebb79e550f3988cb8db936f42-Paper-Datasets_and_Benchmarks.pdf},
volume = {35},
year = {2022}
}
In case of any question or concerns about LEPISZCZE benchmark feel free to contact us:
- Łukasz lukasz.augustyniak@pwr.edu.pl
- Kamil kamil.tagowski@pwr.edu.pl
- Albert albert.sawczyn@pwr.edu.pl
- Denis denis.janiak@pwr.edu.pl
The work was partially supported by (1) the Polish Ministry of Education and Science, CLARIN-PL; (2) the European Regional Development Fund as a part of the 2014-2020 Smart Growth Operational Programme, CLARIN – Common Language Resources and Technology Infrastructure, and (3) the Department of Artificial Intelligence at Wroclaw University of Science.