Standard machine learning systems require massive data and huge processing infrastructures, but the main limitation to their spreading comes from the need of the empirical and rare knowledge of an experienced data scientist able to set and adjust their behaviour over time. The ALLIES project will lay the foundation for development of autonomous intelligent systems sustaining their performance across time. Such unsupervised system will be able to auto-update and perform self-evaluation to be aware of the evolution of its own knowledge acquisition. It should adapt to a changing environment by following a given learning scenario that balances the importance of performance on past and present data to avoid unwanted regression. Such systems could not be developed without adapted metrics and protocols enabling their objective and reproducible evaluation. This evaluation should continuously assess the performance on the given task and quantify the effort required to reach it in terms of unsupervised data collected by the system and of interaction with humans in the case of active-learning. The ALLIES project will develop, evaluate and disseminate those metrics and protocols. They will be available to European actors via an open evaluation platform dedicated to reproducible research. An evaluation campaign and a workshop will be organised to engage the community on this path. By publicly releasing the evaluation protocols and data, by setting up a dedicated evaluation platform and by developing autonomous systems for two tasks: machine translation and speaker diarization, we believe that the ALLIES project will boost the development of intelligent lifelong learning systems in Europe.
Biesialska, M.; Biesialska, K.; Costa-jussà, Marta R. International Conference on Computational Linguistics p. 6523-6541 DOI: 10.18653/v1/2020.coling-main.574 Presentation's date: 2020-12-08 Presentation of work at congresses
Escolano, C.; Costa-jussà, Marta R.; Fonollosa, José A. R. Journal of the Association for Information Science and Technology Vol. 72, num. 2, p. 190-203 DOI: 10.1002/asi.24395 Date of publication: 2020-08-02 Journal article
Biesialska, M.; Rafieian, B.; Costa-jussà, Marta R. Annual Meeting of the Association for Computational Linguistics p. 271-278 DOI: 10.18653/v1/2020.acl-srw.36 Presentation's date: 2020-07-05 Presentation of work at congresses
Basta, C.; Costa-jussà, Marta R.; Fonollosa, José A. R. Widening Natural Language Processing Workshop p. 99-102 DOI: 10.18653/v1/2020.winlp-1.25 Presentation's date: 2020-07-04 Presentation of work at congresses
Biesialska, M.; Guàrdia Fernández, Lluís; Costa-jussà, Marta R. WMT - Conference on Machine Translation p. 185-191 DOI: 10.18653/v1/W19-5424 Presentation's date: 2019-08-02 Presentation of work at congresses
Escolano, C.; Costa-jussà, Marta R.; Fonollosa, José A. R. Annual Meeting of the Association for Computational Linguistics p. 236-242 DOI: 10.18653/v1/P19-2033 Presentation's date: 2019-07-30 Presentation of work at congresses