When training a language model using large amounts of text, much information can be packed into the language model. In addition, the techniques are being developed to pull the desired information out of the language model. These have led to increase attempts to use language models to solve existing AI problems and to define and solve problems in new areas.
We work on large language model technologies so that language models can do more things with more information. We also work on techniques to optimize models for real-world applications and research to increase their reliability when performing tasks. By realizing the high potential of large language models, we aim to develop technologies that enable AI to benefit human life.
Model ArchitectureWe are devising new model architectures to improve the performance of language models. In this research, we explore various neural network architectures, including transformers, and validate their performance.
Pre-trainingWe research technologies for pre-training language models by utilizing large amounts of text data. Various training objectives and transfer learning techniques are applied to maximize the performance of large language models.
Multi-lingual ModelingBy training a language model to understand multiple languages, knowledge written in different languages can be acquired. This allows the lack of knowledge in resource-poor languages can be alleviated.
SafetyThe responses that a language model generates are based on the training data. Traning data can contain biased, unfair, or personal content, and these can cause problems. To handle this, we develop technologies that detect such inappropriate expressions and avoid them
Model CompressionIn order to apply large language models to real-world applications, optimizing the operating costs is important. We reduce the computational costs while minimizing the performance degradation using model compression techniques: knowledge distillation, pruning, quantization, and so on.
Meet our leader!
Jinsik Lee, Head of EXAONE Lab
“All members of EXAONE Lab are united with passion and spirit. When I have a question, I have the qualities of both a professional scientist who persistently digs until I find the answer I want, and a professional engineer who pursues perfect and clean work. In particular, I would like to boast that we are elite members who have valuable experience handling ultra-large AI models that cannot be experienced anywhere else, and who also possess a very high level of expertise in their respective fields in ultra-large AI models. .”