Hymba: A Hybrid-head Architecture for Small Language Models by Dong, Xin, Fu, Yonggan, Diao, Shizhe, Byeon, Wonmin, Chen, Zijia, Ameya Sunil Mahabaleshwarkar, Shih-Yang, Liu, Matthijs Van Keirsbilck, Min-Hung, Chen, Suhara, Yoshi, Lin, Yingyan, Kautz, Jan, Molchanov, Pavlo

Get full text

LLM Pruning and Distillation in Practice: The Minitron Approach by Sreenivas, Sharath Turuvekere, Muralidharan, Saurav, Joshi, Raviraj, Chochowski, Marcin, Ameya Sunil Mahabaleshwarkar, Shen, Gerald, Zeng, Jiaqi, Chen, Zijia, Suhara, Yoshi, Diao, Shizhe, Yu, Chenhan, Wei-Chun, Chen, Ross, Hayley, Olabiyi, Oluwatobi, Aithal, Ashwath, Kuchaiev, Oleksii, Korzekwa, Daniel, Molchanov, Pavlo, Patwary, Mostofa, Shoeybi, Mohammad, Kautz, Jan, Catanzaro, Bryan

Get full text

Nemotron-4 340B Technical Report by Nvidia, Adler, Bo, Agarwal, Niket, Aithal, Ashwath, Anh, Dong H, Bhattacharya, Pallab, Brundyn, Annika, Casper, Jared, Catanzaro, Bryan, Clay, Sharon, Cohen, Jonathan, Das, Sirshak, Dattagupta, Ayush, Delalleau, Olivier, Derczynski, Leon, Dong, Yi, Egert, Daniel, Evans, Ellie, Ficek, Aleksander, Fridman, Denys, Ghosh, Shaona, Ginsburg, Boris, Gitman, Igor, Grzegorzek, Tomasz, Hero, Robert, Huang, Jining, Jawa, Vibhu, Jennings, Joseph, Jhunjhunwala, Aastha, Kamalu, John, Khan, Sadaf, Kuchaiev, Oleksii, LeGresley, Patrick, Li, Hui, Liu, Jiwei, Liu, Zihan, Long, Eileen, Ameya Sunil Mahabaleshwarkar, Majumdar, Somshubra, Maki, James, Martinez, Miguel, Maer Rodrigues de Melo, Moshkov, Ivan, Narayanan, Deepak, Narenthiran, Sean, Navarro, Jesus, Nguyen, Phong, Nitski, Osvald, Noroozi, Vahid, Nutheti, Guruprasad, Parisien, Christopher, Parmar, Jupinder, Patwary, Mostofa, Pawelec, Krzysztof, Wei, Ping, Prabhumoye, Shrimai, Roy, Rajarshi, Saar, Trisha, Vasanth Rao Naik Sabavat, Satheesh, Sanjeev, Jane Polak Scowcroft, Sewall, Jason, Shamis, Pavel, Shen, Gerald, Shoeybi, Mohammad, Sizer, Dave, Smelyanskiy, Misha, Soares, Felipe, Makesh Narsimhan Sreedhar, Su, Dan, Subramanian, Sandeep, Sun, Shengyang, Toshniwal, Shubham, Wang, Hao, Wang, Zhilin, You, Jiaxuan, Zeng, Jiaqi, Zhang, Jimmy, Zhang, Jing, Zhang, Vivienne, Zhang, Yian, Chen, Zhu

Get full text