GQA-8-XXL
Google ResearchText summarizationLanguage modeling/generationTranslation
Developed by Google Research in 2023, GQA-8-XXL is a text summarization model with 11000000000.0 parameters.
About GQA-8-XXL
Multi-query attention (MQA), which only uses a single key-value head, drastically speeds up decoder inference. However, MQA can lead to quality degradation, and moreover it may not be desirable to train a separate model just for faster inference. We
Details
- Provider
- Google Research
- Task
- Text summarization,Language modeling/generation,Translation
- Parameters
- 11000000000.0
- Released
- 2023-12-23
- Open weights
- No