Why is the Inkling model considered a mixture of experts rather than a dense model?

Inkling is categorized as a mixture of experts because, despite having 975 billion total parameters, only a small fraction—approximately 41 billion—are active at any given moment. This architecture differs from the dense models common in earlier years, allowing for greater efficiency and flexibility in performance. This design choice supports the model's primary purpose of being a balanced, adaptable foundation for fine-tuning.