Model Overview DBRX is a transformerbased decoderonly large language model (LLM) that was trained using nexttoken prediction. It uses a finegrained mixtureofexperts (MoE) architecture with 132B total parameters of which 36B parameters are active on
Model Overview DBRX is a transformerbased decoderonly large language model (LLM) that was trained using nexttoken prediction. It uses a finegrained mixtureofexperts (MoE) architecture with 132B total parameters of which 36B parameters are active on
Databricks' dollyv212b, an instructionfollowing large language model trained on the Databricks machine learning platform that is licensed for commercial use. Based on pythia12b, Dolly is trained on ~15k instruction/response fine tuning records [databricksdolly15k](https://github.com/data