This is the Mixtral MoE model, the latest and largest mixture of experts (MoE) language model from Mistral AI. It comes in two flavors: 8x7B and 8x22B. The model utilizes a mixture of 8 expert models, each with 7 resp. 22 billion parameters. During inference, two of these expert models are selected to generate the output.
Chat templates
Top K
Top P
Temperature
Maximum response length in tokens
Progress
Console Output
JSON Example
API Call (Text)
{
"text":"A dialog, where User interacts with an helpful, kind, obedient, honest and very reasonable assistant called Chloe.\nUser: Hello, Chloe.\nChloe: How can I assist you today?\nUser: What is your favourite movie?\nChloe:",
"top_k":40,
"top_p":0.9,
"temperature":0.8,
"client_session_auth_key":"e1b12673-3beb-42eb-bd2c-1747a11dd084",
"wait_for_result":false
}
Progress
{
"success":true,
"job_id":"JID43",
"ep_version":0,
"job_state":"processing",
"progress":{
"job_id":"JID43",
"start_time":1708257097.326644,
"start_time_compute":1708257097.3285873,
"progress":3,
"progress_data":{
"text":"Oh, ",
"num_generated_tokens":3
},
"estimate":1.2,
"queue_position":0,
"num_workers_online":1
}
}
Result
{
"success":true,
"job_id":"JID43",
"ep_version":0,
"job_result":{
"success":true,
"job_id":"JID43",
"ep_version":0,
"text":"Oh, that's a tough one! I have so many favourite movies, but if I had to choose just one, I would say \"Toy Story\". It's such a classic and it always makes me laugh.\n",
"num_generated_tokens":52,
"model_name":"mixtral-8x7b-instruct",
"compute_duration":1.8,
"total_duration":1.8,
"auth":"neo08_NVIDIA A100-PCIE-80GB_0_1",
"worker_interface_version":"AIME-API-Worker-Interface 0.8.1"
},
"job_state":"done",
"progress":{
"job_id":"JID43",
"start_time":1708257097.326644,
"start_time_compute":1708257097.3285873,
"progress":45,
"progress_data":{
"text":"Oh, that's a tough one! I have so many favourite movies, but if I had to choose just one, I would say \"Toy Story\". It's such a classic and it ",
"num_generated_tokens":45
},
"estimate":0,
"queue_position":0,
"num_workers_online":1
}
}