Multiple chat completions for each input message

[apepkuss]

Hi guys,
OpenAI defines a param n in the chat completion request, which is used to specify how many chat completion choices to generate for each input message. Does llama.cpp provide support for multiple chat completions? Thanks a lot!

[dspasyuk]

@apepkuss You can just ask a model to generate you n-number of responses,

[ngxson]

We don't support that for now, but you can have the same result by passing an array of prompts to /completions.

For example, this is equivalent to n=4:

POST /completions

{
  "prompts": [
    "what is 1+1",
    "what is 1+1",
    "what is 1+1",
    "what is 1+1"
  ]
}

Remember to set a high temperature