Bug: Generating in multiple threads at once causes tokens to get mixed together
planned
Naveen
planned
Naveen
under review
Naveen
Ooo, this is weird bug.
Do you want to stop the thread in the middle (this is how ChatGPT handles it) or run multiple threads simultaneously (which might technically take more time)?
Faiz
Naveen: I think both are reasonable solutions (of course, multiple threads is nicer). The practical scenarios in which I've run into this issue are when GPT-4 is rambling on for an extended period and I have some other question. So it might also be good in this vein to just have a button to stop generation within a thread.
Naveen
Faiz: Yeah, I have to add stop generation button.
Long term I want to have parallel multiple threads generation