Faster responses?
I am using the MarinaraSpaghetti_NemoMix-Unleashed-12B model. I have a RTX 3070s but the responses take forever. Is there any way to make it faster? I am new to oobabooga so I did not change any settings.
I am using the MarinaraSpaghetti_NemoMix-Unleashed-12B model. I have a RTX 3070s but the responses take forever. Is there any way to make it faster? I am new to oobabooga so I did not change any settings.