theluyuan/RWKV-Runner
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix the prompt cache that contains potential error

Browse Source
This commit is contained in:
josc146 2023-06-21 16:07:16 +08:00
parent 1465908574
commit 131a7ddf4a
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
backend-python/utils/rwkv.py
View File

@ -325,7 +325,6 @@ The following is a coherent verbose detailed conversation between a girl named {
response += delta
if stop is not None:
if stop in response:
response = response.split(stop)[0]
try:
state_cache.add_state(
state_cache.AddStateBody(
@ -337,6 +336,7 @@ The following is a coherent verbose detailed conversation between a girl named {
)
except HTTPException:
pass
response = response.split(stop)[0]
yield response, "", prompt_token_len, completion_token_len
break
out_last = begin + i + 1