theluyuan/RWKV-Runner
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix the prompt cache that contains potential error

Browse Source
This commit is contained in:
josc146 2023-06-21 16:07:16 +08:00
parent 1465908574
commit 131a7ddf4a
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
backend-python/utils/rwkv.py
View File

@ -325,7 +325,6 @@ The following is a coherent verbose detailed conversation between a girl named {
response += delta response += delta
if stop is not None: if stop is not None:
if stop in response: if stop in response:
response = response.split(stop)[0]
try: try:
state_cache.add_state( state_cache.add_state(
state_cache.AddStateBody( state_cache.AddStateBody(
@ -337,6 +336,7 @@ The following is a coherent verbose detailed conversation between a girl named {
) )
except HTTPException: except HTTPException:
pass pass
response = response.split(stop)[0]
yield response, "", prompt_token_len, completion_token_len yield response, "", prompt_token_len, completion_token_len
break break
out_last = begin + i + 1 out_last = begin + i + 1